Like most industries, the fields of security, access and safety have been transformed by technology, with AI-driven automation presenting a clear opportunity for players seeking growth and leadership when it comes to innovation.

In this respect, these markets know exactly what they want. They require solutions that accurately (without false or negative positives) classify and track people and/or vehicles as well as the precise location and interactions between those objects. They want to have access to accurate data generated by best-of-class solutions irrespective of the sensor modality. And, they need to be able to easily deploy such solutions, at the lowest capex and opex, with the knowledge that they can be integrated with preferred VMSs and PSIMs, be highly reliable, have low install and maintenance overheads and be well supported.

With these needs in mind, camera and computer vision technology providers, solutions providers and systems integrators are forging ahead and have created exemplary ecosystems with established partnerships helping to accelerate adoption. At the heart of this are AI and applications of Convolutional neural networks (CNN), an architecture often used in computer vision deep learning algorithms, which are accomplishing tasks that were extremely difficult with traditional software.

But what about 3D sensing technologies and perception?

The security, safety and access market have an additional crucial need: they must mitigate risk and make investments that deliver for the long-term. This means that if a systems integrator invests in a 3D sensing data perception platform today, it will support their choice of sensors, perception strategies, applications and use cases over time without having to constantly reinvest in alternative computer hardware and perception software each time they adopt new technology or systems.

This begs the question - if the security industry knows what it needs, why is it yet to fully embrace 3D sensing modalities?

Perception strategy

Intelligent perception strategies are yet to evolve which sees designers lock everything down at the design phase Well, one problem facing security, safety and access solutions providers, systems integrators and end-users when deploying first-generation 3D sensing-based solutions is the current approach.

Today, intelligent perception strategies have yet to evolve beyond the status quo which sees designers lock everything down at the design phase, including the choice of the sensor(s), off-the-shelf computer hardware and any vendor-specific or 3rd party perception software algorithms and deep learning or artificial intelligence.

This approach not only builds in constraints for future use-cases and developments, it hampers the level of perception developed by the machine. Indeed, the data used to develop or train the perception algorithms for security, access and safety use cases at design time is typically captured for a narrow and specific set of scenarios or contexts and are subsequently developed or trained in the lab.

Technology gaps

As those in this industry know too well, siloed solutions and technology gaps typically block the creation of productive ecosystems and partnerships while lack of commercial whole products can delay market adoption of new innovation. Perception systems architectures today do not support the real-time adaptation of software and computing engines in the field. They remain the same as those selected during the design phase and are fixed for the entire development and the deployment stage.

Crucially, this means that the system cannot deal with the unknowns of contextually varying real-time situations where contexts are changing (e.g being able to reflex to security situations they haven’t been trained for) and where the autonomous system’s perception strategies need to dynamically adjust accordingly. Ultimately, traditional strategies have non-scalable and non-adaptable competing computing architectures that were not designed to process the next generation of algorithms, deep learning and artificial intelligence required for 3D sensor mixed workloads.

What this means for industries seeking to develop or deploy perception systems, like security, access and safety, is that the available computing architectures are generic and designed for either graphic rendering or data processing. Solutions providers, therefore, have little choice but to promote these architectures heavily into the market. Consequently, the resulting computing techniques are defined by the computing providers and not by the software developers working on behalf of the customer deploying the security solution.

Context…. we don’t know what we don’t know

Perception platform must have the ability to adjust to changes in context, thereby improving the performance post-deployment To be useful and useable in the security context and others, a perception platform must have the ability to adjust to changes in context, can self-optimise and crucially, can self-learn, thereby improving the performance post-deployment. The combinations of potential contextual changes in a real-life environment, such as an airport or military base, are innumerable, non-deterministic, real-time, often analogue and unpredictable.

The moment sensors, edge computing hardware and perception software are deployed in the field, myriad variables such as weather, terrain as well as sensor mounting location and orientation all represent a context shift where the perception systems’ solution is no longer optimal. For example, it might be that a particular sensor system is deployed in an outdoor scenario with heavy foliage.

Because the algorithm development or training was completed in the lab, the moving foliage, bushes or low trees and branches are classified as humans or some other false-positive result. Typically, heavy software customisation and onsite support then ensue, requiring on-site support by solutions vendors where each and every sensor configuration needs to be hand-cranked to deliver something that is acceptable to the end customer.

A new approach for effective perception strategies

Cron AI is building senseEDGE, which represents a significant evolution in the development of sensing to information strategy.  It is a 3D sensing perception and computer vision platform built from the ground up to address and remove the traditional deployment and performance bottlenecks we’ve just described.

senseEDGE is aware of the user application reaction plan indication to trigger an alarm or turning on a CCTV camera The entire edge platform is built around a real-time scalable and adaptable computing architecture that’s flexible enough for algorithms and software to scale and adapt to different workloads and contexts. What’s more, it has real-time contextual awareness, which means that the entire edge platform is, at any time, aware of the external context, the sensor and sensor architecture and the requirements of the user application.

Furthermore, when it produces the object output data, it also aware of the user application reaction plan indication, which could be triggering an alarm or turning on a CCTV camera when a specific action is detected. This approach turns traditional perception strategies on their head: it is software-defined programmable perception and computing architecture, not hardware-defined.

It is free from the constraints imposed by traditional CPU or GPU compute dictated by hardware architecture providers and not limited to the perception built defined during design time. And, being fully configurable, it can be moved from one solution to another, providing computation for different modalities of sensors designed for different use cases or environments, and lower risk of adoption and migration for those developing the security solution. 

Future perception requirements

senseEDGE is also able to scale to future perception requirements, such as algorithms and workloads produced by future sensors as well as computational techniques and neural networks that have yet to be invented. Meanwhile, latency versus throughput is totally software-defined and not limited by providers of computing architecture.

Finally, contextually aware, it is fully connected to the real world where the reflexes adapt to even the subtlest changes in context, which makes all the difference in time and accuracy in critical security situations.

This is how CronAI sees the future of perception. It means that security and safety innovators can now access and invest with low risk in a useable and scalable perception solution that can truly take advantage of current and future 3D sensor modalities.

Download PDF version Download PDF version

Author profile

Neil Huntingdon Group Chief Strategy Officer, CronAI

In case you missed it

How AI and security guards work together using video analytics
How AI and security guards work together using video analytics

How AI and humans can work together is a longstanding debate. As society progresses technologically, there’s always the worry of robots taking over jobs. Self-checkout tills, automated factory machines, and video analytics are all improving efficiency and productivity, but they can still work in tandem with humans, and in most cases, they need to. Video analytics in particular is one impressively intelligent piece of technology that security guards can utilise. How can video analytics help with certain security scenarios? Video analytics tools Before video analytics or even CCTV in general, if a child went missing in a shopping centre, we could only rely on humans. Take a crowded Saturday shopping centre, a complex one with a multitude of shops and eateries, you’d have to alert the security personnel, rely on a tannoy and search party, and hope for a lockdown to find a lost or kidnapped child. With video analytics, how would this scenario play out? It’s pretty mind-blowing. As soon as security is alerted, they can work with the video analytics tools to instruct it precisely With the same scenario, you now have the help of many different cameras, but then there’s the task of searching through all the CCTV resources and footage. That’s where complex search functions come in. As soon as security is alerted, they can work with the video analytics tools to instruct it precisely on what footage to narrow down, and there’s a lot of filters and functions to use. Expected movement direction For instance, they can tick a ‘human’ field, so the AI can track and filter out vehicles, objects etc., and then they can input height, clothing colours, time the child went missing, and last known location. There’s a complex event to check too, under ‘child kidnap’. For a more accurate search, security guards can then add in a searching criterion by drawing the child’s expected movement direction using a visual query function. A unique function like this enables visual criteria-based searches rather than text-based ones. The tech will then narrow down to the images/videos showing the criteria they’ve inputted, showing the object/child that matches the data and filter input. Detecting facial data There are illegal demonstrations and troublesome interferences that police have to deal with A white-list face recognition function is then used to track the child’s route which means the AI can detect facial data that has not been previously saved in the database, allowing it to track the route of a target entity, all in real time. Then, security guards can confirm the child’s route and current location. All up-to-date info can then be transferred to an onsite guard’s mobile phone for them to confirm the missing child’s movement route, face, and current location, helping to find them as quickly as possible. Often, there are illegal demonstrations and troublesome interferences that police have to deal with. Video analytics and surveillance can not only capture these, but they can be used to predict when they may happen, providing a more efficient process in dealing with these types of situations and gathering resources. Event processing functions Picture a public square with a number of entries into the main area, and at each entry point or path, there is CCTV. Those in the control room can set two events for each camera: a grouping event and a path-passing event. These are pretty self-explanatory. A grouping event covers images of seeing people gathering in close proximity and a path-passing event will show when people are passing through or entering. The video analytics tool can look out for large gatherings and increased footfall to alert security By setting these two events, the video analytics tool can look out for large gatherings and increased footfall to alert security or whoever is monitoring to be cautious of protests, demonstrations or any commotion. Using complex event processing functions, over-detection of alarms can also be prevented, especially if there’s a busy day with many passing through. Reducing false alarms By combining the two events, that filters down the triggers for alarms for better accuracy to predict certain situations, like a demonstration. The AI can also be set to only trigger an alarm when the two events are happening simultaneously on all the cameras of each entry to reduce false alarms. There are so many situations and events that video analytics can be programmed to monitor. You can tick fields to monitor any objects that have appeared, disappeared, or been abandoned. You can also check events like path-passing to monitor traffic, as well as loitering, fighting, grouping, a sudden scene change, smoke, flames, falling, unsafe crossing, traffic jams and car accidents etc. Preventing unsafe situations Complex events can include violations of one-way systems, blacklist-detected vehicles Complex events can include violations of one-way systems, blacklist-detected vehicles, person and vehicle tracking, child kidnaps, waste collection, over-speed vehicles, and demonstration detections. The use of video analytics expands our capabilities tremendously, working in real time to detect and help predict security-related situations. Together with security agents, guards and operatives, AI in CCTV means resources can be better prepared, and that the likelihood of preventing unsafe situations can be greatly improved. It’s a winning team, as AI won’t always get it right but it’s there to be the advanced eyes we need to help keep businesses, premises and areas safer.

The EU called for a ban on police use of facial recognition but not commercial use. Why?
The EU called for a ban on police use of facial recognition but not commercial use. Why?

Recently, the European Parliament called for a ban on police use of facial recognition. In the US, too, some cities have restricted police use of facial recognition. The first question that comes to mind is - why ban police from using technology that is allowed to private companies? Point of difference The key difference between the way police use facial recognition and the way commercial facial recognition products work is that: The police get a picture of a suspect from a crime scene and want to find out: "Who is the person in the picture?" That requires as wide a database as possible. Optimally - photos and identities of all the people in the world. Commercial facial recognition products such as those used by supermarkets, football stadiums, or casinos answer different questions: "Is the person in the picture on the employees' list? Is the person in the picture on a watch-list of known shoplifters?" To answer these questions doesn't require a broad database but rather a defined list of employees or a watch-list of specific people against whom there is an arrest warrant or a restraining order. Use of facial recognition AnyVision helps organisations leverage facial recognition ethically to identify known persons of interest "Facial Recognition Apps Should Be Provided to the Police with an Empty Database". This is exactly the subject of the open letter sent by AnyVision, to the British Biometrics and Surveillance Camera Commissioner, Prof. Fraser Sampson, titled: "Facial Recognition Apps Should Be Provided to the Police with an Empty Database". AnyVision recently raised $235M from Softbank and another leading VCs is a visual AI platform company that helps organisations across the globe leverage facial recognition ethically to identify known persons of interest, including shoplifters, felons, and security threats. Ethical use of facial recognition AnyVision CEO Avi Golan wrote, "The ethical use of facial recognition is a thorny one and requires a nuanced discussion. Part of that discussion has to explain how facial recognition works, but, just as important, the discussion must also involve how the technology is used by police departments and what checks and balances are built into their processes.” “We recommend building their watchlists from the ground up based on known felons, persons of interest, and missing persons. Some facial recognition solution providers have scrapped billions of photos and identities of people from social networks, usually without their consent." "Unfortunately, this method of facial recognition has justifiably angered privacy groups and data protection agencies around the globe and damaged the public trust in accuracy and reliability of facial recognition systems.” Preventing invasion of citizen’s privacy We believe an unjustified invasion of citizens' privacy can be prevented, false arrests can be reduced" “We believe that lists of suspects should be limited and justified. In this way, unjustified invasion of citizens' privacy can be prevented, false arrests can be reduced and public confidence in technology can be increased.” Golan added: "AnyVision is willing to share its industry insights and best practices from our vast research experience with leading global players, including name-brand retailers, global hospitality and entertainment companies, and law enforcement agencies from around the world.” Balancing public order and crime prevention “If the regulations set forth by Surveillance Camera Code of Practice are committed to the principles outlined above, then law enforcement agencies can strike the right balance between the need to maintain public order and prevent crime with the rights of every person to privacy and non-discrimination before the law." Recently Clearview AI CEO told Wired; the company has scraped 10 billion photos from the web - 3 times more than was previously known.

Dahua Technology shows how intelligent cameras enhance safety in nursing homes
Dahua Technology shows how intelligent cameras enhance safety in nursing homes

Patient falls in nursing homes are a serious problem. In the United States, for example, around 1,800 elderly people, living in nursing facilities, die each year from injuries related to falls, according to the Nursing Home Abuse Center. Those patients who survive their injuries often have a reduced quality of life and suffer some form of permanent disability. Rise in nursing home patient falls Figures show that between 50% and 75% of nursing home residents suffer falls each year, twice the chances of falling when compared to seniors who live in a regular residential community. It has been a prevalent challenge to detect falls quickly and effectively, especially when these occur in residents’ bedrooms. In the United Kingdom, the Care Quality Commission has recognised that the use of CCTV may be one of the best ways to ensure safety and quality of care. However, using video surveillance also brings into question other security issues, such as privacy and data protection. Dahua’s WizMind technologies WizMind embraces human-based AI (Artificial Intelligence), for a whole host of applications across verticals This is where Dahua Technology’s WizMind technologies come into play. WizMind embraces human-based AI (Artificial Intelligence), for a whole host of applications across verticals, such as retail, energy, finance, transportation and of course, health and social care. Specific to the health and social care sector are deep-learning algorithms, to protect the privacy of the face and body in real-time, and stereo analysis, which combines dual-lens cameras with three-dimensional scene analysis, in order to detect sudden physical movement, such as falls. Stereo video analysis The growth of AI applications has enabled the greater availability of 3D scene analysis solutions, thereby enabling objects and people to be analysed in three dimensions. Dahua Technology’s stereo analysis uses two lenses, in order to capture separate images of the same scene. It then computes the ‘optical parallax’ of spatial points in the two images, providing 3D information of the scene. The stereo vision mimics the depth of view that comes from humans having two eyes, known as binocular vision. Combined with deep-learning algorithm Combined with a deep-learning algorithm, stereo analysis can recognise event patterns, such as falls and other movement-based behaviours, such as people approaching, the detection of an abnormal number of people in an area, and violent behaviour. In nursing and care homes, stereo analysis cameras can help staff monitor residents, in case of emergency and respond to residents’ problems, such as tripping and falls. The cameras can view all three dimensions of subjects and together with its deep-learning algorithm, can immediately alert staff to any unusual or sudden movement, such as would be evident in a fall. Cameras in communal areas and bedrooms With cameras situated both in communal areas and in bedrooms, the staff is able to respond quickly to incidents With cameras situated both in communal areas and in bedrooms, the staff is able to respond quickly to incidents, which may otherwise stay undiscovered for hours. An example of such a scenario is a nursing home in Singapore, which has a capacity of around 400 beds and is divided into 14 separate living environments, with each designed to be a home-like living area. Dahua cameras with intelligent fall detection technology Dahua cameras, such as IPC-HDW8341X-BV-3D with intelligent fall detection technology were installed, including the provision of 167 stereo analysis cameras inside each bedroom. These trigger an alarm, in the case of incidents, such as a fall, allowing immediate response by staff. Not only does this enhance the well-being and safety of residents, but it also can reduce the nursing home’s labour costs. In addition, Stereo Analysis can also be applied in other application scenarios. An underground unmanned bicycle parking garage in Amsterdam, for instance, has installed Dahua Technology’s behaviour analysis cameras, to detect abnormal events and prevent accidents, such as people tripping and falling, or suspicious individuals wandering around the area. Privacy Protection 2.0 technology While monitoring their situation inside the nursing home, Dahua also adopts Privacy Protection 2.0 technology that features masking of human face and body, to protect the residents’ privacy. It involves the restriction of what can be seen in video images and applies equally to live, and recorded images. Digital masking takes place on the front-end device (e.g. network camera). Dahua’s Privacy Protection 2.0 provides real-time occlusion of the body and face and enables users to access recorded videos, without having to overlay faces with mosaic masks. It also offers additional occlusion options, such as irregular polygons, mosaics and coloured blocks, and allows code exporting based on specified targets, ensuring the privacy of subjects. Privacy and security in evidence collection Stereo video analysis and privacy protection come into their own in nursing homes and healthcare facilities Benefits offered include non-pixelated human targets, allowing for privacy and security in evidence collection. The technology also allows for face and human attributes analysis, without breaching people’s privacy, making it ideal for nursing homes. Stereo video analysis and privacy protection come into their own in nursing homes and healthcare facilities. It allows the close monitoring of residents or patients to help ensure their well-being and safety, while at the same time protecting the privacy of often vulnerable individuals. Dahua TechMonth As part of the Dahua TechMonth, this blog highlights how Dahua’s stereo analysis technology, combined with privacy protection, can provide a valuable tool to help staff respond to incidents quickly and efficiently, including falls, without infringing on people’s data protection rights. In the next blog, Dahua Technology will be discussing the WizMind application of human metadata, enabling users to maximise situational awareness and analysis of events.