A technology poised to transform the physical security market is deep learning, which is a neural network approach to machine learning, differentiated by an ability to train using large data sets for greater accuracy. In effect, the system “learns” by looking at lots of data to achieve artificial intelligence (AI).

Phases of deep learning

I heard a lot about AI, including how it can transform the physical security marketplace, when I attended NVIDIA’s GPU Technology Conference (GTC) in San Jose recently.  

Recognising images, including video images, is a big focus of AI. In the past, you needed programmers to spend months telling a computer how to recognise an image. In deep learning, instead of programming the computer, you just show it many different images and it "learns" to distinguish the differences. This is the "training" phase. After the neural network learns about the data, it can then use "inference" to interpret new data based on what it has learned. In effect, if it has seen enough cats before, it will know when a new image is a cat.

Factors enabling AI

Deep learning and AI are fast-growing areas for a wide range of uses – physical security is just one. It is all made possible by the coming together of three factors. One is the availability of lots of data. This is the “big data” we have been hearing about; in effect, a proliferation of sensors (including video cameras) has produced a large enough mass of data to enable systems to be trained effectively. The second factor is the development of new algorithms to train neural networks faster, and the third is the availability of computer hardware (specifically GPUs, graphics processing units), that is capable of rapidly completing the involved calculations. NVIDIA manufactures those GPUs and sponsors the annual GTC conference, all about how they can be used more effectively.

“Deep learning is about teaching technology to understand the world around us in a way that is similar to how we understand it”

Deep learning and neural network computing is everywhere. It is now widely available in on-premises computers, in systems embedded in edge devices, and even in the cloud. The edge is particularly important in the video surveillance market, enabling systems to function despite any bandwidth or latency issues that would limit the effectiveness of a central server-based system. Edge-based functionality also limits concerns about the privacy of information, and eliminates dependence on the availability of 3G connectivity.

NVIDIA AI City initiative

Video analytics applications fall under NVIDIA's “AI City” initiative, which they describe as a combination of "safe cities" (video surveillance, law enforcement, forensics) and "smart cities" (traffic management, retail analytics, resource optimisation). Depending on the application, AI City technology must function in the cloud, on premises and/or at the edge. NVIDIA’s new Metropolis initiative offers AI at every system level, from the Jetson TX2 "embedded supercomputer" available at the edge, to on-premises servers (using NVIDIA’s Tesla and Quadro) to cloud systems (using NVIDIA’s DGX).

“AI City applications need an edge-to-cloud architecture,” says Jesse Clayton, Senior Manager, Product Management, Intelligent Machines, at NVIDIA. “Some applications, such as body cameras and parking entrance applications, have to have AI at the edge. But for other problems, you need to aggregate multiple sources of information, such as using AI on an on-premises server for hundreds of video cameras.”

The sheer volume of installed cameras in the world makes video an AI problem – more than 1 billion cameras worldwide by 2020 will provide 30 billion frames of video per day. The existing limitations of current video systems to adapt and function well in real-world conditions point to a need for better technology, as do the traditional shortcomings of video analytics systems. Video systems can achieve "super-human" results, identifying and classifying images using artificial intelligence.

NVIDIA’s Quadro GPU system enables Avigilon network video recorders (NVRs) to search simultaneously across hundreds of cameras to find images that are similar in appearance
NVIDIA’s GPU Technology Conference offered a chance for Avigilon to interact with others focused on AI

AI in video surveillance

AI is steadily making its way into video surveillance. Multiple security industry partners are using NVIDIA GPUs to boost the effectiveness of their systems. Many companies highlighted their initiatives at ISC West in April and again at NVIDIA’s GPU Technology Conference. Among them are Avigilon’s Appearance Search and BriefCam’s real-time video synopsis system. Hikvision uses the technology for a six-fold improvement detecting pedestrians in the rain, while Dahua is speeding up its licence plate recognition system by five times. Other companies using the technology are UNV Uniview (vehicle classification), SeeQuestor (investigations), Xjera Labs (people and attribute detection) and Sensetime (object detection).

NVIDIA’s Quadro GPU system enables Avigilon network video recorders (NVRs) to search simultaneously across hundreds of cameras to find images that are similar in appearance, such as faces that match an example. The GPU’s fast and efficient processing power, available in a small and affordable form factor, provides a system that is scalable and cost-effective but can run complex algorithms to provide rapid results.

Beyond recognising objects, the
system can also learn about how
objects interact in the environment,
and look for anomalies

“Deep learning is about teaching technology to understand the world around us in a way that is similar to how we understand it,” says Willem Ryan, Senior Director, Global Marketing at Avigilon. “What seem simple to us in terms of how we perceive the world is complex for a machine to do, but a machine learns faster. Deep learning allows you to teach a machine how to make connections that we make every day. Using GPUs, a system can make assumptions and calculations instantaneously.”

Beyond recognising objects, the system can also learn about how objects interact in the environment, and look for anomalies or non-typical events. For example, if the system sees a car go onto a pavement, it could provide an alert.

How will AI develop?

NVIDIA’s GTC conference offered a chance for Avigilon to interact with others focused on AI, and to share Avigilon’s knowledge of the unique AI challenges of the video surveillance market. “This is the heart of the development of AI and deep learning,” said Ryan at the GTC conference. “To be involved and part of this is exciting to Avigilon, and we can expose people here to how AI can be used in a way they may not be familiar with. We have talked to people who didn’t realise how video surveillance happens currently, and how AI is changing it. “

“We want to continue to support the idea of GPU processing and how using it can make video surveillance solutions more effective, and change how people interact with video,” he added. “That’s where we see the impact. There have been challenges we have struggled to overcome in the security industry, and these are the breakthroughs that will help us overcome those challenges. So, we want to be at the forefront and involved in those discussions.”

The impact of AI and deep learning on the physical security industry is only beginning. The full realisation of that impact over the next few years will be fascinating to watch.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

Author profile

Larry Anderson Editor, SecurityInformed.com & SourceSecurity.com

An experienced journalist and long-time presence in the US security industry, Larry is SourceSecurity.com's eyes and ears in the fast-changing security marketplace, attending industry and corporate events, interviewing security leaders and contributing original editorial content to the site. He leads SourceSecurity.com's team of dedicated editorial and content professionals, guiding the "editorial roadmap" to ensure the site provides the most relevant content for security professionals.

In case you missed it

What is the impact of lighting on video performance?
What is the impact of lighting on video performance?

Dark video images contain little or no information about the subject being surveilled. Absence of light can make it difficult to see a face, or to distinguish the color of clothing or of an automobile. Adding light to a scene is one solution, but there are also new technologies that empower modern video cameras to see better in any light. We asked this week’s Expert Panel Roundtable: what impact does lighting have on the performance of video systems?

Alarm.com adapts during pandemic to enable partners to ‘succeed remotely’
Alarm.com adapts during pandemic to enable partners to ‘succeed remotely’

As a cloud-based platform for service providers in the security, smart home and smart business markets, Alarm.com has adapted quickly to changing conditions during the coronavirus pandemic. In the recent dynamic environment, Alarm.com has kept focus on supporting their service provider partners so they can keep local communities protected. “We moved quickly to establish work-from-home protocols to protect our employees and minimise impact on our partners,” says Anne Ferguson, VP of Marketing at Alarm.com. The Customer Operations and Reseller Education (CORE) team has operated without interruption to provide support to partners. Sales teams are utilising webinars and training resources to inform and educate partners about the latest products, tools, and solutions. Alarm.com’s partner tools are essential for remote installations and support of partner accounts. Helping customers remain connected Adapting to challenges of the coronavirus pandemic, Alarm.com is further investing in solutions that help customers remain connected and engaged. The company has created a resource hub called “Succeeding Remotely” that provides tools, tips and news links that partners can use to adapt their business operations. From adjusting sales and installation techniques to maintaining cellular upgrades, Alarm.com is helping partners stay connected to customers remotely, keep their teams trained, and address rapidly evolving customer concerns without rolling trucks.The company has created a resource hub called “Succeeding Remotely “Additionally, after seeing all that our partners are doing to support their local communities in need, we were compelled to highlight those efforts with ongoing videos called Good Connections, which we’re sharing with our partner community to spark more ideas and ways to help,” says Ferguson. “Though our partners have experienced varying degrees of disruption to their business, we’re inspired by their adaptability, ingenuity and resilience,” says Ferguson. “Along with establishing proper safeguards for operating in homes and businesses, our partners are leveraging our support resources more heavily, while our entire staff has worked tirelessly to deliver new, timely resources.” Do-It-Together solutions Alarm.com partners are successfully employing Do-It-Together (DIT) solutions, focusing on 3G-to-LTE upgrades, and pivoting to new verticals like commercial and wellness. Many are also streamlining their business operations and taking advantage of virtual training opportunities to enhance their technicians’ skills and knowledge, says Ferguson. Do-It-Together installs involve depending on customers to perform part or all of the installation process. Partners can send customers fully configured kits with mounting instructions, or technicians may guide customers on a remote video call. Alarm.com’s tools, training and products help partners modify remote installation options depending on each customer’s needs. End users can validate the Alarm.com Smart Gateway with their central station that sensors they have mounted were done correctly using the Alarm.com mobile app Alarm.com Smart Gateway For example, the Alarm.com Smart Gateway can be pre-configured with indoor and outdoor cameras for easy customer installation and to reduce the likelihood of future service calls. Also, end users can validate with their central station that sensors they have mounted were done correctly using the Alarm.com mobile app. “DIT is helping our partners continue onboarding customers and avoid backlogs,” says Ferguson. “We’ve been pleasantly surprised by the resiliency and level of future investment that our residential and commercial partners have shown in the face of adversity,” adds Ferguson. For example, a significant number of business customers have used the slow period to install systems that are typically too disruptive to put in during normal business hours. Similarly, service providers are adopting new technologies or business models, such as cloud-based access control. “They’re often saying to us, ‘I’m going to take this opportunity to make changes to improve our business,’ and have been working closely with us on training and business consulting to support their efforts,“ she says. Shift to the cloud Ferguson sees a growing preference for cloud-managed surveillance and access systems over ones that have historically been run on-premise. The technology itself is attractive, but especially driving change is the enhancement to the daily lives of service providers and customers, which have been strained during this time. “The foundational benefit of our cloud-based solution is the hassle-free, seamless customer experience it delivers,” says Ferguson. “We make this possible by taking ownership of the servers, software maintenance, firmware updates, health monitoring, and more. With cloud technology, these aspects become invisible to the customer and take a lot off their plate, which is more important than ever.” End users can take advantage of Smart Tip video tutorials to help with DIT installations, or they can use the Alarm.com Wellcam to connect with loved ones anywhere.End users can take advantage of Smart Tip video tutorials to help with DIT installations Partners can attend training workshops focused on remote installation tactics, while driving consumer interest in new offerings through Alarm.com’s Customer Connections platform. The goal is to make it simple for partners to stay connected to their customers to maximise lifetime account value. “We are well-positioned to endure the pandemic because of the strength of our partners in their markets along with our investments in technology, hardware and our team,” says Ferguson. “As restrictions slowly lift, there is cautious optimism that the residential, commercial, property management, plumbing/HVAC, builder and other verticals will recover quickly. We believe that as more partners adopt the DIT model and add commercial and wellness RMR, they will find increasing opportunities to deploy security, automation, video, video analytics, access and more throughout their customer base.”

COVID-19 worries boost prospects of touchless biometric systems
COVID-19 worries boost prospects of touchless biometric systems

Spread of the novel coronavirus has jolted awareness of hygiene as it relates to touching surfaces such as keypads. No longer in favour are contact-based modalities including use of personal identification numbers (PINs) and keypads, and the shift has been sudden and long-term. Both customers and manufacturers were taken by surprise by this aspect of the virus’s impact and are therefore scrambling for solutions. Immediate impact of the change includes suspension of time and attendance systems that are touch-based. Some two-factor authentication systems are being downgraded to RFID-only, abandoning the keypad and/or biometric components that contributed to higher security, but are now unacceptable because they involve touching. Touchless biometric systems in demand The trend has translated into a sharp decline in purchase of touch modality and a sharp increase in the demand for touchless systems, says Alex Zarrabi, President of Touchless Biometrics Systems (TBS). Biometrics solutions are being affected unequally, depending on whether they involve touch sensing, he says. Spread of the novel coronavirus has jolted awareness of hygiene as it relates to touching surfaces such as keypads “Users do not want to touch anything anymore,” says Zarrabi. “From our company’s experience, we see it as a huge catalyst for touchless suppliers. We have projects being accelerated for touchless demand and have closed a number of large contracts very fast. I’m sure it’s true for anyone who is supplying touchless solutions.” Biometric systems are also seeing the addition of thermal sensors to measure body temperature in addition to the other sensors driving the system. Fingerscans and hybrid face systems TBS offers 2D and 3D systems, including both fingerscans and hybrid face/iris systems to provide touchless identification at access control points. Contactless and hygienic, the 2D Eye system is a hybrid system that combines the convenience of facial technology with the higher security of iris recognition. The system recognises the face and then detects the iris from the face image and zeros in to scan the iris. The user experiences the system as any other face recognition system. The facial aspect quickens the process, and the iris scan heightens accuracy. TBS also offers the 2D Eye Thermo system that combines face, iris and temperature measurement using a thermal sensor module. TBS's 2D Eye Thermo system combines face, iris and temperature measurement using a thermal sensor module Another TBS system is a 3D Touchless Fingerscan system that provides accuracy and tolerance, anti-spoofing, and is resilient to water, oil, dust and dirt. The 2D+ Multispectral for fingerprints combines 2D sensing with “multispectral” subsurface identification, which is resilient to contaminants and can read fingerprints that are oily, wet, dry or damaged – or even through a latex glove. In addition, the 3D+ system by TBS provides frictionless, no-contact readings even for people going through the system in a queue. The system fills the market gap for consent-based true on-the-fly systems, says Zarrabi. The system captures properties of the hand and has applications in the COVID environment, he says. The higher accuracy and security ratings are suitable for critical infrastructure applications, and there is no contact; the system is fully hygienic. Integration with access control systems Integration of TBS biometrics with a variety of third-party access control systems is easy. A “middleware” subsystem is connected to the network. Readers are connected to the subsystem and also to the corporate access control system. An interface with the TBS subsystem coordinates with the access control system. For example, a thermal camera used as part of the biometric reader can override the green light of the access control system if a high temperature (suggesting COVID-19 infection, for example) is detected. The enrollment process is convenient and flexible and can occur at an enrollment station or at an administration desk. Remote enrollment can also be accomplished using images from a CCTV camera. All templates are encrypted. Remotely enrolled employees can have access to any location they need within minutes. The 3D+ system by TBS provides frictionless, no-contact readings even for people going through the system in a queue Although there are other touchless technologies available, they cannot effectively replace biometrics, says Zarrabi. For example, a centrally managed system that uses a Bluetooth signal from a smart phone could provide convenience, is “touchless,” and could suffice for some sites. However, the system only confirms the presence and “identity” of a smart phone – not the person who should be carrying it. “There has been a lot of curiosity about touchless, but this change is strong, and there is fear of a possible second wave of COVID-19 or a return in two or three years,” says Zarrabi. “We really are seeing customers seriously shifting to touchless.”