A technology poised to transform the physical security market is deep learning, which is a neural network approach to machine learning, differentiated by an ability to train using large data sets for greater accuracy. In effect, the system “learns” by looking at lots of data to achieve artificial intelligence (AI).

Phases of deep learning

I heard a lot about AI, including how it can transform the physical security marketplace, when I attended NVIDIA’s GPU Technology Conference (GTC) in San Jose recently.  

Recognising images, including video images, is a big focus of AI. In the past, you needed programmers to spend months telling a computer how to recognise an image. In deep learning, instead of programming the computer, you just show it many different images and it "learns" to distinguish the differences. This is the "training" phase. After the neural network learns about the data, it can then use "inference" to interpret new data based on what it has learned. In effect, if it has seen enough cats before, it will know when a new image is a cat.

Factors enabling AI

Deep learning and AI are fast-growing areas for a wide range of uses – physical security is just one. It is all made possible by the coming together of three factors. One is the availability of lots of data. This is the “big data” we have been hearing about; in effect, a proliferation of sensors (including video cameras) has produced a large enough mass of data to enable systems to be trained effectively. The second factor is the development of new algorithms to train neural networks faster, and the third is the availability of computer hardware (specifically GPUs, graphics processing units), that is capable of rapidly completing the involved calculations. NVIDIA manufactures those GPUs and sponsors the annual GTC conference, all about how they can be used more effectively.

“Deep learning is about teaching technology to understand the world around us in a way that is similar to how we understand it”

Deep learning and neural network computing is everywhere. It is now widely available in on-premises computers, in systems embedded in edge devices, and even in the cloud. The edge is particularly important in the video surveillance market, enabling systems to function despite any bandwidth or latency issues that would limit the effectiveness of a central server-based system. Edge-based functionality also limits concerns about the privacy of information, and eliminates dependence on the availability of 3G connectivity.

NVIDIA AI City initiative

Video analytics applications fall under NVIDIA's “AI City” initiative, which they describe as a combination of "safe cities" (video surveillance, law enforcement, forensics) and "smart cities" (traffic management, retail analytics, resource optimisation). Depending on the application, AI City technology must function in the cloud, on premises and/or at the edge. NVIDIA’s new Metropolis initiative offers AI at every system level, from the Jetson TX2 "embedded supercomputer" available at the edge, to on-premises servers (using NVIDIA’s Tesla and Quadro) to cloud systems (using NVIDIA’s DGX).

“AI City applications need an edge-to-cloud architecture,” says Jesse Clayton, Senior Manager, Product Management, Intelligent Machines, at NVIDIA. “Some applications, such as body cameras and parking entrance applications, have to have AI at the edge. But for other problems, you need to aggregate multiple sources of information, such as using AI on an on-premises server for hundreds of video cameras.”

The sheer volume of installed cameras in the world makes video an AI problem – more than 1 billion cameras worldwide by 2020 will provide 30 billion frames of video per day. The existing limitations of current video systems to adapt and function well in real-world conditions point to a need for better technology, as do the traditional shortcomings of video analytics systems. Video systems can achieve "super-human" results, identifying and classifying images using artificial intelligence.

NVIDIA’s Quadro GPU system enables Avigilon network video recorders (NVRs) to search simultaneously across hundreds of cameras to find images that are similar in appearance
NVIDIA’s GPU Technology Conference offered a chance for Avigilon to interact with others focused on AI

AI in video surveillance

AI is steadily making its way into video surveillance. Multiple security industry partners are using NVIDIA GPUs to boost the effectiveness of their systems. Many companies highlighted their initiatives at ISC West in April and again at NVIDIA’s GPU Technology Conference. Among them are Avigilon’s Appearance Search and BriefCam’s real-time video synopsis system. Hikvision uses the technology for a six-fold improvement detecting pedestrians in the rain, while Dahua is speeding up its licence plate recognition system by five times. Other companies using the technology are UNV Uniview (vehicle classification), SeeQuestor (investigations), Xjera Labs (people and attribute detection) and Sensetime (object detection).

NVIDIA’s Quadro GPU system enables Avigilon network video recorders (NVRs) to search simultaneously across hundreds of cameras to find images that are similar in appearance, such as faces that match an example. The GPU’s fast and efficient processing power, available in a small and affordable form factor, provides a system that is scalable and cost-effective but can run complex algorithms to provide rapid results.

Beyond recognising objects, the
system can also learn about how
objects interact in the environment,
and look for anomalies

“Deep learning is about teaching technology to understand the world around us in a way that is similar to how we understand it,” says Willem Ryan, Senior Director, Global Marketing at Avigilon. “What seem simple to us in terms of how we perceive the world is complex for a machine to do, but a machine learns faster. Deep learning allows you to teach a machine how to make connections that we make every day. Using GPUs, a system can make assumptions and calculations instantaneously.”

Beyond recognising objects, the system can also learn about how objects interact in the environment, and look for anomalies or non-typical events. For example, if the system sees a car go onto a pavement, it could provide an alert.

How will AI develop?

NVIDIA’s GTC conference offered a chance for Avigilon to interact with others focused on AI, and to share Avigilon’s knowledge of the unique AI challenges of the video surveillance market. “This is the heart of the development of AI and deep learning,” said Ryan at the GTC conference. “To be involved and part of this is exciting to Avigilon, and we can expose people here to how AI can be used in a way they may not be familiar with. We have talked to people who didn’t realise how video surveillance happens currently, and how AI is changing it. “

“We want to continue to support the idea of GPU processing and how using it can make video surveillance solutions more effective, and change how people interact with video,” he added. “That’s where we see the impact. There have been challenges we have struggled to overcome in the security industry, and these are the breakthroughs that will help us overcome those challenges. So, we want to be at the forefront and involved in those discussions.”

The impact of AI and deep learning on the physical security industry is only beginning. The full realisation of that impact over the next few years will be fascinating to watch.

Download PDF version

Author profile

Larry Anderson Editor, SourceSecurity.com

An experienced journalist and long-time presence in the US security industry, Larry is SourceSecurity.com's eyes and ears in the fast-changing security marketplace, attending industry and corporate events, interviewing security leaders and contributing original editorial content to the site. He leads SourceSecurity.com's team of dedicated editorial and content professionals, guiding the "editorial roadmap" to ensure the site provides the most relevant content for security professionals.

In case you missed it

Where is it inappropriate to install video cameras?
Where is it inappropriate to install video cameras?

Video cameras are everywhere, and hundreds more are installed every day. Our society appears to be reaching a point of perpetual surveillance. It certainly feels as if we are always being watched even though it is not yet the case. But as cameras are becoming more common than ever, we are also entering a new era of privacy concerns and sensitivities, as evidenced by GDPR and other such initiatives. We presented this quandary to this week’s Expert Panel Roundtable: Surveillance cameras can go anywhere, right? Where is it “not OK?”

Development of integrated thermal imaging technology into evolving market verticals
Development of integrated thermal imaging technology into evolving market verticals

Global and domestic threats have highlighted the need for tighter security across all verticals. One of the technologies that has redefined situational awareness and intrusion detection is thermal imaging. Once a technology exclusively manufactured for the military operations, thermal cameras today are deployed across hundreds of security applications and continue to see strong demand in existing and emerging commercial markets. With thermal technology, security personnel can see in complete darkness as well as in light fog, smoke and rain Technology overview and early adoption What distinguishes thermal cameras from optical sensors is their ability to produce images based on infrared energy, or heat, rather than light. By measuring the heat signatures of all objects and capturing minute differences between them, thermal cameras produce clear, sharp video despite unfavorable environmental conditions. With thermal technology, security personnel can see in complete darkness as well as in light fog, smoke and rain. Originally a military developed, commercially qualified technology, the first thermal cameras for military and aircraft use appeared in the 1950s. By the 1960s, the technology had been declassified and the first thermal camera for commercial use was introduced. However, it wasn’t until the late 1990s - when FLIR Systems introduced a camera with an uncooled thermal detector - when the technology began to see substantial adoption beyond government defense deployments. Installations at critical infrastructure sites In the 2000s, industrial companies were some of the first adopters of thermal, using the technology for predictive maintenance to monitor overheating and machine malfunctions. In the years following the September 11 terrorist attacks in 2001, there was an increase in thermal camera installations across critical infrastructure sites. Stricter security requirements drove the deployment of thermal cameras for perimeter protection, especially in the nuclear power sector. Thermal cameras produce clear video in daylight, low light or no light scenarios and their sharp images result in higher performing analytics In 2010, the U.S. Nuclear Regulatory Committee released its 73.55 policy, which states nuclear facilities must “provide continuous surveillance, observation and monitoring” as a means to enhance threat detection and deterrence efforts onsite. Because thermal cameras produce clear video in daylight, low light or no light scenarios and because their sharp images result in higher performing analytics, thermal cameras quickly became the preferred option for nuclear facilities. Likewise, following the 2013 sniper attack on PG&E Corporation’s Metcalf transmission substation, the Federal Energy Regulation Commission introduced the Critical Infrastructure Protection Standard 014 (CIP-014). The policy requires utilities to identify threats to mission critical assets and implement a security system to mitigate those risks. This statute also led to more thermal installations in the utility sector as thermal cameras’ long-range capabilities are ideal for detection of approaching targets beyond the fence line. The demand from both industrial and critical infrastructure entities, as well as other factors, helped drive volume production and price reduction for thermal, making the technology more accessible to the commercial security marketplace. Commercial applications In recent years, the increasing affordability of thermal cameras along with the introduction of new thermal offerings has opened the door to new commercial applications for the technology. In the past, thermal cameras were designed for applications with enormous perimeters, where the camera needed to detect a human from 700 meters away. Locations like car dealerships, marinas and construction supply facilities can be protected by precise target detection, thermal analytic cameras providing an early warning to security personnel Today, there are thermal cameras specifically designed for short- to mid-range applications. Developed for small to medium enterprises, these thermal cameras ensure property size and security funds are no longer barriers to adoption. Lumber yards, recreation fields and sports arenas are some of the commercial applications now able to implement thermal cameras for 24-hour monitoring and intrusion detection. Affordable thermal cameras with onboard analytics have become attractive options for commercial businesses Innovation and advancements Innovation and advancements in the core technology have also spurred growth in thermal camera deployment, providing faster image processing, higher resolution, greater video analytic capabilities and better camera performance. In particular, affordable thermal cameras with onboard analytics have become attractive options for commercial businesses that need outdoor, wide area protection. Car dealerships, marinas and construction supply locations all store valuable merchandise and materials outside. Without protection, these assets are vulnerable to vandalism and theft. However, by providing precise target detection, thermal analytic cameras provide an early warning to security personnel so that they can intervene before a crime is committed. By helping to deter just one incident, the thermal solution delivers a clear ROI. New market opportunities Not only are there more thermal cameras in use today than ever before, but there are also more thermal sensors being integrated with other multi-sensor systems, driving the adoption of thermal in new markets. For large perimeter surveillance applications, thermal is repeatedly being integrated with radar and drones to expand situational awareness beyond the point of fixed cameras. Users get immediate, accurate alerts of approaching targets and evidentiary class video for target assessment In the commercial market, thermal imagers are combined with optical sensors, analytics and LED illuminators into one solution that integrates with central monitoring station platforms. By bringing these technologies together, users get immediate, accurate alerts of approaching targets and evidentiary class video for target assessment. The result is a lower number of false positives, reducing the total cost of ownership for the solution. These multi-sensor solutions also feature two-way audio capabilities, which enable remote security officers to act as “virtual guards” and speak to intruders in real-time to dissuade them from illegal activity. The introduction of solutions that integrate all these state-of-the-art technologies under one unit reduces the amount of capital and infrastructure needed for deployment. Consequently, more small businesses and alarm monitoring companies can implement advanced perimeter security technologies like thermal sensors, some for the very first time. Thermal cameras have gone from military defense devices to widespread commercial security cameras Multi-sensor thermal solutions Multi-sensor solutions featuring thermal are quickly gaining traction and opening the door to new business opportunities for the security channel. One of the primary reasons for the strong market interest in these systems is they enable integrators to increase their recurring monthly revenue (RMR). With intense price competition and eroding margins on CCTV equipment, integrators have to rely on RMR to grow their businesses. Offering remote video monitoring services and virtual guarding technologies is one of the best ways to do so.  Additionally, there is a clear demand for it. Central stations are continually looking for new technologies to offer their customers and businesses are interested in economical alternatives to physical guards. In conclusion, thermal cameras have gone from military defense devices to widespread commercial security cameras that are a substantial segment of the outdoor security protection market. From nuclear power plants to construction locations, thermal technology is being implemented to secure sites around the globe.

Highlighting the importance of security integrations and alliances
Highlighting the importance of security integrations and alliances

Most technology companies have one goal in mind: to provide customers with high-quality, affordable products that can efficiently help streamline operations. Whether it's surveillance cameras, video management software, access control technology or any other type of security device, today's leading organisations invest in expertise in these product segments and strive to produce the highest quality solutions. To effectively fulfill this task, technology providers are always searching for emerging components to make their products and services even stronger. Oftentimes, a key aspect necessary to build a comprehensively robust solution involves finding like-minded partners that share a common goal and are willing to work together to create an integration that increases insight and intelligence.The interoperability between systems, devices and different types of applications should be intuitive and fast Key factors for security integrations A basic factor in a partnership is openness. For an integration to perform seamlessly for the end user, the platform through which the technologies converge must follow standard protocols, easily operate with other platforms, allow freedom and customisation, and provide adaptability. The interoperability between systems, devices and different types of applications should be intuitive and fast, enabling more time to be spent on analysing critical data and responding to security events. The puzzle of a complete security solution contains many pieces, and it's often necessary to fuse together aspects from various providers to create a best-in-breed technology offering. When organisations collaborate, the end result is a simplified solution with an increased level of value. As threats become more severe and complex, customers demand solutions that combine different security and business elements into a single interface that can address a wide variety of risks. A unified security system requires a strong collaboration between technology providers and integrated solutions Interconnected security devices Users used to only look at specific security devices - such as cameras or door alarms - as each having a strong, autonomous purpose, but now, every device plays an important interconnected role. And the progression of the Internet of Things (IoT) has made this transition even easier, as maintaining a consistent and uniform communication and interconnectivity between devices has now become the norm. The IoT has also made it so that partnerships must not only exist between manufacturers, but also within the customer's organisational structure. Although exceptionally beneficial, the IoT brings with it an increased amount of cyber vulnerabilities. As security systems are networked to increase flexibility, the door is opened to a number of potential threats that could compromise the entire enterprise. This risk highlights the need for an internal partnership and knowledge sharing between a company's physical security professionals and its IT team. IT experts should be pulled into security decisions and positioned as collaborative partners to assist with software updates, data safety protocols and solving complex network challenges, ultimately leading to a more cyber secure solution.Partnerships are beneficial to both the companies involved and their customers Knowledge sharing and learning Aside from cybersecurity, the latest prominent security attacks and events have focused primarily on soft targets, such as schools, concerts or shopping malls. This has caused many technology providers to venture into different vertical markets, and strong partnerships streamline this crossover. Innovators can extend their geographic reach and purpose through integrations with other like-minded manufacturers or integrators to add new levels of functionalities. Of course, a partnership cannot operate properly and to the best of its ability without a core component: learning. In today's evolving business and risk environment, knowledge is critical. A shared knowledge base can open up new opportunities and lead to the strengthening of security across many levels. A truly powerful, unified security system requires a strong collaboration between technology providers and integrated solutions. Partnerships are beneficial to both the companies involved and their customers, and the results created through these alliances can reach far beyond a user's expectations, offering enhanced flexibility and extensive safety options.