A technology poised to transform the physical security market is deep learning, which is a neural network approach to machine learning, differentiated by an ability to train using large data sets for greater accuracy. In effect, the system “learns” by looking at lots of data to achieve artificial intelligence (AI).

Phases of deep learning

I heard a lot about AI, including how it can transform the physical security marketplace, when I attended NVIDIA’s GPU Technology Conference (GTC) in San Jose recently.  

Recognising images, including video images, is a big focus of AI. In the past, you needed programmers to spend months telling a computer how to recognise an image. In deep learning, instead of programming the computer, you just show it many different images and it "learns" to distinguish the differences. This is the "training" phase. After the neural network learns about the data, it can then use "inference" to interpret new data based on what it has learned. In effect, if it has seen enough cats before, it will know when a new image is a cat.

Factors enabling AI

Deep learning and AI are fast-growing areas for a wide range of uses – physical security is just one. It is all made possible by the coming together of three factors. One is the availability of lots of data. This is the “big data” we have been hearing about; in effect, a proliferation of sensors (including video cameras) has produced a large enough mass of data to enable systems to be trained effectively. The second factor is the development of new algorithms to train neural networks faster, and the third is the availability of computer hardware (specifically GPUs, graphics processing units), that is capable of rapidly completing the involved calculations. NVIDIA manufactures those GPUs and sponsors the annual GTC conference, all about how they can be used more effectively.

“Deep learning is about teaching technology to understand the world around us in a way that is similar to how we understand it”

Deep learning and neural network computing is everywhere. It is now widely available in on-premises computers, in systems embedded in edge devices, and even in the cloud. The edge is particularly important in the video surveillance market, enabling systems to function despite any bandwidth or latency issues that would limit the effectiveness of a central server-based system. Edge-based functionality also limits concerns about the privacy of information, and eliminates dependence on the availability of 3G connectivity.

NVIDIA AI City initiative

Video analytics applications fall under NVIDIA's “AI City” initiative, which they describe as a combination of "safe cities" (video surveillance, law enforcement, forensics) and "smart cities" (traffic management, retail analytics, resource optimisation). Depending on the application, AI City technology must function in the cloud, on premises and/or at the edge. NVIDIA’s new Metropolis initiative offers AI at every system level, from the Jetson TX2 "embedded supercomputer" available at the edge, to on-premises servers (using NVIDIA’s Tesla and Quadro) to cloud systems (using NVIDIA’s DGX).

“AI City applications need an edge-to-cloud architecture,” says Jesse Clayton, Senior Manager, Product Management, Intelligent Machines, at NVIDIA. “Some applications, such as body cameras and parking entrance applications, have to have AI at the edge. But for other problems, you need to aggregate multiple sources of information, such as using AI on an on-premises server for hundreds of video cameras.”

The sheer volume of installed cameras in the world makes video an AI problem – more than 1 billion cameras worldwide by 2020 will provide 30 billion frames of video per day. The existing limitations of current video systems to adapt and function well in real-world conditions point to a need for better technology, as do the traditional shortcomings of video analytics systems. Video systems can achieve "super-human" results, identifying and classifying images using artificial intelligence.

NVIDIA’s Quadro GPU system enables Avigilon network video recorders (NVRs) to search simultaneously across hundreds of cameras to find images that are similar in appearance
NVIDIA’s GPU Technology Conference offered a chance for Avigilon to interact with others focused on AI

AI in video surveillance

AI is steadily making its way into video surveillance. Multiple security industry partners are using NVIDIA GPUs to boost the effectiveness of their systems. Many companies highlighted their initiatives at ISC West in April and again at NVIDIA’s GPU Technology Conference. Among them are Avigilon’s Appearance Search and BriefCam’s real-time video synopsis system. Hikvision uses the technology for a six-fold improvement detecting pedestrians in the rain, while Dahua is speeding up its licence plate recognition system by five times. Other companies using the technology are UNV Uniview (vehicle classification), SeeQuestor (investigations), Xjera Labs (people and attribute detection) and Sensetime (object detection).

NVIDIA’s Quadro GPU system enables Avigilon network video recorders (NVRs) to search simultaneously across hundreds of cameras to find images that are similar in appearance, such as faces that match an example. The GPU’s fast and efficient processing power, available in a small and affordable form factor, provides a system that is scalable and cost-effective but can run complex algorithms to provide rapid results.

Beyond recognising objects, the
system can also learn about how
objects interact in the environment,
and look for anomalies

“Deep learning is about teaching technology to understand the world around us in a way that is similar to how we understand it,” says Willem Ryan, Senior Director, Global Marketing at Avigilon. “What seem simple to us in terms of how we perceive the world is complex for a machine to do, but a machine learns faster. Deep learning allows you to teach a machine how to make connections that we make every day. Using GPUs, a system can make assumptions and calculations instantaneously.”

Beyond recognising objects, the system can also learn about how objects interact in the environment, and look for anomalies or non-typical events. For example, if the system sees a car go onto a pavement, it could provide an alert.

How will AI develop?

NVIDIA’s GTC conference offered a chance for Avigilon to interact with others focused on AI, and to share Avigilon’s knowledge of the unique AI challenges of the video surveillance market. “This is the heart of the development of AI and deep learning,” said Ryan at the GTC conference. “To be involved and part of this is exciting to Avigilon, and we can expose people here to how AI can be used in a way they may not be familiar with. We have talked to people who didn’t realise how video surveillance happens currently, and how AI is changing it. “

“We want to continue to support the idea of GPU processing and how using it can make video surveillance solutions more effective, and change how people interact with video,” he added. “That’s where we see the impact. There have been challenges we have struggled to overcome in the security industry, and these are the breakthroughs that will help us overcome those challenges. So, we want to be at the forefront and involved in those discussions.”

The impact of AI and deep learning on the physical security industry is only beginning. The full realisation of that impact over the next few years will be fascinating to watch.

Download PDF version Download PDF version

Author profile

Larry Anderson Editor, SecurityInformed.com & SourceSecurity.com

An experienced journalist and long-time presence in the US security industry, Larry is SourceSecurity.com's eyes and ears in the fast-changing security marketplace, attending industry and corporate events, interviewing security leaders and contributing original editorial content to the site. He leads SourceSecurity.com's team of dedicated editorial and content professionals, guiding the "editorial roadmap" to ensure the site provides the most relevant content for security professionals.

In case you missed it

The physical side of data protection
The physical side of data protection

The impact of the COVID-19 pandemic has accentuated our digital dependency, on a global scale. Data centres have become even more critical to modern society. The processing and storage of information underpin the economy, characterised by a consistent increase in the volume of data and applications, and reliance upon the internet and IT services. Data centres classed as CNI As such, they are now classed as Critical National Infrastructure (CNI) and sit under the protection of the National Cyber Security Centre (NCSC), and the Centre for the Protection of National Infrastructure (CPNI). As land continues to surge in value, data centre operators are often limited for choice, on where they place their sites and are increasingly forced to consider developed areas, close to other infrastructures, such as housing or industrial sites. Complex security needs One misconception when it comes to data centres is that physical security is straightforward One misconception when it comes to data centres is that physical security is straightforward. However, in practice, things are far more complex. On top of protecting the external perimeter, thought must also be given to factors, such as access control, hostile vehicle mitigation (HVM), protecting power infrastructure, as well as standby generators and localising security devices to operate independently of the main data centre. Face value How a site looks is more important than you may think. Specify security that appears too hostile risks blatantly advertising that you’re protecting a valuable target, ironically making it more interesting to opportunistic intruders. The heightened security that we recommend to clients for these types of sites, include 4 m high-security fences, coils of razor wire, CCTV, and floodlighting. When used together in an integrated approach, it’s easy to see how they make the site appear hostile against its surroundings. However, it must appear secure enough to give the client peace of mind that the site is adequately protected. Getting the balance right is crucial. So, how do you balance security, acoustics and aesthetics harmoniously? Security comes first These are essential facilities and as a result, they require appropriate security investment. Cutting corners leads to a greater long-term expense and increases the likelihood of highly disruptive attacks. Checkpoints Fortunately, guidance is available through independent accreditations and certifications, such as the Loss Prevention Certification Board’s (LPCB) LPS 1175 ratings, the PAS 68 HVM rating, CPNI approval, and the police initiative - Secured by Design (SBD). Thorough technical evaluation and quality audit These bodies employ thorough technical evaluation work and rigorous quality audit processes to ensure products deliver proven levels of protection. With untested security measures, you will not know whether a product works until an attack occurs. Specifying products accredited by established bodies removes this concern. High maintenance Simply installing security measures and hoping for the best will not guarantee 24/7 protection. Just as you would keep computer software and hardware updated, to provide the best level of protection for the data, physical security also needs to be well-maintained, in order to ensure it is providing optimum performance. Importance of testing physical security parameters Inspecting the fence line may seem obvious and straightforward, but it needs to be done regularly. From our experience, this is something that is frequently overlooked. The research we conducted revealed that 63% of companies never test their physical security. They should check the perimeter on both sides and look for any attempted breaches. Foliage, weather conditions or topography changes can also affect security integrity. Companies should also check all fixtures and fittings, looking for damage and corrosion, and clear any litter and debris away. Accessibility When considering access control, speed gates offer an excellent solution for data centres. How quickly a gate can open and close is essential, especially when access to the site is restricted. The consequences of access control equipment failing can be extremely serious, far over a minor irritation or inconvenience. Vehicle and pedestrian barriers, especially if automated, require special attention to maintain effective security and efficiency. Volume control Data centres don’t generally make the best neighbours. The noise created from their 24-hour operation can be considerable. HVAC systems, event-triggered security and fire alarms, HV substations, and vehicle traffic can quickly become unbearable for residents. Secure and soundproof perimeter As well as having excellent noise-reducing properties, timber is also a robust material for security fencing So, how do you create a secure and soundproof perimeter? Fortunately, through LPS 1175 certification and CPNI approval, it is possible to combine high-security performance and up to 28dB of noise reduction capabilities. As well as having excellent noise-reducing properties, timber is also a robust material for security fencing. Seamlessly locking thick timber boards create a flat face, making climbing difficult and the solid boards prevent lines of sight into the facility. For extra protection, steel mesh can either be added to one side of the fence or sandwiched between the timber boards, making it extremely difficult to break through. A fair façade A high-security timber fence can be both, aesthetically pleasing and disguise its security credentials. Its pleasant natural façade provides a foil to the stern steel bars and mesh, often seen with other high-security solutions. Of course, it’s still important that fencing serves its primary purposes, so make sure you refer to certifications, to establish a product’s security and acoustic performance. Better protected The value of data cannot be overstated. A breach can have severe consequences for public safety and the economy, leading to serious national security implications. Countering varied security threats Data centres are faced with an incredibly diverse range of threats, including activism, sabotage, trespass, and terrorism on a daily basis. It’s no wonder the government has taken an active role in assisting with their protection through the medium of the CPNI and NCSC. By working with government bodies such as the CPNI and certification boards like the LPCB, specifiers can access a vault of useful knowledge and advice. This will guide them to effective and quality products that are appropriate for their specific site in question, ensuring it’s kept safe and secure.

Data explosion: Futureproofing your video surveillance infrastructure
Data explosion: Futureproofing your video surveillance infrastructure

Video surveillance systems are producing more unstructured data than ever before. A dramatic decrease in camera costs in recent years has led many businesses to invest in comprehensive surveillance coverage, with more cameras generating more data. Plus, advances in technology mean that the newest (8K) cameras are generating approximately 800% more data than their predecessors (standard definition). Traditional entry-level solutions like network video recorders (NVRs) simply aren’t built to handle massive amounts of data in an efficient, resilient and cost-effective manner. This has left many security pioneers grappling with a data storage conundrum. Should they continue adding more NVR boxes? Or is there another, better, route? Retaining video data In short, yes. To future proof their video surveillance infrastructure, an increasing number of businesses are adopting an end-to-end surveillance architecture with well-integrated, purpose-built platforms for handling video data through its lifecycle. This presents significant advantages in terms of security, compliance and scalability, as well as unlocking new possibilities for data enrichment. All of this with a lower total cost of ownership than traditional solutions. Security teams would typically delete recorded surveillance footage after a few days or weeks Previously, security teams would typically delete recorded surveillance footage after a few days or weeks. However, thanks to increasingly stringent legal and compliance demands, many are now required to retain video data for months or even years. There’s no doubt that this can potentially benefit investigations and increase prosecutions, but it also puts significant pressure on businesses’ storage infrastructure. Data lifecycle management This necessitates a more intelligent approach to data lifecycle management. Rather than simply storing video data in a single location until it’s wiped, an end-to-end video surveillance solution can intelligently migrate data to different storage platforms and media as it ages. So, how does this work? Video is recorded and analysed on a combination of NVR, hyperconverged infrastructure (HCI) and application servers. Then, it’s moved to resilient file storage for a pre-determined period, where it can be immediately retrieved and accessed for review. Finally, based on policies set by heads of security, data is moved from file storage to highly secure, low-cost archive storage such as an object, tape or cloud. Data is moved from file storage to highly secure, low-cost archive storage Long-term storage This process is known as tiering. It allows businesses to use reliable, inexpensive long-term storage for most of their data, whilst still enabling security pioneers to retrieve video data when the need arises, such as during a compliance audit, or to review footage following a security breach. In a nutshell, it offers them the best of both worlds. Scaling your video surveillance infrastructure can be a headache. Businesses that rely on NVRs – even high-end units with 64 or even 96 hard drives – are finding themselves running out of capacity increasingly quickly. In order to scale, security pioneers then have to procure new boxes. With NVRs, this inevitably involves a degree of guesswork. Should they go for the largest possible option, and risk over provisioning? Or perhaps a smaller option, and risk running out of capacity again? Common management console Security pioneers can easily add or remove storage capacity or compute resources – separately or together As businesses add new cameras or replace existing ones, many end up with inadequate surveillance infrastructure made up of multiple NVR boxes along with several application servers for running other surveillance functions such as access control, security photo databases, analytics, etc. This patchwork approach leaves security pioneers scrambling for capacity, maintaining various hardware footprints, repeating updates and checks across multiple systems, and taking up valuable time that could be better spent elsewhere. By contrast, flexible HCI surveillance platforms aggregate the storage and ecosystem applications to run on the same infrastructure and combine viewing under a common management console, avoiding ‘swivel chair’ management workflows. Plus, they offer seamless scalability. Security pioneers can easily add or remove storage capacity or compute resources – separately or together. Data storage solutions Over time, this ensures a lower total cost of ownership. First and foremost, it removes the risk of over provisioning and helps to control hardware sprawl. This in turn leads to hardware maintenance savings and lower power use. Many security pioneers are now looking beyond simple data storage solutions for their video surveillance footage. Meta tags can provide context around data, making it easier to find and access when needed Instead, they’re asking themselves how analysing this data can enable their teams to work faster, more efficiently and productively. Implementing an end-to-end video surveillance architecture enables users to take advantage of AI and machine learning applications which can tag and enrich video surveillance data. These have several key benefits. Firstly, meta tags can provide context around data, making it easier to find and access when needed. Object storage platform For instance, if security teams are notified of a suspicious red truck, they can quickly find data with this tag, rather than manually searching through hours of data, which can feel like looking for a needle in a haystack. Plus, meta tags can be used to mark data for future analysis. This means that as algorithms are run over time, policies can be set to automatically store data in the right location. For example, if a video is determined to contain cars driving in and out of your premises, it would be moved to long-term archiving such as an object storage platform for compliance purposes. If, on the other hand, it contained 24 hours of an empty parking lot, it could be wiped. These same meta tags may be used to eventually expire the compliance data in the archive after it is no longer needed based on policy. Video surveillance architecture Continuing to rely on traditional systems like NVRs will fast become unsustainable for businesses Even if your organisation isn’t using machine learning or artificial intelligence-powered applications to enhance your data today, it probably will be one, three, or even five years down the line. Implementing a flexible end-to-end video surveillance solution prepares you for this possibility. With new advances in technology, the quantity of data captured by video surveillance systems will continue rising throughout the coming decade. As such, continuing to rely on traditional systems like NVRs will fast become unsustainable for businesses. Looking forward, when moving to an end-to-end video surveillance architecture, security pioneers should make sure to evaluate options from different vendors. For true futureproofing, it’s a good idea to opt for a flexible, modular solution, which allow different elements to be upgraded to more advanced technologies when they become available.

How can the security industry provide affordable and cost-effective solutions?
How can the security industry provide affordable and cost-effective solutions?

Cost is a reality to be managed. No matter how powerful or desirable a technology may be to a customer, the sale often comes down to the basic question: Can I afford it? And affordability extends not just to the purchase price, but to the cost of technology over its lifespan. In addition to advances in technology capabilities, the security industry has also achieved inroads to make its offerings more worth the cost. We asked this week’s Expert Panel Roundtable: What is the physical security industry doing to make more affordable and cost-effective technology solutions for end users?