Over the years, video analytics has gained an unfavourable reputation for over-promising and under-delivering in terms of performance. One of the biggest complaints regarding video analytics has been its inability to correctly identify objects in situations which appear trivial to the human observer. In many cases, this has resulted in a tendency to generate substantial numbers of false alarms, while not detecting actual events accurately. This, together with a propensity for complex set-up procedures and much need for manual fine-tuning, has prevented video analytics from becoming a mainstream application deployed on large numbers of cameras.

Machine Learning and video analytics

Machine Learning is a well-established field of research that has existed for decades, which is already present in many products and applications. Machine Learning is based on collecting large amounts of data specific to a particular problem, training a model using this data and then employing this model to process new data. With regard to video analytics, one of the most critical problems impacting accuracy is object classification. Fundamental to improving performance is the capability to teach the algorithm to distinguish between people, animals, different types of vehicles and sources of noise at an extremely high level of accuracy.

Deep Learning algorithms

The recent increased interest in Deep Learning is largely due to the availability of graphical processing units

Until recently there have been minimal applications of Machine Learning used in video analytics products, largely due to high complexity and high resource usage, which made such products too costly for mainstream deployment. However, the last couple of years have seen a tremendous surge in research and advances surrounding a branch of Machine Learning called Deep Learning. Deep Learning is a name used to describe a family of algorithms based on the concept of neural networks. Very loosely speaking, these algorithms try to emulate the functionality of the brain’s neurons, enabling them to learn efficiently from example, and subsequently apply this learning to new data.

The recent increased interest in Deep Learning is largely due to the availability of graphical processing units (GPUs). GPUs can efficiently train and run Deep Learning algorithms, and have allowed the scientific community to accelerate their research and application, bringing them to the point where they exceed the performance of most traditional Machine Learning algorithms across several categories.

Solving object classification with deep learning

This means that Deep Learning can now be used to solve the most crucial problem facing video analytics – object classification – by collecting many thousands of images from hundreds of surveillance cameras, which must first be manually labelled and classified by a human, into a range of categories that include: person, car, bus, truck, bird, vegetation, dog and many more. To achieve the required accuracy rates, such a vast database must be collected and identified from actual surveillance footage.

A crucial component in achieving and maintaining the high performance of Deep Learning-based applications is the ability to continuously update the models as more data is collected
Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics

A Deep Learning algorithm trained on images collected from YouTube, Google Search and elsewhere on the Internet will completely fail in analysing images from surveillance cameras, due to the difference in viewing angles, resolution and image quality. Once enough images are collected, a Deep Learning classifier algorithm can be trained and deployed as part of a video analytics solution, enabling it to practically eliminate most of the existing causes for false alarms.

Due to GPU requirements in order for the algorithms to run efficiently, video analytics solutions using Deep Learning will initially need to run on a server. A few solutions of this nature are already available and are showing a dramatic leap in performance in comparison to traditional video analytics, with a drastic reduction in false alarm rates and a significant increase in detection accuracy. Concurrently, these new solutions do not require manual tweaking by the user and are essentially plug-and-play, making mass deployment a realistic premise.

Surveillance applications of Deep Learning

Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics, but they are by no means the only ones. In the not-too-distant future, we will see Deep Learning enabling as yet not possible video analytics applications, such as identifying objects carried by people, such as a gun, handbag, or a knife, or being able to quickly find people and vehicles with similar appearances across multiple cameras and more.

Over the next few years, we will see a transition of video analytics using Deep Learning running on servers to running inside cameras

Over the next few years, we will see a transition of video analytics using Deep Learning running on servers, to running inside cameras, as powerful, low-cost hardware capable of running Deep Learning becomes more available and a basic function of newer surveillance camera models. This will push the acceptance of video analytics even further, eventually making it a fundamental element of every surveillance camera deployed.

Increasing accuracy with updates

A crucial component in achieving and maintaining the high performance of Deep Learning-based applications, is the ability to continuously update the models as more data is collected so that the models increase in accuracy. This will give an advantage to cloud-based video analytics services, since they can collect vast amounts of data from cameras connected to the service, train new models in the cloud based on this data and then push these new models to cameras at the edge. This continuous improvement cycle will be instrumental in helping video analytics fulfil the promise of improving peoples’ safety and security, by giving surveillance cameras human-level accuracy and a comprehensive understanding of the environment.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

Author profile

In case you missed it

Managing security during unprecedented times of home working
Managing security during unprecedented times of home working

Companies are following government guidance and getting as many people as possible working from home. Some companies will have resisted home working in the past, but I’m certain that the sceptics will find that people can be productive with the right tools no matter where they are. A temporary solution will become permanent. But getting it right means managing risk. Access is king In a typical office with an on-premise data centre, the IT department has complete control over network access, internal networks, data, and applications. The remote worker, on the other hand, is mobile. He or she can work from anywhere using a VPN. Until just recently this will have been from somewhere like a local coffee shop, possibly using a wireless network to access the company network and essential applications. CV-19 means that huge numbers of people are getting access to the same desktop and files, and collaborative communication toolsBut as we know, CV-19 means that huge numbers of people are getting access to the same desktop and files, applications and collaborative communication tools that they do on a regular basis from the office or on the train. Indeed, the new generation of video conferencing technologies come very close to providing an “almost there” feeling. Hackers lie in wait Hackers are waiting for a wrong move amongst the panic, and they will look for ways to compromise critical servers. Less than a month ago, we emerged from a period of chaos. For months hackers had been exploiting a vulnerability in VPN products from Pulse Secure, Fortinet, Palo Alto Networks, and Citrix. Patches were provided by vendors, and either companies applied the patch or withdrew remote access. As a result, the problem of attacks died back.  But as companies race to get people working from home, they must ensure special care is taken to ensure the patches are done before switching VPNs on. That’s because remote desktop protocol (RDP) has been for the most part of 2019, and continues to be, the most important attack vector for ransomware. Managing a ransomware attack on top of everything else would certainly give you sleepless nights. As companies race to get people working from home, they must ensure special care is taken to ensure the patches are done before switching VPNs on Hackers are waiting for a wrong move amongst the panic, and they will look for ways to compromise critical serversExposing new services makes them also susceptible to denial of service attacks. Such attacks create large volumes of fake traffic to saturate the available capacity of the internet connection. They can also be used to attack the intricacies of the VPN protocol. A flow as little as 1Mbps can perturbate the VPN service and knock it offline. CIOs, therefore, need to acknowledge that introducing or extending home working broadens the attack surface. So now more than ever it’s vital to adapt risk models. You can’t roll out new services with an emphasis on access and usability and not consider security. You simply won’t survive otherwise. Social engineering Aside from securing VPNs, what else should CIO and CTOs be doing to ensure security? The first thing to do is to look at employee behaviour, starting with passwords. It’s highly recommended that strong password hygiene or some form of multi-factor authentication (MFA) is imposed. Best practice would be to get all employees to reset their passwords as they connect remotely and force them to choose a new password that complies with strong password complexity guidelines.  As we know, people have a habit of reusing their passwords for one or more online services – services that might have fallen victim to a breach. Hackers will happily It’s highly recommended that strong password hygiene or some form of multi-factor authentication (MFA) is imposedleverage these breaches because it is such easy and rich pickings. Secondly, the inherent fear of the virus makes for perfect conditions for hackers. Sadly, a lot of phishing campaigns are already luring people in with the promise of important or breaking information on COVID-19. In the UK alone, coronavirus scams cost victims over £800,000 in February 2020. A staggering number that can only go up. That’s why CIOs need to remind everyone in the company of the risks of clickbait and comment spamming - the most popular and obvious bot techniques for infiltrating a network. Notorious hacking attempts And as any security specialist will tell you, some people have no ethics and will exploit the horrendous repercussions of CV-19. In January we saw just how unscrupulous hackers are when they started leveraging public fear of the virus to spread the notorious Emotet malware. Emotet, first detected in 2014, is a banking trojan that primarily spreads through ‘malspam’ and attempts to sneak into computers to steal sensitive and private information. In addition, in early February the Maze ransomware crippled more than 230 workstations of the New Jersey Medical Diagnostics Lab and when they refused to pay, the vicious attackers leaked 9.5GB or research data in an attempt to force negotiations. And in March, an elite hacking group tried to breach the World Health Organization (WHO). It was just one of the many attempts on WHO and healthcare organisations in general since the pandemic broke. We’ll see lots more opportunist attacks like this in the coming months.   More speed less haste In March, an elite hacking group tried to breach the World Health Organization (WHO). It was just one of the many attempts on WHOFinally, we also have bots to contend with. We’ve yet to see reports of fake news content generated by machines, but we know there’s a high probability it will happen. Spambots are already creating pharmaceutical spam campaigns thriving on the buying behaviour of people in times of fear from infection. Using comment spamming – where comments are tactically placed in the comments following an update or news story - the bots take advantage of the popularity of the Google search term ‘Coronavirus’ to increase the visibility and ranking of sites and products in search results. There is clearly much for CIOs to think about, but it is possible to secure a network by applying some well thought through tactics. I believe it comes down to having a ‘more speed, less haste’ approach to rolling out, scaling up and integrating technologies for home working, but above all, it should be mixed with an employee education programme. As in reality, great technology and a coherent security strategy will never work if it is undermined by the poor practices of employees.

How does audio enhance security system performance?
How does audio enhance security system performance?

Video is widely embraced as an essential element of physical security systems. However, surveillance footage is often recorded without sound, even though many cameras are capable of capturing audio as well as video. Beyond the capabilities of cameras, there is a range of other audio products on the market that can improve system performance and/or expand capabilities (e.g., gunshot detection.) We asked this week’s Expert Panel Roundtable: How does audio enhance the performance of security and/or video systems? 

How have standards changed the security market?
How have standards changed the security market?

A standard is a document that establishes uniform engineering or technical criteria, methods, processes, and/or practices. Standards surround every aspect of our business. For example, the physical security marketplace is impacted by industry standards, national and international standards, quality standards, building codes and even environmental standards, to name just a few. We asked this week’s Expert Panel Roundtable: How have standards changed the security market as we know it?