The field of artificial intelligence known as machine learning or cognitive computing has in recent years become highly popular. The meteoric rise of “deep learning” technology over the past several years has been truly dramatic in many industries. Industry giants from Google, Microsoft, Facebook, IBM and many others have been pouring massive amounts of investments in this field of artificial intelligence. The machine learning field has exploded on the scene with the breakthrough in the new “deep learning” technology.

Developments in deep learning have ramifications for the physical security industry, too. In video analytics, for example, deep learning has shown promise to improve some difficult problems, although more work is needed. This article will cover the evolving field of deep learning and its potential impact on the security and video surveillance markets.

Evolution of deep learning

The field of deep learning evolved from “artificial neural nets” from the 1980s. In the early years of this branch of artificial intelligence, the neural nets are modelled after a human’s brain, which consists of over 100 billion neurons. The field of neural networks never really took off in the ‘80s and ‘90s due to many reasons. The key limitations of the earlier systems are the difficultly to train the network; and the hardware CPU technologies were too slow to properly train a neural net that can solve meaningful real-world applications.

Over the past several years, real
world applications of deep learning
now encompass many industries
including handwriting recognition
and language translation

The 1980s and 1990s were the dark days of neural network research. Since 2000, the research community of neural nets has really started to garner industry labs’ attention from the breakthroughs in deep learning work in academia at the University of Toronto, NYU, Stanford and others. Over the past several years, real world applications of deep learning now encompass many industries including handwriting recognition, language translation, automatic game (chess/Go) playing, object classification, face recognition, medical image analysis, autonomous driving cars and many other fields.

One example of the excitement with deep learning technology is the recent breakthrough from Google’s AlphaGo, a computer program that for the first time beat a professional human Go player in October, 2015. The sophistication of the Deep learning based program has astonished many in the field of artificial intelligence due the complexity of the ancient Asia GO game, which is considerably more complicated than chess.

 Video surveillance applications for deep learning

Although deep learning has been applied to many industries with breakthrough results compared to legacy systems, not all applications are suitable for deep learning. In the field of video surveillance, several applications stand out that can benefit from deep learning.

Face recognition. Deep learning technology has significantly improved the accuracy rate of face recognition. The National Institute of Standards and Technology (NIST) has conducted Face Recognition Vendor Test (FRVT) test over the past decade. The improvements over the last 20 years of face recognition error rates have decreased by three orders of magnitude, according to an NIST Interagency Report. Most of today’s top-performing commercial face recognition products are based on deep learning. The accuracy has reached 99.9% for controlled environments like airport immigration face recognition applications, according to research by Facebook and Tel Aviv University.

Person and object detection. Person detection and object detection is another area where deep learning has shown tremendous progress. For example, over the past five years, the IMAGENET database has organised the “large scale visual recognition challenge,” in which image software algorithms are challenged to detect, classify and localise a database of over 150,000 photographs collected from Flickr and other search engines. The dataset is labelled into 1,000 object categories. Many deep learning systems are trained with over 1.2 million images from the IMAGENET dataset running on GPU based hardware accelerators. The improvements in accuracy range from 72% to over 90% from 2010 thru 2014. In 2015, all IMAGENET contestants used deep learning techniques.

Deep learning in its infancy has shown a lot of promise in improving some difficult video analytics problems
In the field of video surveillance, several applications stand out that can benefit from deep learning

Deep learning-based video surveillance solutions

A key advantage of deep learning-based algorithms over legacy computer vision algorithms is that deep learning system can be continuously trained and improved with better and more datasets. Many applications have shown that deep learning systems can “learn” to achieve 99.9% accuracy for certain tasks, in contrast to rigid computer algorithms where it is very difficult to improve a system past 95% accuracy.

Deep learning has the true potential
of significantly reducing false-positive
detection events that plague many
security video analytics systems

The second advantage with deep learning system is the “abnormal” event detection. Deep learning systems have shown remarkable ability to detect undefined or unexpected events. This feature has the true potential of significantly reducing false-positive detection events that plague many security video analytics systems. In fact, the inability to reduce false-positive detection rate is the key problem in video surveillance industry; and has to-date prevented the wide scale acceptance of many vendor’s intelligent video analytics solutions.

Open issues with deep learning technology for video

Deep learning in its infancy has shown a lot of promise in improving some hard, and difficult video analytics problems. Much more work needs to be done to fine-tune the generic deep learning system to learn and detect domain specific events that are unique to security-oriented environments.

The second challenge is that the engineering talent for deep learning is in extreme short supply. Most graduates today come from the top universities and upon graduation are immediately snapped up by Internet giants like Google, Facebook, Amazon, Microsoft, etc. The competition for trained machine learning engineers is intense.

The third challenge is that not all video analytics algorithms are best applied with deep learning. There are many legacy computer vision algorithms that have been developed over decades that are very well suited and deployed in commercially successful products. For example, license plate recognition performs very well with computer-vision based algorithms. Industry needs to do more research in hybrid systems the combine the best of computer vision algorithms and deep learning.

What’s next?

Similar to cloud computing and big data technologies, deep learning technology is now emerging as the third wave of rapid advances that have taken over the information industry by storm.

Over the next decade, very few areas in the technology sector will not be touched by the advances of cloud, big data and machine learning. For the video surveillance industry, this is welcome news. The industry has been lacking in innovations that can significantly advance the state-of-the art.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

Author profile

In case you missed it

What is the impact of lighting on video performance?
What is the impact of lighting on video performance?

Dark video images contain little or no information about the subject being surveilled. Absence of light can make it difficult to see a face, or to distinguish the color of clothing or of an automobile. Adding light to a scene is one solution, but there are also new technologies that empower modern video cameras to see better in any light. We asked this week’s Expert Panel Roundtable: what impact does lighting have on the performance of video systems? adapts during pandemic to enable partners to ‘succeed remotely’ adapts during pandemic to enable partners to ‘succeed remotely’

As a cloud-based platform for service providers in the security, smart home and smart business markets, has adapted quickly to changing conditions during the coronavirus pandemic. In the recent dynamic environment, has kept focus on supporting their service provider partners so they can keep local communities protected. “We moved quickly to establish work-from-home protocols to protect our employees and minimise impact on our partners,” says Anne Ferguson, VP of Marketing at The Customer Operations and Reseller Education (CORE) team has operated without interruption to provide support to partners. Sales teams are utilising webinars and training resources to inform and educate partners about the latest products, tools, and solutions.’s partner tools are essential for remote installations and support of partner accounts. Helping customers remain connected Adapting to challenges of the coronavirus pandemic, is further investing in solutions that help customers remain connected and engaged. The company has created a resource hub called “Succeeding Remotely” that provides tools, tips and news links that partners can use to adapt their business operations. From adjusting sales and installation techniques to maintaining cellular upgrades, is helping partners stay connected to customers remotely, keep their teams trained, and address rapidly evolving customer concerns without rolling trucks.The company has created a resource hub called “Succeeding Remotely “Additionally, after seeing all that our partners are doing to support their local communities in need, we were compelled to highlight those efforts with ongoing videos called Good Connections, which we’re sharing with our partner community to spark more ideas and ways to help,” says Ferguson. “Though our partners have experienced varying degrees of disruption to their business, we’re inspired by their adaptability, ingenuity and resilience,” says Ferguson. “Along with establishing proper safeguards for operating in homes and businesses, our partners are leveraging our support resources more heavily, while our entire staff has worked tirelessly to deliver new, timely resources.” Do-It-Together solutions partners are successfully employing Do-It-Together (DIT) solutions, focusing on 3G-to-LTE upgrades, and pivoting to new verticals like commercial and wellness. Many are also streamlining their business operations and taking advantage of virtual training opportunities to enhance their technicians’ skills and knowledge, says Ferguson. Do-It-Together installs involve depending on customers to perform part or all of the installation process. Partners can send customers fully configured kits with mounting instructions, or technicians may guide customers on a remote video call.’s tools, training and products help partners modify remote installation options depending on each customer’s needs. End users can validate the Smart Gateway with their central station that sensors they have mounted were done correctly using the mobile app Smart Gateway For example, the Smart Gateway can be pre-configured with indoor and outdoor cameras for easy customer installation and to reduce the likelihood of future service calls. Also, end users can validate with their central station that sensors they have mounted were done correctly using the mobile app. “DIT is helping our partners continue onboarding customers and avoid backlogs,” says Ferguson. “We’ve been pleasantly surprised by the resiliency and level of future investment that our residential and commercial partners have shown in the face of adversity,” adds Ferguson. For example, a significant number of business customers have used the slow period to install systems that are typically too disruptive to put in during normal business hours. Similarly, service providers are adopting new technologies or business models, such as cloud-based access control. “They’re often saying to us, ‘I’m going to take this opportunity to make changes to improve our business,’ and have been working closely with us on training and business consulting to support their efforts,“ she says. Shift to the cloud Ferguson sees a growing preference for cloud-managed surveillance and access systems over ones that have historically been run on-premise. The technology itself is attractive, but especially driving change is the enhancement to the daily lives of service providers and customers, which have been strained during this time. “The foundational benefit of our cloud-based solution is the hassle-free, seamless customer experience it delivers,” says Ferguson. “We make this possible by taking ownership of the servers, software maintenance, firmware updates, health monitoring, and more. With cloud technology, these aspects become invisible to the customer and take a lot off their plate, which is more important than ever.” End users can take advantage of Smart Tip video tutorials to help with DIT installations, or they can use the Wellcam to connect with loved ones anywhere.End users can take advantage of Smart Tip video tutorials to help with DIT installations Partners can attend training workshops focused on remote installation tactics, while driving consumer interest in new offerings through’s Customer Connections platform. The goal is to make it simple for partners to stay connected to their customers to maximise lifetime account value. “We are well-positioned to endure the pandemic because of the strength of our partners in their markets along with our investments in technology, hardware and our team,” says Ferguson. “As restrictions slowly lift, there is cautious optimism that the residential, commercial, property management, plumbing/HVAC, builder and other verticals will recover quickly. We believe that as more partners adopt the DIT model and add commercial and wellness RMR, they will find increasing opportunities to deploy security, automation, video, video analytics, access and more throughout their customer base.”

COVID-19 worries boost prospects of touchless biometric systems
COVID-19 worries boost prospects of touchless biometric systems

Spread of the novel coronavirus has jolted awareness of hygiene as it relates to touching surfaces such as keypads. No longer in favour are contact-based modalities including use of personal identification numbers (PINs) and keypads, and the shift has been sudden and long-term. Both customers and manufacturers were taken by surprise by this aspect of the virus’s impact and are therefore scrambling for solutions. Immediate impact of the change includes suspension of time and attendance systems that are touch-based. Some two-factor authentication systems are being downgraded to RFID-only, abandoning the keypad and/or biometric components that contributed to higher security, but are now unacceptable because they involve touching. Touchless biometric systems in demand The trend has translated into a sharp decline in purchase of touch modality and a sharp increase in the demand for touchless systems, says Alex Zarrabi, President of Touchless Biometrics Systems (TBS). Biometrics solutions are being affected unequally, depending on whether they involve touch sensing, he says. Spread of the novel coronavirus has jolted awareness of hygiene as it relates to touching surfaces such as keypads “Users do not want to touch anything anymore,” says Zarrabi. “From our company’s experience, we see it as a huge catalyst for touchless suppliers. We have projects being accelerated for touchless demand and have closed a number of large contracts very fast. I’m sure it’s true for anyone who is supplying touchless solutions.” Biometric systems are also seeing the addition of thermal sensors to measure body temperature in addition to the other sensors driving the system. Fingerscans and hybrid face systems TBS offers 2D and 3D systems, including both fingerscans and hybrid face/iris systems to provide touchless identification at access control points. Contactless and hygienic, the 2D Eye system is a hybrid system that combines the convenience of facial technology with the higher security of iris recognition. The system recognises the face and then detects the iris from the face image and zeros in to scan the iris. The user experiences the system as any other face recognition system. The facial aspect quickens the process, and the iris scan heightens accuracy. TBS also offers the 2D Eye Thermo system that combines face, iris and temperature measurement using a thermal sensor module. TBS's 2D Eye Thermo system combines face, iris and temperature measurement using a thermal sensor module Another TBS system is a 3D Touchless Fingerscan system that provides accuracy and tolerance, anti-spoofing, and is resilient to water, oil, dust and dirt. The 2D+ Multispectral for fingerprints combines 2D sensing with “multispectral” subsurface identification, which is resilient to contaminants and can read fingerprints that are oily, wet, dry or damaged – or even through a latex glove. In addition, the 3D+ system by TBS provides frictionless, no-contact readings even for people going through the system in a queue. The system fills the market gap for consent-based true on-the-fly systems, says Zarrabi. The system captures properties of the hand and has applications in the COVID environment, he says. The higher accuracy and security ratings are suitable for critical infrastructure applications, and there is no contact; the system is fully hygienic. Integration with access control systems Integration of TBS biometrics with a variety of third-party access control systems is easy. A “middleware” subsystem is connected to the network. Readers are connected to the subsystem and also to the corporate access control system. An interface with the TBS subsystem coordinates with the access control system. For example, a thermal camera used as part of the biometric reader can override the green light of the access control system if a high temperature (suggesting COVID-19 infection, for example) is detected. The enrollment process is convenient and flexible and can occur at an enrollment station or at an administration desk. Remote enrollment can also be accomplished using images from a CCTV camera. All templates are encrypted. Remotely enrolled employees can have access to any location they need within minutes. The 3D+ system by TBS provides frictionless, no-contact readings even for people going through the system in a queue Although there are other touchless technologies available, they cannot effectively replace biometrics, says Zarrabi. For example, a centrally managed system that uses a Bluetooth signal from a smart phone could provide convenience, is “touchless,” and could suffice for some sites. However, the system only confirms the presence and “identity” of a smart phone – not the person who should be carrying it. “There has been a lot of curiosity about touchless, but this change is strong, and there is fear of a possible second wave of COVID-19 or a return in two or three years,” says Zarrabi. “We really are seeing customers seriously shifting to touchless.”