Data storage devices across the security industry are routinely required to handle an enormous amount and many layers of raw data. As Safe City projects in varying sizes become more prevalent, the number of surveillance nodes has reached the hundreds of thousands. And due to the widespread use of high-definition monitoring, the amount of data involved in security surveillance has increased dramatically in a short time. Efficient collection, analysis, and application of data and the intelligent use of it are becoming ever more critical in this industry. Thus, improving video intelligence appears to be an inevitable, industry-wide goal.

Security users hope that their investment in new products will bring even more benefits beyond simply tracing and tracking persons of interest and evidence collection after a security event. Some examples of added benefits include using the latest technologies to replace the large amount of man-power previously required for searching surveillance footage, detecting anomalous data, and finding ever more efficient ways to allow surveillance to shift from post-incident tracing to alerts during incidents—or even pre-incident alerts. In order to satisfy these demands, new technologies are required. Intelligent video surveillance has been available for many years. However, the outcomes of its application have not been ideal. The emergence of deep learning has enabled these demands to become reality.

The insufficiency of traditional intelligent algorithms

Traditional intelligent video surveillance has especially strict requirements for a scene’s background. The accuracy of intelligent recognition and analysis in comparable scenarios remains inconsistent. This is primarily due to the fact that traditional intelligent video analysis algorithms still have many flaws. In an intelligent recognition and analysis process, such as human facial recognition, two key steps are required: First, features are extracted, and second, “classification learning” is performed.

The degree of accuracy in this first step directly determines the accuracy of the algorithm. In fact, most of the system’s calculation and testing workload is consumed in this part. The features in traditional intelligent algorithms are designed by humans and have always been heavily subjective. More abstract features—those that humans have difficulty comprehending or describing—are inevitably missed. With shifting angles and lighting, and especially when the sample size is enormous, many features can be too difficult to detect. Therefore, while traditional intelligent algorithms perform well in very specific environments, subtle changes (image quality, environment, etc.) yield significant challenges to accuracy.

Target detection and attribute recognition

The second step—classification learning—mainly involves target detection and attribute recognition. As the number of available categories for classification rises, so does the difficulty level. Hence, traditional intelligent analysis technologies are highly accurate in vehicle analysis but not in human and object analysis. For example, in vehicle detection, a distinction is made between a vehicle and a non-vehicle, so the classification is simple and the level of difficulty is low. To recognise vehicle attributes requires recognition of different vehicle designs, logos, and so on. However, there are relatively few of these, making the classification results generally accurate. On the other hand, if recognition is to be performed on human faces, each person is a classification of its own, and the corresponding categories will be extremely numerous—naturally leading to a very high level of difficulty.

The accuracy of intelligent recognition and analysis in comparable scenarios remains inconsistent
Enhanced accuracy is the result of multi-layer learning and extensive data collection

Traditional intelligent algorithms generally use shallow learning models to handle situations with large amounts of data in complex classifications. The analysis results are far from ideal. Furthermore, these results directly restrict the breadth and depth of intelligent applications and further development. Hence the need for increasing the “depth” of intelligence in big data for the security industry is arising.

The advantages of Deep Learning and its algorithms

Traditional intelligent algorithms are designed by humans. Whether or not they are designed well depends greatly on experience and even luck, and this process requires a lot of time. So, is it even possible to get machines to automatically learn some of the features? Yes! This is actually the objective of Artificial Intelligence (AI).

The inspiration for deep learning comes from a human brain’s neural networks. Our brains can be seen as a very complex deep learning model. Brain neural networks are comprised of billions of interconnected neurons; deep learning simulates this structure. These multi-layer networks can collect information and perform corresponding actions. They also possess the ability for object abstraction and recreation. Deep learning is intrinsically different from other algorithms. The way it solves the insufficiencies of traditional algorithms is encompassed in the following aspects.

Algorithmic model for deep learning

The algorithmic model for deep learning has a much deeper structure than the two 3-layered structures of traditional algorithms. Sometimes, the number of layers can reach over a hundred, enabling it to process large amounts of data in complex classifications. Deep learning is very similar to the human learning process, and has a layer-by-layer feature-abstraction process. Each layer will have different “weighting,” and this weighting reflects on what was learned about the images’ “components.” The higher the layer level, the more specific the components. Simulating the human brain, an original signal in deep learning passes through layers of processing; next, it takes a partial understanding (shallow) to an overall abstraction (deep) where we can perceive the object.

Deep learning does not require manual intervention but relies on a computer to extract features by itself. This way it is able to extract as many features from the target as possible, including abstract features that are difficult or impossible to describe. The more features there are, the more accurate the recognition and classification will be. Some of the most direct benefits that deep learning algorithms can bring include achieving comparable or even better-than-human pattern recognition accuracy, strong anti-interference capabilities, and the ability to classify and recognise thousands of features.

Hikvision has operated in the security industry for many years with its own research and development capabilities

Key factors of Deep Learning

In total, there are three main reasons why deep learning only became popular in recent years and not earlier: the scale of data involved, computing power, and network architecture.

Improvements in data-driven algorithm performance have accelerated deep learning in various intelligent applications in a short amount of time. Specifically, with the increase in data scale, algorithmic performance improved as well. Accordingly, user experience has improved and more users are involved, further facilitating a larger scale of data.

Video surveillance data makes up 60% of big data, and the amount is rising at 20% annually. The speed and scale of this achievement is due to the popularisation of high definition video surveillance—HD 1080p is becoming more common, and 4K and higher resolutions are gradually being applied in many important applications.

Hikvision has operated in the security industry for many years with its own research and development capabilities, employing large amounts of real video and image data as training samples. With a large amount of good quality data, and over a hundred team members to label the video images, sample data with millions of categories have been accumulated. With this large amount of quality training data, human, vehicle, and object pattern recognition models will become more and more accurate for video surveillance use.

The deep learning model requires a large amount of samples, making a large amount of calculations inevitable
Enhanced accuracy is the result of multi-layer learning and extensive data collection

Higher computational power

Furthermore, high performance hardware platforms enable higher computational power. The deep learning model requires a large amount of samples, making a large amount of calculations inevitable. In the past, hardware devices were incapable of processing complex deep learning models with over a hundred layers. In 2011, Google’s DeepMind used 1,000 devices with 16,000 CPUs to simulate a neural network with approximately 1 billion neurons. Today, only a few GPUs are required to achieve the same sort of computational power with even faster iteration. The rapid development of GPUs, supercomputers, cloud computing, and other high performance hardware platforms has allowed deep learning to become possible.

Finally, the network architecture plays its own role in advancing deep learning. Through constant optimisation of deep learning algorithms, better target-object recognition can be achieved. For more complex applications such as facial recognition or in scenarios with different lighting, angles, postures, expressions, accessories, resolutions, etc., network architecture will impact the accuracy of recognition, i.e., the more layers in deep learning algorithms, the better the performance.

In 2016, Hikvision achieved the number one position in the Scene Classification category at the ImageNet Large Scale Visual Recognition Challenge 2016. The team from Hikvision Research Institute used inception-style networks and not-so-deep residual networks that perform better in considerably less training time, according to Hikvision’s experiments for training and testing.

Furthermore, Hikvision’s Optical Character Recognition (OCR) Technology, based on Deep Learning and led by the company’s Research Institute, also won the first price in the ICDAR 2016 Robust Reading Competition. The Hikvision team substantially surpassed both strong domestic and foreign competitors in three word-recognition challenges, including born-digital images, focused scene text, and incidental scene text, demonstrating that the word recognition technology by Hikvision reached the world’s top level.

In the past two years, deep learning technology has excelled in speech recognition, computer
vision, voice translation,
and much more

Application of Deep Learning products

In the past two years, deep learning technology has excelled in speech recognition, computer vision, voice translation, and much more. It has even surpassed human capabilities in the areas of facial verification and image classification; hence, it has been highly regarded in the field of video surveillance for the security industry.

In the application of intelligent video in target detection, tracking, and recognition, the rise of deep learning has had a profound influence. When applying those three functions, deep learning potentially touches upon every aspect of the security video surveillance industry: facial detection, vehicle detection, non-motor vehicle detection, facial recognition, vehicle brand recognition, pedestrian detection, human body feature detection, abnormal facial detection, crowd behaviour analysis, multiple target tracking, and so on.

These types of intelligent functions require a series of front-end surveillance cameras, back-end servers and other products which support deep learning algorithms. In small scale applications, front-end cameras can directly operate structured human and vehicle feature extraction, and tens of thousands of human facial images can be stored within the front-end devices to implement direct facial comparison, so as to reduce costs of communicating with a server. In large scale applications, front-end cameras can work with back-end servers. Specifically, the structured video task is handled by front-end devices, reducing the workload for back-end devices; matching and searching efficiency of back-end servers improve as well.

Hikvision new products with Deep Learning

This year, Hikvision will soon introduce a series of products with deep learning technology, such as the DeepInview Series cameras which can accurately detect, recognise, and analyse human, vehicle, and object features and behaviour, and can be widely used in indoor and outdoor scenarios. Another of products worth mentioning is Hikvision’s DeepInmind Series of NVRs which incorporate advanced deep learning algorithms and imitate human thoughts and memory. The DeepInmind products feature an innovative NVR+GPU mode, retaining the advantages of traditional NVRs and additional structured video analysis functions, which together greatly improve the value of video.

Deep learning is the next level of AI development. It is beyond machine learning where supervised classification of features and patterns are set into algorithms. Deep learning incorporates unsupervised or “self-learning” principles. Hikvision is developing this concept in its own analytics algorithms. Enhanced accuracy is the result of multi-layer learning and extensive data collection. Application of this algorithm into face recognition, vehicle recognition, human recognition, and other platforms will significantly advance the performance of analytics.

Download PDF version

In case you missed it

The new alliance of humans and robotics in security solutions in 2018
The new alliance of humans and robotics in security solutions in 2018

  The past year has proved to be a year full of many changes both within our industry and for Robotic Assistance Devices (RAD). While we have experienced increased adoption of artificial intelligence-based solutions, the industry has been challenged with an ever-evolving technology landscape. Protecting enterprise organisations from both cyber and physical security threats will be an ongoing challenge the industry must grapple with. Greater adoption of robotic solutions To address the physical security challenges, we saw a greater adoption of robotic solutions across the board. Our massive industry started to make the change: Shifting from an uneducated view of this advanced technology to increased interest about artificial intelligence across multiple markets including guarding companies, integrators and, most importantly, end-users. In 2017 there was a greater adoption of robotic solutions across the board With security-guard robots, security directors now have access to additional tools to meet their performance and budget goals. Currently, we see a great adoption with progressive guarding companies, which are signing up to have RAD as their robotic guarding partner. RAD deployed its first robot this year, and we look forward to deploying many more as we work with our customers to customise our robotic solution to their needs. Human collaboration with robotics I believe our industry is at the beginning stages of what could be a serious paradigm shift in how we rely on a combination of humans and technology to do a job. We've seen that in video analytics and the wide variety of solutions available on the market today. The trend has progressed beyond video analytics and into robotics, and that will continue to evolve into 2018 and beyond. As we continue to build on the success of our security guard robot solution, we look forward to expanding our product offerings to meet the security needs of our customers.

CES 2018: Security technologies influencing the consumer electronics market
CES 2018: Security technologies influencing the consumer electronics market

Security is more-than-ever linked to consumer electronics, especially in the residential/smart home market. CES 2018 in Las Vegas is therefore brimming with news that will have a direct impact on the security market, today and especially looking into the future. Products for the future of security   CES is a giant trade show for consumer electronics with 2.75 million net square feet of exhibitor space and featuring more than 3,900 exhibitors, including 900 startups - in contrast, ISC West has some 1,000 exhibitors. During the week-long show welcoming 170,000-plus attendees from 150 countries, more than 20,000 new products are being launched. The products incorporate ingredient technologies such as artificial intelligence and 5G that will also be familiar elements as the future of the security industry unfolds. Familiar players at security shows also have a presence at CES, and many consumer technologies on display offer a glimpse of what’s ahead for security The areas of consumer electronics and security are closely intertwined. For example, Apple recently expanded near-field communication (NFC) support to include the NDEF (NFC Data Exchange Format), which will likely accelerate the adoption of smartphones for access control credentialing. In another recent development, Amazon acquired Blink, a home security camera startup that offers wireless home security systems. The acquisition aligns with Amazon’s effort to offer more home devices. Key security technologies at CES 2018 Familiar players at security shows also have a presence at CES. For example, Bosch is highlighting its “Simply. Connected” portfolio of smart city technology to transform security as well as urban mobility, air quality and energy efficiency. Many consumer technologies on display offer a glimpse of what’s ahead for security. Are Panasonic’s 4K OLEDs with HDR10+ format or Sony’s A8F OLED televisions a preview of the future of security control room monitors? At CES, Johnson Controls is announcing support for Apple HomeKit now offered in their DSC iotega wireless security and automation solution. Consumers can manage both their security system and also other home automation abilities using Apple’s Home app, or Siri on their iPhone, iPad or Apple Watch. Interlogix is announcing new features and components of its UltraSync SmartHome system, including hands-free voice control, high-definition cameras, an LTE cellular module and soon-to-be-released doorbell camera. The areas of consumer electronics and security are closely intertwined Developments in crime awareness ADT has a high profile at CES, including the launch of its ADT Go mobile app, equipped with 24/7 emergency response from ADT’s live monitoring agents and backed by Life360’s location technology, providing emergency response, family connectivity, safety assistance and crime awareness.  ADT is also unveiling a video doorbell and expanding its monitoring to cybersecurity. IC Realtime is introducing Ella, a cloud-based deep-learning search engine that augments surveillance systems with natural language search capabilities across recorded video footage. Ella enables any surveillance or security cameras to recognise objects, colours, people, vehicles and animals. Ella was designed using the technology backbone of Camio, a startup founded by ex-Googlers who designed a simpler way to apply searching to streaming video feeds. It’s a “Google for video:” Users can type in queries such as “white truck” to find every relevant video clip. Smarter homes and smarter computers Do-it-yourself smart home security company Abode Systems announces iota, an all-in-one system giving customers more freedom and flexibility to build out and monitor their smart home. The new form factor has a built-in full-HD resolution camera enabling customers to see and hear what’s going on in their home 24/7 while a built-in gateway supports hundreds of devices to make homes more convenient, safer and more secure. There is also support for Apple HomeKit. Highly programmable and high-performance platforms will no doubt play a role in the future of video surveillance systems in our market  The Z-Wave Alliance will host 30-plus leading smart home brands in the Z-Wave pavilion at CES. A full walk-through home will demonstrate different brands working together to create one cohesive smart home experience. Sigma Designs unveils its 700-Series Z-Wave platform, including numerous performance and technology enhancements in energy-efficiency and RF performance. Personal protection in attendance Self-defence product company SABRE will debut a combination pepper spray with dual sound-effect personal alarm that “alternates between the traditional wailing sound and a primal scream, while a strobe blinks 19 times per second to disorient assailants.” SABRE’s Modern Fake Security Camera includes “sleek, realistic design to deter would-be thieves.” Chip maker Ambarella is introducing the CV1 4K Stereovision Processor with CFflow Computer Vision Architecture. The chip combines environmental perception with advances in deep neural network processing for a variety of applications, including video security cameras and fully autonomous drones. At CES, applications will focus on automotive uses, including advanced driver assistance systems (ADAS), self-driving, electronic mirror and surround view systems. The highly programmable and high-performance platform will no doubt play a role in the future of video surveillance systems in our market. A full walk-through home will demonstrate different brands working together to create one cohesive smart home experience Extending home security and efficiency  The Ring whole-house security ecosystem creates a “Ring of Security” around homes and neighbourhoods. Products include “Stick Up” indoor/outdoor security cameras, integrated LED lighting, a “Ring Alarm” integrated bundle for $199 including a base station, keypad, contact sensor, and Z-Wave extender. “Ring Protect Plans” include 24/7 professional monitoring. The “Streety” phone app, from Vivint Smart Home, extends home security into the neighbourhood. Streety makes it easy for neighbours to monitor neighbourhood activity through a network of shared residential cameras. They can keep an eye on kids, cars and property through live video feeds and use recorded video clips to investigate incidents. A new device making its debut at CES is the Walker “commercialised biped robot,” from UBTECH Robotics, which provides a complete home butler service and is designed to ease the day-to-day operations of a busy home or office. The varied of functions includes video surveillance monitoring, security patrol monitoring, motion detection and “instant alarm,” as well as dancing and playing games with children. The company says Walker will “bridge the gap between technologies that were once only available in scientific research institutions and everyday people.”

How IoT and Cloud-based security will make cities safer in 2018
How IoT and Cloud-based security will make cities safer in 2018

In 2017 we saw a lot of new construction projects, and many existing buildings upgraded their security systems to include high-resolution cameras and better-quality recording systems. Because the economy is stronger, many businesses and municipalities increased their security budgets for large-scale and public projects due to terrorism threats in public places.   Smart cities became more popular One of the bigger trends we saw in 2017 is the growing popularity of smart cities and the adoption of public safety systems in both North American and Europe. This includes many cities creating wireless network infrastructure for public WiFi connectivity and for their surveillance network. Oftentimes smart cities develop because of an initial safe city initiative and then cities start to leverage the same infrastructure for more applications. Impact of terrorism Unfortunately, we saw a growth in terrorism attacks in 2017 in Europe and the United States. This has had a significant impact on security in public spaces where large groups of people congregate for entertainment, shopping and sporting events, all of which are now potential targets. We started to see cities install bollards on streets to prevent trucks from driving up on people on sidewalks and video surveillance systems so that police can monitor public spaces in real time. An example was the SuperBowl LIVE venue in Houston, which held several large outdoor events. To help monitor these events the city deployed a mmWave wireless network system for the surveillance cameras which were installed to monitor this area. Cybersecurity a growing concern In addition to terrorism threats, cybersecurity has become a growing concern and focus. More and more manufacturers, including Siklu, have begun to develop secure systems that are extremely difficult for hackers to gain access to because an encrypted network is no longer enough. The devices on the network also have to be secure. There is a growing shift towards younger generations wanting to live in the city where they have access to public transportation, restaurants and entertainment Looking ahead to 2018, the security market should expect to see continued growth in the use of video analytics for proactive surveillance purposes and more technology that leverages the intelligence of this data. Also, there is a growing shift towards younger generations wanting to live in the city where they have access to public transportation, restaurants and entertainment. They also expect to live in a safer environment and this is where the smart city approach comes into play with the introduction of WiFi in parks and public spaces, along with surveillance systems. These two solutions and services can now sit on the same network, thanks to better connectivity options and interference free solutions, such as mmWave wireless radios. Embracing new technology Next year the winners will be those who embrace new technology and do not solely focus on security. It’s important to embrace other IoT devices and recognise that video as a service is growing in demand. Cloud-based solutions are also growing for both video storage and monitoring management systems. The losers will be those who are not willing to embrace new technology, those who offer poor service and those who don’t expand their business to include professional services. Siklu success Siklu’s security business has doubled year over year, and there are now more than 100 cities globally with a Siklu radio deployed. This is because there is an increasing acceptance of our mmWave wireless technology and people are starting to recognise the benefits our systems provide when compared with installing new fiber or a traditional WiFi system. We recently introduced a new point-to-multipoint solution called MultiHaul™, which utilises immune narrow beams within a point-to-multipoint network topology and enables interference free connectivity and complete security. The solution’s 90-degree scanning antenna auto-aligns multiple terminal units from a single base unit, serving multiple locations while reducing installation times to minutes instead of hours by a single person and the total cost of ownership for end users.