Data storage devices across the security industry are routinely required to handle an enormous amount and many layers of raw data. As Safe City projects in varying sizes become more prevalent, the number of surveillance nodes has reached the hundreds of thousands. And due to the widespread use of high-definition monitoring, the amount of data involved in security surveillance has increased dramatically in a short time. Efficient collection, analysis, and application of data and the intelligent use of it are becoming ever more critical in this industry. Thus, improving video intelligence appears to be an inevitable, industry-wide goal.

Security users hope that their investment in new products will bring even more benefits beyond simply tracing and tracking persons of interest and evidence collection after a security event. Some examples of added benefits include using the latest technologies to replace the large amount of man-power previously required for searching surveillance footage, detecting anomalous data, and finding ever more efficient ways to allow surveillance to shift from post-incident tracing to alerts during incidents—or even pre-incident alerts. In order to satisfy these demands, new technologies are required. Intelligent video surveillance has been available for many years. However, the outcomes of its application have not been ideal. The emergence of deep learning has enabled these demands to become reality.

The insufficiency of traditional intelligent algorithms

Traditional intelligent video surveillance has especially strict requirements for a scene’s background. The accuracy of intelligent recognition and analysis in comparable scenarios remains inconsistent. This is primarily due to the fact that traditional intelligent video analysis algorithms still have many flaws. In an intelligent recognition and analysis process, such as human facial recognition, two key steps are required: First, features are extracted, and second, “classification learning” is performed.

The features in traditional intelligent algorithms are designed by humans and have always been heavily subjective

The degree of accuracy in this first step directly determines the accuracy of the algorithm. In fact, most of the system’s calculation and testing workload is consumed in this part. The features in traditional intelligent algorithms are designed by humans and have always been heavily subjective. More abstract features—those that humans have difficulty comprehending or describing—are inevitably missed. With shifting angles and lighting, and especially when the sample size is enormous, many features can be too difficult to detect. Therefore, while traditional intelligent algorithms perform well in very specific environments, subtle changes (image quality, environment, etc.) yield significant challenges to accuracy.

Target detection and attribute recognition

The second step—classification learning—mainly involves target detection and attribute recognition. As the number of available categories for classification rises, so does the difficulty level. Hence, traditional intelligent analysis technologies are highly accurate in vehicle analysis but not in human and object analysis. For example, in vehicle detection, a distinction is made between a vehicle and a non-vehicle, so the classification is simple and the level of difficulty is low. To recognise vehicle attributes requires recognition of different vehicle designs, logos, and so on. However, there are relatively few of these, making the classification results generally accurate. On the other hand, if recognition is to be performed on human faces, each person is a classification of its own, and the corresponding categories will be extremely numerous—naturally leading to a very high level of difficulty.

The accuracy of intelligent recognition and analysis in comparable scenarios remains inconsistent
Enhanced accuracy is the result of multi-layer learning and extensive data collection

Traditional intelligent algorithms generally use shallow learning models to handle situations with large amounts of data in complex classifications. The analysis results are far from ideal. Furthermore, these results directly restrict the breadth and depth of intelligent applications and further development. Hence the need for increasing the “depth” of intelligence in big data for the security industry is arising.

The advantages of Deep Learning and its algorithms

Traditional intelligent algorithms are designed by humans. Whether or not they are designed well depends greatly on experience and even luck, and this process requires a lot of time. So, is it even possible to get machines to automatically learn some of the features? Yes! This is actually the objective of Artificial Intelligence (AI).

The inspiration for deep learning comes from a human brain’s neural networks. Our brains can be seen as a very complex deep learning model. Brain neural networks are comprised of billions of interconnected neurons; deep learning simulates this structure. These multi-layer networks can collect information and perform corresponding actions. They also possess the ability for object abstraction and recreation. Deep learning is intrinsically different from other algorithms. The way it solves the insufficiencies of traditional algorithms is encompassed in the following aspects.

Algorithmic model for deep learning

The algorithmic model for deep learning has a much deeper structure than the two 3-layered structures of traditional algorithms. Sometimes, the number of layers can reach over a hundred, enabling it to process large amounts of data in complex classifications. Deep learning is very similar to the human learning process, and has a layer-by-layer feature-abstraction process. Each layer will have different “weighting,” and this weighting reflects on what was learned about the images’ “components.” The higher the layer level, the more specific the components. Simulating the human brain, an original signal in deep learning passes through layers of processing; next, it takes a partial understanding (shallow) to an overall abstraction (deep) where we can perceive the object.

Deep learning does not require manual intervention but relies on a computer to extract features by itself. This way it is able to extract as many features from the target as possible, including abstract features that are difficult or impossible to describe. The more features there are, the more accurate the recognition and classification will be. Some of the most direct benefits that deep learning algorithms can bring include achieving comparable or even better-than-human pattern recognition accuracy, strong anti-interference capabilities, and the ability to classify and recognise thousands of features.

Hikvision has operated in the security industry for many years with its own research and development capabilities

Key factors of Deep Learning

In total, there are three main reasons why deep learning only became popular in recent years and not earlier: the scale of data involved, computing power, and network architecture.

Improvements in data-driven algorithm performance have accelerated deep learning in various intelligent applications in a short amount of time. Specifically, with the increase in data scale, algorithmic performance improved as well. Accordingly, user experience has improved and more users are involved, further facilitating a larger scale of data.

Video surveillance data makes up 60% of big data, and the amount is rising at 20% annually. The speed and scale of this achievement is due to the popularisation of high definition video surveillance—HD 1080p is becoming more common, and 4K and higher resolutions are gradually being applied in many important applications.

Hikvision has operated in the security industry for many years with its own research and development capabilities, employing large amounts of real video and image data as training samples. With a large amount of good quality data, and over a hundred team members to label the video images, sample data with millions of categories have been accumulated. With this large amount of quality training data, human, vehicle, and object pattern recognition models will become more and more accurate for video surveillance use.

The deep learning model requires a large amount of samples, making a large amount of calculations inevitable
Enhanced accuracy is the result of multi-layer learning and extensive data collection

Higher computational power

Furthermore, high performance hardware platforms enable higher computational power. The deep learning model requires a large amount of samples, making a large amount of calculations inevitable. In the past, hardware devices were incapable of processing complex deep learning models with over a hundred layers. In 2011, Google’s DeepMind used 1,000 devices with 16,000 CPUs to simulate a neural network with approximately 1 billion neurons. Today, only a few GPUs are required to achieve the same sort of computational power with even faster iteration. The rapid development of GPUs, supercomputers, cloud computing, and other high performance hardware platforms has allowed deep learning to become possible.

Finally, the network architecture plays its own role in advancing deep learning. Through constant optimisation of deep learning algorithms, better target-object recognition can be achieved. For more complex applications such as facial recognition or in scenarios with different lighting, angles, postures, expressions, accessories, resolutions, etc., network architecture will impact the accuracy of recognition, i.e., the more layers in deep learning algorithms, the better the performance.

In 2016, Hikvision achieved the number one position in the Scene Classification category at the ImageNet Large Scale Visual Recognition Challenge 2016. The team from Hikvision Research Institute used inception-style networks and not-so-deep residual networks that perform better in considerably less training time, according to Hikvision’s experiments for training and testing.

Furthermore, Hikvision’s Optical Character Recognition (OCR) Technology, based on Deep Learning and led by the company’s Research Institute, also won the first price in the ICDAR 2016 Robust Reading Competition. The Hikvision team substantially surpassed both strong domestic and foreign competitors in three word-recognition challenges, including born-digital images, focused scene text, and incidental scene text, demonstrating that the word recognition technology by Hikvision reached the world’s top level.

In the past two years, deep learning technology has excelled in speech recognition, computer
vision, voice translation,
and much more

Application of Deep Learning products

In the past two years, deep learning technology has excelled in speech recognition, computer vision, voice translation, and much more. It has even surpassed human capabilities in the areas of facial verification and image classification; hence, it has been highly regarded in the field of video surveillance for the security industry.

In the application of intelligent video in target detection, tracking, and recognition, the rise of deep learning has had a profound influence. When applying those three functions, deep learning potentially touches upon every aspect of the security video surveillance industry: facial detection, vehicle detection, non-motor vehicle detection, facial recognition, vehicle brand recognition, pedestrian detection, human body feature detection, abnormal facial detection, crowd behaviour analysis, multiple target tracking, and so on.

These types of intelligent functions require a series of front-end surveillance cameras, back-end servers and other products which support deep learning algorithms. In small scale applications, front-end cameras can directly operate structured human and vehicle feature extraction, and tens of thousands of human facial images can be stored within the front-end devices to implement direct facial comparison, so as to reduce costs of communicating with a server. In large scale applications, front-end cameras can work with back-end servers. Specifically, the structured video task is handled by front-end devices, reducing the workload for back-end devices; matching and searching efficiency of back-end servers improve as well.

Hikvision new products with Deep Learning

This year, Hikvision will soon introduce a series of products with deep learning technology, such as the DeepInview Series cameras which can accurately detect, recognise, and analyse human, vehicle, and object features and behaviour, and can be widely used in indoor and outdoor scenarios. Another of products worth mentioning is Hikvision’s DeepInmind Series of NVRs which incorporate advanced deep learning algorithms and imitate human thoughts and memory. The DeepInmind products feature an innovative NVR+GPU mode, retaining the advantages of traditional NVRs and additional structured video analysis functions, which together greatly improve the value of video.

Deep learning is the next level of AI development. It is beyond machine learning where supervised classification of features and patterns are set into algorithms. Deep learning incorporates unsupervised or “self-learning” principles. Hikvision is developing this concept in its own analytics algorithms. Enhanced accuracy is the result of multi-layer learning and extensive data collection. Application of this algorithm into face recognition, vehicle recognition, human recognition, and other platforms will significantly advance the performance of analytics.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

In case you missed it

What is the role of higher education to create next-gen security leaders?
What is the role of higher education to create next-gen security leaders?

Traditionally, security industry professionals have often come from backgrounds in law enforcement or the military. However, the industry is changing, and today’s security professionals can benefit from a variety of backgrounds and educational disciplines. The industry’s emphasis on technology solutions suggests a need for more students of computer science, engineering and other technology fields. The closer integration of security with related disciplines within the enterprise suggests a need to prepare through a broad array of educational pursuits. We asked this week’s Expert Panel Roundtable: What is the role of higher education to create the next generation of physical security leaders?

Transport security: utilising the cloud to manage passenger flow and improve health & safety
Transport security: utilising the cloud to manage passenger flow and improve health & safety

Throughout the COVID-19 pandemic, ensuring the safety of passengers and staff aboard public transport has been an ongoing concern. The scenes of underground trains, still packed with commuters as infection rates soared, will have raised alarm bells with bus and train managers, transport officials and government representatives alike. Now, as infection rates hold steady and people slowly return to the workplace, a rise in commuter levels, coupled with a need for strong infection control protocols, is putting a strain on an already overburdened transport system. Managing passenger flow through bus terminals and train stations, while ensuring adherence to social distancing and mask-wearing policies, can be a difficult task. On buses and trains, staff have the unenviable task of challenging any individual who flouts the rules, while attempting to maintain safe operation for the benefit of all passengers. This is where advances in digital surveillance technologies can play an important role in enhancing security, improving operations and supporting the customer facing teams in their day to day roles.  The power of the cloud Keeping businesses afloat and people connected throughout the pandemicCloud or hosted technology has played an important part in keeping businesses afloat and people connected throughout the pandemic. When it comes to physical security such as video surveillance and access control, today’s cloud-enabled systems are far removed from the outdated CCTV and manual access control technologies employed in the past. Cloud connectivity brings with it many benefits, from a security, operational and also business intelligence point of view, thanks to the powerful data that these solutions produce which can be used to inform decision making. The advantages of cloud-based physical security technologies are many, and have wide ranging applications for all areas of the transport sector; across stations, transport hubs and vehicles. When used to support staff and complement existing processes, such systems can prove invaluable for transport professionals in helping to create a safer working environment, promoting confidence among personnel and passengers, and assuring passengers who are fearful about the current pandemic that all possible precautions are being taken during their journey. Managing occupancy across bus and rail Monitoring the movement of staff and passengers is an essential part of being able to maintain a safe operation. Through the utilisation of surveillance cameras at entrances and exit points, as well as at key areas within transport terminals and on the transport mode itself, occupancy thresholds can be determined to ensure passenger numbers do not exceed safe limits. Network surveillance cameras, accessed via mobile device, can enable transport officials to check passenger flow in real-time, while live alerts to warn that health and safety protocols are being breached, enable swift drafting of security or operations personnel to address the situation. Live alerts to warn that health and safety protocols are being breached Through internet of things (IoT) connectivity, additional devices can be easily added to complement the surveillance solution and unlock further benefits. Network audio speakers can be triggered to play pre-recorded messages to alert or inform passengers. Similarly, frictionless access control, enabling customers and staff to move ‘hands-free’ through gateways and ticket checkpoints to avoid viral spread, is made possible by having an access reader which is activated, for example, via QR codes on a mobile phone. And when access readers are integrated with surveillance cameras, this will act as a second layer of authentication to grant or refuse access based on valid staff credentials. Improving security in challenging times Such technologies, interconnected and able to share data, can be used to more effectively report in real time on activity that threatens to have an adverse effect on passengers, staff and the transport environment. Significant parts of the rail network are relatively unmonitored, and inevitably these areas are more vulnerable to vandalism. Similarly, on bus services, abuse of passengers and staff, and acts of criminal behaviour remain a concern. By alerting security staff to a developing situation before it occurs, an incident can be dealt with quickly, minimising disruption to transport services. Cloud based technology can be relied on Cloud based technology can be relied on to not only help improve current services, around passenger occupancy in the current pandemic, but also to help transport officials plan for the security challenges of the future. Simple customisation and easy scalability, plus software upgrades and firmware updates to ensure the system is always up to date and operational, form essential components of a future proof solution which is capable of bringing peace of mind to the transport industry. Additionally, predicted future benefits include the potential for customers to check transport occupancy levels via a mobile app. This would inform them of particularly busy times of passenger transit, allowing more choice over when and where to travel based on real-time data, and ultimately helping to even out passenger numbers to balance journeys and greatly improve efficiency and flow. In a busy world where the demands on our rail and bus networks are now impacted by the COVID-19 pandemic, and indeed the possibility for further related challenges in the future, such cloud-connected technologies represent a worthwhile investment.

In a cybersecurity 'wild west', look for the sheriffs!
In a cybersecurity 'wild west', look for the sheriffs!

As the media often reports, the world of cybersecurity can be seen like the ‘Wild West’. There’s now a wide range of Internet of Things (IoT) devices connected to the web, making this a hot topic. Among these devices are security cameras. IoT devices are computers that use software that makes them vulnerable. As the famous cybersecurity evangelist Mikko Hypponen says, "If a device is smart, it's vulnerable!" Hypponen is right. On a daily basis, new vulnerabilities are found in software, regardless of the manufacturer. In 2019, more than 12,000 vulnerabilities worldwide were made public and reported as a CVE (Common Vulnerability and Exposure) in the National Vulnerability Database (NVD). Unfortunately, vulnerabilities are a given. What really matters is how a company deals with and resolves vulnerabilities. Cybersecurity vulnerabilities Awareness of cybersecurity vulnerabilities is vitally important Awareness of cybersecurity vulnerabilities is vitally important to protect you, your business and the Internet, but it’s also important to understand that a vulnerability is not synonymous with “backdoor”, and is not necessarily indicative of “cheap quality.” But there are companies out there that are embedding safeguards into their development processes to reduce the risks. You could see them as ‘Sheriffs’, taking steps to make this Wild West a little safer.   Why Hikvision chooses ‘Secure-by-Design’ Security cameras, like all other IoT devices, are vulnerable to cyberattacks. Fortunately, manufacturers of IoT devices can significantly reduce these vulnerabilities during the production of devices, using a process called ‘Secure-by-Design’. Implementation of Secure-by-Design requires a commitment on the part of the manufacturer’s management team and a serious investment in resources and technology, which can result in a longer production process and a higher cost of the IoT device. Cost is often the reason why some IoT device manufacturers do not use Secure-by-Design (and are indeed cheaper).  Hikvision is a producer of IoT devices that takes security and privacy very seriously and has implemented Secure-by-Design in its production process. Management supports this process and has even set up a dedicated internal cybersecurity structure charged with product cybersecurity. This group is also the central point of contact for all other cybersecurity matters. The Hikvision Security Development Life Cycle (HSDLC) is an essential part of Hikvision's cybersecurity program. Cybersecurity checks take place at every stage of product development — from concept to delivery. Cybersecurity checks take place at every stage of product development For example, product testing takes place during the verification phase, the company also regularly invites well-known security companies and public testing platforms to conduct penetrating testing. Does this mean that all Hikvision products are immune to hacking? No, that guarantee cannot be given, but the HSDLC is a testament to a manufacturer that makes every effort to produce products that are as cyber secure as possible.  Source code transparcency centre In addition to the Secure-by-Design process, Hikvision opened a Source Code Transparency Center (SCTC) lab in California in 2018, being the industry’s first-of-its-kind lab to open such a centre. At this centre, U.S., Canadian government and law enforcement agencies can view and evaluate the source code of Hikvision IoT devices (IP cameras and network video recorders). It’s important to emphasise that no product is 100 percent secure. Hikvision has a Vulnerability Management Program in place when a vulnerability is discovered in a product. To date, vulnerabilities that have been reported to Hikvision and/or made publicly known, have been patched in the latest Hikvision firmware, and are readily available on the Hikvision website. In addition, Hikvision is a CVE CNA, and has committed to continuing to work with third-party white-hat hackers and security researchers, to find, patch and publicly release updates to products in a timely manner. These vulnerabilities are collected in the National Vulnerability Database (NVD) and are public. Hikvision recommends that customers who are interested in purchasing security cameras inquire about a manufacturer’s cybersecurity practices and if they have an established Vulnerability Management Program.    Cybersecurity questions to consider  The cybersecurity of IoT devices is a topic that needs to be addressed in a serious way and it should play an essential role in the product development process, beginning at the concept phase of an IoT product. This requires time, investment and knowledge.  Consider the following questions: Do I trust the manufacturer of a low-cost security camera? Does this manufacturer have a dedicated cybersecurity organisation? How does this manufacturer handle vulnerabilities?   These are the questions that everyone should ask themselves when making a purchase, be it a camera or any other IoT product.  There is no absolute 100% guarantee of security, but Hikvision has industry-leading practices to ensure the cybersecurity for its cameras. Cooperation, with its customers, installers, distributers and partners, and full transparency are key elements to successfully secure IoT devices. When you read cybersecurity news, we invite you to look beyond the headlines, and really get to know the companies that produce the IoT devices. Before you buy a security camera or any IoT device, check out the manufacturer’s cybersecurity practices, look for a company with a robust vulnerability management program, a company that aligns itself with Secure-by-Design and Privacy-by-Design and a company that employs cybersecurity professionals who are ready and eager to answer your questions. Remember, there are Sheriffs out there, as well as bandits.