The deep learning phenomenon continues to excite the IT world, with computing power now at the level where it can be properly used in practical applications.

Hikvision has been at the forefront of applying the technology in the surveillance industry and beyond. It has already released its first set of products that harness the power of artificial intelligence (AI).

The concept of deep learning takes inspiration from the way the human brain works. Our brain is a very complex deep learning model. Brain neural networks are comprised of billions of interconnected neurons; deep learning simulates this structure. These multi-layer networks can collect information and perform corresponding actions according to analysis of that information.The ability of deep learning to enhance the recognition of human beings makes it a great addition to the security arsenal

Reducing false alarms

In the past two years, the technology has excelled in speech recognition, computer vision, voice translation, and much more. It has even surpassed human capabilities in the areas of facial verification and image classification; hence, it has been highly regarded in the field of video surveillance for the security industry.

Its ability to enhance the recognition of human beings - distinguishing them from animals, for example - makes the technology a great addition to the security arsenal. This is especially relevant in a world where false alarms account for 94%-99% of all alarms, according to police and fire service statistics.

Deep learning for data processing

Deep learning is intrinsically different from other algorithms. The way it solves the insufficiencies of traditional algorithms is encompassed in the following aspects.

The algorithmic model for deep learning has a much deeper structure than the traditional algorithms. Sometimes, the number of layers can reach over a hundred, enabling it to process large amounts of data in complex classifications.

Deep learning is very similar to the human learning process and has a layer-by-layer feature-abstraction process. Each layer will have different 'weighting', and this weighting reflects on what was learnt about the images’ 'components'.The more features there are, the more accurate the recognition and classification will be

Feature classification and recognition

The higher the layer level, the more specific the components. Just like the human brain, an original signal in deep learning passes through layers of processing; next, it takes a partial understanding (shallow) to an overall abstraction (deep) where it can perceive the object.

Deep learning does not require manual intervention but relies on a computer to extract features by itself. This way, it is able to extract as many features from the target as possible, including abstract features that are difficult or impossible to describe.

The more features there are, the more accurate the recognition and classification will be. Some of the most direct benefits that deep learning algorithms can bring include achieving comparable or even better-than-human pattern recognition accuracy, strong anti-interference capabilities, and the ability to classify and recognise thousands of features.

Conventional surveillance challenges

Conventional surveillance systems mostly detect moving targets without further analysis. Even smart IP cameras can only map individual points on a shape one by one, making it difficult to calibrate some features (e.g. forehead or cheek), thus decreasing accuracy. 

For perimeter security, for example, other technologies can be (and are) used to provide more comprehensive security. But they all have their downsides. Infrared emission detectors can be ‘jumped over’ but are also prone to false alarms caused by animals. Electronic fences can be a safety hazard and are limited in certain areas. Some of these solutions can also be expensive and complicated to install.Frequent false alarms are always an issue for end-users, who need to spend time to investigate each one

Identifying false alarms

Objects such as animals, leaves, or even light can cause false alarms, so being able to identify the presence of a human shape really improves the accuracy of perimeter VCA functions.

Frequent false alarms are always an issue for end-users, who need to spend time to investigate each one, potentially delaying any necessary response and generally affecting efficiency. 

Imagine, for example, a scenario where it’s relatively quiet – a location at night where there are few cars and people around. Even here, there could be 50 false alarms in a night. Assuming it takes 2-3 minutes to check out a false alarm, and that just 3 out of the 50 warrants more attention – say 15 minutes each.

A guard either needs to check the system and look back at the alert, or someone needs to be dispatched to the location and look around, checking if anyone has indeed ‘entered without permission’. In most organisations, these would need to be reported/recorded too, adding to the overall time spent on this ‘false alarm’.

Increasing recognition accuracy

Deep learning, however, makes a big difference. With a large amount of good quality data from the cameras and other sources, like the Hikvision Research Institute, and over a hundred data cleaning team members to label the video images, sample data with millions of categories within the industry have been accumulated. With this large amount of quality training data, human, vehicle, and object pattern recognition models become more and more accurate for video surveillance use.

Based on a series of experiments, the recognition accuracy of solutions using the deep learning algorithm increased accuracy by 38% - applying this to the previous example, that’s a saving of nearly one hour each night. This makes deep learning technology a great advantage in a perimeter security solution, with much more accurate line crossing, intrusion, entrance and exit detection.The value of deep learning technology stretches further than traditional security

Accurate motion detection

The value of deep learning technology stretches further than traditional security. For example, tracking movement patterns of individuals can see if they are ‘loitering’ and a potential threat in the future.

A threshold could be set to five metres radius of movement, or ten seconds of staying in the same place. If the person passes either threshold, an alarm could be triggered. The solution tracks the individual and compares this behaviour to a database to see if it recognises a pattern.

Another application would be in a scenario where ‘falling down’ could be a threat, like an elderly care home. If a height threshold was set at 0.5m and duration time 10 seconds, for example, the solution would be able to see a person falling down (as they go below 0.5m) and might be in trouble (if they ‘stay down’ for longer than 10 seconds).

AI-powered deep learning

The solution uses the parameters set to compare with its database and raise an alarm. With features and benefits like these, it’s easy to see how many smart applications could be catered for by deep learning technology.Artificial intelligence has massive potential

To sum up 10,000-strong R&D Centre is pushing the boundaries of surveillance solutions and bringing even more benefits to them. Artificial intelligence has massive potential, and Hikvision is always exploring new ways to apply this exciting technology throughout the security industry and beyond.

Intelligent search

Hikvision deep learning solutions are available at three levels:

  • DeepinView camera can conduct target tracking, grading and capturing when an alarm is triggered.
  • ‘Traditional’ IP camera using a DeepinMind NVR will add the function of searching intelligently by picture, saving time on searching for targets compared with a regular NVR.
  • DeepinView camera and a DeepinMind NVR delivers a full power solution, with the camera sending the information to the NVR, which can then analyse it. This accelerates recording and false alarm filtering.
Download PDF version

In case you missed it

School security benefits from advanced communication technology
School security benefits from advanced communication technology

With the recent tragic events in Florida, it’s evident that schools require more tools to help ensure their students’ safety. With that, school and municipal officials all over the country are looking for more advanced ways to combat gun violence. While there is no perfect solution for the myriad of threats and emergencies with which our schools are confronted, many have looked to technology to help improve communications before, during and after incidents. For schools across the state of Arkansas; Nassau County, New York; Snohomish County, Washington; New Castel County, Delaware; Limestone County, Alabama and scores of others, the answer was the implementation of technologies that connect school personnel directly with local police, fire and EMS, and designated individuals at the school. Communication tools have proven invaluable when a potential active shooter situation was being discussed on social media Key to these schools’ choice in technology was the recognition that while the most traumatic of threats is the active assailant, any technology investment should be just as effective in handling the more frequent day-to-day incidents. Communication technologies for incident management How have technologies such as mobile panic buttons and anonymous texting helped impact school safety? Here are a few examples: In Limestone County, Alabama, 9-1-1 Director Brandon Wallace led an effort to implement technology tools across the county to help prevent and more quickly notify personnel of possible emergency situations. Communication tools have proven invaluable especially when a potential active shooter situation was being discussed on social media. Advanced technology integration The technology not only connected directly to emergency personnel, but also ensured that school faculty were able to communicate with one another during a potential emergency and account for students. Following the tragic shooting in Newtown, Connecticut, the school superintendent Dr. Joseph V. Erardi, Jr. chose to make communication technology a part of their new safety plan. The integration of advanced technology has given staff and teachers a greater sense of safety with not only active shooter events but also events like medical emergencies that require fast action and a quick response from public safety officials. Trainings ensure that staff and students are prepared for any type of situation and be on the same page in an emergency situation Implementing enhanced safety measures What are some lessons learned from these schools that can be applied to protect students in other areas? Here are some tips for implementing more safety measures into schools: Make sure your solution has a daily use. Unfortunately, many great technologies have sat unused when emergencies struck simply because those involved weren’t familiar with them or were under extreme duress and forgot about them. Training is an obvious necessity, but finding solutions with daily use cases (such as value in medical emergencies) can have a huge impact not only on ROI but also during an incident. Evaluate past incidents. Response during past incidents can always help future plans. Whether incidents have been handled well or have room for improvement, it’s important to continue to develop incident responses. For Limestone County, Alabama, the use of technology in their response plan was first tested during a medical emergency which helped to confirm the continued use within the school. Knowing the ease of use and responsiveness of emergency response tools, the county decided to build upon the technology already in place to help thwart future incidents. Train staff on the newest measures. Snohomish County, Washington holds trainings with teachers and staff, alongside local emergency personnel to prepare for active shooter incidents. Trainings ensure that staff and students can be prepared for any type of situation and be on the same page in the event of an emergency. Especially as new technology is introduced, integrating the tools with staff first will help ensure greater adoption throughout the process. Integrate practice drills. Fire drills are a common part of the school year; why not implement practicing other scenarios which may affect your school? Not only will this help with preparedness but will also highlight any measures that might need to be adjusted. New tools can then be tested to ensure that all staff and students are comfortable in the event they will need to utilise it in the future. Expand those involved in your drills to be those who will actually be involved in an incident. All too often, drills are siloed and don’t include outside agencies. Re-evaluating safety procedures Schools across the country can learn a lot from districts that have implemented and actually used new communications technologies addressing school safety, which are leading the way in how teachers and faculty are preparing to keep students safe. However, it will remain important to re-evaluate safety procedures and integrate technology to help ensure that these steps remain effective. As the tools continue to advance, the available safety measures will only continue to grow.

Artificial intelligence is changing intrusion detection dynamics in the security industry
Artificial intelligence is changing intrusion detection dynamics in the security industry

With the ever-growing availability of video data thanks to the low cost of high-resolution video cameras and storage, artificial intelligence (AI) and deep learning analytics now have become a necessity for the physical security industry, including access control and intrusion detection. Minimising human error and false positives are the key motivations for applying AI technologies in the security industry. What is artificial intelligence? Artificial intelligence is the ability of machines to learn from experience using a multi-layer neural network, which mimics the human brain, in order to recognise items and patterns and make decisions without human interference. The human brain is estimated to have 86 billion neurons; in comparison, the newest Nvidia GPU Volta has 21 billion transistors (the equivalence of a neuron), which offers the performance of hundreds of CPUs for deep learning.AI can learn continuously 24 hours per day every day, constantly acquiring, retaining and improving its knowledge In addition, unlike humans, AI can learn continuously 24 hours per day every day, constantly acquiring, retaining and improving its knowledge. With such enormous processing power, machines using Nvidia GPU and similar chips can now distinguish faces, animals, vehicles, languages, parts of speech, etc. Depending on the required complexity, level of details, acceptable error margin, and learning data quality, AI can learn new objects within as fast as a few seconds using Spiking Neural Network (SNN) to a few weeks using Convolution Neural Network (CNN). While both SNN and CNN offer advantages and drawbacks, they outperform tradition security systems without AI in terms of efficiency and accuracy. According to the research reports of MarketsandMarkets, the market size of perimeter intrusion detection systems is projected to increase from 4.12 billion USD in 2016 to 5.82 billion USD in 2021 at a Compound Annual Growth Rate (CAGR) of 7.1%. Meanwhile, the predicted market of AI in security (both cyber security and physical security) will grow from 3.92 billion USD in 2017 to 34.81 billion USD by 2025, i.e., with an impressive CAGR of 31.38%. Legacy perimeter intrusion detection systems Legacy perimeter intrusion detection systems (PIDSs) are typically set up with the following considerations: Geographical conditions: landscape, flora, fauna, climate (sunrise, sunset, weather conditions, etc.), whether there are undulations in the terrain that would block the field of view of cameras Presence or lack of other layers of physical protection or barriers Integration with other systems in the security network: camera, storage, other defensive lines (door, lock, alarm, etc.) Types of alarm triggers and responses System complexity: intrusion detection with various types of sensors, e.g., microwave sensors, radar sensors, vibration sensors, acoustic sensors, etc. Length of deployment Local regulations: privacy protection, whether the cameras/sensors must be visible/hidden/buried, etc., electromagnetic interferences that may affect other systems such as oil rigs or power plants Human involvement: on-site personnel arrangement, human monitoring, human action in response to alarms AI object detection can easily distinguish different types of people and objects Pain points and benefits of AI The conditions listed above correspond to certain requirements of an intrusion detection systems: minimal false alarm, easy setup and maintenance, easy integration, and stable performance.AI by nature is designed to learn, adapt itself and evolve to work in multiple conditions: it should be integrated with existing video recording systems  Minimal false alarms: False alarms lead to increased cost and inefficiency but are the main problem of PIDSs without AI technology, where animals, trees, shadows, and weather conditions may trigger the sensors. AI object detection can easily distinguish different types of people and objects, e.g., in a region set up to detect people, a car driving by, a cat walking by, or a person’s shadow will not trigger the alarm. Therefore, the amount of false alarms can be reduced by 70% to orders of magnitude. Easy setup and maintenance: Legacy PIDSs without AI must account for terrain, line of sight of cameras, sensor locations; any changes to the system would require manual effort to recalculate such factors and may disturb other components in the system. In contrast, AI PIDSs enable the system administrator to access the entire system or individual cameras from the control room, configure the region and object of interest in the field of view of cameras within minutes, and adjust with ease as often as necessary. Computing knowledge and even specific security training are not required to set up a secured PIDS with AI because AI PIDS is designed to relieve humans from knowing the inner working of machines. Easy integration with complementary technologies: Legacy PIDS without AI relies on physical technology, which are often proprietary and require complete overhaul of systems to function smoothly. On the other hand, AI by nature is designed to learn, adapt itself and evolve to work in multiple conditions, so AI PIDS is easily integrated with existing video recording (camera) and storage (NVR) systems. AI also eliminates the need for physical wireless or fiber-based sensors; instead, it functions based on the videos captured by cameras. Furthermore, AI enables easy and instantaneous combinations of multiple layers of defense, e.g., automatic triggering of door lock, camera movement focusing and access control as soon as a specified object is detected in the region of interest, all set up with a click of a button.  Stable performance and durability: Legacy PIDSs without AI requires complicated setup with multiple components in order to increase detection accuracy. More components mean a higher probability of malfunction in the system, including exposure to damages (e.g., sensors can be destroyed) and delay in detection, while human monitoring is inconsistent due to human fatigue (studies have shown that a person can concentrate in mundane tasks for only up to 20 minutes, and the attention span decreases even more rapidly when humans are faced with multiple items at once, e.g., multiple camera monitoring screens). AI significantly reduces, if not completely eliminates the need for human involvement in the intrusion detection system once it is set up. In addition, AI reduces the risk of system malfunction by simplifying the hardware sensors needed. Minimising human error and false positives are the key motivations for applying AI technologies in the security industry Additional benefits of AI in intrusion detection Artificial Intelligence is undeniably reshaping every business and weaving into every aspect of daily lifeMaximal detection capability: The most advanced AI intrusion detection system today provides an all-in-one solution to distinguish any combination of alarm-triggering criteria beyond perimeter protection. Using AI, the system administrator can configure as many zones with different settings and object of interests as necessary, which include detections for specific colors or attributes (e.g., person not wearing the required uniform or carrying food/drink), numbers and dwell time (e.g., group of more than 5 people loitering), or movements (e.g., cars moving faster than the speed limit). In addition, AI can accurately pinpoint the location of event occurrence by displaying the camera that records the event in near real time, i.e., with few-second delays. Lower security operation cost: By minimising the number of false positives and human involvement while maximising ease of use and stability, AI intrusion detection systems significantly decrease the total cost of ownership. Companies can reduce the large security personnel overhead and cost of complicated and expensive legacy PIDSs systems. McKinsey Global report in June 2017 shows that proactive AI adopters can realize up to 15% increase in profit margin across various industries. Artificial Intelligence is undeniably reshaping every business and weaving into every aspect of daily life. In security, legacy systems are giving way to AI-based systems, and the first enterprises to adopt AI-based systems will soon, if not immediately, benefit from such investment. By Paul Sun, CEO of IronYun, and Mai Truong, Marketing Manager of IronYun

How physical security consultants ensure cybersecurity for end users
How physical security consultants ensure cybersecurity for end users

Cybersecurity talk currently dominates many events in the physical security industry. And it’s about time, given that we are all playing catch-up in a scary cybersecurity environment where threats are constant and constantly evolving. I heard an interesting discussion about cybersecurity recently among consultants attending MercTech4, a conference in Miami hosted by Mercury Security and its OEM partners. The broad-ranging discussion touched on multiple aspects of cybersecurity, including the various roles of end user IT departments, consultants, and integrators. Factors such as training, standardisation and pricing were also addressed as they relate to cybersecurity. Following are some edited excerpts from that discussion.  The role of the IT department Pierre Bourgeix of ESI Convergent: Most enterprises usually have the information technology (IT) department at the table [for physical security discussions], and cybersecurity is a component of IT. The main concern for them is how any security product will impact the network environment. The first thing they will say, is “we have to ensure that there is network segmentation to prevent any potential viruses or threats or breaches from coming in.” The main concern for IT departments is how any security product will impact the network environment”They want to make sure that any devices in the environment are secure. Segmentation is good, but it isn’t an end-all. There is no buffer that can be created; these air gaps don’t exist. Cyber is involved in a defensive matter, in terms of what they have to do to protect that environment. IT is more worried about the infrastructure. The role of consultants and specifiers Phil Santore of DVS, division of Ross & Baruzzini: As consultants and engineers, we work with some major banks. They tell us if you bring a new product to the table, it will take two to three months before they will onboard the product, because they will run it through [cybersecurity testing] in their own IT departments. If it’s a large bank, they have an IT team, and there will never be anything we [as consultants] can tell them that they don’t already know. But we all have clients that are not large; they’re museums, or small corporations, or mom-and-pop shops. They may not be as vulnerable from the international threat, but there are still local things they have to be concerned about. It falls on us as consultants to let them know what their problems are. Their IT departments may not be that savvy. We need to at least make them aware and start there. Wael Lahoud of Goldmark Security Consulting: We are seeing more and more organisations having cybersecurity programs in place, at different maturity levels. At the procurement stage, we as consultants must select and specify products that have technology to enable cybersecurity, and not choose products that are outdated or incompatible with cybersecurity controls. We also see, from an access control perspective, a need to address weaknesses in databases. Specifying and having integrators that can harden the databases, not just the network itself, can help. The impact of physical security products on the network environment was a dominant topic at the MercTech4 consultants roundtable discussion The need for standards on cybersecurity Jim Elder of Secured Design: I’d like to know what standards we as specifiers can invoke that will help us ensure that the integrator of record has the credentials, knows what standards apply, and knows how to make sure those standards are maintained in the system. I’m a generalist, and cybersecurity scares the hell out of me.We’re not just talking about access to cameras, we are talking about access to the corporate network and all the bad things that can happen with that. My emphasis would be on standards and compliance with standards in the equipment and technology that is used, and the way it is put in. It can be easier for me, looking at some key points, to be able to determine if the system has been installed in accordance. We are seeing more and more organisations having cybersecurity programs in place, at different maturity levels"I’m taking the position of the enforcement officer, rather than the dictator. It would be much better if there were focused standards that I could put into the specification— I know there are some – that would dictate the processes, not just of manufacturing, but of installation of the product, and the tests you should run accordingly. Pierre Bourgeix: With the Security Industry Association (SIA), we are working right now on a standard that includes analysed scoring on the IT and physical side to identify a technology score, a compliance score, a methodology, and best-of-breed recommendation. Vendor validation would be used to ensure they follow the same process. We have created the model, and we will see what we can do to make it work. Terry Robinette of Sextant: If a standard can be written and it’s a reasonable process, I like the idea of the equipment meeting some standardised format or be able to show that it can withstand the same type of cyber-attack a network switch can withstand. We may not be reinventing the wheel. IT is the most standardised industry you will ever see, and security is the least standardised. But they’re merging. And that will drive standardisation. Jim Elder: I look to Underwriters Laboratory (UL) for a lot of standards. Does the product get that label? I am interested in being able to look at a box on the wall and say, “That meets the standard.” Or some kind of list with check-boxes; if all the boxes are checked I can walk out and know I have good cybersecurity threat management. IT is the most standardised industry you will ever see, and security is the least standardised" The role of training Phil Santore: Before you do any cybersecurity training, you would need to set the level of cybersecurity you are trying to achieve. There are multiple levels from zero to a completely closed network. Wael Lahoud: From an integrator’s perspective, cybersecurity training by the manufacturer of product features would be the place to start – understanding how to partner the database, and the encryption features. We see integrators that know these features are available – they tick the boxes – but they don’t understand what they mean. Cybersecurity is a complex topic, and the risk aspects and maturity levels vary by organisation. That would be a good starting point. The role of integrators Wael Lahoud: Integrators like convenience; less time means more money. So, we see some integrators cut corners. I think it is our role (as consultants) to make sure corners are not cut. If you rely solely on integrators, it will always be the weak password, the bypass. We have seen it from small projects to large government installations. It’s the same again and again. Even having an internal standard within an organisation, there may be no one overseeing that and double-checking. Tools will help, but we are not there at this point. I will leave it up to manufacturers to provide the tools to make it easy for consultants to check, and easier for integrators to use the controls. Cybersecurity is a complex topic, and the risk aspects and maturity levels vary by organisation - so training is very important The impact of pricing Pierre Bourgeix: The race to the cheapest price is a big problem. We have well-intended designs and assessments that define best-of-breed and evaluate what would be necessary to do what the client needs. But once we get to the final point of that being implemented, the customer typically goes to the lowest price – the lowest bidder. That’s the biggest issue. You get what you pay for at the end of the day. With standards, we are trying to get to the point that people realise that not all products are made the same, not all integrators do the same work. We hope that through education of the end user, they can realise that if they change the design, they have to accept the liability.It’s not just the product that’s the weakest link, it’s the whole process from design to securing that product and launching it" The big picture Wael Lahoud: The Windows platform has a lot of vulnerabilities, but we’re still using it, even in banks. So, it’s not just the product that’s the weakest link, it’s the whole process from design to securing that product and launching it. That’s where the cybersecurity program comes into play. There are many vulnerable products in the market, and it’s up to professionals to properly secure these products and to design systems and reduce the risk. Pierre Bourgeix: The access port to get to data is what hackers are looking for. The weakest link is where they go. They want to penetrate through access control to get to databases. The golden ring is the data source, so they can get credentialing, so they can gain access to your active directory, which then gives them permissions to get into your “admin.” Once we get into “admin,” we get to the source of the information. It has nothing to do with gaining access to a door, it has everything to do with data. And that’s happening all the time.