Remember the old adage “The whole is greater than the sum of its parts?” Nowhere is that truism more evident than when you add network video to the current generation of Internet of Things (IoT) solutions. Whether we’re talking about industrial IoT applications, “Smart – X” (city, building, parking etc.) or retail operations, integrating network video into the solution provides value far beyond simple situational awareness.

Optimising sophisticated video technology

When video systems first moved from analogue to digital and then became part of the IoT world, they were primarily used to provide visual validation of sensor-detected events. For instance, if an industrial controller sensed an environmental issue such as a temperature exceeding set threshold maximum limits, the sensor would trigger the management software to notify the operator that this event had occurred. The operator could then pull up the video feed of the closest camera and observe the area remotely. While this application is simple, it shows how video enhances sensor management. 

As edge devices, such as sensors and network video become more intelligent, the interactions between systems are growing in sophistication and generating even greater value than each system could provide on its own. 

To appreciate how these smart applications are being used to improve overall efficiencies and profitability, let’s delve into three areas where they’re being deployed: intelligent buildings, smart cities, and smart retailing.  

Lights can automatically turn on or off, brighten or dimmed, to eliminate wasteful energy consumption
By overlaying intelligent operational sensors with intelligent video, it’s now possible to automate lighting levels based on motion detection

Video-based operational analytics

Applying intelligent monitoring to environmental equipment (HVAC) makes it easy for building owners and property managers to determine existing operating costs based on current equipment performance. They can then compare that amount to the cost of upgrades and potential cost savings over time.

Lighting is another significant operating cost within building management. By overlaying intelligent operational sensors with intelligent video (light sensors), it’s now possible to automate lighting levels based on motion detection. Lights can automatically turn on or off, brighten or dimmed, to eliminate wasteful energy consumption. With the addition of occupancy analytics via intelligent video, property managers can determine what caused the motion and learn other operational details such as occupancy counts. Did someone walk through and area causing lighting to turn on or up? Did they dwell in this area? These specifics can help managers efficiently optimise lighting controls and reduce the overall operating cost of the property.

Businesses are also using smart applications to optimise allocation of desk space and conference areas. For instance, intelligent video can determine conference room occupancy (in use, number of people in room, free space even though showing booked) far better than stand-alone motion sensors. When tied to automated room assignment systems, the additional statistics provided by video analytics might suggest room changes based on room size and number of attendees through back-office applications such as Microsoft Outlook.

These examples are just a few of a growing list of available video-based operational analytics currently on the market.

Video analytics in smart cities

Initial forays into smart city technologies such as smart lighting, smart grid, smart parking and so on relied on standalone sensor technologies. Their capabilities were good but limited. Smart Lighting for instance would use basic light detectors to turn street lighting. Smart Parking and traffic systems would use weight sensors to trigger vehicle counts, traffic signal changes or determine if a parking space was in use and paid for. Augmenting these applications with intelligent video and analytics, however, opens up a whole new world of additional details. In Smart Lighting, the video sensor can now trigger a change in lighting based on rules such as vehicular and pedestrian events. Video analytics can yield additional metadata such as vehicle type (commercial versus public use). Smart Parking becomes much more effective when you can begin to provide vehicle detail such as vehicle type or other information based on licence plate recognition. These additional details can help parking lots operate more efficiently and offer value-added services like space reservation and open space location notifications. 

. In Smart Lighting, the video sensor can now trigger a change in lighting based on rules such as vehicular and pedestrian events
Augmenting smart city applications with intelligent video and analytics opens up a whole new world of additional details

Smart Grid offers some less obvious but equally valuable system augmentation capabilities. We often associate Smart Grid with simple automated meter reading but these systems also traverse critical power infrastructure. Solution providers in this arena are now offering heightened asset and perimeter protection via integration of network-based radar detection with video and audio analytics. This strategic mix of technologies can be used to minimise false detection alarms, turn on/off or change lighting levels and point cameras to areas of interest for extremely effective and cost-effective perimeter security.

Network video for retail intelligence

Retailing was one of the earliest adopters of smart device integration with network video and video analytics to support loss prevention and customer safety. They’ve been using video to analyse customer traffic and behaviour in order to improve product placement, increase product sales, as well as cross-sell related items. Adding programmable “Digital Signage” to the mix created new opportunities to display targeted messages based on viewer demographics about additional products and services of potential interest.

Integrating network video with point-of-sale terminals to reconcile cash register receipts, adding heat mapping analytics to study customer foot traffic patterns, measuring check out wait times to increase employee productivity and efficiency as well as improve the customer experience are just some of the ways retailers have applied the principles of IoT to their advantage. Overlay intelligent building controls and you can see the exponential power of integrating intelligent video with other IoT devices and systems.

Overlay intelligent building controls and you can see the exponential power of integrating intelligent video with other IoT devices and system
Retailing was one of the earliest adopters of smart device integration with network video and video analytics to support loss prevention and customer safety

Minimising metadata overload

Smart application integration produces an enormous amount of metadata. Collecting, transporting and synthesising this data into meaningful business intelligence can be daunting. It requires disciplined use of resources from the network infrastructure transporting the data locally to the various cloud technologies (private cloud, hybrid cloud, public cloud) storing and disseminating it securely. 

Generally smart sensor data is fairly light weight in terms of actual data transmitted. Adding video elements can significantly increase bit-rate (bandwidth and storage) requirements. This highlights the need for the video to be more intelligent and interactive with the intelligent sensor and edge device technologies so that resources can be used more efficiently. Smart applications let you do that. You can fine tune video rules and optimise transmission based on retention value. You can program the video to sensor triggers or events, transmitting lower frame rate and resolution video for less interesting video and increasing the video settings when higher quality video is more relevant and valuable based on these sensor triggers.

The back-end collectors of sensor metadata are becoming more mainstream and easier to operate.  In many sectors, service providers are offering management of this sensor output “As a Service.” 

As smart IoT technology continues to mature, the benefits of integration between network video systems and other network solutions will only get better. We’re already seeing greater efficiency in operations as well as higher quantifiable returns on investment through cost savings and more in-depth, usable business intelligence.

Download PDF version

Author profile

Vince Ricco Business Development Manager, Axis Communications

In case you missed it

What is the changing role of training in the security industry?
What is the changing role of training in the security industry?

Even the most advanced and sophisticated security systems are limited in their effectiveness by a factor that is common to all systems – the human factor. How effectively integrators install systems and how productively users interface with their systems both depend largely on how well individual people are trained. We asked this week’s Expert Panel Roundtable: What is the changing role of training in the security and video surveillance market?

What is AI Face Search? Benefits over facial recognition systems
What is AI Face Search? Benefits over facial recognition systems

When a child goes missing in a large, crowded mall, we have a panicking mom asking for help from the staff, at least a dozen cameras in the area, and assuming the child has gone missing for only 15 minutes, about 3 hours’ worth of video to look through to find the child. Typical security staff response would be to monitor the video wall while reviewing the footage and making a verbal announcement throughout the mall so the staff can keep an eye out for her. There is no telling how long it will take, while every second feels like hours under pressure. As more time passes, the possible areas where the child can be will widen, it becomes more time-consuming to search manually, and the likelihood of finding the child decreases. What if we can avoid all of that and directly search for that particular girl in less than 1 second? Artificial neural networks are improving every day and now enable us to search for a person across all selected camera streamsWith Artificial Intelligence, we can. Artificial neural networks are improving every day and now enable us to search for a person across all selected camera streams in a fraction of a second, using only one photo of that person. The photo does not even have to be a full frontal, passport-type mugshot; it can be a selfie image of the person at a party, as long as the face is there, the AI can find her and match her face with the hundreds or thousands of faces in the locations of interest. The search result is obtained in nearly real time as she passes by a certain camera. Distinguishing humans from animals and statues The AI system continuously analyses video streams from the surveillance cameras in its network, distinguishes human faces from non-human objects such as statues and animals, and much like a human brain, stores information about those faces in its memory, a mental image of the facial features so to speak. When we, the system user, upload an image of the person of interest to the AI system, the AI detects the face(s) in that image along with their particular features, search its memory for similar faces, and shows us where and when the person has appeared. We are in control of selecting the time period (up to days) and place (cameras) to search, and we can adjust the similarity level, i.e., how much a face matches the uploaded photo, to expand or fine-tune the search result according to our need. Furthermore, because the camera names and time stamps are available, the system can be linked with maps to track and predict the path of the person of interest. AI Face Search is not Face Recognition for two reasons: it protects people’s privacy, and it is lightweight Protecting people’s privacy with AI Face Search  All features of face recognition can be enabled by the system user, such as to notify staff members when a person of interest is approaching the store AI Face Search is not Face Recognition for two reasons: it protects people’s privacy, and it is lightweight. First, with AI Face Search, no names, ID, personal information, or lists of any type are required to be saved in the system. The uploaded image can be erased from the system after use, there is no face database, and all faces in the camera live view can be blurred out post-processing to guarantee GDPR compliance. Second, the lack of a required face database, a live view with frames drawn around the detected faces and constant face matching in the background also significantly reduces the amount of computing resource to process the video stream, hence the lightweight. Face Search versus Face Recognition AI Face Search Face Recognition Quick search for a particular person in video footage Identify everyone in video footage Match detected face(s) in video stream to target face(s) in an uploaded image Match detected face(s) in video stream to a database Do not store faces and names in a database Must have a database with ID info Automatically protect privacy for GDPR compliance in public places May require additional paperwork to comply with privacy regulations Lightweight solution Complex solution for large-scale deployment Main use: locate persons of interest in a large area Main use: identify a person who passes through a checkpoint Of course, all features of face recognition can be enabled by the system user if necessary, such as to notify staff members when a person of interest is approaching the store, but the flexibility to not have such features and to use the search tool as a simple Google-like device particularly for people and images is the advantage of AI Face Search.Because Face Search is not based on face recognition, no faces and name identifications are stored Advantages of AI Face Search Artificial Intelligence has advanced so far in the past few years that its facial understanding capability is equivalent to that of a human. The AI will recognise the person of interest whether he has glasses, wears a hat, is drinking water, or is at an angle away from the camera. In summary, the advantages of Face Search: High efficiency: a target person can be located within a few seconds, which enables fast response time. High performance: high accuracy in a large database and stable performance, much like Google search for text-based queries. Easy setup and usage: AI appliance with the built-in face search engine can be customised to integrate to any existing NVR/VMS/camera system or as a standalone unit depending on the customer’s needs. The simple-to-use interface requires minimal training and no special programming skills. High-cost saving: the time saving and ease of use translate to orders of magnitude less manual effort than traditionally required, which means money saving. Scalability: AI can scale much faster and at a wider scope than human effort. AI performance simply relies on computing resource, and each Face Search appliance typically comes with the optimal hardware for any system size depending on the customer need, which can go up to thousands of cameras. Privacy: AI Face Search is not face recognition. For face recognition, there are privacy laws that limits the usage. Because Face Search is not based on face recognition, no faces and name identifications are stored, so Face Search can be used in many public environments to identify faces against past and real-time video recordings. AI Face Search match detected face(s) in video stream to target face(s) in an uploaded image Common use cases of AI Face Search In addition to the scenario of missing child in a shopping mall, other common use cases for the AI Face Search technology include: Retail management: Search, detect and locate VIP guests in hotels, shopping centres, resorts, etc. to promptly attend to their needs, track their behaviour pattern, and predict locations that they tend to visit. Crime suspect: Quickly search for and prove/disprove the presence of suspects (thief, robber, terrorist, etc.) in an incident at certain locations and time. School campus protection: With the recent increase in number of mass shootings in school campuses, there is a need to identify, locate and stop a weapon carrier on campus as soon as possible before he can start shooting. Face Search will enable the authorities to locate the suspect and trace his movements within seconds using multiple camera feeds from different areas on campus. Only one clear image of the suspect’s face is sufficient. In the race of technology development in response to business needs and security concerns, AI Face Search is a simple, lightweight solution for airports, shopping centres, schools, resorts, etc. to increase our efficiency, minimise manual effort in searching for people when incidents occur on site, and actively prevent potential incidents from occurring. By Paul Sun, CEO of IronYun, and Mai Truong, Marketing Manager of IronYun

Importance of establishing security standards for K12 school security
Importance of establishing security standards for K12 school security

As we approach National Safe Schools Week (October 21-27), it is appropriate for a conversation to begin regarding establishing standards for K12 school security. Currently no standards exist for assisting schools navigate the complexity of understanding what they need, how much it will cost and how they will secure their learning environments. Security industry experts The Partner Alliance for Safer Schools (PASS) is one of the organisations at the forefront of establishing security standards for schools. In 2014, the Security Industry Association (SIA) and the National Systems Contractors Association (NSCA) formed PASS, which brought together a cross functional group of members including school officials, safe schools’ consultants, law enforcement and security industry experts to collaborate and develop a coordinated approach to protecting K-12 students and staff. School administrators are often contacted repeatedly by organisations with multiple safety and security products PASS has provided valuable insights regarding an ‘All Hazards’ approach to school safety and security. In fact, PASS suggests that school administrators are challenged with two decisions: Determining what they need to do How to prioritise Safe school environment School administrators are experts in running schools and providing education. However, most are not security experts and do not understand the complexity of implementing a comprehensive physical security and safety program across their districts. Still, they are often contacted repeatedly by organisations with multiple safety and security products. School administrators are experts in running schools and providing education, but most are not security experts  Some of these organisations recognise their products are just pieces of a safe school environment puzzle and how they fit in, whereas others focus on specific applications and do not understand how their specific solutions may affect life safety codes and Americans with Disabilities Act law. (Note: Many ‘barricade devices’ fall into this latter category and actually introduce liability concerns with the unintended consequences of their use.)Schools incorporate evacuation drills as part of their emergency preparedness plans and practice on a regular basis Even for experts, the plethora of options and disparate systems required to integrate a safety and security approach at schools is daunting. The ongoing challenge is integrating access control, video, mass notification, and/or visitor management products into a single, effective, and appropriate system the owner can understand, utilise, and afford and that meet local codes and ADA laws. In the absence of standards, schools are likely to amass a collection of devices that do not constitute a comprehensive solution. Lack of consensus In years past, the our industry and commercial buildings adhered to legacy codes – like Building Officials and Code Administrators International Inc. (BOCA), Uniform Building Code (UBC), Southern Building Code Congress International Inc. (SBBCI), and International Conference of Building Officials (ICBO) – which have traditionally been revised every three years, while local jurisdictions decided what versions to adopt and enforce. Currently, however, there is a move toward the International Building Code (IBC), which is published by the International Code Council (ICC) and includes standards and guidance for commercial buildings on doors, windows, and other openings.A risk assessment is the next step toward developing a comprehensive security plan, and begins with developing a trend analysis Still, despite this migration of codes from a patchwork of local decisions to global guidelines, there remains a lack of consensus around school security. The current fragmented approach causes confusion regarding how new schools are designed and how to retrofit existing school buildings, whose average age is 45+ years. Right protection equipment One can point to the fact that there hasn’t been one student lost in a school fire in over 50 years as testament to standards like NFPA 80 and NFPA 101 being referenced in model building codes. Additionally, schools incorporate evacuation drills as part of their emergency preparedness plans and practice on a regular basis. It’s not just having the right protection equipment in the building, it’s also having a procedural layer in place to make sure everyone knows their roles and responsibilities in the event of fire. The stress of the actual event can limit ones’ ability to think clearly. Practice makes perfect. Why would we approach school security any differently? School security is a team effort, and it is important to understand all the areas security impacts and involves School security is a team effort. It is important to understand all the areas security impacts and involves. PASS suggests starting with a basic team consisting of: Security Director Local Law Enforcement School Administrator Integrator Door and Hardware Consultant IT Director Comprehensive security plan Quantifying and mitigating risk are the jobs of security professionals and school administrators A risk assessment is the next step toward developing a comprehensive security plan. This often begins with conducting a trend analysis requiring the collection of data from a variety of public and private sources. The challenge is to pull these pieces into a usable and easily understood format that provides a guide for current and future risk concerns. Risk assessment and mitigation can never eliminate risk. Quantifying and mitigating risk are the jobs of security professionals and school administrators. Data from the following sources can help measure risk: Campus: Review incident report trends for at least the past 36 months. Area and city: Review crime data from local law enforcement for the surrounding neighborhood and city. Screening procedures: How is hiring conducted? Anonymous tip reporting systems: Enabling students, staff members, parents and the community to anonymously alert administrators to perceived and actual threats. Social media monitoring: such monitoring can provide important information that can be used to identify risks. Monitoring social media could help measure risk for school safety Delay adversarial behaviors These assessments can then be incorporated into the best practice approach of Layered Security. Layered security combines best practice components within each layer that effectively deter, detect and delay adversarial behaviors. Layered security works from the outside in. As one layer is bypassed, another layer provides an additional level of protection. The asset being protected is at the center of the layers – students, staff and authorised visitors. PASS defines five layers of Security:As one layer is bypassed, another layer provides an additional level of protection District Wide Property Perimeter Parking Lot Perimeter Building Perimeter Classroom/Interior Perimeter Appropriate Tier target Each layer can be broken down into Tier levels with Tier 1 being basic and Tier 4 being the highest level of security (Figure 1) . It is important to understand that the demographics of individual school buildings varies, even within the same district. Security experts will quickly point out that ‘if you’ve seen one school, you’ve seen one school’. The assessments will determine the appropriate Tier target. Figure 1 Each layer includes essential protective elements, or components, of security. Every layer does not necessarily include all seven of these common components, and a layer may include additional components unique to that particular layer. Safety and security components Policies & Procedures People (roles & training) Architectural Communication Access Control Video Surveillance Detection and Alarms While components are not listed in a priority order, three components included in all layers are policies and procedures, the roles and training of people, and communication. These components often perform a function in every layer and every tier in each layer. Three tools come together in the PASS approach as outlined in the new 4th Edition of the PASS Guidelines (Figure 2) - the Layers are established and defined, a Checklist/Assessment breaks down each layer into tiered best practices which then tie into the guidelines where a narrative explains each best practice in more detail. Figure 2  Schools need not reinvent the wheel when it comes to school security planning. Following the best practices of Risk Assessments and Layered Security will ensure that every school building in a district will have a unique and comprehensive plan that is tailored to their individual needs.