Remember the old adage “The whole is greater than the sum of its parts?” Nowhere is that truism more evident than when you add network video to the current generation of Internet of Things (IoT) solutions. Whether we’re talking about industrial IoT applications, “Smart – X” (city, building, parking etc.) or retail operations, integrating network video into the solution provides value far beyond simple situational awareness.

Optimising sophisticated video technology

When video systems first moved from analogue to digital and then became part of the IoT world, they were primarily used to provide visual validation of sensor-detected events. For instance, if an industrial controller sensed an environmental issue such as a temperature exceeding set threshold maximum limits, the sensor would trigger the management software to notify the operator that this event had occurred. The operator could then pull up the video feed of the closest camera and observe the area remotely. While this application is simple, it shows how video enhances sensor management. 

As edge devices, such as sensors and network video become more intelligent, the interactions between systems are growing in sophistication and generating even greater value than each system could provide on its own. 

To appreciate how these smart applications are being used to improve overall efficiencies and profitability, let’s delve into three areas where they’re being deployed: intelligent buildings, smart cities, and smart retailing.  

Lights can automatically turn on or off, brighten or dimmed, to eliminate wasteful energy consumption
By overlaying intelligent operational sensors with intelligent video, it’s now possible to automate lighting levels based on motion detection

Video-based operational analytics

Applying intelligent monitoring to environmental equipment (HVAC) makes it easy for building owners and property managers to determine existing operating costs based on current equipment performance. They can then compare that amount to the cost of upgrades and potential cost savings over time.

Lighting is another significant operating cost within building management. By overlaying intelligent operational sensors with intelligent video (light sensors), it’s now possible to automate lighting levels based on motion detection. Lights can automatically turn on or off, brighten or dimmed, to eliminate wasteful energy consumption. With the addition of occupancy analytics via intelligent video, property managers can determine what caused the motion and learn other operational details such as occupancy counts. Did someone walk through and area causing lighting to turn on or up? Did they dwell in this area? These specifics can help managers efficiently optimise lighting controls and reduce the overall operating cost of the property.

Businesses are also using smart applications to optimise allocation of desk space and conference areas. For instance, intelligent video can determine conference room occupancy (in use, number of people in room, free space even though showing booked) far better than stand-alone motion sensors. When tied to automated room assignment systems, the additional statistics provided by video analytics might suggest room changes based on room size and number of attendees through back-office applications such as Microsoft Outlook.

These examples are just a few of a growing list of available video-based operational analytics currently on the market.

Video analytics in smart cities

Initial forays into smart city technologies such as smart lighting, smart grid, smart parking and so on relied on standalone sensor technologies. Their capabilities were good but limited. Smart Lighting for instance would use basic light detectors to turn street lighting. Smart Parking and traffic systems would use weight sensors to trigger vehicle counts, traffic signal changes or determine if a parking space was in use and paid for. Augmenting these applications with intelligent video and analytics, however, opens up a whole new world of additional details. In Smart Lighting, the video sensor can now trigger a change in lighting based on rules such as vehicular and pedestrian events. Video analytics can yield additional metadata such as vehicle type (commercial versus public use). Smart Parking becomes much more effective when you can begin to provide vehicle detail such as vehicle type or other information based on licence plate recognition. These additional details can help parking lots operate more efficiently and offer value-added services like space reservation and open space location notifications. 

. In Smart Lighting, the video sensor can now trigger a change in lighting based on rules such as vehicular and pedestrian events
Augmenting smart city applications with intelligent video and analytics opens up a whole new world of additional details

Smart Grid offers some less obvious but equally valuable system augmentation capabilities. We often associate Smart Grid with simple automated meter reading but these systems also traverse critical power infrastructure. Solution providers in this arena are now offering heightened asset and perimeter protection via integration of network-based radar detection with video and audio analytics. This strategic mix of technologies can be used to minimise false detection alarms, turn on/off or change lighting levels and point cameras to areas of interest for extremely effective and cost-effective perimeter security.

Network video for retail intelligence

Retailing was one of the earliest adopters of smart device integration with network video and video analytics to support loss prevention and customer safety. They’ve been using video to analyse customer traffic and behaviour in order to improve product placement, increase product sales, as well as cross-sell related items. Adding programmable “Digital Signage” to the mix created new opportunities to display targeted messages based on viewer demographics about additional products and services of potential interest.

Integrating network video with point-of-sale terminals to reconcile cash register receipts, adding heat mapping analytics to study customer foot traffic patterns, measuring check out wait times to increase employee productivity and efficiency as well as improve the customer experience are just some of the ways retailers have applied the principles of IoT to their advantage. Overlay intelligent building controls and you can see the exponential power of integrating intelligent video with other IoT devices and systems.

Overlay intelligent building controls and you can see the exponential power of integrating intelligent video with other IoT devices and system
Retailing was one of the earliest adopters of smart device integration with network video and video analytics to support loss prevention and customer safety

Minimising metadata overload

Smart application integration produces an enormous amount of metadata. Collecting, transporting and synthesising this data into meaningful business intelligence can be daunting. It requires disciplined use of resources from the network infrastructure transporting the data locally to the various cloud technologies (private cloud, hybrid cloud, public cloud) storing and disseminating it securely. 

Generally smart sensor data is fairly light weight in terms of actual data transmitted. Adding video elements can significantly increase bit-rate (bandwidth and storage) requirements. This highlights the need for the video to be more intelligent and interactive with the intelligent sensor and edge device technologies so that resources can be used more efficiently. Smart applications let you do that. You can fine tune video rules and optimise transmission based on retention value. You can program the video to sensor triggers or events, transmitting lower frame rate and resolution video for less interesting video and increasing the video settings when higher quality video is more relevant and valuable based on these sensor triggers.

The back-end collectors of sensor metadata are becoming more mainstream and easier to operate.  In many sectors, service providers are offering management of this sensor output “As a Service.” 

As smart IoT technology continues to mature, the benefits of integration between network video systems and other network solutions will only get better. We’re already seeing greater efficiency in operations as well as higher quantifiable returns on investment through cost savings and more in-depth, usable business intelligence.

Download PDF version

Author profile

Vince Ricco Business Development Manager, Axis Communications

In case you missed it

2018 FIFA World Cup Russia integrates safety, security and service
2018 FIFA World Cup Russia integrates safety, security and service

The 2018 FIFA World Cup tournament is bringing 32 national teams and more than 400,000 foreign football fans from all over the world to 12 venues in 11 cities in Russia. Fans are crowding into cities including Moscow, St. Petersburg and Kazan. Given continuing global concerns about terrorism, security is top-of-mind. Protection of the World Cup games in Russia is focusing on an “integrated safety, security and service approach,” according to officials. Combining the term “security” with the terms “safety” and “service” is not an accident. An aggressive security stance is necessary, but at the end of the day, fan safety is paramount, and a service-oriented approach ensures a positive fan experience. Medical responders will be working side-by-side with police and antiterrorism personnel. Risk management best practices We asked Sean T. Horner and Ben Joelson, directors of the Chertoff Group, a global advisory firm focused on best practices in security and risk management, to comment on security at FIFA World Cup 2018. Although not involved in securing the 2018 World Cup, the Chertoff Group is experienced at securing large events and enterprises using risk management, business practices and security. Integration is another important aspect of protecting the games, says Horner. The use of multiple resources, including Russian military, intelligence and law enforcement, will be closely integrated to provide the best security for the large-scale event in each of the host cities, he says. The approach will be centralised and flexible, with resource deployment guided by effective situational awareness. Primary security and emergency operations centres will be dispersed throughout each host city “There is a unified command structure at the Russian Federation level, and they will keep resources in reserve and shift them as needed to various events and venues based on any specific intelligence, in effect deploying resources where threats are greatest,” says Joelson. “There will also be some regional commands, and resources will incorporate a spectrum of police and military personnel ranging from the ‘cop on the beat’ to the Spetsnaz, the Russian ‘special forces'.” Primary security and emergency operations centres will be dispersed throughout each host city, and additional forces can be shifted as necessary, he notes. Role of law enforcement In Russia, the lines of separation between law enforcement and the military are not as stark as in the United States, for example, where military forces are restricted from deployment for domestic law enforcement by the Posse Comitatus Act. In Russia, there is no such restriction.  A broad range of technology will play a role at the World Cup, Horner and Joelson agree. Technology will be used primarily as a force multiplier and a decision-support tool for security personnel. There are robust CCTV systems in many Russian cities, and mobile CCTV systems, such as camera towers or mobile security centres on wheels, will also be deployed. Technologies will include infrared cameras, flood lights, and ferromagnetic screening systems to scan hundreds of individuals as they walk by. In some locations, facial recognition systems will be used, tied into various intelligence, military and law enforcement databases of known bad actors. Behaviour analytics will be used as a decision-support tool. In addition to security in public areas, private CCTV systems in hotels, at transportation hubs, and inside the venues themselves will be leveraged. Video analytics and detection will help personnel review live view of people who may be acting suspiciously or who leave a bag unattended. In some locations, facial recognition systems will be used, tied into various intelligence, military and law enforcement databases of known bad actors Rigorous anti-terrorism measures A Fan ID card is required to enter the 2018 World Cup Tournament, even for Russian residents. The Russians have an aggressive stance against domestic terrorism, which will also help ensure the safety of the World Cup games, say Horner and Joelson. Terrorist group ISIS has promised “unprecedented violence” at the games, but they make similar threats at every major global event. Russia has been an active force disrupting ISIS in Syria, and experts suggest that losing ground geographically could lead to addition “asymmetric” terrorist attacks. However, Russia is leveraging all their intelligence resources to identify any plots and deploying their security apparatus to disrupt any planned attacks, experts say. Russia’s rigorous anti-terrorism measures include a total ban on planes and other flying devices (such as drones) around the stadiums hosting the World Cup. Private security In addition to military, intelligence and law enforcement personnel, private security will play a have a high profile during the 2018 World Cup in Russia. Private security personnel will be on the front lines in hotels and in “fan zones.” They will operate magnetometers at entrances, perform bag checks, enforce restrictions on hand-carried items, etc. Private security will be especially important to the “guest experience” aspects of protecting the games. Private security will be especially important to the “guest experience” aspects of protecting the games Another private security function at the World Cup is executive protection of dignitaries and high-net-worth individuals who will be attending. Executive protection professionals will arrive early, conduct advanced security assessments before VIPs arrive, and secure trusted and vetted transportation (including armoured cars in some cases.) VIPs will include both Russian citizens and foreign (including U.S.) dignitaries attending the games. Private security details will be out in force. Aggressive security approach Overeager and outspoken fans are a part of the football culture, but Russia will deploy a near-zero tolerance policy against hooliganism and riots. An overwhelming force presence will take an aggressive approach to curbing any civil disturbances, and offenders will be removed quickly by Russian security forces. Strict restrictions on the sale and consumption of alcohol will be enforced in the venue cities before and after the matches. Officials will also be cognisant of the possibility of a riot or other event being used as a distraction to draw attention from another area where a terrorist event is planned. It will be a delicate balance between deploying an aggressive security approach and preserving the fan experience. Joelson notes that freedom of speech is not as valued in Russia as in other parts of the world, so the scales will be even more tipped toward security. “The last thing they want is for things to get out of control,” says Horner. “The event is putting Russia on the world stage, and they want visitors to walk away safely after having a great time and wanting to go back in the future.” Attendees should also have good situational awareness, and keep their heads up, scanning crowds and identifying unsafe situations" Precautions for World Cup attendees Attendees to the World Cup in Russia should take some basic precautions, Horner and Joelson agree. For example, Russia requires a translated, notarised letter explaining any prescription drugs. The country has a more aggressive foreign intelligence environment, so visitors cannot depend on their data being private. Joelson recommends the usual “social media hygiene” and privacy settings. Visitors should not post information about their travel plans or locations, and it’s best to travel with a disposable mobile phone that does not contain personal information. Location tracking should be deactivated. Travellers should also beware of talking and sharing information with others, or of saying anything derogatory. “They should also have good situational awareness, and keep their heads up, scanning crowds and identifying unsafe situations,” says Joelson. “If you bring a personal electronic device, you should expect that it has been compromised,” says Horner. Text messages and email will not be private, and he suggests creating an email address used only for travel. Don’t leave drinks unattended. Travellers from the U.S. should register at the Smart Traveler Enrollment Program (STEP) operated by the U.S. State Department. “Plan before you travel and before you get to the airport,” says Horner.

The benefits and challenges of in-camera audio analytics for surveillance solutions
The benefits and challenges of in-camera audio analytics for surveillance solutions

Audio is often overlooked in the security and video surveillance industry. There are some intercom installations where audio plays a key role, but it’s not typically thought about when it comes to security and event management. Audio takes a back seat in many security systems because audio captured from a surveillance camera can have a different impact on the privacy of those being monitored. Audio surveillance is therefore subject to strict laws that vary from state to state. Many states require a clearly posted sign indicating audio recording is taking place in an area before a person enters. Analytic information derived from audio can be a useful tool and when implemented correctly, removes any concerns over privacy or legal compliance. Audio analytics on the edge overcomes legal challenges as it never passes audio outside of the camera Focused responses to events Audio analytics processed in the camera, has been a niche and specialised area for many installers and end users. This could be due to state laws governing audio recording, however, audio analytics on the edge overcomes legal challenges as it never passes audio outside of the camera Processing audio analytics in-camera provides excellent privacy since audio data is analysed internally with a set of algorithms that only compare and assess the audio content. Processing audio analytics on the edge also reduces latency compared with any system that needs to send the raw audio to an on-premises or cloud server for analysis. Audio analytics can quickly pinpoint zones that security staff should focus on, which can dramatically shorten response times to incidents. Audio-derived data also provides a secondary layer of verification that an event is taking place which can help prioritise responses from police and emergency personnel. Having a SoC allows a manufacturer to reserve space for specialised features, and for audio analytics, a database of reference sounds is needed for comparison Microphones and algorithms Many IP-based cameras have small microphones embedded in the housing while some have a jack for connecting external microphones to the camera. Microphones on indoor cameras work well since the housing allows for a small hole to permit sound waves to reach the microphone. Outdoor cameras that are IP66 certified against water and dust ingress will typically have less sensitivity since the microphone is not exposed. In cases like these, an outdoor microphone, strategically placed, can significantly improve outdoor analytic accuracy. There are several companies that make excellent directional microphones for outdoor use, some of which can also combat wind noise. Any high-quality external microphone should easily outperform a camera’s internal microphone in terms of analytic accuracy, so it is worth considering in areas where audio information gathering is deemed most important. In-built audio-video analytics Surveillance cameras with a dedicated SoC (System on Chip) have become available in recent years with in-built video and audio analytics that can detect and classify audio events and send alerts to staff and emergency for sounds such as gunshots, screams, glass breaks and explosions. Having a SoC allows a manufacturer to reserve space for specialised features. For audio analytics, a database of reference sounds is needed for comparison. The camera extracts the characteristics of the audio source collected using the camera's internal or externally connected microphone and calculates its likelihood based on the pre-defined database. If a match is found for a known sound, e.g., gunshot, explosion, glass break, or scream, an event is triggered, and the message is passed to the VMS. If a match is found for a known sound, e.g., gunshot, explosion, glass break, or scream, an event is triggered, and the message is passed to the VMS Configuring a camera for audio analytics Audio detectionThe first job of a well-configured camera or camera/mic pair is to detect sounds of interest while rejecting ancillary sounds and noise below a preset threshold. Each camera must be custom configured for its particular environment to detect audio levels which exceed a user-defined level. Since audio levels are typically greater in abnormal situations, any audio levels exceeding the baseline set levels are detected as being a potential security event. Operators can be notified of any abnormal situations via event signals allowing the operator to take suitable measures. Finding a baseline of background noise and setting an appropriate threshold level is the first step. Installers should be able to enable or disable the noise reduction function and view the results to validate the optimum configuration during setup Noise reductionA simple threshold level may not be adequate enough to reduce false alarms depending on the environment where a camera or microphone is installed. Noise reduction is a feature on cameras that can reduce background noise greater than 55dB-65dB for increased detection accuracy. Installers should be able to enable or disable the noise reduction function and view the results to validate the optimum configuration during setup. With noise reduction enabled, the system analyses the attenuated audio source. As such, the audio source classification performance may be hindered or generate errors, so it is important to use noise reduction technology sparingly. Audio source classificationIt’s important to supply the analytic algorithm with a good audio level and a high signal-to-noise ratio to reduce the chance of generating false alarms under normal circumstances. Installers should experiment with ideal placement for both video as well as audio. While a ceiling corner might seem an ideal location for a camera, it might also cause background audio noise to be artificially amplified. Many cameras provide a graph which visualises audio source levels to allow for the intuitive checking of noise cancellation and detection levels. Analytics take privacy concerns out of the equation and allow installers and end users to use camera audio responsibly Messages and eventsIt’s important to choose a VMS that has correctly integrated the camera’s API (application programming interface) in order to receive comprehensive audio analytic events that include the classification ID (explosion, glass break, gunshot, scream). A standard VMS that only supports generic alarms, may not be able to resolve all of the information. More advanced VMS solutions can identify different messages from the camera. Well configured audio analytics can deliver critical information about a security event, accelerating response times and providing timely details beyond video-only surveillance. Analytics take privacy concerns out of the equation and allow installers and end users to use camera audio responsibly. Hanwha Techwin's audio source classification technology, available in its X Series cameras, features three customisable settings for category, noise cancellation and detection level for optimum performance in a variety of installation environments.

How important is packaging in the commercial security market?
How important is packaging in the commercial security market?

High-quality products are the building blocks of successful physical security systems. How they are packaged may sometimes be seen as an unimportant detail or an afterthought. But should it be? Effective packaging can serve many functions, from creating a favorable customer impression to ensuring the product isn’t damaged in transit. Packaging can also contribute to ease of installation. On the negative side, excess packaging can be an environmental concern, especially for customers who are sensitive to green factors or to minimising waste. We asked this week’s Expert Panel Roundtable: Is packaging of products important in the commercial security market? Why or why not?