Audio is often overlooked in the security and video surveillance industry. There are some intercom installations where audio plays a key role, but it’s not typically thought about when it comes to security and event management. Audio takes a back seat in many security systems because audio captured from a surveillance camera can have a different impact on the privacy of those being monitored.

Audio surveillance is therefore subject to strict laws that vary from state to state. Many states require a clearly posted sign indicating audio recording is taking place in an area before a person enters. Analytic information derived from audio can be a useful tool and when implemented correctly, removes any concerns over privacy or legal compliance.

Audio analytics on the edge overcomes legal challenges as it never passes audio outside of the camera

Focused responses to events

Audio analytics processed in the camera, has been a niche and specialised area for many installers and end users. This could be due to state laws governing audio recording, however, audio analytics on the edge overcomes legal challenges as it never passes audio outside of the camera

Processing audio analytics in-camera provides excellent privacy since audio data is analysed internally with a set of algorithms that only compare and assess the audio content. Processing audio analytics on the edge also reduces latency compared with any system that needs to send the raw audio to an on-premises or cloud server for analysis.

Audio analytics can quickly pinpoint zones that security staff should focus on, which can dramatically shorten response times to incidents. Audio-derived data also provides a secondary layer of verification that an event is taking place which can help prioritise responses from police and emergency personnel.

Having a SoC allows a manufacturer to reserve space for specialised features, and for audio analytics, a database of reference sounds is needed for comparison

Microphones and algorithms

Many IP-based cameras have small microphones embedded in the housing while some have a jack for connecting external microphones to the camera. Microphones on indoor cameras work well since the housing allows for a small hole to permit sound waves to reach the microphone. Outdoor cameras that are IP66 certified against water and dust ingress will typically have less sensitivity since the microphone is not exposed. In cases like these, an outdoor microphone, strategically placed, can significantly improve outdoor analytic accuracy.

There are several companies that make excellent directional microphones for outdoor use, some of which can also combat wind noise. Any high-quality external microphone should easily outperform a camera’s internal microphone in terms of analytic accuracy, so it is worth considering in areas where audio information gathering is deemed most important.

In-built audio-video analytics

Surveillance cameras with a dedicated SoC (System on Chip) have become available in recent years with in-built video and audio analytics that can detect and classify audio events and send alerts to staff and emergency for sounds such as gunshots, screams, glass breaks and explosions. Having a SoC allows a manufacturer to reserve space for specialised features. For audio analytics, a database of reference sounds is needed for comparison.

The camera extracts the characteristics of the audio source collected using the camera's internal or externally connected microphone and calculates its likelihood based on the pre-defined database. If a match is found for a known sound, e.g., gunshot, explosion, glass break, or scream, an event is triggered, and the message is passed to the VMS.

Each camera must be custom configured for its particular environment to detect audio levels which exceed a user-defined level
If a match is found for a known sound, e.g., gunshot, explosion, glass break, or scream, an event is triggered, and the message is passed to the VMS

Configuring a camera for audio analytics

Audio detection
The first job of a well-configured camera or camera/mic pair is to detect sounds of interest while rejecting ancillary sounds and noise below a preset threshold. Each camera must be custom configured for its particular environment to detect audio levels which exceed a user-defined level. Since audio levels are typically greater in abnormal situations, any audio levels exceeding the baseline set levels are detected as being a potential security event. Operators can be notified of any abnormal situations via event signals allowing the operator to take suitable measures. Finding a baseline of background noise and setting an appropriate threshold level is the first step.

Installers should be able to enable or disable the noise reduction function and view the results to validate the optimum configuration during setup

Noise reduction
A simple threshold level may not be adequate enough to reduce false alarms depending on the environment where a camera or microphone is installed. Noise reduction is a feature on cameras that can reduce background noise greater than 55dB-65dB for increased detection accuracy. Installers should be able to enable or disable the noise reduction function and view the results to validate the optimum configuration during setup. With noise reduction enabled, the system analyses the attenuated audio source. As such, the audio source classification performance may be hindered or generate errors, so it is important to use noise reduction technology sparingly.

Audio source classification
It’s important to supply the analytic algorithm with a good audio level and a high signal-to-noise ratio to reduce the chance of generating false alarms under normal circumstances. Installers should experiment with ideal placement for both video as well as audio. While a ceiling corner might seem an ideal location for a camera, it might also cause background audio noise to be artificially amplified. Many cameras provide a graph which visualises audio source levels to allow for the intuitive checking of noise cancellation and detection levels.

Analytics take privacy concerns out of the equation and allow installers and end users to use camera audio responsibly

Messages and events
It’s important to choose a VMS that has correctly integrated the camera’s API (application programming interface) in order to receive comprehensive audio analytic events that include the classification ID (explosion, glass break, gunshot, scream). A standard VMS that only supports generic alarms, may not be able to resolve all of the information. More advanced VMS solutions can identify different messages from the camera.

Well configured audio analytics can deliver critical information about a security event, accelerating response times and providing timely details beyond video-only surveillance. Analytics take privacy concerns out of the equation and allow installers and end users to use camera audio responsibly. Hanwha Techwin's audio source classification technology, available in its X Series cameras, features three customisable settings for category, noise cancellation and detection level for optimum performance in a variety of installation environments.

Download PDF version

Author profile

In case you missed it

Artificial intelligence: why you should enable deep learning and video analytics
Artificial intelligence: why you should enable deep learning and video analytics

Constantly optimising deep learning algorithms yields better video analytics performance, even in complex applications such as facial recognition or in scenarios with variable lighting, angles, postures, expressions, accessories, resolution, etc. Deep learning, a form of artificial intelligence (AI), holds the potential to enable video analytics to deliver on long-promised, but not often delivered performance. Our AI series continues here with part 2. Adapting existing hardware Today, low-cost system-on-chip (SoC) camera components enable deep neural network (DNN) processing for the next generation of intelligent cameras, thus expanding the availability of AI processing to a broader market. AI software can even add learning capabilities by adapting existing hardware to AI applications AI software can even add learning capabilities by adapting existing hardware to AI applications. Today’s smartphones include cameras, gyroscopes and accelerometers to provide sufficient data to drive AI applications. Software can adapt existing hardware to transform them into AI devices capable of continuous learning in the field. Inside a video camera, real-time deep learning processing can be used to detect discarded objects, issue loitering alarms and detect people or objects entering a pre-defined field. Data capture form to appear here! Detect anomalous data Additional capabilities are applicable to demanding environments and mission-critical applications, such as the perimeter protection of airports, critical infrastructures and government buildings, border patrol, ship-tracking and traffic-monitoring (e.g. wrong-way detection, traffic-counts and monitoring roadsides for parked cars: all vital video security solutions). IoT is transforming the lowly security camera from a device that simply captures images, into an intelligent sensor that plays an integral role in gathering the kind of vital business data that can be used to improve commercial operations in areas beyond security. For example, cities are transitioning into smart cities. Deep learning enables systems to search surveillance footage, to detect anomalous data, and to shift surveillance from post-incident response to providing alerts during, or even before, an event. The ability of deep learning for video analytics is much more sophisticated and accurate Make critical decisions Deep learning can eliminate previous video analytics limitations such as dependence on a scene’s background. Deep learning is also more adept than humans at discerning subtle changes in an image. The ability of deep learning for video analytics is much more sophisticated – and accurate – than the programmed approaches previously employed to identify targets. AI is a timely solution in an age when there is more video surveillance than ever. There are too many cameras and too much recorded video for security operators to keep pace with. On top of that, people have short attention spans. AI is a technology that doesn’t get bored and can analyse more video data than humans. Systems are designed to bring the most important events and insight to users’ attention, freeing them to do what they do best: make critical decisions. Multiple camera streams AI can reduce information overload to enable humans to work with the data more efficiently The video benefits reflect the larger goal of AI to amplify human skills. AI can reduce information overload to enable humans to work with the data more efficiently. Another benefit is faster search, and new systems make searching video as easy as searching the internet. AI enables specific people or cameras to be located quickly across all the cameras at a site. Searching can be directed by a reference images or by physical descriptors such as gender or clothing colour. Consider a scenario of a child missing from a crowded shopping mall: Every second can seem like hours, and artificial intelligence and neural networks can enable a rapid search among multiple camera streams using only one photo of the child. The photo does not have to be a full-frontal passport-type photos; it could be a selfie from a party as long as the face is there. Intrusion detection scenario AI can find her and match her face from among hundreds of thousands of faces captured from video, in nearly real time. AI can also continuously analyse video streams from the surveillance cameras in its network, distinguishing human faces from non-human objects such as statues and animals. Privacy concerns are minimal as there is no ID or personal information on the photo, and the image can be erased after use. And there is no database of stored images.    In a perimeter security/intrusion detection scenario, an AI-driven video system can avoid false alarms by easily distinguishing different types of people and objects, e.g., in a region set up to detect people, a car driving by, a cat walking by, or a person’s shadow will not trigger the alarm. Part three coming soon. If you missed part one, see it here.

3 key security tips for public event planners
3 key security tips for public event planners

Public spaces in cities and suburbs are important places for community development and promoting outdoor recreation. These areas may include main streets, parks, promenades, band shells and fields. Such locations are often utilised by public event planners for community activities, including summer festivals, wintertime ice skating rink installations, music concerts and art fairs. As the year draws to a close, holiday and Christmas markets as well as major New Year’s Eve events, present cities with constant public event security needs. The public nature of these events increases risks of incidents with high-speed vehicles that put attendees in danger. Fortunately, there are three ways for public space managers to prevent casualty-causing collisions and further promote the use of local public areas. Developing an effective action plan    When strategising how to react to an alert, think about what time of the year and time of day the event is occurring It is important to have a plan developed before an incident or accident occurs. Warning systems, utilising doppler radar and digital loop technologies, alert guards to abnormal vehicle velocity changes in the surrounding area. Managers of public areas should organise a meeting with public safety authorities and local agencies to discuss what must immediately occur when a high-speed vehicle is approaching a public event. When strategising how to react to an alert, think about what time of the year and time of day the event is occurring. Having such a reaction plan in place combines technology and strategic planning to ensure everyone is on the same page to effectively target a threat and promote overall event safety. Securing public areas  Ideally, there will be no need to implement a well-conceived action plan. After all, taking preventive measures to secure public areas where events take place is important to keep people safe from accidental vehicle collisions and intentional attacks. Protect attendees by clearly separating pedestrian and vehicle locations using security devices such as – Barricades Portable barriers Bollards Install guard booths  Avoid the risk of vandalism and theft, making sure people are safe when walking back to the cars at night by keeping parking areas illuminated with flood lights. Install guard booths with employees who monitor activity in the parking area and who are prepared to react if an alert is triggered. Furthermore, prevent accidental collisions by clearly marking the parking area with informative warning signs and using barricades to direct traffic. These three tips can be used by public area managers to promote security at the next community event. Additionally, the technologies used to secure an event can also be used as infrastructure for year-round security. Installing gates that shut when the public space is closed or using aesthetically pleasing bollards are steps any public area manager can take to promote community safety.

Choosing your security entrance installation in line with your company culture
Choosing your security entrance installation in line with your company culture

The extensive analysis and discussion preceding any decision to implement a new physical security solution – whether it’s hardware, software or a combination of both – often focuses on technology, ROI and effectiveness. When it comes to deciding what type of security entrances to install at your facility, you will almost certainly also consider the aesthetics of the product, along with throughput and, if you’re smart, you’ll also look into service concerns. Each of these factors has its important place within the evaluation process, and none should be overlooked as they all have a significant effect on how well your entrances will perform once they are installed. Culture influences door solution decisions How significant will the change from current entrances to security entrances be for employees? Still, one additional factor actually trumps everything: if you have not considered your organisation’s culture in choosing a security entrance, you may be missing the most important piece of the puzzle. Culture is a part of every other decision factor when selecting an entry solution. Before you make a decision about what type of entrance to deploy, you need to consider and understand the values, environment and personality of your organisation and personnel. For example, how significant will the change from current entrances to security entrances be for employees? If people are accustomed to simply walking through a standard swinging door with no access control, this will be a culture change. Beyond this, whether you are considering a type of turnstile, a security revolving door or possibly a mantrap portal, simply walking through it will be a significant change as well. Training employees on door security You’ll want to know whether employees have ever used security entrances before. If these types of entrances are in place in another part of the facility, or in a facility they’ve worked in at an earlier time, the adjustment will not be as great as if they’ve never used them at all. Consider, too, how your personnel typically react to changes like this in the organisation or at your facility. They may be quite adaptable, in which case there will be less work to do in advance to prepare them. However, the opposite may also be true, which will require you to take meaningful steps in order to achieve buy-in and train employees to properly use the new entrances. With the increased importance of workplace security, discussing new entrances with  workforces will help maintain a safer environment Communicate through the decision-making process All of this will need to be communicated to your staff, of course. There are a number of ways to disseminate information without it appearing to come down as a dictate. Your personnel are a community, so news about changes should be shared rather than simply decreed. As part of this process, you’ll need to give some thought to the level of involvement you want for your staff in the decision-making process. Finally, do not overlook the special needs among your personnel population. You undoubtedly have older individuals on staff, as well as disabled persons and others who bring service animals to the office. Entrances need to be accessible to all, and you never want to be in the position of having a gap in accessibility pointed out to you by the individual who has been adversely affected. New security entrance installation By communicating early and often with your personnel, you can alleviate a great deal of the anxiety Once you have made the decision about which security entrances to install, training your personnel on how to use the new security entrances – both before and after the installation – will help to smooth the transition. Because workplace security is such a big issue right now, it makes sense to discuss the new entrances in the context of helping to maintain a safer environment. They will prevent violent individuals from entering, decrease theft, and most of all, promote greater peace of mind during the workday. If you can help them take control of their own safety in a responsible way, you have achieved much more than just a compliant workforce. By communicating early and often with your personnel, you can alleviate a great deal of the anxiety and concern that surrounds a significant change in the work environment. Schedule group meetings Consider your employees; what type of communications do they respond best to? A few suggestions to educate staff on the benefits of the new entrances include: Typically, you would communicate a general message 2-3 months in advance and then provide more specific information (for example, impacts to fire egress, using certain entrances during construction) in a follow up message closer to the installation date. Schedule group meetings to: announce the rationale for increased security, share statistics on crime, review the new security changes that are coming, show drawings/photos of the new doors/turnstiles, and show the orientation videos available from the manufacturer. These meetings are an excellent way to work through user questions and directly address any concerns. Once the installation of a new security system is complete, it is a good idea to have an "ambassador" on board to help employees use these new systems Ensure you monitor public areas If you are implementing a lot of new changes, such as a new access control system, new guard service and security entrances, you might consider hosting a ‘security fair’ on a given day and have the selected vendors come for a day with tabletop displays to meet employees and answer questions during their lunch. This could be a great way to break the ice in a large organisation. Make user orientation videos (provided by the manufacturer) available in several ways, for example: Intranet Site Monitors in public areas—lounges, cafeteria, hallways, etc. Send to all staff as email attachments Immediately after installation, once the doors or turnstiles are operational but before they are put into service, train ‘ambassadors’ on how to use the door/turnstile. Have these people monitor and assist employees during peak traffic times. What is the ultimate success of the installation? By communicating clearly and openly with your population you can greatly facilitate adoption and satisfaction If you have thousands of employees, consider dividing them into groups and introduce the new entrance to one group at a time (Group A on Monday, Group B on Tuesday, etc.) to allow a little extra orientation time. Place user education ‘quick steps’ posters next to the door/turnstiles for a few weeks to help employees remember the basic steps and guidelines, e.g., ‘stand in front of the turnstile, swipe badge, wait for green light, proceed.’ Ask your manufacturer to provide these or artwork. While there are always going to be people who are resistant to change, by communicating clearly and openly with your population you can greatly facilitate adoption and satisfaction. Your responsiveness to any issues and complaints that arise during and after the implementation is equally fundamental to the ultimate success of the installation.