When a child goes missing in a large, crowded mall, we have a panicking mom asking for help from the staff, at least a dozen cameras in the area, and assuming the child has gone missing for only 15 minutes, about 3 hours’ worth of video to look through to find the child. Typical security staff response would be to monitor the video wall while reviewing the footage and making a verbal announcement throughout the mall so the staff can keep an eye out for her.

There is no telling how long it will take, while every second feels like hours under pressure. As more time passes, the possible areas where the child can be will widen, it becomes more time-consuming to search manually, and the likelihood of finding the child decreases. What if we can avoid all of that and directly search for that particular girl in less than 1 second?

Artificial neural networks are improving every day and now enable us to search for a person across all selected camera streamsWith Artificial Intelligence, we can. Artificial neural networks are improving every day and now enable us to search for a person across all selected camera streams in a fraction of a second, using only one photo of that person.

The photo does not even have to be a full frontal, passport-type mugshot; it can be a selfie image of the person at a party, as long as the face is there, the AI can find her and match her face with the hundreds or thousands of faces in the locations of interest. The search result is obtained in nearly real time as she passes by a certain camera.

Distinguishing humans from animals and statues

The AI system continuously analyses video streams from the surveillance cameras in its network, distinguishes human faces from non-human objects such as statues and animals, and much like a human brain, stores information about those faces in its memory, a mental image of the facial features so to speak.

When we, the system user, upload an image of the person of interest to the AI system, the AI detects the face(s) in that image along with their particular features, search its memory for similar faces, and shows us where and when the person has appeared.

We are in control of selecting the time period (up to days) and place (cameras) to search, and we can adjust the similarity level, i.e., how much a face matches the uploaded photo, to expand or fine-tune the search result according to our need. Furthermore, because the camera names and time stamps are available, the system can be linked with maps to track and predict the path of the person of interest.

The AI system continuously analyses video streams from the surveillance cameras in its network, distinguishing, human faces from non-human objects
AI Face Search is not Face Recognition for two reasons: it protects people’s privacy, and it is lightweight

Protecting people’s privacy with AI Face Search 

All features of face recognition can be enabled by the system user, such as to notify staff members when a person of interest is approaching the store AI Face Search is not Face Recognition for two reasons: it protects people’s privacy, and it is lightweight. First, with AI Face Search, no names, ID, personal information, or lists of any type are required to be saved in the system. The uploaded image can be erased from the system after use, there is no face database, and all faces in the camera live view can be blurred out post-processing to guarantee GDPR compliance.

Second, the lack of a required face database, a live view with frames drawn around the detected faces and constant face matching in the background also significantly reduces the amount of computing resource to process the video stream, hence the lightweight.

Face Search versus Face Recognition

AI Face Search

Face Recognition

Quick search for a particular person in video footage

Identify everyone in video footage

Match detected face(s) in video stream to target face(s) in an uploaded image

Match detected face(s) in video stream to a database

Do not store faces and names in a database

Must have a database with ID info

Automatically protect privacy for GDPR compliance in public places

May require additional paperwork to comply with privacy regulations

Lightweight solution

Complex solution for large-scale deployment

Main use: locate persons of interest in a large area

Main use: identify a person who passes through a checkpoint

Of course, all features of face recognition can be enabled by the system user if necessary, such as to notify staff members when a person of interest is approaching the store, but the flexibility to not have such features and to use the search tool as a simple Google-like device particularly for people and images is the advantage of AI Face Search.Because Face Search is not based on face recognition, no faces and name identifications are stored

Advantages of AI Face Search

Artificial Intelligence has advanced so far in the past few years that its facial understanding capability is equivalent to that of a human. The AI will recognise the person of interest whether he has glasses, wears a hat, is drinking water, or is at an angle away from the camera.

In summary, the advantages of Face Search:

  1. High efficiency: a target person can be located within a few seconds, which enables fast response time.
  2. High performance: high accuracy in a large database and stable performance, much like Google search for text-based queries.
  3. Easy setup and usage: AI appliance with the built-in face search engine can be customised to integrate to any existing NVR/VMS/camera system or as a standalone unit depending on the customer’s needs. The simple-to-use interface requires minimal training and no special programming skills.
  4. High-cost saving: the time saving and ease of use translate to orders of magnitude less manual effort than traditionally required, which means money saving.
  5. Scalability: AI can scale much faster and at a wider scope than human effort. AI performance simply relies on computing resource, and each Face Search appliance typically comes with the optimal hardware for any system size depending on the customer need, which can go up to thousands of cameras.
  6. Privacy: AI Face Search is not face recognition. For face recognition, there are privacy laws that limits the usage. Because Face Search is not based on face recognition, no faces and name identifications are stored, so Face Search can be used in many public environments to identify faces against past and real-time video recordings.
Face Search can be used in many public environments to identify faces against past and real-time video recordings.
AI Face Search match detected face(s) in video stream to target face(s) in an uploaded image

Common use cases of AI Face Search

In addition to the scenario of missing child in a shopping mall, other common use cases for the AI Face Search technology include:

  • Retail management: Search, detect and locate VIP guests in hotels, shopping centres, resorts, etc. to promptly attend to their needs, track their behaviour pattern, and predict locations that they tend to visit.
  • Crime suspect: Quickly search for and prove/disprove the presence of suspects (thief, robber, terrorist, etc.) in an incident at certain locations and time.
  • School campus protection: With the recent increase in number of mass shootings in school campuses, there is a need to identify, locate and stop a weapon carrier on campus as soon as possible before he can start shooting. Face Search will enable the authorities to locate the suspect and trace his movements within seconds using multiple camera feeds from different areas on campus. Only one clear image of the suspect’s face is sufficient.

In the race of technology development in response to business needs and security concerns, AI Face Search is a simple, lightweight solution for airports, shopping centres, schools, resorts, etc. to increase our efficiency, minimise manual effort in searching for people when incidents occur on site, and actively prevent potential incidents from occurring.

By Paul Sun, CEO of IronYun, and Mai Truong, Marketing Manager of IronYun

Download PDF version

In case you missed it

Enhance traditional security systems within your smart home
Enhance traditional security systems within your smart home

Market dynamics are changing the U.S. residential security market, creating new business models that better appeal to the approximately 70% of households without a security system. Smart home adjacencies have helped revitalise the traditional security industry, and alternative approaches to systems and monitoring for the security industry are emerging, including a new batch of DIY systems. Growth in the residential security market and its position as the channel for smart home solutions have attracted numerous new entrants. Telecoms, cable operators, and CE (consumer electronics) manufacturers are joining traditional security players as they compete to fulfill consumer demand for safety and security. Connected products also provide a layer of competition as consumers must decide whether having category devices such as doorbell video cameras, networked cameras, and other products suffice for their security. Increasingly competitive landscape Smart home services can provide additional revenue streams for the security industry For instance, IP cameras are a highly popular smart home device rooted in security, and Parks Associates estimates 7.7 million standalone and all-in-one networked/IP cameras will be sold in the U.S. in 2018, with $889M in revenues. Product owners may feel their security needs are fulfilled with this single purchase, as such dealers and service providers are under increasing pressure to communicate their value proposition to consumers. Categorically, each type of player is facing competition uniquely—national, regional, and local dealers all have a different strategy for overcoming the increasingly competitive landscape. Smart home services can provide additional revenue streams for the security industry. In Parks Associates’ 2017 survey of U.S. security dealers, 58% report that smart home service capabilities enable extra monthly revenue. Almost half of dealers also note they have to offer smart home devices and services in order to keep up with their competition. While white-label devices are acceptable in some instances, dealers need to integrate with hero products whenever possible when those exist for a category. For dealers who have added smart home devices and services are all potential benefits and good for business Improved customer engagement That 2017 survey also revealed 36% of security dealers that offer interactive services report security system sales with a networked camera and 16% report sales with a smart thermostat. For dealers who have added smart home devices and services, enhanced system utility, increased daily value, and improved customer engagement with the system are all potential benefits and good for business. Security has served as the most productive channel for smart home solutions, mainly because the products create natural extensions of a security system’s functions and benefits, but as smart home devices, subsystems, and controllers expand their functionality, availability, and DIY capabilities, many standalone devices constitute competition to classical security. Particularly viable substitute devices include IP cameras, smart door locks, smart garage doors, or a combination of these devices. Products that are self-installed offer both convenience and cost savings, and these drivers are significant among DIY consumers—among the 6% of broadband households that installed a security system themselves, 39% did it to save money. Enhance traditional security Self-installable smart home devices may resonate with a segment of the market who want security While many security dealers believe substitute offerings are a threat, some dealers do not find such devices an existential threat but instead view them as another path to consumer awareness. They argue that the difference between smart product substitutes and traditional security is that of a solution that provides knowledge versus a system that gives one the ability to act on that knowledge. A common theme among professional monitoring providers is that a homeowner who is aware of events happening in the home does not necessarily have a secure and protected household. For example, a Nest camera, a DIY product, notifies a consumer via smartphone about events in the home when it detects motion, but only when the notification is opened and identified will a consumer be able to act on the related event. Self-installable smart home devices may resonate with a segment of the market who want security but are unwilling to adopt professional monitoring; however, providers can leverage these devices to enhance traditional security features and communicate the value of professional monitoring. Smart home devices and features, while posing a threat to some security companies, are a potential way forward to increased market growth Increased market growth A key counterstrategy for security dealers and companies is to leverage their current, powerful role as the prime channel for smart home devices. Many security dealers now include smart home devices with their security systems to complement their offerings and increase system engagement. For example, as of Q4 2017, nearly 70% of U.S. broadband households that were very likely to purchase a security system in the next 12 months reported that they want a camera to be included as part of their security system purchase. In response, many security system providers now offer IP cameras as optional enhancements for their systems. Smart home devices and features, while posing a threat to some security companies, are a potential way forward to increased market growth. Security dealers have an opportunity to become more than a security provider but a smart home solutions provider rooted in safety. Provide status updates Comcast has entered both the professionally monitored security market and the market for smart home services The alternative is to position as a provider of basic security with low price as the key differentiator. Comcast has entered both the professionally monitored security market and the market for smart home services independent of security. It has discovered that monetising smart home value propositions through recurring revenue becomes increasingly challenging as the value extends further away from life safety. Since the security industry remains the main channel for smart home services, security dealers are in a unique position to leverage that strength. Value propositions must shift from the traditional arming and disarming of a system to peace-of-mind experiences that builds off the benefits of smart devices in the home to provide status updates (e.g., if the kids arrived home safely) and monitoring at will (e.g., checking home status at any time to see a pet or monitor a package delivery). These types of clear value propositions and compelling use cases, which resonate with consumer and motivate them to expand beyond standalone products, will help expand the home security market.

What is the value of "free" video management systems?
What is the value of "free" video management systems?

They say that every choice has a cost. It's a basic principle that, economically speaking, nothing is free. If it doesn't cost actual money, it may be expensive in terms of time, attention and/or effort. These are interesting observations to keep in mind as one peruses the various "free" video management system (VMS) offerings available on the market. Some are provided by camera companies to unify their products into a "system", even if it's a small one. Other free VMS offerings are entry-level versions offered by software companies with the intent of the customer upgrading later to a paid version. For more insights, we asked this week's Expert Panel Roundtable: What is the value of “free” video management systems (VMSs) and how can a customer decide whether “free” is the right price for them?

The ongoing challenge of IT and data risk management
The ongoing challenge of IT and data risk management

Managing IT and data risk is a challenging job. When we outsource our IT, applications and data processing to third-parties more and more every day, managing that risk becomes almost impossible. No longer are our data and systems contained within an infrastructure that we have full control over. We now give vendors our data, and allow them to conduct operations on our behalf.  The problem is, we don’t control their infrastructure, and we can never fully look under the hood to understand and vet their ability to protect our data and operations. We have to fully understand how important this issue is, and ensure we have the right governance, processes and teams to identify and mitigate any risks found in our vendors. No longer are our data and systems contained within an infrastructure that we have full control over Today, everything is connected. Our own networks have Internet of Things (IoT) devices.  We have VPN connections coming in, and we aren’t always sure who is on the other end of that connection. It is a full-time job just to get a handle on our own risk. How much harder, and how much larger should our teams and budgets be, to truly know and trust that our vendors can secure those devices and external connections?  For every device and application we have internally, it is very difficult to even keep an accurate inventory. Do all of our vendors have some special sauce that allows them to overcome the traditional challenges of securing internal and vendor-connected networks? They are doing the same thing we are – doing our best with the limited human and financial resources allocated by our organisation. Risk stratification and control objectives  The benefits of outsourcing operations or using a vendor web application are clear. So how can we properly vet those vendors from an IT risk perspective?  The very first thing we need to put in place is Risk Stratification. Risk Stratification presents a few targeted questions in the purchasing process. These questions include – what type of data will be shared? How much of this data? Will the data be hosted by a vendor? Will this hosting be in the US or offshored? Has the vendor ever had a data breach? These questions allow you to quickly discern if a risk assessment is needed and if so, what depth and breadth.  Risk stratification allows you to make decisions that not only improve your team’s efficiency, but also ensure that you are not being a roadblock to the business Risk stratification allows you to make decisions that not only improve your team’s efficiency, but also ensure that you are not being a roadblock to the business. With risk stratification, you can justify the extra time needed to properly assess a vendor’s security.  And in the assessment of a vendor’s security, we have to consider what control objectives we will use. Control objectives are access controls, policies, encryption, etc. In healthcare, we often use the HITRUST set of control objectives. In assessing against those control objectives, we usually use a spreadsheet.  Today, there are many vendors who will sell us more automated ways to get that risk assessment completed, without passing spreadsheets back and forth. These solutions are great if you can get the additional budget approved.  Multi-factor authentication  Even if we are using old-fashioned spreadsheets, we can ensure that the questions asked of the vendor include a data flow and network/security architecture document.  We want to see the SOC2 report if they are hosting their solution in Amazon, etc. If they are hosting it within their own datacentre, we absolutely want to see a SOC2 Type II report. If they haven’t done that due diligence, should that be a risk for you?  Today, we really need to be requiring our vendors to have multi-factor authentication on both their Internet-facing access, as well as their privileged internal access to our sensitive data. I rate those vendors who do not have this control in place as a high risk. We’ve recently seen breaches that were able to happen because the company did not require administrators or DBAs to use a 2-factor authentication into sensitive customer data sources.  In the assessment of a vendor’s security, one has to consider what control objectives to use This situation brings up the issue of risk acceptance. Who in your organisation can accept a high risk? Are you simply doing qualitative risk assessment – high, medium and low risks? Or are you doing true quantitative risk analysis? The latter involves actually quantifying those risks in terms of likelihood and impact of a risk manifesting, and the dollar amount that could impact your organisation.   So is it a million dollars of risk? Who can accept that level of risk? Just the CEO? These are questions we need to entertain in our risk management programs, and socialised within your organisation.  This issue is so important – once we institute risk acceptance, our organisation suddenly starts caring about the vendors and applications we’re looking to engage.  If they are asked to accept a risk without some sort of mitigation, they suddenly care and think about that when they are vetting future outsourced solutions. Quantitative risk analysis involves quantifying risks in terms of likelihood and impact of a risk manifesting Risk management process  In this discussion, it is important to understand how we think of, and present, the gaps we identify in our risk management processes. A gap is not a risk. If I leave my front door unlocked, is that a control gap or a risk? It is a gap – an unlocked door. What is the risk?  The risk is the loss of property due to a burglary or the loss of life due to a violent criminal who got in because the door was unlocked. When we present risks, we can’t say the vendor doesn’t encrypt data. The risk of the lack of encryption is fines, loss of reputation, etc. due to the breach of data. A gap is not a risk.  Once we’ve conducted our risk analysis, we must then ensure that our contracts protect our organisation? If we’re in healthcare, we must determine if the vendor is, in fact, a true HIPAA Business Associate, and if so we get a Business Associate Agreement (BAA) in place. I also require my organisation to attach an IT Security Amendment to these contracts. The IT Security Amendment spells out those control objectives, and requires each vendor to sign off on those critical controls. We are responsible for protecting our organisation’s IT and data infrastructure – today that often means assessing a 3rd-party’s security controls One final note on risk assessments – we need to tier our vendors. We tier them in different ways – in healthcare a Tier 1 vendor is a vendor who will have our patient information on the Internet. Tiering allows us to subject our vendors to re-assessment. A tier 1 vendor should be re-assessed annually, and may require an actual onsite assessment vs. a desk audit. A tier 2 vendor is re-assessed every 2 years, etc. We are responsible for protecting our organisation’s IT and data infrastructure – today that often means assessing a 3rd-party’s security controls. We must be able to fully assess our vendors while not getting in the way of the business, which needs to ensure proper operations, financial productivity and customer satisfaction. If we truly understand our challenge of vendor risk management, we can tailor our operations to assess at the level needed, identify and report on risks, and follow-up on any risks that needed mitigated.