When a child goes missing in a large, crowded mall, we have a panicking mom asking for help from the staff, at least a dozen cameras in the area, and assuming the child has gone missing for only 15 minutes, about 3 hours’ worth of video to look through to find the child. Typical security staff response would be to monitor the video wall while reviewing the footage and making a verbal announcement throughout the mall so the staff can keep an eye out for her.

There is no telling how long it will take, while every second feels like hours under pressure. As more time passes, the possible areas where the child can be will widen, it becomes more time-consuming to search manually, and the likelihood of finding the child decreases. What if we can avoid all of that and directly search for that particular girl in less than 1 second?

Artificial neural networks are improving every day and now enable us to search for a person across all selected camera streamsWith Artificial Intelligence, we can. Artificial neural networks are improving every day and now enable us to search for a person across all selected camera streams in a fraction of a second, using only one photo of that person.

The photo does not even have to be a full frontal, passport-type mugshot; it can be a selfie image of the person at a party, as long as the face is there, the AI can find her and match her face with the hundreds or thousands of faces in the locations of interest. The search result is obtained in nearly real time as she passes by a certain camera.

Distinguishing humans from animals and statues

The AI system continuously analyses video streams from the surveillance cameras in its network, distinguishes human faces from non-human objects such as statues and animals, and much like a human brain, stores information about those faces in its memory, a mental image of the facial features so to speak.

When we, the system user, upload an image of the person of interest to the AI system, the AI detects the face(s) in that image along with their particular features, search its memory for similar faces, and shows us where and when the person has appeared.

We are in control of selecting the time period (up to days) and place (cameras) to search, and we can adjust the similarity level, i.e., how much a face matches the uploaded photo, to expand or fine-tune the search result according to our need. Furthermore, because the camera names and time stamps are available, the system can be linked with maps to track and predict the path of the person of interest.

The AI system continuously analyses video streams from the surveillance cameras in its network, distinguishing, human faces from non-human objects
AI Face Search is not Face Recognition for two reasons: it protects people’s privacy, and it is lightweight

Protecting people’s privacy with AI Face Search 

All features of face recognition can be enabled by the system user, such as to notify staff members when a person of interest is approaching the store AI Face Search is not Face Recognition for two reasons: it protects people’s privacy, and it is lightweight. First, with AI Face Search, no names, ID, personal information, or lists of any type are required to be saved in the system. The uploaded image can be erased from the system after use, there is no face database, and all faces in the camera live view can be blurred out post-processing to guarantee GDPR compliance.

Second, the lack of a required face database, a live view with frames drawn around the detected faces and constant face matching in the background also significantly reduces the amount of computing resource to process the video stream, hence the lightweight.

Face Search versus Face Recognition

AI Face Search

Face Recognition

Quick search for a particular person in video footage

Identify everyone in video footage

Match detected face(s) in video stream to target face(s) in an uploaded image

Match detected face(s) in video stream to a database

Do not store faces and names in a database

Must have a database with ID info

Automatically protect privacy for GDPR compliance in public places

May require additional paperwork to comply with privacy regulations

Lightweight solution

Complex solution for large-scale deployment

Main use: locate persons of interest in a large area

Main use: identify a person who passes through a checkpoint

Of course, all features of face recognition can be enabled by the system user if necessary, such as to notify staff members when a person of interest is approaching the store, but the flexibility to not have such features and to use the search tool as a simple Google-like device particularly for people and images is the advantage of AI Face Search.Because Face Search is not based on face recognition, no faces and name identifications are stored

Advantages of AI Face Search

Artificial Intelligence has advanced so far in the past few years that its facial understanding capability is equivalent to that of a human. The AI will recognise the person of interest whether he has glasses, wears a hat, is drinking water, or is at an angle away from the camera.

In summary, the advantages of Face Search:

  1. High efficiency: a target person can be located within a few seconds, which enables fast response time.
  2. High performance: high accuracy in a large database and stable performance, much like Google search for text-based queries.
  3. Easy setup and usage: AI appliance with the built-in face search engine can be customised to integrate to any existing NVR/VMS/camera system or as a standalone unit depending on the customer’s needs. The simple-to-use interface requires minimal training and no special programming skills.
  4. High-cost saving: the time saving and ease of use translate to orders of magnitude less manual effort than traditionally required, which means money saving.
  5. Scalability: AI can scale much faster and at a wider scope than human effort. AI performance simply relies on computing resource, and each Face Search appliance typically comes with the optimal hardware for any system size depending on the customer need, which can go up to thousands of cameras.
  6. Privacy: AI Face Search is not face recognition. For face recognition, there are privacy laws that limits the usage. Because Face Search is not based on face recognition, no faces and name identifications are stored, so Face Search can be used in many public environments to identify faces against past and real-time video recordings.
Face Search can be used in many public environments to identify faces against past and real-time video recordings.
AI Face Search match detected face(s) in video stream to target face(s) in an uploaded image

Common use cases of AI Face Search

In addition to the scenario of missing child in a shopping mall, other common use cases for the AI Face Search technology include:

  • Retail management: Search, detect and locate VIP guests in hotels, shopping centres, resorts, etc. to promptly attend to their needs, track their behaviour pattern, and predict locations that they tend to visit.
  • Crime suspect: Quickly search for and prove/disprove the presence of suspects (thief, robber, terrorist, etc.) in an incident at certain locations and time.
  • School campus protection: With the recent increase in number of mass shootings in school campuses, there is a need to identify, locate and stop a weapon carrier on campus as soon as possible before he can start shooting. Face Search will enable the authorities to locate the suspect and trace his movements within seconds using multiple camera feeds from different areas on campus. Only one clear image of the suspect’s face is sufficient.

In the race of technology development in response to business needs and security concerns, AI Face Search is a simple, lightweight solution for airports, shopping centres, schools, resorts, etc. to increase our efficiency, minimise manual effort in searching for people when incidents occur on site, and actively prevent potential incidents from occurring.

By Paul Sun, CEO of IronYun, and Mai Truong, Marketing Manager of IronYun

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

In case you missed it

Managing security during unprecedented times of home working
Managing security during unprecedented times of home working

Companies are following government guidance and getting as many people as possible working from home. Some companies will have resisted home working in the past, but I’m certain that the sceptics will find that people can be productive with the right tools no matter where they are. A temporary solution will become permanent. But getting it right means managing risk. Access is king In a typical office with an on-premise data centre, the IT department has complete control over network access, internal networks, data, and applications. The remote worker, on the other hand, is mobile. He or she can work from anywhere using a VPN. Until just recently this will have been from somewhere like a local coffee shop, possibly using a wireless network to access the company network and essential applications. CV-19 means that huge numbers of people are getting access to the same desktop and files, and collaborative communication toolsBut as we know, CV-19 means that huge numbers of people are getting access to the same desktop and files, applications and collaborative communication tools that they do on a regular basis from the office or on the train. Indeed, the new generation of video conferencing technologies come very close to providing an “almost there” feeling. Hackers lie in wait Hackers are waiting for a wrong move amongst the panic, and they will look for ways to compromise critical servers. Less than a month ago, we emerged from a period of chaos. For months hackers had been exploiting a vulnerability in VPN products from Pulse Secure, Fortinet, Palo Alto Networks, and Citrix. Patches were provided by vendors, and either companies applied the patch or withdrew remote access. As a result, the problem of attacks died back.  But as companies race to get people working from home, they must ensure special care is taken to ensure the patches are done before switching VPNs on. That’s because remote desktop protocol (RDP) has been for the most part of 2019, and continues to be, the most important attack vector for ransomware. Managing a ransomware attack on top of everything else would certainly give you sleepless nights. As companies race to get people working from home, they must ensure special care is taken to ensure the patches are done before switching VPNs on Hackers are waiting for a wrong move amongst the panic, and they will look for ways to compromise critical serversExposing new services makes them also susceptible to denial of service attacks. Such attacks create large volumes of fake traffic to saturate the available capacity of the internet connection. They can also be used to attack the intricacies of the VPN protocol. A flow as little as 1Mbps can perturbate the VPN service and knock it offline. CIOs, therefore, need to acknowledge that introducing or extending home working broadens the attack surface. So now more than ever it’s vital to adapt risk models. You can’t roll out new services with an emphasis on access and usability and not consider security. You simply won’t survive otherwise. Social engineering Aside from securing VPNs, what else should CIO and CTOs be doing to ensure security? The first thing to do is to look at employee behaviour, starting with passwords. It’s highly recommended that strong password hygiene or some form of multi-factor authentication (MFA) is imposed. Best practice would be to get all employees to reset their passwords as they connect remotely and force them to choose a new password that complies with strong password complexity guidelines.  As we know, people have a habit of reusing their passwords for one or more online services – services that might have fallen victim to a breach. Hackers will happily It’s highly recommended that strong password hygiene or some form of multi-factor authentication (MFA) is imposedleverage these breaches because it is such easy and rich pickings. Secondly, the inherent fear of the virus makes for perfect conditions for hackers. Sadly, a lot of phishing campaigns are already luring people in with the promise of important or breaking information on COVID-19. In the UK alone, coronavirus scams cost victims over £800,000 in February 2020. A staggering number that can only go up. That’s why CIOs need to remind everyone in the company of the risks of clickbait and comment spamming - the most popular and obvious bot techniques for infiltrating a network. Notorious hacking attempts And as any security specialist will tell you, some people have no ethics and will exploit the horrendous repercussions of CV-19. In January we saw just how unscrupulous hackers are when they started leveraging public fear of the virus to spread the notorious Emotet malware. Emotet, first detected in 2014, is a banking trojan that primarily spreads through ‘malspam’ and attempts to sneak into computers to steal sensitive and private information. In addition, in early February the Maze ransomware crippled more than 230 workstations of the New Jersey Medical Diagnostics Lab and when they refused to pay, the vicious attackers leaked 9.5GB or research data in an attempt to force negotiations. And in March, an elite hacking group tried to breach the World Health Organization (WHO). It was just one of the many attempts on WHO and healthcare organisations in general since the pandemic broke. We’ll see lots more opportunist attacks like this in the coming months.   More speed less haste In March, an elite hacking group tried to breach the World Health Organization (WHO). It was just one of the many attempts on WHOFinally, we also have bots to contend with. We’ve yet to see reports of fake news content generated by machines, but we know there’s a high probability it will happen. Spambots are already creating pharmaceutical spam campaigns thriving on the buying behaviour of people in times of fear from infection. Using comment spamming – where comments are tactically placed in the comments following an update or news story - the bots take advantage of the popularity of the Google search term ‘Coronavirus’ to increase the visibility and ranking of sites and products in search results. There is clearly much for CIOs to think about, but it is possible to secure a network by applying some well thought through tactics. I believe it comes down to having a ‘more speed, less haste’ approach to rolling out, scaling up and integrating technologies for home working, but above all, it should be mixed with an employee education programme. As in reality, great technology and a coherent security strategy will never work if it is undermined by the poor practices of employees.

Security technology and AI: A powerful duo in the fight against COVID-19
Security technology and AI: A powerful duo in the fight against COVID-19

A person infected with the Coronavirus (COVID-19) infects an average of 2.5 other people within five days. You do not need to be a mathematician to realise that early detection of infected people is key to successful pandemic containment. The aim of effective containment strategies is therefore not so much to reduce the number of absolute cases as it is to extend the time frame within which they occur. Without effective containment measures, the virus spreads rapidly and is beyond the capacity of the health care system. However, if infection rates can be minimised through early detection and rapid, targeted identification of further infections, cases will continue to occur over a longer period of time and remain within the capacity of the health care system. Identifying, testing and results For example, the goal of many countries is to carry out as many Corona tests as possible to quickly identify infected people. It is then necessary to identify and reach potentially-infected people and isolate them in quarantine. This is a tried and tested procedure. But this method also costs valuable time in the fight against the virus and has many unknowns. The determination of a concrete test result alone sometimes takes up to 48 hours due to limited laboratory capacity. Added to this is the imprecise and slow procedure for determining contact persons. Or do you still remember exactly who and where you shook hands with in the last ten days - and could you provide information on this? Security technology to the rescue When it comes to the time factor, security technology can be a great help. Thermal imaging cameras and temperature sensors, for example, can help to detect a person with elevated body temperatures. Fever can also be one of the symptoms in those infected with the Coronavirus. At neuralgic points such as airports and train stations, or at entrances to hospitals, thermal imaging cameras can quickly reveal which people have fever. Presumably infected people can be easily separated and asked about other symptoms. Physical security technology can make a great contribution here. Dr. Frank Gillert, a professor at the University of Applied Sciences in Wildau, Germany states, however, as one of the leading scientists for logistics-centric security research, he demands "rapid innovation in dealing with situations like COVID-19 should be a priority". He sees enormous potential in the possibilities of IT and artificial intelligence; "We should use the disruptive changes that are currently taking place and that are challenging global orders to strengthen the significance in IT infrastructure development and also in security technology development.“ The goal in a global crisis And he is right: In global crises such as the Corona pandemic, security-related deficits become apparent and space is created for technical innovations. The goal of governments and companies is to restore security and save human lives as quickly as possible. The German data analytics powerhouse G2K, for example, has developed a Corona Detection & Containment System (CDCS) that is ready for immediate use in record time. Detection takes place in combination with AI-supported data analysis to specifically identify virus hotspots and distribution routes, as well as to identify other potentially infected persons. When developing the system, the focus was on two questions: How do I detect a suspected infected person in crowded environments and even more importantly, how do I quickly and comprehensively determine the person's contacts and previous whereabouts, and find correlations and patterns in this information? The data experts of the Berlin-based company found the answer in the combination of physical security technology and their existing data analytics platform. The G2K system The system is based on G2K's scalable IoT platform "Situational Awareness Builder" (SAB), which is already in use in several projects worldwide and sets standards in process automation and process optimisation, including security management. As soon as a person with fever is detected by the system, he or she can be immediately screened to avoid contact with other people and thus prevent possible new infections, i.e. to interrupt the chain of infection. For this purpose, stationary thermal imaging cameras or smartphones equipped with a temperature sensor accessory can be used. The potentially infected person must now be registered and referred to a doctor or hospital for further specific diagnostic measures. The entire process is covered by a mobile G2K application. A combination of security and medicine The platform can bring together available hospital capacity, infection reports, movement and contact profiles and provide an excellent picture of the source of infection. Thus, medically necessary isolations can be implemented quickly. At the same time, infected patients can use the app to document their recovery and become actively involved. All this data is centrally managed and analysed, using deep learning methods. This provides crisis managers with a single monitoring, control and resource management tool that enables immediate action to be taken to combat the spread of the virus and gives officials full transparency on the status of the pandemic. Karsten Neugebauer, founder and CEO of the company behind the solution, explains his commitment as follows "A few weeks ago we too were faced with increasing difficulties due to the Corona crisis. As we have a strong presence in Europe in particular, we had to struggle with postponed project starts and limited resources". But instead of burying their heads in the sand, G2K's dedicated team decided to declare war on the virus." "In our entrepreneurial duty, we, therefore, decided to use our available technology and equip it to fight COVID-19. Our team has been working day and night over the last few weeks to expand our software platform to enable us to contain the pandemic quickly and effectively. Politicians must now immediately push ahead with the unbureaucratic implementation of prevention and control measures such as our CDCS to ensure the stability of our public systems," demands Karsten Neugebauer. The pandemic continues As the COVID-19 pandemic spreads from continent to continent, researchers around the world are working to develop antidotes to the virus. As long as this has not been found, the spread of the virus must be slowed down internationally. Only by this can system-relevant infrastructure be held consistently. Combining modern physical security technology with platform technology and artificial intelligence provides an excellent possibility to slow down the current and for sure, future pandemics.

Emergency response and notification systems: Crucial for improving hospital security
Emergency response and notification systems: Crucial for improving hospital security

When violence or a life-threatening incident occurs, hospitals and other healthcare institutions are often in the crosshairs. Hospitals increasingly face a reality of workplace violence, attacks on patients, and threats to doctors and other support staff. And even if violence happens outside a hospital – such as an active shooter at a public place – the local hospital must be prepared to respond to an influx of injured victims. When conflicts arise inside a hospital, there is an urgent need to lock the facility down quickly. Security professionals and their teams need access control options that allow lockdowns to occur at the touch of a button. Lockdown capabilities are an important aspect of safety and security for hospitals, doctor’s offices and medical facilities The need for mass notification is also growing in the healthcare environment Fire alarm public address system The need for mass notification – another aspect of responding in an emergency – is also growing in the healthcare environment. Various systems can communicate through the fire alarm public address (PA) system to notify people in an emergency, or, alternately, to use email notification, text messaging, pagers, smart phones and/or personal computers (PCs). In lockdown situations, access control systems provide an emergency button with various triggers in the system – a hospital can lockdown specific units or the entire facility. Data capture form to appear here! Jim Stankevich, Global Manager – Healthcare Security, Johnson Controls/Tyco Security Products, points out that the safety of hospital staff, particularly nurses, cannot be overlooked. In the emergency room, 55 percent of nurses are assaulted in some way each year, which is a high percentage. The safety of nurses and all hospital staff deserves more attention. Duress/emergency notification technology Stankevich says one solution is to use duress/emergency notification technology: staff can carry and wear a ‘panic button” or have a two-key combination on their computer as an alarm trigger. When the staff member hits the panic button, a direct message can be sent to security, alerting security staff about the event and requiring a response. There has been an increase in demand for the safety and security of patients, staff and visitors at healthcare institutions, as evidenced by the recent CMS (Centers for Medicare and Medicaid Services) Emergency Preparedness Rule. As of Nov. 17, 2017, healthcare institutions that participate in Medicare or Medicaid must demonstrate compliance with the rule. Emergency preparedness systems A major challenge in compliance to this rule is balancing patient safety with comfort At its core, the rule seeks to establish national emergency preparedness requirements to ensure adequate planning for both natural and man-made disasters, and coordination with federal, state, tribal, regional and local emergency preparedness systems. A major challenge in compliance to this rule is balancing patient safety with comfort. Institutions should consider two-way communication that enables leadership to disseminate targeted messages quickly and efficiently, while arming all employees with a tool that can alert the appropriate staff should an incident occur. Solutions like this enable swift communication of issues without disturbing patients and visitors unless necessary. Effective response to emergencies “Fortunately, hospitals and their security departments are generally well equipped to respond to most emergency situations”, said John M. White, president/CEO of Protection Management, a consultant who works with hospitals to address their security needs. During the Ebola scare in 2014, however, hospitals had to re-examine their plans to ensure they were prepared to meet the challenges specific to rare and deadly disease. “Hospitals are prepared for most things, but Ebola seemed to have caught the whole world off guard, so people responded in different ways,” says White, who previously was security director of two multi-campus medical facilities before becoming a consultant. Hospital security Hospitals made adjustments to their emergency programs to determine how best to handle Ebola patients" He adds, “Hospitals made adjustments to their emergency programs to determine how best to handle Ebola patients and to protect other patients and staff. It was a new threat that healthcare organisations had not specifically addressed.” A particular concern was the possibility of an infected person walking into an emergency room and infecting other people and/or requiring facility decontamination. One role the hospital security department plays in such an emergency is to control access to the facility and to control visitors’ movements once they are inside the facility, says White. If the Ebola scare had progressed to the point that a hospital would need to screen patients, security would be positioned at the front entrance to help with that screening and, if necessary, to direct patients to a specific area for quarantine. Protective equipment Security might also need to wear protective equipment to handle a patient who is resistant to treatment, for example. There are often interactions between security personnel and the general public, a scenario that becomes more complicated if Ebola or a similar infection is likely. In general, security would be tasked with maintaining order and keeping people where they need to be, freeing up the medical professionals to do their jobs more efficiently, says White. To prepare for the impact of the Ebola scare, hospitals addressed various training and equipment needs and adjusted their disaster/emergency response plans. Read parts two and three of our heathcare mini series here and here.