Facial recognition has a long history dating back to the 1800s. To track down criminals, such as infamous bandits Jesse Woodson James and Billy the Kid, law enforcement would place “Wanted Alive or Dead” posters advertising bounties and soliciting public cooperation to help locate and even apprehend the alleged criminals. In addition to the bounty, these posters would include a photo and brief description of the crime, which would then be circulated to law enforcement agencies around the country and displayed in every US Post Office to speed up apprehension.

Facial recognition

Advancements in artificial intelligence and biometric technology have led to the widespread use of computerised facial recognition

Today, technology such as social media, television and other more specialised communication networks play a more influential role in the recognition process. Advancements in artificial intelligence and biometric technology, including the development of Machine Learning capabilities, have led to increased accuracy, accessibility and the widespread use of computerised facial recognition. The significance of this means that facial recognition can occur on an even larger scale and in more challenging environments.

This article will explore key milestones and technological advances that have resulted in the modern incarnation of facial recognition, before discussing the capabilities of cutting-edge “one-to-many” technology which is increasingly being used by counter-terror defence, police and security forces around the world.

Technology inception and developments

The technology was able to match 40 faces an hour, which was considered very impressive at the time

The 1960s marked the start of computerised facial recognition, when Woodrow Wilson (Woody) Bledsoe developed a way to classify faces using gridlines. Bledsoe’s facial recognition still required a large amount of human involvement because a person had to extract the co-ordinates of the face’s features from a photograph and enter this information into a computer. The technology was able to match 40 faces an hour (each face took approximately 90 seconds to be matched) which was considered very impressive at the time.

By the end of the 1960s, facial recognition had seen further development at the Stanford Research Institute where the technology proved to outperform humans in terms of accuracy of recognition (humans are notoriously bad at recognising people they don’t know). By the end of the century, the leading player in the field was a solution that came out of the University of Bochum in Germany – and the accuracy of this technology was such that it was even sold on to bank and airport customers.

From this stage on, the facial recognition market began to blossom, with error rates of automatic facial recognition systems decreasing by a factor of 272 from 1993 to 2010 according to US Government-sponsored evaluations.

The safety and security market requires near instant feedback on who a person matched against a watchlist is
The aim for facial technology is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware

Modern usage of facial recognition

Fast-forward to the modern day and facial recognition has become a familiar technology when using applications such as the iPhone X’s Face ID capability or MasterCard Identity Check, passport e-gates at airports and other security and access control points. These solutions implement a consensual form of identity verification, as the user has a vested interest in being identified.

This is a “one-to-one” facial recognition event, one person in front of the camera being compared to one identity either on a passport or the app. In these scenarios, the hardware is specifically developed for the application at hand, therefore technically much easier to accomplish.

Facial recognition can now be used in a variety of governmental and commercial environments The safety and security world brings a much more complex problem to solve – how to pick out a face in a moving and changing environment and compare it to several faces of interest. “One-to-many” facial recognition is a much harder problem to solve.

It’s even more challenging when the aim is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware. And unlike in the 1960’s where identifying a face every 90 seconds was acceptable; the safety and security market requires near instant feedback on who a person matched against a watchlist is.

Security and safety applications

The idea behind all facial recognition technologies is broadly the same: you start with an image of a person’s face (ideally a high quality one, although machine learning means that to a point we can now even use video without reducing accuracy). A fully front facing image is best, think a passport photo, but machine learning and new software has made this more flexible.

An algorithm converts this image into a numeric template, which cannot be converted back to an image and so represents a secure one way system. Every numeric template is different, even if it started out as an image of the same person, although templates from the same person are more similar than templates from different people.

The accuracy of facial recognition continues to increase alongside deployments in more challenging and complex environments What happens next sounds simple although the technology is extremely complex: templates of people’s faces are taken in real time and compared to those in the database. The technology identifies individuals by matching the numeric template of their face with all the templates saved in a database in a matter of seconds or milliseconds. To put this into perspective, imagine you are at the turnstiles of a busy train station looking for a person on the run.

Today’s facial recognition technology would be able to identify that person should they pass in view of a CCTV camera, as well as notify the police of any additional persons of interest, whether they are a known terrorist or missing vulnerable person on an entirely separate watch list.

Because of technical progression, facial recognition can now be used in a variety of governmental and commercial environments, from identifying barred hooligans attempting entry at a football stadium or helping self-excluded gamblers at casino to overcome addiction.

Real-time assessments

The latest evolution of facial recognition pits the technology against an even more challenging application – directly matching individuals from body worn cameras for real time recognition for police officers on the beat. This capability equips first responders with the ability to detect a person from a photo and verify their identity with assurance.

The broader implication for this means that every interaction, such as stop and search or arrest, can be supported by real-time facial recognition which will see cases of mistaken identity driven down on the streets. First responders can now for the first time be deployed and furnished with the ability to identify wider groups of people of interest with a degree of accuracy that previously relied only on the fallible human memory.

As the accuracy of the technology continues to increase alongside deployments in more challenging and complex environments, its ability to support government initiatives and law enforcement means the debate about the lawful and appropriate use of facial recognition must be addressed. Facial recognition should not be everywhere looking for everyone, but when used properly it has the potential to improve public safety and we should make the most of its potential.

Download PDF version

Author profile

In case you missed it

Virtual worlds disrupt building security & facility management
Virtual worlds disrupt building security & facility management

From satellite imagery to street views to indoor mapping, technology has disrupted our past world. This has left us dependent upon new ways to visualise large spaces. This new world has brought many benefits and risks. But what does that mean for the security professional or facility manager today and what technologies can be used to secure buildings and improve facility operations? A brief history of 3D technology Starting May 5, 2007 (inception 2001), Google rolled out Google Street View to augment Google Maps and Google Earth; documenting some of the most remote places on earth using a mix of sensors (Lidar/GSP/Radar/Imagery). The mission to map the world moved indoors May 2011 with Google Business Photos mapping indoor spaces with low cost 360° cameras under the Trusted Photographer program. In the earlier days, 3D scanning required a high level of specialisation, expensive hardware and unavailable computing power With the growth of 3D laser scanning from 2007 onwards, the professional world embraced scanning as effective method to create digitised building information modelling (BIM), growing fast since 2007. BIM from scanning brought tremendous control, time and cost savings through the design and construction process, where As-Built documentation offered an incredible way to manage large existing facilities while reducing costly site visits. In the earlier days, 3D scanning required a high level of specialisation, expensive hardware, unavailable computing power and knowledge of architectural software. Innovation during the past 8 year, have driven ease of use and lower pricing to encourage market adoption. Major investments in UAVs in 2014 and the commercial emergence of 360° photography began a new wave of adoption. While 3D scanners still range from $20K – $100K USD, UAVs can be purchased for under $1K USD and 360° cameras for as low as $100. UAVs and 360° cameras also offer a way to document large spaces in a fraction of the time of terrestrial laser scanners with very little technical knowledge.  Access to building plans, satellite imagery, Google Street View, indoor virtual tours and aerial drone reconnaissance prove effective tools to bad actors The result over the past 10+ years of technology advancement has been a faster, lower cost, more accessible way to create virtual spaces. However, the technology advances carry a major risk of misuse by bad actors at the same time. What was once reserved to military personal is now available publicly. Access to building plans, satellite imagery, Google Street View, indoor virtual tours and aerial drone reconnaissance prove effective tools to bad actors. Al Qaeda terror threats using Google Maps, 2007 UK troops hit by terrorists in Basra, 2008 Mumbai India attacks, 2016 Pakistan Pathankot airbase attacks, ISIS attacks in Syria using UAVs, well-planned US school shootings and high casualty attacks show evidence that bad actors frequently leverage these mapping technologies to plan their attacks. The weaponization of UAVs is of particular concern to the Department of Homeland Security: "We continue to face one of the most challenging threat environments since 9/11, as foreign terrorist organisations exploit the internet to inspire, enable or direct individuals already here in the homeland to commit terrorist acts."   Example comparison of reality capture on the left of BIM on the right. A $250 USD 360° camera was used for the capture in VisualPlan.net software What does this mean for the security or facility manager today? An often overlooked, but critical vulnerability to security and facility managers is relying on inaccurate drawing. Most facilities managers today work with outdated 2D plan diagrams or old blueprints which are difficult to update and share.Critical vulnerability to security and facility managers is relying on inaccurate drawing Renovations, design changes and office layout changes leave facility managers with the wrong information, and even worse is that the wrong information is shared with outside consultants who plan major projects around outdated or wrong plans. This leads to costly mistakes and increased timelines on facility projects.  Example benefits of BIM There could be evidence of a suspect water value leak which using BIM could be located and then identified in the model without physical inspection; listing a part number, model, size and manufacture. Identification of vulnerabilities can dramatically help during a building emergency. First Responders rely on facilities managers to keep them updated on building plans and they must have immediate access to important building information in the event of a critical incident. Exits and entrances, suppression equipment, access control, ventilation systems, gas and explosives, hazmat, water systems, survival equipment and many other details must be at their fingertips. In an emergency situation this can be a matter of life or death. Example benefit of reality capture First Responders rely on facilities managers to keep them updated on building plans A simple 360° walk-through can help first responders with incident preparedness if shared by the facility manager. Police, fire and EMS can visually walk the building, locating all critical features they will need knowledge of in an emergency without ever visiting the building. You don’t require construction accuracy for this type of visual sharing. This is a solution and service we offer as a company today. Reality capture is rapidly becoming the benchmark for facility documentation and the basis from which a security plan can be built. Given the appropriate software, plans can be easily updated and shared.  They can be used for design and implementation of equipment, training of personnel and virtual audits of systems or security assessments by outside professionals. Our brains process visual information thousands of times faster than text. Not only that, we are much more likely to remember it once we do see it. Reality capture can help reduce the need for physical inspections, walk-throughs and vendor site-visits but more importantly, it provides a way to visually communicate far more effectively and accurately than before. But be careful with this information. You must prevent critical information falling into the hands of bad actors. You must watch out for bad actors attempting to use reality capture as a threat, especially photo/video/drones or digital information and plans that are posted publicly. Have a security protocol to prevent and confront individuals taking photos or video on property or flying suspect drones near your facility and report to the authorities. Require authorisation before capturing building information and understand what the information will be used for and by who.There are a number of technologies to combat nefarious use of UAVs today Nefarious use of UAVs There are a number of technologies to combat nefarious use of UAVs today, such as radio frequency blockers and jammers, drone guns to down UAVs, detection or monitoring systems. Other biometrics technologies like facial recognition are being employed to counter the risk from UAVs by targeting the potential operators. UAVs are being used to spy and monitor for corporate espionage and stealing intellectual property. They are also used for monitoring security patrols for the purpose of burglary. UAVs have been used for transport and delivery of dangerous goods, delivering weapons and contraband and have the ability to be weaponised to carry a payload.Investigating reality capture to help with accurate planning and visualisation of facilities is well worth the time The Federal Aviation Administration has prevented UAV flights over large event stadiums, prisons and coast guard bases based on the risks they could potentially pose, but waivers do exist. Be aware that it is illegal today to use most of these technologies and downing a UAV, if you are not Department of Justice or Homeland Security, could carry hefty penalties. Facility managers must have a way to survey and monitor their buildings for threats and report suspicious UAV behaviours immediately to authorities. At the same time, it’s critical to identify various potential risks to your wider team to ensure awareness and reporting is handled effectively. Having a procedure on how identify and report is important. Investigating reality capture to help with accurate planning and visualisation of facilities is well worth the time. It can help better secure your facilities while increasing efficiencies of building operations. Reality capture can also help collaboration with first responders and outside professionals without ever having to step a foot in the door. But secure your data and have a plan for bad actors who will try to use the same technologies for nefarious goals.

Intellectual honesty: the growth of Cobalt Robotics and robots in security
Intellectual honesty: the growth of Cobalt Robotics and robots in security

The best route to greater adoption of robotics in the field of physical security is intellectual honesty, says Travis Deyle, CEO and co-founder of Cobalt Robotics. “Robots are not a panacea, so we must be clear and honest about capabilities and use cases,” he says. “If you are dishonest, people will lose faith. We must have clear expectations about what’s feasible today and possible tomorrow.” The robotics tide is turning in the security market, which is notoriously slow to embrace new technologies. “The tone has changed at recent security events,” says Deyle. “Previously, robots were thought of as a science experiment. But now, there are big-name users wanting to discuss proof of concept. It has evolved from being a novelty to now it’s time to give it a serious look. They want us to help them sell the concept up the chain of command. It’s helpful to have conversations with other parts of the company because it has an impact on the culture of the company.” The robotics tide is turning in the security market, which is notoriously slow to embrace new technologies Cobalt’s robots are purpose-built for a specific use case: providing after-hours support and security for corporate locations. Indoor environments, confined and controlled, present fewer navigation challenges for robots, which can quickly become familiar with the surroundings and navigate easily through an office space. Indoor robots can provide benefits beyond security, too, such as facility management, promoting employee health and safety, and emergency response. Cobalt's human-centred design Cobalt’s robots also interact well with people. They are friendly and approachable and make employees feel safe and secure. The human-centered design promotes that interaction, and a real person (located remotely) can enter into any interaction instantly as needed. “We combine machines with people,” says Deyle. “We allow the machine to do what it does best, such as dull and boring activities, and add the flexibility and cultural relevancy of having a person there.” Cobalt’s robots also interact well with people, they are friendly and approachable and make employees feel safe and secure When a robot is deployed, it performs a brief mapping phase (about an hour), in which it moves around and builds up a “map” of its space and develops its patrol route. Over time, it lingers more in areas where it encounters more incidents. There are 60 sensors on the robot, including day/night cameras, high-resolution thermal cameras, a card reader that integrates with the corporate access control system, a microphone, and environmental sensors for temperature and humidity.  The robot builds models of what’s normal in its environment in terms of people, sound, motion, open doors and windows, and even leaks and spills. And then it detects anomalies and sends relevant notifications to Cobalt specialists, who respond and manage any events in real time. The machine provides unwavering attention, perfect recall, and accountability. Cobalt robots have been designed to help bridge the problems faced with utilising guards and cameras  Accommodating various anomalies  The Cobalt robot is designed to blend into a high-end office environment, with flexible fabric and a corporate design aesthetic. It is stable beyond 45-degrees, so it’s hard to topple over. The 5-foot-2-inch robot can see over desks and cubicles. It is designed to bridge the gap between guards, who are expensive and underutilised during uneventful night shifts, and cameras, which are unable to respond to nuanced situations. Cobalt Robotics already has customers in defense, finance and manufacturing, and a handful of Fortune 500 companies are looking at the service Autonomous navigation uses artificial intelligence (AI) and machine learning to avoid static and dynamic obstacles. Over time, the robot accommodates various anomalies such as loud machinery noise, and “semantic mapping” adds intelligence to its map. When the robot figures out that a picture on the wall is not a real person, for example, it stores that information for future reference.  The technologies enabling robotics in the indoor environment are mature – there have been variations of security robots in operation for decades. What has changed is the costs of the technologies, which are now inexpensive enough to make a robot affordable to businesses. Cobalt Robotics offers an all-inclusive service providing hardware, software, service and maintenance as well as the remote human interface. All together, the service is a third to half the cost of a man-guard, and it bills monthly, says Deyle. Cobalt Robotics offers an all-inclusive service providing hardware, software, service and maintenance as well as the remote human interface Cobalt Robotics already has customers in defense, finance and manufacturing, and a handful of Fortune 500 companies are looking at the service. They are currently operational in the San Francisco Bay area and Chicago and will be in six other geographies in the next three months (in response to customer needs). Uses include offices, museums, warehouses, technology centres, and innovation centres.  A former Google employee, Deyle’s experience in robotics goes back to his Ph.D. studies at Georgia Tech, where he worked on developing a robot to deliver healthcare to homebound patients. Deyle and Cobalt Robotics co-founder Erik Schluntz departed Google in 2016 to form Cobalt Robotics. In just 12 months, Cobalt went from the initial idea to paid robot deployments.

Which segments are under-served in the physical security industry?
Which segments are under-served in the physical security industry?

Physical security technologies operate successfully in many different markets, but in which markets do they fall short? Physical security is a difficult challenge that can sometime defy the best efforts of manufacturers, integrators and end users. This is especially the case in some of the more problematic markets and applications where even the best technology has to offer may not be good enough, or could it be that the best technology has not been adequately applied? We asked this week’s Expert Panel Roundtable to reflect on instances when the industry may fall short: Which segments of the physical security industry are most under-served and why?