A tipping point is defined as: “The point at which a series of changes becomes significant enough to cause a larger, more important change”. In the same way that IP video changed surveillance a decade ago, our industry is now feeling the impact of recent developments in Artificial Intelligence, Machine Learning, Deep Learning, Big Data, and Intelligent Video Analysis.

Keyword definitions

Let’s start with a few more definitions. Artificial Intelligence (AI) deals with the simulation of intelligent behaviour in computers. Machine Learning (ML) deals with developing computer algorithms that access data and use it to learn for themselves. Neural networks are computer systems that loosely mimic human brain operation.

Deep Learning is a subset of ML based on neural networks that has been proven to provide breakthrough capabilities in many problems that were previously unsolvable, and Big Data, or metadata, refers to huge amounts of structured and/or unstructured data -- in our case, the immense quantities of video information being generated daily by security cameras deployed in cities around the world. Deep Learning is tipped to change Intelligent Video Analysis (IVA), the digital video technology integrated with analytical software that is a basic tool for our industry.

AI in surveillance

Traditionally, the main benefit of surveillance cameras is the ability to collect evidence for debriefing or investigation, as well as the ability to view events remotely in real-time. A decade ago, video analytics technologies were introduced to solve the problem of human inattention -- computers don’t get tired, bored or distracted, and can monitor a camera continuously.

And then, camera costs dropped, deployment skyrocketed, and video management systems began collecting reams of useless, costly unstructured data. AI technology seemed to answer the pressing new industry needs of how to use this Big Data effectively, make a return on the investment in expensive storage, while maintaining (or even lowering) human capital costs

Three limiting factors

All this was theoretical, however, as multiple technological barriers prevented AI solutions from real-world utilisation. Despite decades of research on how to cause a computer to accurately recognise different objects in a video stream, the quality of the results, especially in urban environments, was, to put it mildly, underwhelming.

Deep Learning software must be able to differentiate between different objects and under various circumstances
Deep Learning has matured to the point where it can accurately detect and classify objects both in still images and in video

AI was limited primarily by these three factors: 

  • Lack of understanding -- The software must be able to differentiate between different objects (person, vehicle, animal, etc.), and under various circumstances (day, night, seasonal weather conditions, etc.).

  • Inability to learn -- Traditional IVA applications relied on a rule-based approach that required software configuration -- by a human operator -- for each monitoring camera and each type of alert. Although effective in some scenarios, the exponential growth in camera counts rendered this approach impractical, given the amount of manual labor required to configure, reconfigure, and maintain rules.

  • High cost -- The hard truth is that budgets for security and safety will always be constrained. Until recently, implementing real-time AI was extremely cost-prohibitive, sometimes requiring a 1:1 server to camera ratio.

Meeting challenges

That was yesterday. Today, the application of AI in security applications has reached its tipping point, meeting the above-mentioned challenges. 

  • Understanding -- Deep Learning has matured to the point where it can accurately detect and classify objects both in still images and in video. DL technology is fast becoming the basic building block for IVA.

  • Ability to Learn -- As an AI solution collects and analyses data over time, it creates metadata that describes all objects in each video stream. Machine Learning techniques process this metadata to generate models for “normally observed” behaviour. These models are applied in real-time to detect behaviours deviating from the norm. Only those flagged as suspicious events require review by a human operator. This technique allows the solution to scale to an unlimited number of cameras, with no need for a human to configure each new device.

  • Lower cost -- The rapid increase in GPU computational capacity, coupled with mass market adoption, has lowered server costs to a reasonable level. Today, with the correct implementation, a single server can be deployed across hundreds and even thousands of cameras.

The convergence of Deep Learning for video analysis, advances in AI for fully automated event detection, plus the significant reduction in cost to implement these techniques – including cloud-based software as a service (SaaS) models -- means that the fully automated video surveillance solution for cities is fast becoming a reality. We’ll see more of this type of solution being deployed over the coming months, and within the next few years, it will be standard in any Smart City deployment.

Download PDF version

Author profile

In case you missed it

What are the security challenges of the hospitality market?
What are the security challenges of the hospitality market?

Hospitality businesses work to provide a safe and pleasant customer experience for their guests. Hotels offer a “home away from home” for millions of guests every day around the world. These are businesses of many sizes and types, providing services ranging from luxury accommodations to simple lodging for business travelers to family vacation experiences. Hospitality businesses also include restaurants, bars, movie theaters and other venues. Security needs are varied and require technologies that span a wide spectrum. We asked this week’s Expert Panel Roundtable: What are the security challenges of the hospitality market?

Physical and cyber security precautions when travelling
Physical and cyber security precautions when travelling

Surveillance systems can track the locations of mobile phone users and spy on their calls, texts and data streams. The Washington Post has reported on such systems that are being turned against travellers around the world, according to security experts and U.S. officials. The summer season highlights the need to take extra precautions when travelling. When travelling anywhere in the world, for business or pleasure, citizens need to be aware of and alert to looming physical and cybersecurity threats. To elaborate on expert security tips, strategies and advice for traveling this summer, we presented several questions to The Chertoff Group, a global security advisory firm that enables clients to navigate changes in security risk, technology and policy. Chris Duvall, Senior Director at The Chertoff Group, offers insights into cybersecurity concerns, physical security precautions, and recommends digital resources/apps for consumers while traveling. Q: How are security risks – physical and digital – changing? Why are threats greater today than five years ago? The exponential number of headlines over the past few years is a strong indication that both physical and digital risks are evolving and increasing Duvall: The exponential number of headlines over the past few years is a strong indication that both physical and digital risks are evolving and increasing. The scope, severity and complexity of physical and cyber risks are increasing and becoming more dangerous and destructive. This is especially true for those travelling outside the U.S. On the physical side, threat actors are actively seeking “soft targets” – public events, social settings, mass audience venues, etc. – to communicate their message, sow chaos and inflict catastrophic harm. On the digital or cyber side, we have seen a shift from “thrill hacking,” to an increase of “hacking as a business” (through credential compromise and ransomware), to an increase in “hacking for harm” - with the rise of “nuke ware” and ransomware without a clear financial motivation. Q. What specific precautions should a traveller take to protect their calls, texts and data streams from being spied on? Duvall: When travelling abroad, we recommend to our clients that their personnel and executives should practice good internet and social media hygiene. Some best practices include: Avoid using public Wi-Fi services—unless you use private VPN service for encryption Increase the privacy setting on your technical devices Disable location identifiers on apps Create a new (unlinked) email for internet correspondence Consider purchasing international MyFi devices to decrease the risk of getting your personal identification information (PII) or protected healthcare information (PHI) stolen  Use temporary (i.e. burner) phones to protect your data and your contacts Q. What cybersecurity concerns are likely to impact travellers? Are the threats greater outside the United States or in any specific parts of the world? Significant precautions should be taken to protect personal electronic devices (PEDs) and the data connected to PEDs Duvall: The international cybersecurity landscape has grown increasingly dynamic, with threats posed by government authorities (in some countries), terrorists, insurgents, and criminals, requiring travelers to be proactive and vigilant. U.S. citizens, particularly executives of U.S.-based technology companies, must be aware that they are considered high-value targets for nation-state intelligence services and criminally-motivated bad actors. Many countries will go to great lengths and expense to acquire and exploit proprietary information from U.S.-based companies, and views U.S. executives visiting the country as “soft” targets of opportunity. As such, significant precautions should be taken to protect personal electronic devices (PEDs) and the data connected to PEDs. The tactics, techniques and procedures (TTPs) utilised by bad actors are often covert and nearly undetectable by the affected person. Threat actors routinely access, monitor and utilise Wi-Fi networks at hotels and in public spaces to compromise target devices. Other targeting methods include luggage searches, extensive questioning, and unnecessary inspection and downloading of information from personal electronic devices. There are numerous, high-risk countries for which the U.S. Government warns travellers to be wary of mobile malware, mobile device privacy attacks and hot spots for mobile botnets. The U.S. Department of State has the most recent and up-to-date list. For example, the U.S. Government has investigated numerous incidents in which U.S. travellers’ PEDs (personal and company devices) have been compromised by Russian authorities while transiting Russian airports, left unattended in public spaces and in travellers’ hotel rooms.  When travelling to an unfamiliar place, research your destination to understand the local roads and transportation, geography, local roads, culture, etiquette and laws Q: What physical security precautions should a traveller take? Duvall: Here are some useful precautions: When traveling to an unfamiliar place, research your destination to understand the local roads and transportation, geography, local roads, culture, etiquette and laws. Protect your personal information and travel itinerary as much as possible. Limit the amount of jewelry worn, cash, credit cards and electronic devices carried while traveling. Avoid staying on the ground floor of a hotel. Consider choosing a room on the 2nd through 7th floors as these rooms may be more difficult to break into than those on the ground level, but still able to be accessed by fire/emergency response equipment. Never answer your hotel room door for anyone until you’ve determined who they are, why they are at your door, and if it is necessary for you to open the door to interact with them. Carry a rubber door stop/wedge with you to install on the room side of the door before you go to bed. Vary your patterns and routines when venturing out in to a new location, do not become predictable. Politely decline offers of food or drink from strangers (If you do accept beverages, ensure that they are in sealed containers and that there is no evidence of tampering). Never discuss your itinerary, personal, business or other sensitive information where others can hear you. Q: How can companies be proactive in protecting their business travellers? Companies should educate their employees on the importance of maintaining good internet hygiene while travelling abroad Duvall: When travelling on business, companies should provide their employees with clean computers and cell phones before departure. Upon return, the company should immediately wipe the computer clean to prevent any malicious threats from penetrating the company’s internal, cyber-infrastructure. Additionally, companies should educate their employees on the importance of maintaining good internet hygiene and recommend their employees disconnect from social media platforms while travelling abroad. Some general tips to recommend to your employees when travelling abroad include: Register in the Smart Traveler Enrollment Program (https://step.state.gov/step/) Visit Travel.State.Gov to view travel related information specific to the country or countries you’re visiting, including local US Embassy or Consulate contact information, as well as current travel advisories and alerts. Always leave a copy of your transportation and hotel itinerary and driver’s license (or passport if travelling internationally) with a family member or trusted friend. Always use a baggage tag with a protective cover Avoid using public Wi-Fi services Q: What digital resources and/or apps might a traveller benefit from (and how)? Duvall: The Chertoff Group recommends researching the below travel-related apps before departing on a trip: TravWell: This app provides destination-specific vaccine recommendations, a checklist of what you need to do to prepare for travel, and a customisable healthy travel packing list. The app can store travel documents, keep records of medications and immunisations, and set reminders to get vaccine booster doses or take medicines. My TSA: This app provides real-time updates on airport delays. It includes how long security lines are at various airports; information about what you can and cannot bring onto an airplane; and a frequently-asked question list, including new advanced imaging technology. Border Wait Time: The app provides estimated wait times and open-lane status at land ports of entry, which may be particularly helpful when in an area with multiple crossings. Mobile Pass: The Mobile Passport app speeds you through U.S. Customs and Border Protection at (1) cruise port and (24) airports Q: As a security expert, what’s your best advice for travellers? Duvall: At the end of the day, travel security is not rocket science. Simply put, travellers need to: Be aware and situationally alert at all times. Be aware and situationally alert to the location of your luggage and carry-ons at all times. Don’t access unknown, unsecured or public Wi-Fi if at all possible. Turn off “auto connect” features and institute stringent privacy controls as much as possible. Try to “blend in” – you don’t have to try to look like a local but travellers should avoid gaudy and expensive attire wherever possible. Use your common sense – if an offer, invitation or opportunity seems to good to be true... it probably is.

Mass transit security evolves with modern security solutions
Mass transit security evolves with modern security solutions

As anyone who has ever flown on a commercial airline since 2001 knows, security measures at airports are well enforced and the emphasis on traveller safety is all around the airport and its grounds. Mass transportation, meanwhile, presents a special but not any less significant challenge when it comes to determining security issues. These facilities need to develop the means to protect a constantly changing and large population of passengers. And unlike airports these facilities often have hundreds of points of entry and exit on multiple modes—buses, subways, light rail, commuter trains, even ferries. About 2 million Americans will use the nation’s airways on a given work day, while 35 million people will board some form of public transportation. In fact, statistics have shown that nearly 11 billion trips are taken on public transportation every year. In some large metropolitan areas in North America where mass transit is well established, more than 20 percent of the area’s inhabitants get around via public transportation.About 2 million Americans will use the nation’s airways on a given work day, while 35 million people will board some form of public transportation Solving mass transit security For transportation officials and their security providers, solving the mass transit security issue begins with determining the key concerns and then creating the proper responses via security systems, policies and procedures to mitigate the risks. Although vandalism and graffiti are very visible signs of criminal behaviour in mass transit settings such as bus stops and subway stations, this is not where transportation officials typically focus their energy. Fences and gates can secure out-of-service buses and train cars, as can remote surveillance methods to keep such vandalism at a minimum. Instead, it is the day-to-day safety and security of transit riders and employees that should become the highest priority. This begins with creating the safest environment possible that is highlighted with appropriate signage and, when necessary, audible warnings, and supporting that with technology, such as surveillance cameras, that will document what has happened if an incident occurs.Analytics can also be useful in alerting security about other suspicious behaviours at a transit stop, such as an untended bag or package Crime prevention in transportation Analytics can also be useful in alerting security about other suspicious behaviours at a transit stop, such as an untended bag or package Incidents of concern within a transit setting can take several forms, ranging from legitimate accidents or crimes to false claims such as faked fall down the stairs to potential and actual suicides. Bus and subway stations also have become magnets for homeless people who may put themselves and others in harm’s way by trying to access less secure public areas within a station as temporary shelters. If someone is injured on a subway platform and the transit provider is held liable, it could be on the hook for hundreds of thousands, if not millions of dollars. Suicides are a major concern for operators, with personnel now being trained to look for individuals who seem distressed, are loitering in the area or are intentionally putting themselves in a dangerous situation, such as standing too close to the edge of a platform. The deployment of video analytics, which can be programmed to send alerts when certain pre-set actions occur, can help determine when such dangerous behaviours come into play. Analytics can also be useful in alerting security about other suspicious behaviours at a transit stop, such as an untended bag or package or a person going into a restricted area. Whether it is on the bus, train or ferry or at the stops themselves, cameras and intuitive video management systems are the key to both active and forensic transit security. Some cities use buses that are up to 60 feet long and those can be equipped with up to a dozen cameras Train security and safety By using the proper cameras and recording systems in a transit environment, quick-acting personnel can locate a person of interest who boarded a train at one station, follow him during his trip and produce a crisp, clear identifiable image at the end. Those setting up the system thus should keep in mind proper camera positioning, resolution and motion-based changes to framerates or other compression settings. A typical 30-foot bus often has six cameras—one each at the front and middle doors, two more within the bus and then one looking forward and another looking behind the bus. The latter two are important in the event of accidents to verify liability. Some cities use buses that are up to 60 feet long and those can be equipped with up to a dozen cameras.Train stations often deploy high-definition cameras to better support facial recognition software to get that actionable image Train cars are similarly equipped with two to four cameras to view activity down the centre aisle. Within the stations themselves, there can be from 15 to 30 or more cameras capturing wide-angle shots. Train stations, which have a restricted point of egress, often deploy high-definition cameras to better support facial recognition software to get that actionable image. Installing the right technology for the solution Although bandwidth and storage can be a concern, with motion-based recording, the resolution can be bumped up during event, resulting in a 1-megapixel stream jumping to 4 or even 8mbps when needed. By changing the resolution on demand, end users can cut their storage needs significantly. Transportation settings often rely on the same technology used in other security installations, primarily mini dome cameras, although there are some mini transit domes built specifically for the environment with the proper aesthetics. Because of vandalism threats, transit typically avoids pendant mounts, which can be more easily grabbed and damaged. Temperature ratings for cameras also come into play in cold climates with cameras often getting outdoor exposure.Today’s new buses and trains are constructed with the cameras onboard and newer stations also take security into consideration at the earliest design stage As trains and buses move along their routes, especially those that service outlying areas, Internet connectivity becomes an issue as well. Because it may be difficult for video to be sent in transit, security bus barns are equipped with Wi-Fi so video from onboard cameras can be downloaded at the end of the day. And the use of hardened recorders at the stations allows security personnel to retrieve recorded video. Transit security with modern technology Today’s new buses and trains are constructed with the cameras onboard and newer stations also take security into consideration at the earliest design stage. Older infrastructure from long-standing subway and bus terminals can prove to be a challenge when adding security, but these issues aren’t insurmountable. Often the solution is to add more cameras to cover the same square footage because of less-than-ideal sight lines and to place conduit wherever it works best, which may mean positioning it under platforms or in other out-of-the-way places within older stations. Looking ahead, transit security will continue to evolve, not only as new stations and modes of transportation are added to the system, but in terms of communicating with commuters. People can expect to get mass notification alerts on their mobile devices, and those same devices can provide vital data to transportation entities to better develop their overall systems.