Articles by Zvika Ashani
2017 witnessed a continued decline in the cost of cameras. While this creates a challenge for camera companies, it creates two clear opportunities: (1) Product differentiation now relies more heavily on software rather than camera parameters, which drives more focus and rapid innovation on the software side, and (2) cameras have become more affordable which encourages an increase in the adoption rate and size of surveillance projects. Artificial Intelligence surveillance applications Additionally, 2017 has been the breakout year for real-world implementations of Artificial Intelligence (AI). The surveillance industry was not left behind with almost weekly announcements of various new products claiming to employ AI, to some extent. New and incumbent video analytics vendors are talking about employing deep learning to provide features and accuracy not previously attainable. While there has been a lot of hype, few companies actually shipped products successfully employing AI. In 2018 we are going to continue hearing a lot about AI (with a focus on Deep Learning) video products. We can expect to see a gradual increase in successful field deployments leading to a shift in customer expectations. Highly accurate people and vehicles detection will be considered commonplace. Demand will increase for complex applications: tracking in urban environments, anomaly detection, and smart search. New and incumbent video analytics vendors are talking about employing deep learning to provide features and accuracy not previously attainable Cloud-based video analytics One of the major challenges with developing Deep Learning-based applications is access to real-word data and the ability to train the applications to work in any environment. Companies with access to relevant datasets, that can iterate their solutions quickly, will come out on top. Cloud-based solutions are a significant advantage in this case, as they allow for continuous updates and easy collection of vast amounts of data. While Agent Vi was a pioneer in implementing cloud-based video analytics, we encountered some concerns around cloud adoption during 2017, especially from traditional municipal and enterprise customers. We expect this to gradually change, as customers realise that cloud implementations are more cost-effective, easier to deploy and maintain, and in many cases, even more secure than traditional on-premise deployments. In partnership with the leading cloud providers, we help carry this message to our customers and will gradually see a shift in the acceptance of cloud-based solutions in the traditional security markets.
A tipping point is defined as: “The point at which a series of changes becomes significant enough to cause a larger, more important change”. In the same way that IP video changed surveillance a decade ago, our industry is now feeling the impact of recent developments in Artificial Intelligence, Machine Learning, Deep Learning, Big Data, and Intelligent Video Analysis. Keyword definitions Let’s start with a few more definitions. Artificial Intelligence (AI) deals with the simulation of intelligent behaviour in computers. Machine Learning (ML) deals with developing computer algorithms that access data and use it to learn for themselves. Neural networks are computer systems that loosely mimic human brain operation. Deep Learning is a subset of ML based on neural networks that has been proven to provide breakthrough capabilities in many problems that were previously unsolvable, and Big Data, or metadata, refers to huge amounts of structured and/or unstructured data -- in our case, the immense quantities of video information being generated daily by security cameras deployed in cities around the world. Deep Learning is tipped to change Intelligent Video Analysis (IVA), the digital video technology integrated with analytical software that is a basic tool for our industry. AI in surveillance Traditionally, the main benefit of surveillance cameras is the ability to collect evidence for debriefing or investigation, as well as the ability to view events remotely in real-time. A decade ago, video analytics technologies were introduced to solve the problem of human inattention -- computers don’t get tired, bored or distracted, and can monitor a camera continuously. And then, camera costs dropped, deployment skyrocketed, and video management systems began collecting reams of useless, costly unstructured data. AI technology seemed to answer the pressing new industry needs of how to use this Big Data effectively, make a return on the investment in expensive storage, while maintaining (or even lowering) human capital costs Three limiting factors All this was theoretical, however, as multiple technological barriers prevented AI solutions from real-world utilisation. Despite decades of research on how to cause a computer to accurately recognise different objects in a video stream, the quality of the results, especially in urban environments, was, to put it mildly, underwhelming. Deep Learning has matured to the point where it can accurately detect and classify objects both in still images and in video AI was limited primarily by these three factors: Lack of understanding -- The software must be able to differentiate between different objects (person, vehicle, animal, etc.), and under various circumstances (day, night, seasonal weather conditions, etc.). Inability to learn -- Traditional IVA applications relied on a rule-based approach that required software configuration -- by a human operator -- for each monitoring camera and each type of alert. Although effective in some scenarios, the exponential growth in camera counts rendered this approach impractical, given the amount of manual labor required to configure, reconfigure, and maintain rules. High cost -- The hard truth is that budgets for security and safety will always be constrained. Until recently, implementing real-time AI was extremely cost-prohibitive, sometimes requiring a 1:1 server to camera ratio. Meeting challenges That was yesterday. Today, the application of AI in security applications has reached its tipping point, meeting the above-mentioned challenges. Understanding -- Deep Learning has matured to the point where it can accurately detect and classify objects both in still images and in video. DL technology is fast becoming the basic building block for IVA. Ability to Learn -- As an AI solution collects and analyses data over time, it creates metadata that describes all objects in each video stream. Machine Learning techniques process this metadata to generate models for “normally observed” behaviour. These models are applied in real-time to detect behaviours deviating from the norm. Only those flagged as suspicious events require review by a human operator. This technique allows the solution to scale to an unlimited number of cameras, with no need for a human to configure each new device. Lower cost -- The rapid increase in GPU computational capacity, coupled with mass market adoption, has lowered server costs to a reasonable level. Today, with the correct implementation, a single server can be deployed across hundreds and even thousands of cameras. The convergence of Deep Learning for video analysis, advances in AI for fully automated event detection, plus the significant reduction in cost to implement these techniques – including cloud-based software as a service (SaaS) models -- means that the fully automated video surveillance solution for cities is fast becoming a reality. We’ll see more of this type of solution being deployed over the coming months, and within the next few years, it will be standard in any Smart City deployment.
Over the years, video analytics has gained an unfavourable reputation for over-promising and under-delivering in terms of performance. One of the biggest complaints regarding video analytics has been its inability to correctly identify objects in situations which appear trivial to the human observer. In many cases, this has resulted in a tendency to generate substantial numbers of false alarms, while not detecting actual events accurately. This, together with a propensity for complex set-up procedures and much need for manual fine-tuning, has prevented video analytics from becoming a mainstream application deployed on large numbers of cameras. Machine Learning and video analytics Machine Learning is a well-established field of research that has existed for decades, which is already present in many products and applications. Machine Learning is based on collecting large amounts of data specific to a particular problem, training a model using this data and then employing this model to process new data. With regard to video analytics, one of the most critical problems impacting accuracy is object classification. Fundamental to improving performance is the capability to teach the algorithm to distinguish between people, animals, different types of vehicles and sources of noise at an extremely high level of accuracy. Deep Learning algorithms The recent increased interest in Deep Learning is largely due to the availability of graphical processing units Until recently there have been minimal applications of Machine Learning used in video analytics products, largely due to high complexity and high resource usage, which made such products too costly for mainstream deployment. However, the last couple of years have seen a tremendous surge in research and advances surrounding a branch of Machine Learning called Deep Learning. Deep Learning is a name used to describe a family of algorithms based on the concept of neural networks. Very loosely speaking, these algorithms try to emulate the functionality of the brain’s neurons, enabling them to learn efficiently from example, and subsequently apply this learning to new data. The recent increased interest in Deep Learning is largely due to the availability of graphical processing units (GPUs). GPUs can efficiently train and run Deep Learning algorithms, and have allowed the scientific community to accelerate their research and application, bringing them to the point where they exceed the performance of most traditional Machine Learning algorithms across several categories. Solving object classification with deep learning This means that Deep Learning can now be used to solve the most crucial problem facing video analytics – object classification – by collecting many thousands of images from hundreds of surveillance cameras, which must first be manually labelled and classified by a human, into a range of categories that include: person, car, bus, truck, bird, vegetation, dog and many more. To achieve the required accuracy rates, such a vast database must be collected and identified from actual surveillance footage. Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics A Deep Learning algorithm trained on images collected from YouTube, Google Search and elsewhere on the Internet will completely fail in analysing images from surveillance cameras, due to the difference in viewing angles, resolution and image quality. Once enough images are collected, a Deep Learning classifier algorithm can be trained and deployed as part of a video analytics solution, enabling it to practically eliminate most of the existing causes for false alarms. Due to GPU requirements in order for the algorithms to run efficiently, video analytics solutions using Deep Learning will initially need to run on a server. A few solutions of this nature are already available and are showing a dramatic leap in performance in comparison to traditional video analytics, with a drastic reduction in false alarm rates and a significant increase in detection accuracy. Concurrently, these new solutions do not require manual tweaking by the user and are essentially plug-and-play, making mass deployment a realistic premise. Surveillance applications of Deep Learning Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics, but they are by no means the only ones. In the not-too-distant future, we will see Deep Learning enabling as yet not possible video analytics applications, such as identifying objects carried by people, such as a gun, handbag, or a knife, or being able to quickly find people and vehicles with similar appearances across multiple cameras and more. Over the next few years, we will see a transition of video analytics using Deep Learning running on servers to running inside cameras Over the next few years, we will see a transition of video analytics using Deep Learning running on servers, to running inside cameras, as powerful, low-cost hardware capable of running Deep Learning becomes more available and a basic function of newer surveillance camera models. This will push the acceptance of video analytics even further, eventually making it a fundamental element of every surveillance camera deployed. Increasing accuracy with updates A crucial component in achieving and maintaining the high performance of Deep Learning-based applications, is the ability to continuously update the models as more data is collected so that the models increase in accuracy. This will give an advantage to cloud-based video analytics services, since they can collect vast amounts of data from cameras connected to the service, train new models in the cloud based on this data and then push these new models to cameras at the edge. This continuous improvement cycle will be instrumental in helping video analytics fulfil the promise of improving peoples’ safety and security, by giving surveillance cameras human-level accuracy and a comprehensive understanding of the environment.
Agent Video Intelligence (Agent Vi), the global provider of video analytics solutions, has integrated the new 2nd Generation Intel Xeon Scalable processors in its innoVi hosted video analytics service. The latest generation of Intel Xeon Scalable processors deliver a leap in processor performance, memory bandwidth, and advanced AI capabilities. Agent Vi’s innoVi Edge, a compact appliance that serves as a cloud gateway and transforms IP cameras into ‘smart’ cameras, utilises the Intel Distribution of OpenVINO toolkit to deliver AI-powered video analytics by performing deep learning inference at the edge. On-premise or private cloud deployments innoVi is available for on-premise or private cloud deployments for customers with restrictions on use of third-party cloud hosting servicesinnoVi’s patented architecture distributes video processing between the innoVi Edge appliance and the innoVi service running on the cloud while reducing the data required for edge-to-cloud transmission. innoVi is also available for on-premise or private cloud deployments for customers with restrictions on use of third-party cloud hosting services. Now, Agent Vi’s next generation of innoVi Edge devices with 2nd Generation Intel Xeon Scalable processors will be able to handle a significantly larger number of cameras thus reducing total cost of ownership for end users. Reduces bandwidth and operating costs Agent Vi CTO Zvika Ashani said, “Moving a portion of deep learning inference from the cloud to innoVi Edge reduces both required bandwidth and cloud operating costs. Now, with the availability of 2nd Generation Intel Xeon Scalable processors, we will be able to support more cameras with less hardware, translating to customer savings and improved ROI.” To learn more about Agent Vi, a member of the Intel IoT Solution Alliance, visit the Agent Vi pod at the Milestone booth (#18053) at ISC West in Las Vegas on 10-12 April.
If you’ve been paying attention over the last twelve months, you will have noticed that deep learning techniques and artificial intelligence (AI) are making waves in the physical security market, with manufacturers eagerly adopting these buzzwords at the industry's biggest trade shows. With all the hype, security professionals are curious to know what these terms really mean, and how these technologies can boost real-world security system performance. The growing number of applications of deep learning technology and AI in physical security is a clear indication that these are more than a passing fad. This review of some of our most comprehensive articles on these topics shows that AI is an all-pervasive trend that the physical security industry will do well to embrace quickly. Here, we examine the opportunities that artificial intelligence presents for smart security applications, and look back at how some of the leading security companies are adapting to respond to rapidly-changing expectations: What is deep learning technology? Machine Learning involves collecting large amounts of data related to a problem, training a model using this data and employing this model to process new data. Recently, there have been huge advances in a branch of Machine Learning called Deep Learning. This describes a family of algorithms based on neural networks. These algorithms are able to learn efficiently from example, and subsequently apply this learning to new data. Here, Zvika Ashani explains how deep learning technology can boost video surveillance systems. Relationship between deep learning and artificial intelligence With deep learning, you can show a computer many different images and it will "learn" to distinguish the differences. This is the "training" phase. After the neural network learns about the data, it can then use "inference" to interpret new data based on what it has learned. For example, if it has seen enough cats before, the system will know when a new image is a cat. In effect, the system “learns” by looking at lots of data to achieve artificial intelligence (AI). Larry Anderson explores how new computer hardware - the Graphic Processing Unit (GPU) – is making artificial intelligence accessible to the security industry. Improving surveillance efficiency and accuracy with AI Larry Anderson explains how the latest technologies from Neurala and Motorola will enable the addition of AI to existing products, changing an existing solution from a passive sensor to a device that is “active in its thinking.” The technology is already being added to existing Motorola body-worn-cameras to enable police officers to more efficiently search for objects or persons of interest. In surveillance applications, AI could eliminate the need for humans to do repetitive or boring work, such as look at hours of video footage. Intelligent security systems overcome smart city surveillance challenges AI technology is expected to answer the pressing industry questions of how to use Big Data effectively and make a return on the investment in expensive storage, while maintaining (or even lowering) human capital costs. However, until recently, these expectations have been limited by factors such as a limited ability to learn, and high ongoing costs. Zvika Ashani examines how these challenges are being met and overcome, making artificial intelligence the standard in Smart City surveillance deployments. Combining AI and robotics to enhance security operations With the abilities afforded by AI, robots can navigate any designated area autonomously to keep an eye out for suspicious behaviour or alert first responders to those who may need aid. This also means that fewer law enforcement and/or security personnel will have be pulled from surrounding areas. While drones still require a human operator to chart their flight paths, the evolution of artificial intelligence (AI) is increasing the capabilities of these machines to work autonomously, says Steve Reinharz. Future of artificial intelligence in the security industry Contributors to SourceSecurity.com have been eager to embrace artificial intelligence and its ability to make video analytics more accurate and effective. Manufacturers predicted that deep learning technology could provide unprecedented insight into human behaviour, allowing video systems to more accurately monitor and predict crime. They also noted how cloud-based systems hold an advantage for deep learning video analytics. All in all, manufacturers are hoping that AI will provide scalable solutions across a range of vertical markets.
Agent Video Intelligence (Agent Vi), global provider of video analytics solutions, has announced the launch of its breakthrough Anomaly Detection capability, as part of Agent Vi’s cloud-based Software as a Service (SaaS), innoVi. Combining Agent Vi’s extensive research into Artificial Intelligence (AI) and development of Deep Learning-powered algorithms with the company’s 15 years of experience in providing cutting-edge Video Analytics solutions, the new Anomaly Detection is a robust self-learning capability that instantly alerts users to atypical incidents. innoVi turns unstructured video data into structured metadata by detecting and tracking all objects in the camera’s field of view innoVi cloud-based SaaS While innoVi’s Anomaly Detection can be applied to surveillance systems of all sizes, it is particularly valuable for large-scale deployments such as Smart Cities, where the sheer volume of cameras prevents video feeds from being monitored continuously. In such scenarios, and without pre-definition by the user, Agent Vi’s Anomaly Detection can quickly learn the regular movement and traffic in a scene, and alert to irregular incidents, such as a crowd forming or running, a traffic accident or interruption, and more. innoVi turns unstructured video data into structured metadata by detecting and tracking all objects in the camera’s field of view, and then further analyses the metadata to discover typical object types, motion patterns and models. Anomaly Detection continuously analyses the scene in real-time and compares it to the learned models, to identify and alert users to irregular incidents that may have otherwise gone undetected. This is achieved through autonomous and continuous learning by innoVi without any pre-definition by the user, eliminating the need for manual rule configuration and testing. innoVi boasts Anomaly Detection as well as highly accurate, user-defined rules for Real-Time Event Detection Real-time monitoring and anomaly detection innoVi boasts Anomaly Detection as well as highly accurate, user-defined rules for Real-Time Event Detection, making it an optimal solution for scenes where the events of interest are predicted and well-defined, as well as scenes with continuous motion where it is impossible to predict the range of irregularities that may require user attention. Implemented over scalable cloud infrastructure and applicable to any type of surveillance camera, innoVi is the ideal solution for any type of surveillance deployment, especially large Smart City installations. Agent Vi CTO, Zvika Ashani, commented that “By utilising the latest advances in AI and unsupervised learning, we are able to deliver a solution that fits our customers’ highest priority, namely, increasing their ability to quickly respond to unforeseen security and safety events, and improving overall management of their cities and facilities.” Agent Vi’s Anomaly Detection will be on display at ISC West, 11-13th April 2018 in Las Vegas, NV, USA, at the Agent Vi pod within the Axis Communications booth (#14051).
SourceSecurity.com’s most trafficked articles in 2017 reflected changing trends in the market, from facial detection to drones, from deep learning to body worn cameras. Again in 2017, the most well-trafficked articles posted at SourceSecurity.com tended to be those that addressed timely and important issues in the security marketplace. In the world of digital publishing, it’s easy to know what content resonates with the market: Our readers tell us with their actions; i.e., where they click. Let’s look back at the Top 10 articles posted at SourceSecurity.com in 2017 that generated the most page views. They are listed in order here with the author’s name and a brief excerpt. MOBOTIX is increasingly positioning itself as a specialist in high-quality IP surveillance software 1. MOBOTIX Aims High with Cybersecurity and Customer-Focused Solutions [Jeannie Corfield] With a new CEO and Konica Minolta on board, MOBOTIX is set for expansion on a global scale. But how much growth can we expect for a company like MOBOTIX in an increasingly commoditised surveillance market, where many of the larger players compete on price as a key differentiator? While MOBOTIX respects those players, the German manufacturer wants to tell a different story. Rather than competing as a camera hardware manufacturer, MOBOTIX is increasingly positioning itself as a specialist in high-quality IP surveillance software – camera units are just one part of an intelligent system. When MOBOTIX succeeds in telling this story, partners understand that it’s not about the price. 2. ‘Anti-Surveillance Clothing’ Creates a New Wrinkle in Facial Detection [Larry Anderson] The latest challenge to facial recognition technology is “anti-surveillance clothing,” aimed at confusing facial recognition algorithms as a way of preserving “privacy.” The clothing, covered with ghostly face-like designs to specifically trigger face-detection algorithms, are a backlash against the looming possibility of facial recognition being used in retail environments and for other commercial purposes. 3. Drone Terror: How to Protect Facilities and People [Logan Harris] Already, rogue groups such as ISIS have used low cost drones to carry explosives in targeted attacks. Using this same method, targeting high profile locations to create terror and panic is very possible. Security professionals and technologists are working furiously to address the gaps in drone defence. Compact Surveillance Radar (CSR) is a security technology addressing the problems with other types of detection. CSR, like traditional radar, has the benefit of being able to detect and track foreign objects in all weather conditions, but at a fraction of the size and cost. The last couple of years have seen a tremendous surge in research and advances surrounding a branch of Machine Learning called Deep Learning 4. Deep Learning Algorithms Broaden the Scope of Video Analytics [Zvika Anshani] Until recently there have been minimal applications of Machine Learning used in video analytics products, largely due to high complexity and high resource usage, which made such products too costly for mainstream deployment. However, the last couple of years have seen a tremendous surge in research and advances surrounding a branch of Machine Learning called Deep Learning. The recent increased interest in Deep Learning is largely due to the availability of graphical processing units (GPUs). GPUs can efficiently train and run Deep Learning algorithms 5. Body Worn Cameras: Overcoming the Challenges of Live Video Streaming [Mark Patrick] Most body camera manufacturers, that are trying to stream, attempt to use these consumer technologies; but they don’t work very well in the field, which is not helpful when you need to see what is happening, right now, on the ground. The video must be of usable quality, even though officers wearing the cameras may be moving and experiencing signal fluctuations – most mobile video produces significant delays and signal breakups. Video and audio must always remain in sync so there’s no confusion about who said what. Therefore, special technology is required that copes with poor and varying bandwidths to allow a real-time view of the scene and support immediate decision-making by local and remote team members and support teams moving to the scene. 6. QinetiQ Demonstrates New Privacy-Protecting Body Scanner for Crowded Places [Ron Alalouff] QinetiQ has developed a scanner that can be used in crowded places without having to slow down or stop moving targets. The body scanner, capable of detecting hidden explosives or weapons on a person, has been demonstrated publicly in the United Kingdom for the first time. SPO-NX from QinetiQ – a company spun out of the UK’s Defence Evaluation and Research Agency (DERA) in 2001 – can quickly screen large groups of people for concealed weapons or explosives in a passive, non-intrusive way, without needing people to stop or slow down. 7. ISC West 2017: How Will IT and Consumer Electronics Influence the Security Industry? [Fredrik Nilsson] A good way to predict trends [at the upcoming ISC West show] is to look at what’s happening in some larger, adjacent technology industries, such as IT and consumer electronics. Major trends on these fronts are the most likely to influence what new products will be launched in the electronic security industry. Proof in point is H.264, an advanced compression technology ratified in 2003 and adopted as the new standard by the consumer industry a few years later. By 2009, it became the new compression standard for the video surveillance industry as well. By drawing data from a number of different sources and subsystems, it is possible to move towards a truly smart environment 8. Integrating Security Management into Broader Building Systems [Gert Rohrmann] Security solutions should be about integration not isolation. Many organisations are considering their existing processes and systems and looking at how to leverage further value. Security is part of that focus and is a central component in the move towards a more integrated approach, which results in significant benefits. By drawing data from a number of different sources and subsystems, including building automation, it is possible to move towards a truly smart environment. 9. How to Use Video Analytics and Metadata to Prevent Terrorist Attacks [Yury Akhmetov] How we defend and prevent terrorism must be based on intelligent processing of information, and an early awareness of potential threats – and effective preventive action – may eliminate most attacks. Video analytics, automated surveillance and AI decision-making will change the rules of the struggle between civilians and terrorists by making attempted attacks predictable, senseless and silent. To what extent can technology investigate and prevent terror crimes considering the latest technology innovations? 10. Next Generation Video Analytics: Separating Fact from Fiction [Erez Goldstein] ‘Next generation video analytics’ is a catchy marketing phrase, is how much substance is behind it? Video analytics as a technology has been with us for many years, but there has always been an air of confusion and mystery around it, in large part created by Hollywood movies, where every camera is connected, an operator can search the network and locate the villain in a matter of seconds. I am pleased to say that, in many respects, fact has caught up with fiction, with the newest video analytics solutions that are now on the market focusing on search and specifically real-time search. These solutions have been tried, tested and proven to help reduce search time from hours to minutes and even seconds.
Agent Video Intelligence, a provider of video analytics solutions, announced that it has joined NVIDIA’s Metropolis Software Partner Program. Agent Vi employs NVIDIA GPUs to deliver cutting-edge intelligent video analysis via deep learning algorithms, and being a part of the Metropolis Partner Program will enable Agent Vi to better leverage Deep Learning technology to create safer and smarter cities. Deep learning technology Agent Vi was a pioneer in the use of deep learning technology on cloud-based GPUs for video analytics applications in the realm of surveillance – a topic that will be presented by Agent Vi’s CTO, Mr. Zvika Ashani, at GTC Israel, on 18 October 2017, in Tel Aviv. Agent Vi offers innoVi™ for Smart Cities – a cloud-based video analytics Software-as-a-Service (SaaS) that transforms the enormous number of cameras deployed across cities into smart IoT devices that become the core of the city's ability to improve security, safety, and incident response. Enhanced situational awareness Employing advanced AI and machine learning algorithms that continuously monitor video feeds captured by the city’s surveillance cameras, innoVi for Smart Cities enables city authorities to become immediately aware of a wide range of events of interest as they unfold, enhancing the operator’s situational awareness and ability to dispatch the most appropriate response. The solution also allows law enforcement agencies and city authorities to make sense of massive amounts of data hidden in recorded video by utilising advanced big data algorithms and an automated video search engine for rapid investigations. “With more than a decade of experience in deploying thousands of video analytics installations worldwide, we were among the first companies to develop deep learning algorithms utilising NVIDIA’s cloud-based GPUs,” commented CTO of Agent Vi, Mr. Zvika Ashani. "Agent Vi delivers the highest level of automated video analysis that enables the effective management of extremely large surveillance systems, even in crowded city environments.”
Improvements in the technology have lowered – but not eliminated – concerns about false alarms False alarms have plagued the video analytics market since the beginning. Improvements in the technology have lowered – but not eliminated – concerns about false alarms. Companies providing video analytics systems say the question isn’t whether false alarms can be eliminated, but rather how they can be managed. “We’re still very far away from a day of zero false alarms,” says Zvika Ashani, chief technology officer (CTO), Agent Video Intelligence (Agent Vi). “Maybe someday when enough computing power is available and algorithms are at the level of sophistication of humans, but we’re still not there.” However, significant strides have been made in the last half dozen or so years, Ashani adds. Today when customers install the Agent Vi system and configure it, they can quickly reach a working state with mostly “true positives,” he says. In other words, false positives can be managed. The ability to achieve a workable level of false positives is “the difference between robust, high-end analytics and what you can get inside a camera at a low cost,” he adds. “People realise they can’t work with [in-camera analytics], so they come to us for a more reliable solution.” iOmniscient’s Nuisance Alarm Minimization System (NAMS) uses artificial intelligence to analyse what is or isn’t a threat – what’s a man or what’s a dog or what’s just a light changing? The system also helps to deal with the “noise” in a video image. “You never get to zero (false alarms), but if you reduce it several levels of magnitude, it’s tolerable,” says Dr. Rustom Kanga, CEO of iOmniscient. For example, if false alarms can be reduced from 20 a day to one every few days, it’s more manageable. "You never get to zero (false alarms), but if you reduce it several levels of magnitude, it’s tolerable" says Dr. Rustom Kanga, CEO of iOmniscient Kanga also contends there are times when false alarms are more tolerable because of the environment. For example, a face recognition analytic looking for shoplifters in the totally uncontrolled environment of a shopping mall might only be 70 percent accurate. However, identifying seven out of 10 in a crowd of 30,000 is better than 0 (and 3 out of 10 false positives is manageable). In markets that require perimeter protection and real-time alerting, there is still a lot of wariness about the success of video analytics, concedes Brian Lane, director of marketing, 3VR. In these markets, analytics need to be nearly 100 percent accurate for them to be useful; otherwise, security directors tire of all the false alarms and eventually turn the analytics off. In markets that require less accuracy, analytics are gaining ground, Lane says. For example, in business intelligence, analytics require less accuracy than, for example, identifying a former employee entering a building. Business intelligence relies on trends, not a single data point.
Analytics at the edge provide the ability to process what is happening in a field of view and discern if a relevant alert is triggered There are multiple benefits to using video analytics at the edge (i.e., near or inside the camera). For one thing, analytics at the edge provides the ability to process what is happening in a field of view and discern if a relevant alert is triggered. This can be faster and less expensive than the original video analytics model of using a separate dedicated server. However, there isn’t one right solution, as a video analytics' complexity and a camera’s processing power are not always aligned. Some analytics can begin the analysis at the camera and also utilise a server to balance the workload. Others may be best used in server-only models. Speed of alert is of importance, as results that are not urgent may not dictate a powerful camera. Another variable is whether the system needs actual video of an event or just information (metadata) from that video. When recorded video is not required at a server, intelligent cameras at the edge help lessen the required bandwidth, says Brian Lane, director of marketing, 3VR. He says intelligent cameras and the cloud go hand-in-hand. For example, only metadata is needed when counting people; therefore, intelligent cameras can do all the processing in the camera, and only the metadata is sent to the cloud. For security, only a low-bandwidth stream is sent to the cloud, while the high-resolution video is stored at the camera. When video is required, the edge advantage becomes far less, since the video must reach the server to be recorded, adds Lane. Having analytics such as face and demographics at the server level keeps the cost of the cameras low since the processor on the server does most of the work. Processing power on servers is far cheaper than having a robust processor in each camera. Analytics that require a lot of processing power greatly increase the cost of the cameras, since they must have a robust processor. When the processing takes place at the server level, the customer can keep overall costs down by using far cheaper cameras and using a centralised server-based system. Edge-based analytic cameras offer a host of benefits to facilities that need to monitor large perimeters, complex campus environments or geographically dispersed open spaces Sometimes, a combination is optimal. For example, Agent Vi has a patented approach that enables analytics processing both at the server and distributed to the edge. The Agent VI system operates on a server between the camera and the video management system (VMS), analysing video streams and providing output of that analysis. A software module called “Vi Agent” runs inside video encoders and cameras at the edge (including brands such as Axis, Samsung, Hikvision, and Vivotek). The Agent Vi software completes “preprocessing” at the edge and sends information to the server, which completes the process and provides the output. Unlike strictly edge-based analytics, the approach is not limited by processing power and memory in the camera. Compared to server-only installations, the system is more scalable (by a factor of 10 to 20 compared to server-based systems), says Zvika Ashani, chief technology officer (CTO), Agent Video Intelligence (Agent Vi). The Vi Agent and server are the same for various verticals; various functionalities are activated per user based on license keys, with various licensing at different price points. Ashani notes a trend in the market of camera vendors turning their cameras into open platforms to allow software vendors to load analytics (and other applications) onto the cameras. Previously, software vendors had to work closely with camera vendors, even creating special software versions. “Today, the cameras are not yet at the level of an iPhone or Android [platform], but they are much more open and there is greater variety in terms of applications you can load,” he says. Ipsotek has always seen edge-based analytics as an interesting alternative to traditional server-based (centralised) solutions. Edge deployment lends itself to a distributed solution where infrastructure is not available, hence where transmitting video of high quality to a centralised server is not an option. Transport (road/rail) has been a major beneficiary of edge-based analytics technology, says Dr. Boghos Boghossian, CTO, Ipsotek. The lack of infrastructure results in a need for a more complex management of rules and possibly more challenging environmental aspects. In order to operate advanced video analytics solutions at the edge, a suitable hardware platform should be provided with enough processing power. However, often at the edge, the system must be rugged and should operate at high temperature extremes; consequently, the availability of such a hardware platform is less likely. There isn’t one right solution,as a video analytic’s complexityand a camera’s processing powerare not always aligned “Because of these issues, most manufacturers have opted to offer only basic analytics solutions at the edge,” says Boghossian. “Ipsotek took a different route, and through the use of digital signal processing technology, has managed to move its technology to the edge with no compromise to performance, feature list or robustness.” Ipsotek has been offering cloud-based systems to a number of large customers for a few years. The interesting correlation is the larger adoption of cloud-based solutions in projects based on edge analytics due to the lack of infrastructure and therefore reverting to cloud storage for data management. This trend may soon be overtaken by cloud-based video analytics, which is waiting for sufficient affordable bandwidth to stream video to the cloud at the required speed and quality. Edge-based analytics run on raw video data as opposed to encoded video on the server, allowing the analytics to gather more sensitive and accurate data, says Maor Mishkin, director, Video Analytics Product Champion, DVTEL. In addition, it allows the analytics to control the sensor and enable optimised video input for the analytic engine. Edge-based analytic cameras offer a host of benefits to facilities that need to monitor large perimeters, complex campus environments or geographically dispersed open spaces. Edge-based analytic devices do not rely on servers or third-party software. This reduces the network bandwidth requirements while maintaining performance at the highest level. In addition, when technology developers offer a complete solution that ties in edge analytics and video management, users benefit from a single, tightly integrated solution, which means there is less opportunity for failure, Mishkin says.
Intelligent searches of video archives provide investigators faster access to any needed video clip That video analytics can be immensely useful in forensics is relatively less known. However, forensic search capabilities offered by some modern video analytics solutions can not only save investigators significant amounts of time but also help them find results more accurately. These solutions leverage facial recognition and advanced object tracking, demographic analytics, license plate recognition capabilities and other such powerful features to take forensic investigation to a whole new level. Video analytics have earned a reputation as solutions that can provide real-time intelligence to enable immediate response. However, another aspect of video analytics is how the technology can be used for forensics. Basically, intelligent searches of video archives provide investigators faster access to any needed video clip based on the content of the video. It’s a monumental improvement over the old days of searching for hours while rewinding and fast-forwarding videotape. The use of video analytics for forensics is a less well-known benefit of the technology, says Zvika Ashani, chief technology officer (CTO), Agent Video Intelligence (Agent Vi). “Most people associate it with events, like detecting a person approaching a perimeter,” says Ashani. “But video analytics can also be useful in a forensic sense. It can help an investigator who needs to go into a video archive and find evidence of an event.” Defining queries with analytics speeds up forensic investigationAnalytics-based forensics enable investigators to define their queries. For example, a user might want to see all the blue trucks in an area in a specific time frame. “In seconds, we can provide results in terms of thumbnails – images that fit that query,” says Ashani. “They can zoom in and see what they’re looking for. Some customers understand this, and for some, it’s something new.” Most of the video recorded on an NVR/HVR (network video recorder/hybrid video recorder) is never viewed, says Brian Lane, director of marketing, 3VR. But, when the time comes that a short piece of video is needed and there are thousands of hours of recorded video, analytics can greatly simplify the task. ”For example, say you are looking for a male between 40 and 50 years old, wearing a red shirt, headed west,” says Lane. “Using analytics on a 3VR system, you can type these parameters into the search fields, and the system will send back video of the closest results. If you know the face of the suspect, using a face analytic makes the search even easier. In a world where time is money, analytics can be used to save countless hours searching through video.” Using facial, advanced object tracking, licence plate, and demographics analytics, a user can search for colour, speed, direction, size, age and gender, or a licence plate number or a face 3VR’s VisionPoint VMS uses video analytics to greatly reduce the amount of time needed to search through video. Using facial, advanced object tracking, licence plate, and demographics analytics, a user can search for colour, speed, direction, size, age and gender, or a licence plate number or a face. If a customer calls his bank to complain that money is missing from his account, an investigator can use 3VR’s VisionPoint VMS and type in the customer’s account number into the system. The system will display any video associated with a transaction using that account number. If there is video that doesn’t match that of the customer’s face, using 3VR’s face analytic, the bank can search through all of the bank’s video to see if the suspect has been in the bank using other stolen cards. The alternative is to get dates and times from the customer as to when they believe they were at the bank and search though thousands of hours of video. “Imagine how much video you would have to search through if the bank had 16 cameras and 90 days of storage?” says Lane. “If the bank has multiple branches, the investigator can also search across the various branches for the subject.” Tracking individuals and movements in forensic modeOne of Ipsotek’s latest generation developments is Tag and Track, a video content analysis-based tracking system that operates on a network of overlapping and non-overlapping CCTV cameras to track a “tagged” individual. In the forensic mode of operation, the system can be used to analyse hours of incident-related video footage in minutes to produce a detailed account of individual movements in the surveillance area. This product has the ability to identify behaviour, selecting persons/objects of interest and enabling them to be tracked going forward. The system can also analyse where that person has come from, the path they have previously taken through the network and therefore where they have been. This system can analyse thousands of hours of video footage within minutes, therefore assisting greatly in investigations in which tracking evidence is essential. It is possible to search for a new face within historical data, providing information on when and where a new person of interest was in a facility over the previous month Facial recognition provides forensic capabilitiesAnother recent development is Ipsotek’s use of facial recognition technology to provide forensic functionalities. For example, it is possible to search for a new face within historical data, providing information on when and where a new person of interest was in a facility over the previous month. In an operational role, once a person has been tagged, the system takes over and automatically follows the person, intuitively waiting and reacquiring them should they disappear into areas not covered by the CCTV network. The system will also track the tagged individual backwards in time to show where that person has been. Tagging is either performed by operators using the Graphical User Interface or automatically with certain video content analysis and/or other if triggers (e.g., intrusion) have been satisfied or another sensor input trips (e.g., facial recognition). The system memorises the appearance of the tagged individual (based on a number of different visual identifiers) and tracks this collection of information. The Tag and Track system automatically learns the geometry of the camera network and understands where an object is likely to reappear when it goes out of one camera view.
The better the sensors, the better the analytics Garbage in, garbage out. The familiar cliché is just as applicable to the area of video analytics as any other field of computing. You simply must have a high-quality image in order to achieve a high-functioning analytics system. The good news is that video cameras, which are the sensors in video analytics systems, are providing images that are better than ever, offering higher quality – and more data – for use by video analytics. For analytics that require a higher resolution to achieve superior results, megapixel cameras provide video that allows for better face recognition, clearer licence plate numbers, reliable age and gender of customers, and other uses. These help prevent false positives and increase reliability in forensic searches, says Brian Lane, director of marketing, 3VR. When Ipsotek considers a video analytics-based solution, 50 percent of that solution is reliant on the selection of the appropriate sensor (camera). With the emerging technologies of thermal, megapixel and advances in camera processing, this half of the solution is more readily achieved, says Dr. Boghos Boghossian, CTO, Ipsotek. In some areas like face recognition, the illumination of the face in challenging environmental conditions is key to the success of the solution. Therefore, Ipsotek has been evaluating cutting edge camera technology provided by Ipsotek’s technology partners to assist consultants and solution partners to design successful solutions for every growing video analytics market. The better the sensors, the better the analytics, agrees Dr. Rustom Kanga, CEO of iOmniscient, and lower costs of thermal cameras make them a good choice. However, cameras that provide higher-resolution images require more computing power, bandwidth, and storage, which complicates their use with analytics. In general, the resolution is downgraded to the least resolution possible to detect the activity the analytics system is looking for. For analytics that require a higher resolution to achieve superior results, megapixel cameras provide video that allows for better face recognition, clearer licence plate numbers, reliable age and gender of customers, and other uses iOmniscient has a new technology called IQ Hawk that “pulls out of the image what is important,” says Kanga. It accesses higher resolution only for areas of interest in the photo – such as using higher resolution of a face or licence plate viewed from a distance to enable facial or license plate recognition. The rest of the image is used at lower resolution. If there are three people in a video frame, IQ Hawk presents all three faces in high-res to enable identification. “With IQ Hawk, we can dynamically look at an image at high and low resolution, based on what’s important,” says Kanga. In terms of using higher-resolution cameras with analytics, Zvika Ashani, chief technology officer (CTO), Agent Video Intelligence (Agent Vi), says it is important to consider the “lowest common denominator” in terms of usable resolution. For example, a megapixel camera might have a clearer image in good sunlight; but at night time, the image will suffer, and could be worse than a low-resolution image. “More pixels don’t mean more detection quality,” he says. “The more pixels you have, the more processing power you need inside the camera.” Therefore, high-resolution images may even be “downscaled” to a lower resolution for analysis to minimise the amount of data to be managed. Higher resolution can also introduce additional noise in many cases. Some higher-resolution cameras have video analytics built in. DVTEL’s new ioimage HD Analytic IP cameras provide HD broadcast-quality IP video coupled with built-in military-grade analytics. These high-resolution, low-bandwidth cameras, available in both HD 1080p and 720p, are optimized for outdoor conditions and available with predictable storage. The cameras have enhanced low-light and no-light capabilities, high sensitivity, and true wide dynamic range. A new analytics feature provides a reduced false alarm rate for people standing upright, which benefits applications that don’t need sophisticated detection of camouflaged or crawling intruders. ioimage analytics now have improved detection distance, which allows for fewer cameras needed to cover the same area.
The aviation and transportation industries are using video analytics to provide operational cost savings and performance enhancement Video analytics are now increasingly being used for the critical infrastructure, airports, transportation and city surveillance sectors, among other high-value markets. These markets need robust video analytics solutions that can be integrated into an overall security solution to deliver totally reliable results without any significant level of nuisance alerts. “We expect these markets to continue growing in hand with increasing advances in research and development for video analytics,” says Bill Flind, CEO, Ipsotek. Aviation and transportation industries are good examples where video analytics provide additional benefits including operational cost savings and performance enhancement, Flind says. Security personnel who were previously required to physically man certain areas can now be freed up to enhance security elsewhere on a site. Transportation and city surveillance sectors benefit from improved safety, compliance and business intelligence through early detection of potential hazards as they develop, triggering alerts for action before incidents occur. The market has been generally slow to adopt video analytics, says Flind. “We believe this is due to the historically disappointing results from many of the early technologies (that were over-promised and under-delivered),” he says. A good example is the many times when a “basic” system has been deployed onto what the customer had believed was a simple environment (e.g., perimeter fence of an airport or a power station), and then that system has become unusable because it generates hundreds (in some cases, thousands) of nuisance alarms each day. Flind adds: “The change that we are now seeing is that most potential customers now understand that a far more advanced technology will deliver an excellent solution into these environments – specifically, it is now widely understood that the power of a scenario-based solution like Ipsotek’s is required to deliver a robust solution even in these simple applications.” “What we are seeing now in the marketplace is that video analytics is being specified, but specified in a discerning and intelligent way, where the client and the client’s trusted advisors have an appreciation of the difference in the capability of the different technologies,” Flind says. 3VR’s VisionPoint VMS uses face, licence plate, advanced object tracking, loitering and demographics analytics to search for suspects Ipsotek delivers scenario-based video analytics across a wide range of applications in the commercial and public sectors. Deployments include perimeter protection, intrusion detection, investigation and forensics, and the management of traffic, crowds and operations. The patented scenario-based approach creates an exact description of the target behaviour, thereby giving dependable real alerts and dramatically reduced false alarms. Ipsotek has received “i-LIDS Primary” accreditation by the UK Home Office, signifying it is “deployable as the sole system for perimeter protection for sterile zone intrusion.” This recognition of quality and reliable video analytics provides assurance that high-security sites and projects will be monitored to the utmost level of protection. Installations include Transport for London (TfL) traffic management, London Eye, the O2 Arena, Network Rail, the Australian Parliament, plus various international airports and other critical buildings and infrastructures around the world. Meeting stringent customer requirements Another video analytics provider, Agent Vi, is strong at the mid- to high-end of the market, where customer requirements are more stringent. Vertical markets include critical infrastructure and transportation hubs (including airports, seaports, train stations and railroad operations), the enterprise market, and retail. Agent Vi’s rules-based system includes some preconfigured types of objects the system can detect and behaviours it can analyse. The integrator can specify the analytics rules needed on a per-camera basis. For example, one camera might alert if a crowd forms and another might alert if a vehicle stops in the area. A user interface allows operators to set the rules, or they can rely on a systems integrator to set the system up. In the enterprise market, applications might include perimeter security, or detecting people where they shouldn’t be, or detecting tailgating through access control points. Analytics can address safety issues, such as detecting if something is blocking an emergency exit or a vehicle is parked in a no-loading zone. “We provide a fairly extensive tool box, and a lot of times we don’t know the exact configuration or application,” says Zvika Ashani, chief technology officer (CTO), Agent Video Intelligence (Agent Vi). “It’s so versatile.” Video analytics company iOmniscient is active is 30 vertical markets, including transportation (airports, railways, seaports), retail banking, hotels, casinos, the Secret Service, Defence Department, etc. The company is also active in Smart City applications, including Singapore and other locations in the Middle East and Asia. In the North American market, iOmniscient supplies the Chicago Transit Authority, the Mexico City airport, prisons in Vancouver, various oil and gas applications, universities, museums, etc. iOmniscient systems can operate centrally or at the edge, depending on the application. However, at the edge, the software is installed in a black box alongside the camera rather than in the camera because chips inside cameras do not have enough computing power. “When we say edge, we mean near the camera not in the camera,” says Dr. Rustom Kanga, CEO of iOmniscient. Consider overall system costs In assessing the costs of video analytics, the traditional practice of considering costs “per camera” is no longer relevant. Rather, the cost of the entire system should be considered, and also how it might be offset by cost savings achieved in other areas. Kanga contends one Smart City application was completed at zero incremental cost – the cost of the system was offset completely by a 70 to 80 percent reduction in storage, bandwidth and computing requirements. “The hardware is half the cost of the system, so if you can reduce it by 80 percent, the whole system can come in at zero incremental cost,” he says. “Our system is very cost-competitive.” "If you don’t have video analytics, you are wasting your money putting in cameras" “If you don’t have video analytics, you are wasting your money putting in cameras,” says Kanga. “If you have 1,000 cameras in a large environment, no one will sit and watch them. They are only useful after the event. Video analytics enables systems to become a useful tool to prevent incidents and to allow a fast response. With video analysis, video becomes a tool that you can use in real time.” Analytics are used to solve a specific problem, says Brian Lane, director of marketing, 3VR. For example, when there are bandwidth restraints, cameras can be set up to send only low-bandwidth video until the analytic detects a car or person. The analytic tells the camera to then send a high-resolution stream to the server for the duration of the event. This helps keep bandwidth and storage requirements low. 3VR’s Customer Insights business is used by retailers, banks, food service, hospitality and other markets to track customers and understand buying habits. Through an interactive dashboard, retailers and others can harness 3VR analytics to provide business intelligence they can use to help increase sales or improve operations. 3VR’s VisionPoint VMS loss prevention and security video management software (VMS) uses face, licence plate, advanced object tracking, loitering and demographics analytics to search for suspects. The price of analytics is dropping, and many companies are lowering the barrier to entry by allowing customers to pay monthly or yearly, rather than all at once, says Lane. But, while implementing analytics has become easier, many integrators are not trained on how to set up and install analytics, leading to a frustrated customer. 3VR has specific tried-and-true methods for setting up cameras for analytics, and its integrators are trained on analytic implementation through online and on-site classes. 3VR provides the hardware for highly accurate people counting, and customers can pay for retail analytics monthly. Proactive perimeter security According to DVTEL, video analytics provide an ideal solution for proactive perimeter security in the critical infrastructure, transportation and commercial/industrial verticals, as well as general security for healthcare, gaming and other facilities. Going beyond the typical forensic video capabilities, analytics can help users catch criminals during a security event. The technology provides the best ratio of false alarm rates to probability of detection, as well as the most reliable and cost-effective solution for intrusion detection, when compared to legacy perimeter and fence sensors, typical surveillance devices and other video content analysis solutions. DVTEL’s Site Viewer eliminates the need to use a Video Management System to monitor and control remote sites. This makes it suitable for remote sites that cannot install a PC (such as power utility installations, solar farms, cellular and communication facilities, and construction sites) and that have limited network bandwidth. Today’s most effective security systems incorporate other sensors, including perimeter intrusion detection, alarms, remote video monitoring thermal imagery and video analytics, says Maor Mishkin, director, Video Analytics Product Champion, DVTEL. In today’s market, end users demand integrated systems that combine intelligent detection systems to specifically fit their business and risk profile, he adds. By streamlining decision-making and even automating some protocols, video analytics enable organisations to respond more quickly to events By streamlining decision-making and even automating some protocols, video analytics enable organisations to respond more quickly to events, using fewer personnel than would be required with traditional surveillance systems. In addition, ioimage video analytics enable more accurate detection and fewer false alarms by reliably differentiating between legitimate threats and other movement, such as tree branches blowing in the wind. Perhaps most compelling, video analytics has the potential to decrease storage costs for many users, since they are able to record at lower resolution until the system detects suspicious movement. Tight integration at the edge High-profile perimeter security breaches at critical sites have underscored the risks associated with physical security breaches and the potentially disastrous consequences. Now organizations – some as a result of pending regulations such as NERC CIP for utilities – are seeking reliable and cost-effective solutions using video analytics to detect intruders and send alarms about security incidents in real time. SightLogix SightSensors are smart thermal cameras with embedded video analytics powered by a high degree of video processing. By tightly integrating the imager, analytics and video processing, SightSensors are able to deliver accurate detection in all weather, climates, and conditions. Some verticals adopting video analytics for outdoor intrusion detection include electrical utilities, specifically substations; airports and other transportation organizations, such as bridges and rail applications; datacentres; and ports. These sites require protection and 24/7 detection for large, open areas that are difficult to patrol. Video analytics with thermal detection cameras combine detection with video verification in a single solution, allowing for a fast and precise response. Beyond security, video analytics combined with outdoor thermal cameras opens up a new world of understanding, from early detection scenarios to looking at behaviour that increases operational efficiency.