Imagine a home surveillance camera monitoring an elderly parent and anticipating potential concerns while respecting their privacy. Imagine another camera predicting a home burglary based on suspicious behaviors, allowing time to notify the homeowner who can in turn notify the police before the event occurs—or an entire network of cameras working together to keep an eye on neighborhood safety.

Artificial Intelligence vision chips

A new gen of AI vision chips are pushing advanced capabilities such as behavior analysis and higher-level security

There's a new generation of artificial intelligence (AI) vision chips that are pushing advanced capabilities such as behavior analysis and higher-level security to the edge (directly on devices) for a customisable user experience—one that rivals the abilities of the consumer electronics devices we use every day.

Once considered nothing more than “the eyes” of a security system, home monitoring cameras of 2020 will leverage AI-vision processors for high-performance computer vision at low power consumption and affordable cost—at the edge—for greater privacy and ease of use as well as to enable behavior analysis for predictive and preemptive monitoring.

Advanced home monitoring cameras

With this shift, camera makers and home monitoring service providers alike will be able to develop new edge-based use cases for home monitoring and enable consumers to customise devices to meet their individual needs. The result will be increased user engagement with home monitoring devices—mirroring that of cellphones and smart watches and creating an overlap between the home monitoring and consumer electronics markets.

A quick step back reminds us that accomplishing these goals would have been cost prohibitive just a couple of years ago. Face recognition, behavior analysis, intelligent analytics, and decision-making at this level were extremely expensive to perform in the cloud. Additionally, the lag time associated with sending data to faraway servers for decoding and then processing made it impossible to achieve real-time results.

Cloud-based home security devices

The constraints of cloud processing certainly have not held the industry back, however. Home monitoring, a market just seven years young, has become a ubiquitous category of home security and home monitoring devices. Consumers can choose to install a single camera or doorbell that sends alerts to their phone, a family of devices and a monthly manufacturer’s plan, or a high-end professional monitoring solution.

While the majority of these devices do indeed rely on the cloud for processing, camera makers have been pushing for edge-based processing since around 2016. For them, the benefit has always been clear: the opportunity to perform intelligent analytics processing in real-time on the device. But until now, the balance between computer vision performance and power consumption was lacking and camera companies weren’t able to make the leap. So instead, they have focused on improving designs and the cloud-centric model has prevailed.

Hybrid security systems

Even with improvements, false alerts result in unnecessary notifications and video recording

Even with improvements, false alerts (like tree branches swaying in the wind or cats walking past a front door) result in unnecessary notifications and video recording— cameras remain active which, in the case of battery powered cameras, means using up valuable battery life.

Hybrid models do exist. Typically, they provide rudimentary motion detection on the camera itself and then send video to the cloud for decoding and analysis to suppress false alerts. Hybrids provide higher-level results for things like people and cars, but their approach comes at a cost for both the consumer and the manufacturer.

Advanced cloud analytics

Advanced cloud analytics are more expensive than newly possible edge-based alternatives, and consumers have to pay for subscriptions. In addition, because of processing delays and other issues, things like rain or lighting changes (or even bugs on the camera) can still trigger unnecessary alerts.

And the more alerts a user receives, the more they tend to ignore them—there are simply too many. In fact, it is estimated that users only pay attention to 5% of their notifications. This means that when a package is stolen or a car is burglarised, users often miss the real-time notification—only to find out about the incident after the fact. All of this will soon change with AI-based behavior analysis, predictive security, and real-time meaningful alerts.

Predictive monitoring while safeguarding user privacy

These days, consumers are putting more emphasis on privacy and have legitimate concerns about being recorded while in their homes. Soon, with AI advancements at the chip level, families will be able to select user apps that provide monitoring without the need to stream video to a company server, or they’ll have access to apps that record activity but obscure faces.

Devices will have the ability to only send alerts according to specific criteria. If, for example, an elderly parent being monitored seems particularly unsteady one day or seems especially inactive, an application could alert the responsible family member and suggest that they check in. By analysing the elderly parent’s behavior, the application could also predict a potential fall and trigger an audio alert for the person and also the family.

AI-based behavior analysis

Ability to analyse massive amounts of data locally and identify trends is a key advantage of AI at the edge

The ability to analyse massive amounts of data locally and identify trends or perform searches is a key advantage of AI at the edge, for both individuals and neighborhoods. For example, an individual might be curious as to what animal is wreaking havoc in their backyard every night.

In this case, they could download a “small animal detector” app to their camera which would trigger an alert when a critter enters their yard. The animal could be scared off via an alarm and—armed with video proof—animal control would have useful data for setting a trap.

Edge cameras

A newly emerging category of “neighborhood watch” applications is already connecting neighbors for significantly improved monitoring and safety. As edge cameras become more commonplace, this category will become increasingly effective.

The idea is that if, for example, one neighbor captures a package thief, and then the entire network of neighbors will receive a notification and a synopsis video showing the theft. Or if, say, there is a rash of car break-ins and one neighbor captures video of a red sedan casing their home around the time of a recent incident, an AI vision-based camera could be queried for helpful information:

Residential monitoring and security

The camera could be asked for a summary of the dates and times that it has recorded that particular red car. A case could be made if incident times match those of the vehicle’s recent appearances in the neighborhood. Even better, if that particular red car was to reappear and seems (by AI behavior analysis) to be suspicious, alerts could be sent proactively to networked residents and police could be notified immediately.

Home monitoring in 2020 will bring positive change for users when it comes to monitoring and security, but it will also bring some fun. Consumers will, for example, be able to download apps that do things like monitor pet activity. They might query their device for a summary of their pet’s “unusual activity” and then use those clips to create cute, shareable videos. Who doesn’t love a video of a dog dragging a toilet paper roll around the house?

AI at the Edge for home access control

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring, and it’s an application that is expected to take off soon. With smart biometrics, cameras will be able to recognise residents and then unlock their smart front door locks automatically if desired, eliminating the need for keys.

And if, for example, an unauthorised person tries to trick the system by presenting a photograph of a registered family member’s face, the camera could use “3D liveness detection” to spot the fake and deny access. With these and other advances, professional monitoring service providers will have the opportunity to bring a new generation of access control panels to market.

Leveraging computer vision and deep neural networks

Ultimately, what camera makers strive for is customer engagement and customer loyalty. These new use cases—thanks to AI at the edge—will make home monitoring devices more useful and more engaging to consumers. Leveraging computer vision and deep neural networks, new cameras will be able to filter out and block false alerts, predict incidents, and send real-time notifications only when there is something that the consumer is truly interested in seeing.

AI and computer vision at the edge will enable a new generation of cameras that provide not only a higher level of security but that will fundamentally change the way consumers rely on and interact with their home monitoring devices.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

Author profile

William Xu Senior Product Marketing Manager, Ambarella, Inc.

In case you missed it

What is the impact of lighting on video performance?
What is the impact of lighting on video performance?

Dark video images contain little or no information about the subject being surveilled. Absence of light can make it difficult to see a face, or to distinguish the color of clothing or of an automobile. Adding light to a scene is one solution, but there are also new technologies that empower modern video cameras to see better in any light. We asked this week’s Expert Panel Roundtable: what impact does lighting have on the performance of video systems?

Alarm.com adapts during pandemic to enable partners to ‘succeed remotely’
Alarm.com adapts during pandemic to enable partners to ‘succeed remotely’

As a cloud-based platform for service providers in the security, smart home and smart business markets, Alarm.com has adapted quickly to changing conditions during the coronavirus pandemic. In the recent dynamic environment, Alarm.com has kept focus on supporting their service provider partners so they can keep local communities protected. “We moved quickly to establish work-from-home protocols to protect our employees and minimise impact on our partners,” says Anne Ferguson, VP of Marketing at Alarm.com. The Customer Operations and Reseller Education (CORE) team has operated without interruption to provide support to partners. Sales teams are utilising webinars and training resources to inform and educate partners about the latest products, tools, and solutions. Alarm.com’s partner tools are essential for remote installations and support of partner accounts. Helping customers remain connected Adapting to challenges of the coronavirus pandemic, Alarm.com is further investing in solutions that help customers remain connected and engaged. The company has created a resource hub called “Succeeding Remotely” that provides tools, tips and news links that partners can use to adapt their business operations. From adjusting sales and installation techniques to maintaining cellular upgrades, Alarm.com is helping partners stay connected to customers remotely, keep their teams trained, and address rapidly evolving customer concerns without rolling trucks.The company has created a resource hub called “Succeeding Remotely “Additionally, after seeing all that our partners are doing to support their local communities in need, we were compelled to highlight those efforts with ongoing videos called Good Connections, which we’re sharing with our partner community to spark more ideas and ways to help,” says Ferguson. “Though our partners have experienced varying degrees of disruption to their business, we’re inspired by their adaptability, ingenuity and resilience,” says Ferguson. “Along with establishing proper safeguards for operating in homes and businesses, our partners are leveraging our support resources more heavily, while our entire staff has worked tirelessly to deliver new, timely resources.” Do-It-Together solutions Alarm.com partners are successfully employing Do-It-Together (DIT) solutions, focusing on 3G-to-LTE upgrades, and pivoting to new verticals like commercial and wellness. Many are also streamlining their business operations and taking advantage of virtual training opportunities to enhance their technicians’ skills and knowledge, says Ferguson. Do-It-Together installs involve depending on customers to perform part or all of the installation process. Partners can send customers fully configured kits with mounting instructions, or technicians may guide customers on a remote video call. Alarm.com’s tools, training and products help partners modify remote installation options depending on each customer’s needs. End users can validate the Alarm.com Smart Gateway with their central station that sensors they have mounted were done correctly using the Alarm.com mobile app Alarm.com Smart Gateway For example, the Alarm.com Smart Gateway can be pre-configured with indoor and outdoor cameras for easy customer installation and to reduce the likelihood of future service calls. Also, end users can validate with their central station that sensors they have mounted were done correctly using the Alarm.com mobile app. “DIT is helping our partners continue onboarding customers and avoid backlogs,” says Ferguson. “We’ve been pleasantly surprised by the resiliency and level of future investment that our residential and commercial partners have shown in the face of adversity,” adds Ferguson. For example, a significant number of business customers have used the slow period to install systems that are typically too disruptive to put in during normal business hours. Similarly, service providers are adopting new technologies or business models, such as cloud-based access control. “They’re often saying to us, ‘I’m going to take this opportunity to make changes to improve our business,’ and have been working closely with us on training and business consulting to support their efforts,“ she says. Shift to the cloud Ferguson sees a growing preference for cloud-managed surveillance and access systems over ones that have historically been run on-premise. The technology itself is attractive, but especially driving change is the enhancement to the daily lives of service providers and customers, which have been strained during this time. “The foundational benefit of our cloud-based solution is the hassle-free, seamless customer experience it delivers,” says Ferguson. “We make this possible by taking ownership of the servers, software maintenance, firmware updates, health monitoring, and more. With cloud technology, these aspects become invisible to the customer and take a lot off their plate, which is more important than ever.” End users can take advantage of Smart Tip video tutorials to help with DIT installations, or they can use the Alarm.com Wellcam to connect with loved ones anywhere.End users can take advantage of Smart Tip video tutorials to help with DIT installations Partners can attend training workshops focused on remote installation tactics, while driving consumer interest in new offerings through Alarm.com’s Customer Connections platform. The goal is to make it simple for partners to stay connected to their customers to maximise lifetime account value. “We are well-positioned to endure the pandemic because of the strength of our partners in their markets along with our investments in technology, hardware and our team,” says Ferguson. “As restrictions slowly lift, there is cautious optimism that the residential, commercial, property management, plumbing/HVAC, builder and other verticals will recover quickly. We believe that as more partners adopt the DIT model and add commercial and wellness RMR, they will find increasing opportunities to deploy security, automation, video, video analytics, access and more throughout their customer base.”

COVID-19 worries boost prospects of touchless biometric systems
COVID-19 worries boost prospects of touchless biometric systems

Spread of the novel coronavirus has jolted awareness of hygiene as it relates to touching surfaces such as keypads. No longer in favour are contact-based modalities including use of personal identification numbers (PINs) and keypads, and the shift has been sudden and long-term. Both customers and manufacturers were taken by surprise by this aspect of the virus’s impact and are therefore scrambling for solutions. Immediate impact of the change includes suspension of time and attendance systems that are touch-based. Some two-factor authentication systems are being downgraded to RFID-only, abandoning the keypad and/or biometric components that contributed to higher security, but are now unacceptable because they involve touching. Touchless biometric systems in demand The trend has translated into a sharp decline in purchase of touch modality and a sharp increase in the demand for touchless systems, says Alex Zarrabi, President of Touchless Biometrics Systems (TBS). Biometrics solutions are being affected unequally, depending on whether they involve touch sensing, he says. Spread of the novel coronavirus has jolted awareness of hygiene as it relates to touching surfaces such as keypads “Users do not want to touch anything anymore,” says Zarrabi. “From our company’s experience, we see it as a huge catalyst for touchless suppliers. We have projects being accelerated for touchless demand and have closed a number of large contracts very fast. I’m sure it’s true for anyone who is supplying touchless solutions.” Biometric systems are also seeing the addition of thermal sensors to measure body temperature in addition to the other sensors driving the system. Fingerscans and hybrid face systems TBS offers 2D and 3D systems, including both fingerscans and hybrid face/iris systems to provide touchless identification at access control points. Contactless and hygienic, the 2D Eye system is a hybrid system that combines the convenience of facial technology with the higher security of iris recognition. The system recognises the face and then detects the iris from the face image and zeros in to scan the iris. The user experiences the system as any other face recognition system. The facial aspect quickens the process, and the iris scan heightens accuracy. TBS also offers the 2D Eye Thermo system that combines face, iris and temperature measurement using a thermal sensor module. TBS's 2D Eye Thermo system combines face, iris and temperature measurement using a thermal sensor module Another TBS system is a 3D Touchless Fingerscan system that provides accuracy and tolerance, anti-spoofing, and is resilient to water, oil, dust and dirt. The 2D+ Multispectral for fingerprints combines 2D sensing with “multispectral” subsurface identification, which is resilient to contaminants and can read fingerprints that are oily, wet, dry or damaged – or even through a latex glove. In addition, the 3D+ system by TBS provides frictionless, no-contact readings even for people going through the system in a queue. The system fills the market gap for consent-based true on-the-fly systems, says Zarrabi. The system captures properties of the hand and has applications in the COVID environment, he says. The higher accuracy and security ratings are suitable for critical infrastructure applications, and there is no contact; the system is fully hygienic. Integration with access control systems Integration of TBS biometrics with a variety of third-party access control systems is easy. A “middleware” subsystem is connected to the network. Readers are connected to the subsystem and also to the corporate access control system. An interface with the TBS subsystem coordinates with the access control system. For example, a thermal camera used as part of the biometric reader can override the green light of the access control system if a high temperature (suggesting COVID-19 infection, for example) is detected. The enrollment process is convenient and flexible and can occur at an enrollment station or at an administration desk. Remote enrollment can also be accomplished using images from a CCTV camera. All templates are encrypted. Remotely enrolled employees can have access to any location they need within minutes. The 3D+ system by TBS provides frictionless, no-contact readings even for people going through the system in a queue Although there are other touchless technologies available, they cannot effectively replace biometrics, says Zarrabi. For example, a centrally managed system that uses a Bluetooth signal from a smart phone could provide convenience, is “touchless,” and could suffice for some sites. However, the system only confirms the presence and “identity” of a smart phone – not the person who should be carrying it. “There has been a lot of curiosity about touchless, but this change is strong, and there is fear of a possible second wave of COVID-19 or a return in two or three years,” says Zarrabi. “We really are seeing customers seriously shifting to touchless.”