Agentic AI will finally solve physical security’s oldest problem

A security camera installed today has more AI processing power than the systems that guided early autonomous vehicle prototypes. And yet the operator who mounts that camera on a wall will, in all likelihood, never use most of that capability. 

Industry surveys bear this out: a wide gap persists between the number of security professionals who believe AI can improve outcomes and the much smaller share who have adopted it operationally. The reason has nothing to do with the silicon and everything to do with how the industry has asked people to configure these systems.

The problem is not that the industry lacks algorithms. The problem is that physical security has never found a scalable way to personalise systems for each site.

The personalisation dilemma hiding in plain sight

The problem is that physical security has never found a scalable way to personalise systems for each site

A surveillance deployment at an airport, a retail chain, a school campus, and a logistics yard can look strikingly similar in hardware terms. Each installation uses image sensors, edge processors, network connectivity, and a management layer. What changes is what the operator cares about.

At a school entrance, the priority might be perimeter approach after hours and controlled access during the day. At a loading dock, the concern is tailgating, vehicle dwell time, and safety incidents near forklifts. At an airport, the operator may need queue-flow analytics one moment, unattended-item detection the next, and then a search for a specific person of interest carrying a particular bag. At a retail store, loss prevention teams want to correlate customer flow patterns with point-of-sale data and identify suspicious behaviour near high-value merchandise.

This range of needs forces a reality that the industry has acknowledged in principle but never resolved in practice: the application pool across the market is vast, yet each individual site typically requires only a narrow set of outcomes. Each deployment needs personalisation once, at commissioning, and then again whenever the environment or the risk profile shifts.

The app store that never became a market

For the better part of a decade, the industry’s most visible answer to the personalisation problem was the “app store” model. The logic was straightforward: curate a marketplace of trained neural network algorithms, let integrators browse a catalogue, and download the right analytic for each job. Queue counting for a passport control hall. License plate recognition for a parking structure. Occupancy monitoring for a conference room. The concept borrowed directly from the consumer smartphone approach.

In practice, it never matched physical security’s purchasing and operating rhythm. A phone owner discovers and downloads new apps continuously. A physical security deployment selects one or two analytics functions at installation and rarely revisits them.

Another maintenance burden

A queue-counting algorithm trained on airport data is excellent at queue counting

The economic incentive to maintain, curate, and update a broad catalogue across a fragmented ecosystem of camera OEMs, VMS platforms, and system integrators never materialised when the average buyer drew from only a thin slice of it. And the question of who would operate such a marketplace across that fragmented landscape was never satisfactorily answered.

The deeper issue is that distribution was not the hard part. Personalisation was. A queue-counting algorithm trained on airport data is excellent at queue counting. It does not naturally become a general-purpose security tool for whatever the operator needs next. Once a model is trained for a narrow task, adaptation requires another project, another integration cycle, and another maintenance burden.

AI-enabled cameras

The examples that do exist are instructive. Schiphol Airport in the Netherlands has used trained camera systems for over a decade to measure queue length at passport control and alert staff when additional counters should open. Rome trailed AI-enabled cameras to track pedestrian wait times at crosswalks, measure bus queue length, and monitor parking occupancy to support active transport and reduce vehicle emissions.

These are effective, well-regarded deployments. They also illustrate the limitation: each required its own trained model, its own integration effort, and its own maintenance cycle. The queue-counting camera at Schiphol cannot be redeployed to detect an abandoned bag. That is a separate algorithm, a separate procurement, and a separate project.

What changes with agentic AI

Applied to physical security, this translates into a simpler commissioning experience

Agentic AI points to a fundamentally different approach. An agentic system can receive goals expressed in natural language, determine the appropriate actions to fulfil those goals, execute those actions using available tools, and verify the results. Applied to physical security, this translates into a simpler commissioning experience: the operator expresses intent in plain language, and the system configures itself to achieve that intent.

Consider the practical implications. An installer commissioning cameras at a retail location could type or speak a set of instructions: “Alert the manager if more than five people are waiting at checkout for longer than two minutes.” A facilities director could ask the system to “Track vehicles that enter the east parking lot after 9 p.m. and flag any that remain for more than 30 minutes.” A school security coordinator might specify: “Notify campus police if anyone approaches the perimeter fence between midnight and 5 a.m.”

Appropriate perception capabilities

None of these instructions require the operator to select a specific analytic from a catalogue, configure a detection model, or define pixel-level zones in a complex VMS interface. The system interprets the intent, selects the appropriate perception capabilities, configures thresholds and context, and validates behaviour over time. When the operator’s needs change, a new instruction replaces the old one. The camera hardware stays the same. The AI adapts.

This is the core of the shift: minimal user input, maximum flexibility, and a security system that personalises itself without requiring the operator to navigate the traditional customise-certify-deploy cycle.

Vision language models make it practical

A conventional neural network trained for people counting can count people

The enabling technology is the vision language model, or VLM. A VLM combines visual encoders with language reasoning, allowing it to interpret images or video in the context of natural language prompts. This is a qualitative leap beyond traditional convolutional neural networks, which classify or detect predefined objects and have no mechanism for open-ended interpretation.

A conventional neural network trained for people counting can count people. It cannot distinguish between a crowd of commuters exiting a train station and a crowd assembling in protest. A VLM, by integrating contextual reasoning with visual analysis, can draw inferences that a task-specific model cannot. It can assess behavioural patterns, interpret spatial relationships, and respond to queries about scenes it has never been explicitly trained to analyse. Where a neural network might register two people carrying objects, a VLM could infer whether the scene suggests travellers with luggage or workers transporting equipment, provided the visual context supports that inference.

Supporting multimodal input

This matters in physical security because operational questions are rarely phrased as taxonomy labels. Operators want to express outcomes. They want to say “show me anything unusual near the loading bay after hours,” and the system should be able to reason about what “unusual” means given the site context.

VLMs also support multimodal input. Audio cues such as a raised voice, a scream, an alarm, or breaking glass can contribute to scene interpretation when paired with video. In security applications, where events routinely unfold across both visual and auditory channels, this capability adds a meaningful layer of situational awareness.

The edge constraint that forces discipline

Large language models in the cloud use hundreds of billions of parameters and consume hundreds of watts

None of this works if the architecture assumes data centre conditions. Most surveillance cameras operate under strict power and thermal limits. Power over Ethernet (PoE), the standard delivery mechanism, typically provides between 15 and 30 watts depending on the PoE class, and only a fraction of that budget is available for AI processing after the sensor, ISP, video encoder, and network stack have taken their share. In many installations, the AI workload must fit within a few watts.

Large language models in the cloud use hundreds of billions of parameters and consume hundreds of watts. That scale does not translate to a camera mounted on a pole or embedded in a ceiling tile. For agentic AI to work at the edge of a physical security network, the models must be compact, efficient, and designed for the purpose.

Neural network acceleration

This is where smaller, domain-specific VLMs become essential. Models trained on industry-relevant image and text datasets, combined with techniques such as pruning, quantisation, and parameter-efficient fine-tuning, can deliver meaningful visual reasoning within the compute and memory constraints of an edge processor. The result is a VLM that fits inside a camera’s power budget and still responds to natural language instructions with useful accuracy.

Ambarella’s CVflow AI architecture, now in its third generation, was designed for this class of workload. The architecture integrates advanced neural network acceleration with high-resolution image signal processing and video encoding on a single system-on-chip, allowing cameras to run complex AI inference alongside their core imaging functions without exceeding the thermal and power boundaries that define edge deployments.

The company's latest addition to its portfolio, the 4-nanometer CV7, runs CNNs and vision language models concurrently across multiple video streams while consuming 20 percent less power than its predecessor. For infrastructure and robotic applications requiring heavier models, the 5-nanometer N1 family supports multimodal LLMs in multi-camera configurations.

Distributing intelligence across far edge, near edge, and cloud

This tier must respond in milliseconds and operate within a fixed power envelope

A workable agentic architecture for physical security distributes intelligence across three tiers, each matched to the processing demands and latency requirements of its role. At the far edge, inside the camera itself, the processor handles real-time perception: object detection, tracking, zone logic, and initial event classification.

This tier must respond in milliseconds and operate within a fixed power envelope. At the near edge, on a local gateway or network video recorder, a more capable processor orchestrates across multiple cameras, maintains state, correlates events, retrieves site-specific policies and procedures, and classifies incidents requiring more context than any single camera provides. At the cloud/server tier, available when connectivity permits, the system accesses heavier models for forensic analysis, fleet-wide analytics, model updates, and long-horizon reporting.

Periodic cloud access

This tiered approach keeps the most time-sensitive decisions local, where latency is lowest and data privacy is strongest. It also means agentic capabilities can scale incrementally. A small installation might run entirely at the far edge with periodic cloud access. A large campus might employ all three tiers, with near-edge orchestration coordinating PTZ patrol patterns across dozens of cameras while the cloud generates shift summaries and updates models based on fleet-wide telemetry.

In practice, a security workflow built on this pattern often combines real-time detection at the far edge, behaviour-tree orchestration at the near edge for multi-camera coordination, local retrieval over site playbooks, and conservative safe-mode escalation when system confidence is low. The discipline of deterministic guardrails and structured verification loops is essential in security operations, where unpredictable system behaviour is not acceptable.

A hybrid future, with VLMs orchestrating specialist models

The transition to agentic AI does not eliminate specialised neural networks

The transition to agentic AI does not eliminate specialised neural networks. Purpose-trained models will continue to deliver superior accuracy for well-defined, high-frequency tasks such as license plate recognition, face matching, and fire and smoke detection. In a mature agentic system, the VLM acts as an orchestrator. It handles open-ended perception and natural language interaction while routing to specialised models when a task demands their precision.

A PTZ camera at a transportation hub might receive the instruction “monitor the west concourse for unattended items.” The VLM interprets the request, manages the interface, and reasons over broader scene context.

Real-time video processing

When it identifies a candidate object, it routes to a dedicated abandoned-item classifier optimised for that specific validation step. The VLM orchestrates. The specialist model validates. The operator receives a refined, actionable alert.

That hybrid pattern places specific demands on the silicon. The processor must support both traditional CNN inference and generative AI workloads simultaneously while maintaining real-time video processing within the same power envelope. The value of a tightly integrated SoC, one that combines an advanced ISP, a deep learning accelerator, and a video encoder on a single die, is that it eliminates the multi-chip complexity and power overhead that would otherwise make this approach impractical at the edge.

Making agentic AI deployable for the ecosystem

Ambarella’s Developer Zone, launched at CES 2026, provides a centralised portal of tools

Physical security is built on a broad ecosystem of camera OEMs, VMS providers, independent software vendors, module builders, and system integrators. For agentic AI to reach the market at scale, these participants need model-ready tooling, reference workflows, and a practical path from prototype to production.

This is where developer ecosystems become part of the story. Ambarella’s Developer Zone, launched at CES 2026, provides a centralised portal of tools, optimised AI models, agentic blueprints, low-code templates, and documentation aimed at accelerating edge AI application development on Ambarella’s SoCs.

Common software stack

ISVs and integrators can evaluate models, prototype applications, and deploy using a common software stack that spans the company’s CV7 and N1 SoC families through the Cooper development platform. That consistency across the product range reduces per-project engineering cost and accelerates time-to-market for partners building perception and analytics solutions.

The point is broader than any single portal: agentic systems require components that have already been tested and optimised for edge deployment, so that integrators can focus on solving their customers' problems rather than rebuilding the AI pipeline from scratch. The ecosystem participants who lead the transition to agentic AI in physical security will be the ones with access to tooling that fits into their existing development and deployment processes.

What comes next

Physical security has searched for years for a scalable answer to personalisation

Physical security has searched for years for a scalable answer to personalisation. The app store model did not provide it. Manual configuration, while functional on a per-site basis, scales poorly across large portfolios of cameras and changing operational requirements.

Agentic AI offers a credible path forward because it aligns with how operators actually think. They express outcomes, not model specifications. They want systems that adapt to new requirements without repeated engineering cycles.

Traditional neural networks

With VLMs as the interface layer, smaller domain-specific models at the far edge, orchestration at the near edge, and disciplined verification loops throughout, personalisation can become a standard part of deployment rather than a custom project.

The building blocks are now in place. Power-efficient edge AI processors can run VLMs and traditional neural networks simultaneously. Developer ecosystems are maturing to support rapid prototyping and deployment. Reference architectures for distributing intelligence across far-edge, near-edge, and cloud tiers are solidifying. For an industry that already installs vast numbers of AI-capable cameras each year, the opportunity is to make the intelligence already embedded in those endpoints genuinely usable for the people who rely on them every day.