Download PDF version Contact company

Gaps in video from dropped video frames are a common problem. A web search on the phrase “dropped video frames” turns up links oriented towards gamers, YouTube videographers and even the world of video surveillance, discussing why frame drops happen and providing advice for coping with them.

On Windows computers, video frame drops are a fact of life. Much of the advice can help reduce the number of frame drops, though never completely eliminate them. If you are an amateur filmmaker, and you drop key video frames, you can always do another “take.”

Not so in the 24/7 world of security surveillance. You never know which few seconds of video might be critical in determining what really happened, who was really there and who could be held liable.General use IT products like a Windows computer are architected to be event-driven and time-shared

Causes of video frame drops in surveillance systems

To begin to address this problem, it’s important to have a better understanding of the major causes of video frame drops in a video surveillance system.

Many video surveillance systems stream the output from the video cameras over a network to an off-the-shelf computer such as a Windows machine, which acts as the video server and runs your Video Management Software (VMS). The video server stores all the video files to racks of disk drives (disk drives being an affordable choice for the large amount of storage needed in a multi-camera system). Let’s focus on these major sources of video frame drops:

  • Video servers dropping incoming video frames when various system tasks compete for resources
  • Disk heads in your disk drives that can’t write the incoming video files fast enough to keep up with the continuous stream of data

    Many integrators who use off-the-shelf Windows computers will ensure the computer is dedicated to being the video server

Video processing overload

General IT products like a Windows computer are architected to be event-driven and time-shared. They strive for fairness and are designed to share the processing capacity so that no single task is starved. Instead, all tasks get a smaller proportion of processing capacity.

This means new events, like a keystroke or automatic start of a common “background” task or system service, can interrupt and delay ongoing tasks like the handling of your incoming video streams.

In the networked world, that is usually a good choice, since the network protocols will recover by resending the data and the latency that adds is not a problem. This does not work with video. Your cameras are continuing to send new video, so even if you could resend, this would simply bog down the computer still further and cause even more frames to drop.

If the CPU, memory or disk handling required by both the new event and your ongoing video processing exceeds the capacity of the computer, its default way of dealing with the overload is to drop data coming in over its network connections.

By using disk systems that are optimised for video storage, you can improve your odds
Disk drives are optimised for reading and writing data quickly, generally in a variety of block sizes depending on what is being stored

Dropped video frame testing

Recently, we started monitoring and testing for dropped video frames on off-the-shelf Windows video servers. In our tests, just clicking on a Windows disk management command resulted in almost 4,000 frames being dropped, an 8-second gap in the video record – and as we all know, a lot can happen in 8 seconds.

Even we were surprised by how often and how consistently frames were dropped, coinciding with various demands on the servers from things like background tasks. Almost always, these drops happened as a cluster, multiple seconds of contiguous video simply gone, a much more serious issue than dropping a few random frames here or there.The best solution is to use a machine and operating system that are optimised to be part of a video surveillance system

Many integrators who use off-the-shelf Windows computers will ensure the computer is dedicated to being the video server, and they will try to tweak the Windows operating system to reduce the number of background tasks, like removing any third-party antivirus programs or performance programs. This reduces the number of video frame drops but does not come close to eliminating all of them.

Video surveillance system solution

The best solution is to use a machine and operating system that are optimised to be part of a video surveillance system, designed to funnel resources as needed to handling the incoming video streams versus attempting to share resources evenly across all tasks or dropping incoming video data when the system is overloaded. By giving priority to the incoming video data and being sized correctly for the number and type of cameras, these systems are much more robust and less prone to dropping frames.

Optimising storage for surveillance

The previous sections looked at how video frames get dropped before your VMS even sees them. However, frames can go missing on the other side of the system when the storage system cannot keep up with the incoming video data.

Most systems use some form of disk storage; because to store all the video from multiple cameras running 24/7 can require a massive amount of storage – frequently into the terabytes – and disk storage is still affordable on this scale.

Disk drives are optimised for reading and writing data quickly, generally in a variety of block sizes depending on what is being stored. If you think about a database, various data records are written once, read many times as people look up data and modified (written again). Storing the video output from multiple cameras breaks this model in several ways:

  • The block sizes for video data are larger than typical data files
  • The blocks are written once and very seldom read, so it is very write-intensive
  • New files from multiple cameras are continuously coming in to be written
    When you need high performance and reliability from a system that faces heavy demands, off-the-shelf may simply not be good enough

Avoid missing key video

All these factors keep the disk head extremely busy and lead to longer seek times. Essentially, the disk head cannot write fast enough and cannot keep up with the incoming stream of video data to be stored. The result is that some video frames get dropped.

By using disk systems that are optimised for video storage, you can improve your odds. Solutions that improve head seek time, consolidate multiple video files into one file to reduce the number of writes needed and find other ways for the disk head to work more efficiently can greatly reduce the chances that frames might be dropped.

The off-the-shelf components that are frequently used for video surveillance systems are general purpose machines that do a decent job of supporting a small system with a minimal number of cameras. However, even in a small system with only 3 to 4 cameras, our testing has shown that video frame drop is a given, due partly to Windows’ bias to share resources among tasks and default to dropping incoming data/video streams when busy, and partly to off-the-shelf disk drives that cannot keep up with the write-intensive nature of storing video feeds.

These issues are due to the size and nature of video files, but are not unique to surveillance systems. In the film industry, they too use custom machines as their video servers to handle incoming streams from their digital cameras and store those streams to custom storage hardware. When you need high performance and reliability from a system that faces heavy demands, off-the-shelf – though less expensive initially – may simply not be good enough. Only you can determine how much tolerance you have to the risk of missing key video sequences, and whether your potential liability and the risk of lawsuits is increased beyond an acceptable measure by a surveillance system that captures less than 100% of the action.

Download PDF version Download PDF version

Author profile

Sean Chang Co-Founder, CEO & President, Rasilient Systems, Inc.

Sean has been the co-founder, CEO, and President of Rasilient Systems since its inception in 2001 in Silicon Valley, California. Before starting Rasilient, Sean worked at Lucent, HP, HP Labs, and several startups. Sean’s career centers on video and Internet technologies. While at HP and HP Lab, he worked on Video On Demand and associated MPEG Transport technologies. He also led the development of LMDS broadband wireless system in Lucent. While at MCNC, he participated in IETF Video Conferencing work Group which let to RTSP and SIP protocols. He has five patents and 20 plus publications in video and broadband. Sean received his PhD in Electrical and Computer Engineering from Illinois Institute of Technology in Chicago in 1991.

In case you missed it

The physical side of data protection
The physical side of data protection

The impact of the COVID-19 pandemic has accentuated our digital dependency, on a global scale. Data centres have become even more critical to modern society. The processing and storage of information underpin the economy, characterised by a consistent increase in the volume of data and applications, and reliance upon the internet and IT services. Data centres classed as CNI As such, they are now classed as Critical National Infrastructure (CNI) and sit under the protection of the National Cyber Security Centre (NCSC), and the Centre for the Protection of National Infrastructure (CPNI). As land continues to surge in value, data centre operators are often limited for choice, on where they place their sites and are increasingly forced to consider developed areas, close to other infrastructures, such as housing or industrial sites. Complex security needs One misconception when it comes to data centres is that physical security is straightforward One misconception when it comes to data centres is that physical security is straightforward. However, in practice, things are far more complex. On top of protecting the external perimeter, thought must also be given to factors, such as access control, hostile vehicle mitigation (HVM), protecting power infrastructure, as well as standby generators and localising security devices to operate independently of the main data centre. Face value How a site looks is more important than you may think. Specify security that appears too hostile risks blatantly advertising that you’re protecting a valuable target, ironically making it more interesting to opportunistic intruders. The heightened security that we recommend to clients for these types of sites, include 4 m high-security fences, coils of razor wire, CCTV, and floodlighting. When used together in an integrated approach, it’s easy to see how they make the site appear hostile against its surroundings. However, it must appear secure enough to give the client peace of mind that the site is adequately protected. Getting the balance right is crucial. So, how do you balance security, acoustics and aesthetics harmoniously? Security comes first These are essential facilities and as a result, they require appropriate security investment. Cutting corners leads to a greater long-term expense and increases the likelihood of highly disruptive attacks. Checkpoints Fortunately, guidance is available through independent accreditations and certifications, such as the Loss Prevention Certification Board’s (LPCB) LPS 1175 ratings, the PAS 68 HVM rating, CPNI approval, and the police initiative - Secured by Design (SBD). Thorough technical evaluation and quality audit These bodies employ thorough technical evaluation work and rigorous quality audit processes to ensure products deliver proven levels of protection. With untested security measures, you will not know whether a product works until an attack occurs. Specifying products accredited by established bodies removes this concern. High maintenance Simply installing security measures and hoping for the best will not guarantee 24/7 protection. Just as you would keep computer software and hardware updated, to provide the best level of protection for the data, physical security also needs to be well-maintained, in order to ensure it is providing optimum performance. Importance of testing physical security parameters Inspecting the fence line may seem obvious and straightforward, but it needs to be done regularly. From our experience, this is something that is frequently overlooked. The research we conducted revealed that 63% of companies never test their physical security. They should check the perimeter on both sides and look for any attempted breaches. Foliage, weather conditions or topography changes can also affect security integrity. Companies should also check all fixtures and fittings, looking for damage and corrosion, and clear any litter and debris away. Accessibility When considering access control, speed gates offer an excellent solution for data centres. How quickly a gate can open and close is essential, especially when access to the site is restricted. The consequences of access control equipment failing can be extremely serious, far over a minor irritation or inconvenience. Vehicle and pedestrian barriers, especially if automated, require special attention to maintain effective security and efficiency. Volume control Data centres don’t generally make the best neighbours. The noise created from their 24-hour operation can be considerable. HVAC systems, event-triggered security and fire alarms, HV substations, and vehicle traffic can quickly become unbearable for residents. Secure and soundproof perimeter As well as having excellent noise-reducing properties, timber is also a robust material for security fencing So, how do you create a secure and soundproof perimeter? Fortunately, through LPS 1175 certification and CPNI approval, it is possible to combine high-security performance and up to 28dB of noise reduction capabilities. As well as having excellent noise-reducing properties, timber is also a robust material for security fencing. Seamlessly locking thick timber boards create a flat face, making climbing difficult and the solid boards prevent lines of sight into the facility. For extra protection, steel mesh can either be added to one side of the fence or sandwiched between the timber boards, making it extremely difficult to break through. A fair façade A high-security timber fence can be both, aesthetically pleasing and disguise its security credentials. Its pleasant natural façade provides a foil to the stern steel bars and mesh, often seen with other high-security solutions. Of course, it’s still important that fencing serves its primary purposes, so make sure you refer to certifications, to establish a product’s security and acoustic performance. Better protected The value of data cannot be overstated. A breach can have severe consequences for public safety and the economy, leading to serious national security implications. Countering varied security threats Data centres are faced with an incredibly diverse range of threats, including activism, sabotage, trespass, and terrorism on a daily basis. It’s no wonder the government has taken an active role in assisting with their protection through the medium of the CPNI and NCSC. By working with government bodies such as the CPNI and certification boards like the LPCB, specifiers can access a vault of useful knowledge and advice. This will guide them to effective and quality products that are appropriate for their specific site in question, ensuring it’s kept safe and secure.

Data explosion: Futureproofing your video surveillance infrastructure
Data explosion: Futureproofing your video surveillance infrastructure

Video surveillance systems are producing more unstructured data than ever before. A dramatic decrease in camera costs in recent years has led many businesses to invest in comprehensive surveillance coverage, with more cameras generating more data. Plus, advances in technology mean that the newest (8K) cameras are generating approximately 800% more data than their predecessors (standard definition). Traditional entry-level solutions like network video recorders (NVRs) simply aren’t built to handle massive amounts of data in an efficient, resilient and cost-effective manner. This has left many security pioneers grappling with a data storage conundrum. Should they continue adding more NVR boxes? Or is there another, better, route? Retaining video data In short, yes. To future proof their video surveillance infrastructure, an increasing number of businesses are adopting an end-to-end surveillance architecture with well-integrated, purpose-built platforms for handling video data through its lifecycle. This presents significant advantages in terms of security, compliance and scalability, as well as unlocking new possibilities for data enrichment. All of this with a lower total cost of ownership than traditional solutions. Security teams would typically delete recorded surveillance footage after a few days or weeks Previously, security teams would typically delete recorded surveillance footage after a few days or weeks. However, thanks to increasingly stringent legal and compliance demands, many are now required to retain video data for months or even years. There’s no doubt that this can potentially benefit investigations and increase prosecutions, but it also puts significant pressure on businesses’ storage infrastructure. Data lifecycle management This necessitates a more intelligent approach to data lifecycle management. Rather than simply storing video data in a single location until it’s wiped, an end-to-end video surveillance solution can intelligently migrate data to different storage platforms and media as it ages. So, how does this work? Video is recorded and analysed on a combination of NVR, hyperconverged infrastructure (HCI) and application servers. Then, it’s moved to resilient file storage for a pre-determined period, where it can be immediately retrieved and accessed for review. Finally, based on policies set by heads of security, data is moved from file storage to highly secure, low-cost archive storage such as an object, tape or cloud. Data is moved from file storage to highly secure, low-cost archive storage Long-term storage This process is known as tiering. It allows businesses to use reliable, inexpensive long-term storage for most of their data, whilst still enabling security pioneers to retrieve video data when the need arises, such as during a compliance audit, or to review footage following a security breach. In a nutshell, it offers them the best of both worlds. Scaling your video surveillance infrastructure can be a headache. Businesses that rely on NVRs – even high-end units with 64 or even 96 hard drives – are finding themselves running out of capacity increasingly quickly. In order to scale, security pioneers then have to procure new boxes. With NVRs, this inevitably involves a degree of guesswork. Should they go for the largest possible option, and risk over provisioning? Or perhaps a smaller option, and risk running out of capacity again? Common management console Security pioneers can easily add or remove storage capacity or compute resources – separately or together As businesses add new cameras or replace existing ones, many end up with inadequate surveillance infrastructure made up of multiple NVR boxes along with several application servers for running other surveillance functions such as access control, security photo databases, analytics, etc. This patchwork approach leaves security pioneers scrambling for capacity, maintaining various hardware footprints, repeating updates and checks across multiple systems, and taking up valuable time that could be better spent elsewhere. By contrast, flexible HCI surveillance platforms aggregate the storage and ecosystem applications to run on the same infrastructure and combine viewing under a common management console, avoiding ‘swivel chair’ management workflows. Plus, they offer seamless scalability. Security pioneers can easily add or remove storage capacity or compute resources – separately or together. Data storage solutions Over time, this ensures a lower total cost of ownership. First and foremost, it removes the risk of over provisioning and helps to control hardware sprawl. This in turn leads to hardware maintenance savings and lower power use. Many security pioneers are now looking beyond simple data storage solutions for their video surveillance footage. Meta tags can provide context around data, making it easier to find and access when needed Instead, they’re asking themselves how analysing this data can enable their teams to work faster, more efficiently and productively. Implementing an end-to-end video surveillance architecture enables users to take advantage of AI and machine learning applications which can tag and enrich video surveillance data. These have several key benefits. Firstly, meta tags can provide context around data, making it easier to find and access when needed. Object storage platform For instance, if security teams are notified of a suspicious red truck, they can quickly find data with this tag, rather than manually searching through hours of data, which can feel like looking for a needle in a haystack. Plus, meta tags can be used to mark data for future analysis. This means that as algorithms are run over time, policies can be set to automatically store data in the right location. For example, if a video is determined to contain cars driving in and out of your premises, it would be moved to long-term archiving such as an object storage platform for compliance purposes. If, on the other hand, it contained 24 hours of an empty parking lot, it could be wiped. These same meta tags may be used to eventually expire the compliance data in the archive after it is no longer needed based on policy. Video surveillance architecture Continuing to rely on traditional systems like NVRs will fast become unsustainable for businesses Even if your organisation isn’t using machine learning or artificial intelligence-powered applications to enhance your data today, it probably will be one, three, or even five years down the line. Implementing a flexible end-to-end video surveillance solution prepares you for this possibility. With new advances in technology, the quantity of data captured by video surveillance systems will continue rising throughout the coming decade. As such, continuing to rely on traditional systems like NVRs will fast become unsustainable for businesses. Looking forward, when moving to an end-to-end video surveillance architecture, security pioneers should make sure to evaluate options from different vendors. For true futureproofing, it’s a good idea to opt for a flexible, modular solution, which allow different elements to be upgraded to more advanced technologies when they become available.

How can the security industry provide affordable and cost-effective solutions?
How can the security industry provide affordable and cost-effective solutions?

Cost is a reality to be managed. No matter how powerful or desirable a technology may be to a customer, the sale often comes down to the basic question: Can I afford it? And affordability extends not just to the purchase price, but to the cost of technology over its lifespan. In addition to advances in technology capabilities, the security industry has also achieved inroads to make its offerings more worth the cost. We asked this week’s Expert Panel Roundtable: What is the physical security industry doing to make more affordable and cost-effective technology solutions for end users?