Machine learning focuses on the development of computer programmes that can teach themselves to grow and change through exposure to new data
The need for security convergence and shared threat intelligence is markedly increasing

“Converged security” has been a buzz phrase for more than a decade, but the industry is just now starting to reap the rewards. Converged security recognises that truly comprehensive organisational risk management involves the integration of two distinct security functions that have largely been siloed in the past: information security (network operations centre or NOC) and physical security (security operations centre or SOC). In fact, “siloed” may be a nicer way of saying that these people historically have had no desire or ability to work together.

NOC and SOC convergence

That situation has been acceptable in the past but the need, and in some cases requirement, for security convergence and shared threat intelligence is markedly increasing and clearly more important today than ever before. The recent slew of successful attacks that all had predictive indicators that were overlooked because of highly segmented data collection and analysis are solemn reminders that the vulnerabilities are real. Organisations are tasked with keeping people and other assets safe, and to do that effectively, they must encourage cooperation between both the NOC and SOC functions, as they are inextricably linked. In the most recent tragedies, there were unlinked predictors on the cyber side that were discovered after the fact. In the past, physical assets merited the most attention in security protection, but today’s organisations are data driven and many of these traditionally physical assets are now information-based.

These two security worlds are markedly different. Security in a NOC often is focused on information like raw network traffic, security and audit logs, and other similarly abstract data that requires some interpretation as to what it could possibly mean. Points of emphasis in a SOC are video camera feeds and recordings, physical identity and access logs, fire safety, and many other important but largely tangible data. In an optimal security environment, the NOC and the SOC rely on each other, so today’s security professionals must be aware of the goals of the “other side.”

Modern threats are linked to
each
other, meaning that
there’s rarely a
physical threat
that didn’t originate
from a
network touchpoint at some

point during planning or
execution

What’s driving (or enabling) convergence on the IT side for many organisations is the ongoing analogue-to-IP video conversion that started some time ago and some heavy investments in IT infrastructure (often for other areas of the business), which have led to easier access to sensor connectivity. This, combined with the continuously decreasing cost of network bandwidth and data storage, has removed the last big obstacles to widespread use. Further pressure on the outcomes comes from the intelligence perspective where modern threats are linked to each other, meaning that there’s rarely a physical threat that didn’t originate from a network touchpoint at some point in the planning or execution phase. That reality has led in some cases to the obliteration of walls between NOCs and SOCs, creating a “fusion centre” or “SNOC.”

Convergence challenges

Although necessary, there are some notable challenges to convergence, best served through the integration of people, processes, preferred solutions in the cyber and physical security space, and the analysts’ knowledge base, meaning that security officers, for example, have different training than cyber analysts.

Different “personalities” often are observed within organisations that are tasked with security. The cyber team, for example, might be comprised of millennials who have highly technical skill sets because they grew up in the Internet age (digital natives). Those in physical security, on the other hand, might be comprised of former city/state law enforcement, former military or government service protecting physical assets, who are often more senior and didn’t grow up with technology at their fingertips. As a result, these personalities sometimes don’t “mix” naturally, so extra effort is needed to break down barriers that isolate the roles into separate business units, in completely separate operation centres, or sometimes on opposite sides of the country.

Security in a NOC often is focused on information like raw network traffic, security and audit logs, and other similarly abstract data
Cyber and physical security professionals often have different knowledge, personalities and training that hinder cooperation

Because these operators/analysts come from different backgrounds, have different areas of responsibility, and because their response workflows rarely intersect, a question emerges: Would a typical operator in the SOC think to even call the NOC if the operator saw something suspicious that could relate to the cyber side? Some NOCs are unaware that the SOC even exists, and if they are, they don’t know what the SOC is monitoring. The key to success is cross-training. For greater context and threat identification/mitigation, operators need to be familiar with the physical and logical risk, solutions, and cross-escalations.

The challenge of going traditional

Even in a converged security environment, traditional security detection systems produce a range of challenges in keeping organisations secure. Among them are:

Weak, independent alert streams: Most threat detection systems today are limited to a single data type – physical or cyber – and often these best-of-breed solutions are niched into a specific use (or division) within the department. For example, a large metropolitan transportation authority might have a physical security team with a dedicated fare evasion department – and, thus, leverage multiple cutting-edge solutions, including some machine learning, in support of a very specific objective rather than looking holistically at how to apply the technology across the organisation.

Cost of alarm investigation: Operators are inundated with data and “false alarms” in their day-to-day work. For example, on a large, urban college campus, SOC operators are responding to 911 (blue phones), LPR, unit dispatch, video analytics, and access control. The challenge of data inundation and false alarms cause them to average just over three minutes in issuing the required acknowledgement. In some cases this is actually considered a “good” response time. 

In short, false alarms without
context  or relevance and data
inundation require enormous
time and resources from
organisations

In another example, a major metropolitan city‘s police department, operators attempt to proactively keep the public safe and direct response resources by monitoring almost 2,000 cameras online (easily 48,000 hours of recorded footage per day). In an 18-month period, only one time did they actually catch an anomalous event as it happened with a camera operator looking at the monitor at the exact right time. Every other incident had to be found after the fact. In short, false alarms without context or relevance and data inundation require enormous time and resources from organisations and in most cases, are making real-time or even rapid response impossible.

Interpretability of alerts: Even when an alert is issued, the hardest thing to figure out through many systems is (1) why the alert was issued and (2) is there a recommended action/workflow.

Alerting rules start bad but get worse: Considering data inundation and the incidence of false alarms, traditional systems don’t adjust themselves to stop providing alerts that eventually are deemed to not be useful and don’t teach themselves to provide more relevant alerts that merit further investigation. Over time, a system that started with a large volume of alerts and a manageable amount of false alarms eventually becomes a system of mostly false alarms.

A machine learning system will connect to known threat libraries to help classify new anomalies and recommend mitigation steps
The next attack will not look like the last so we need an intelligent system that will identify the unexpected

How machine learning can help solve the challenges

So given the limitations of traditional systems, increasingly machine learning security systems are being used to address the challenges. Machine learning is a facet of artificial intelligence that provides computers with the ability to learn without being explicitly programmed or configured. Machine learning focuses on the development of computer programs that can teach themselves to grow and change through exposure to new data. Giant Gray’s Graydient platform, for example, leverages machine learning in its integration with video, SCADA or cyber technology to reduce false alarms in “teaching itself” what’s normal behaviour in a given setting. Machine learning addresses the limitations of traditional systems by:

Reducing the cost of alarm investigation with intelligent prioritisation: In a traditional rules-based system, the logic is largely black and white. There is either a violation of a rule or there isn’t. All alerts are treated equally. In an unsupervised machine learning system, the logic to determine the likelihood or “unusualness” of an event can be based on an ever-evolving body of highly detailed knowledge. As a result, it offers the ability to rate the unusualness of any given event. With the ability to dynamically rank alerts, those alerts can be classified based on this unusualness score. 

Machine learning focuses on
the development of computer
programs that can teach
themselves to grow and
change through exposure to
new data

A typical machine learning event ranking in a given period might be: Four alarms requiring acknowledgement, seven worth investigating, and 10 informational-only alerts that don’t create tickets. A perfectly configured traditional rules-based system in the same period would generate 21 equally ranked alerts that all require human interpretation. That said, optimally configured rules are rare and get worse with time, so the expectation might be to expect hundreds of equally ranked alerts in the same period that all need human review.

Context through combining traditionally disconnected alert sources: Machine learning systems leverage a composite sensor, a collection of individual sensors of various types that the system will learn and alarm as a whole based on the relationships between the member sensors. For example: When Object-A exhibits this behaviour, Object-B typically exhibits another behaviour within a certain time. The system will alert if the expected correlated action doesn’t occur.

External threat-intelligence: A system will connect to known threat libraries to help classify new anomalies and recommend mitigation steps. No one likes to see an “unknown” or “unk” classification, so many of the leading SIEMs have this functionality built in.

Automatic self-improvement: Feedback loops must be guarded and learned. There always will be risk that a human’s input can corrupt a learning system, which could result in undesirable output. This risk is mitigated with continuous learning, where we‘re either reinforcing memories or driving memory decay (forgetting) based on what we see. This approach adapts to changing conditions and can prevent long-term, heavy handed feedback.

Why machine learning is required in security

  • There is no baseline training data we can use to create reliable system rules or to train supervised learning systems;
  • We cannot manually keep pace with change, so we have to have a system that continuously adapts, learning the new environment or condition and forgetting the old;
  • Modelling and rules are the most effective they will ever be on the day they’re programmed. The next attack will not look like the last so we need an intelligent system that will identify the unexpected;
  • The most dangerous threats we all face are the ones that have never been seen before. They can’t be predicted, and therefore, we cannot program a rule or build a model for something that we can’t quantify.
Download PDF version

Author Profile

Cody Falcon Vice President, Solutions & Services, Giant Gray

In case you missed it

Drawbacks of PenTests and ethical hacking for the security industry
Drawbacks of PenTests and ethical hacking for the security industry

PenTesting, also known as “ethical hacking” or “white-hat hacking,” has always been viewed as the “sexy” side of cybersecurity, a task that is far more exciting than monitoring systems for intrusions, shoring up defenses, or performing compliance audits. Numerous security conferences are devoted to the fine art of attempting to hack into systems – with an owner’s full knowledge and permission – and reporting on the results. At an organisational level within businesses, they also value PenTesting under the premise that it allows them to identify security vulnerabilities before cyber criminals can. There are some regulatory requirements like PCI-DSS that require penetration assessments as part of their PCI compliance. However, many organisations have come to over-rely on PenTesting, thinking that if all the issues were identified in a PenTest, they’re good to go. Not only is this not helping them improve their security posture, it is also leaving them with a false sense of security. A penetration test is a simulated, live attack on your environment by a white-hat hacker What is PenTesting? A penetration test is a simulated, live attack on your environment by a white-hat hacker, customised to address specific problem areas, such as web-based applications, mobile applications and infrastructure services like border VPNs and firewalls. The PenTest may include different types of attacks based on the requested scope from an organisation so that the tester attempts to come at each system from all sides, the way a cyber-criminal would. The goal is to identify which systems and data the tester was able to access and how an organisation can address the vulnerabilities that allowed them to get in. The limitations of PenTesting There is great value in performing periodic PenTests, which is why PCI DSS and other security standards mandate them. However, PenTesting has three significant limitations: PenTesting does not provide solutions Let’s be honest: No one likes reading technical reports, but typically, that's the only deliverable provided by a PenTester. The value of a PenTesting report varies wildly based on the scope of the testing, the PenTester’s technical expertise and their writing ability. The tester may miss some things, or not clearly convey their findings. Additionally, a PenTest is a snapshot in time and the PenTester could miss changes in the systems, configurations, attack vectors and application environments. Even if your system “passes” a PenTest, will it crumble in the face of a brand new, more powerful attack vector that emerges a week later? The worst type of “PenTest report” consist of an analyst producing nothing more than the results of a vulnerability scan. Even if the PenTester produces a well-written, comprehensive report filled with valuable, actionable information, it’s up to your organisation to take the action, which leads to the next limitation of PenTesting. The value of a PenTesting report varies wildly based on the scope of the testing, the PenTester’s technical expertise and their writing ability PenTesters only exploit vulnerabilities and do not promote change PenTesting does not highlight the missing links in your organisation's technology stack that could help you address your security vulnerabilities. This is often in the guise of being agnostic to the technologies that exist because their expertise is only offensive security – unless, of course, the performing company has “magic software” to sell you. PenTests also do not help to develop your organisational processes. Additionally, they do not ensure that your employees have the knowledge and training needed to treat the identified fixes. Worst of all, if your in-house expertise is limited, any security issues that are identified during a PenTest aren't validated, which leads to a misrepresentation of their magnitude and severity while giving your team a false sense of security. PenTesters are self-serving Too often, PenTesting pits the assessment team against the organisation; the goal of the assessment team is to find the best way to "shame" the business into remediation, purchasing the testing company’s “magic software”, then call it a day. Once the PenTesters find, for example, a privilege escalation or a way to breach PII, they stop looking for other issues. The testers then celebrate the success of finding a single “flag”. In the meantime, the business is left in a precarious situation, since other unidentified issues may be lurking within their systems. Shifting the paradigm of PenTesting The goal of PenTesters is to find the best way to "shame" the business into purchasing the testing company’s “magic software”, then call it a day Penetration testing can uncover critical security vulnerabilities, but it also has significant limitations and it’s not a replacement for continuous security monitoring and testing. This is not to say that all PenTesting is bad. PenTesting should be integrated into a comprehensive threat and vulnerability management programme so that identified issues are addressed. The purpose of a mature vulnerability management programme is to identify, treat and monitor any identified vulnerabilities over its lifecycle. Vulnerability management programme Additionally, a vulnerability management programme requires the multiple teams within an organisation to develop and execute on the remediation plan to address the vulnerability. A mature threat and vulnerability management plan takes time and is helpful to partner with a managed security services provider (MSSP) to help you in the following areas: Improve your cyber-risk management program so that you can identify and efficiently address vulnerabilities in your infrastructure, applications and other parts within your organisation’s ecosystem on a continuous basis; Perform retests to validate any problems identified through a vulnerability scan or a PenTest assessment; Ensure that your in-house staff has the knowledge, skills and tools they need to respond to incidents. Cyber risk management and remediation is a "team sport." While periodic testing conducted by an external consultant satisfies compliance requirements, it is not a replacement for continuous in-house monitoring and testing. To ensure that your systems are secure, you must find a partner who not only performs PenTesting but also has the engineering and development experience to assist you in fixing these types of complex problems in a cost-effective manner and ensuring that your systems are hardened against tomorrow’s attacks.

Has the gap closed between security fiction and security reality?
Has the gap closed between security fiction and security reality?

Among its many uses and benefits, technology is a handy tool in the fantasy world of movie and television thrillers. We all know the scene: a vital plot point depends on having just the right super-duper gadget to locate a suspect or to get past a locked door. In movies and TV, face recognition is more a super power than a technical function. Video footage can be magically enhanced to provide a perfect image of a license plate number. We have all shaken our heads in disbelief, and yet, our industry’s technical capabilities are improving every day. Are we approaching a day when the “enhanced” view of technology in movies and TV is closer to the truth? We asked this week’s Expert Panel Roundtable: How much has the gap closed between the reality of security system capabilities and what you see on TV (or at the movies)?

How moving to Security as a Service benefits both providers and end users
How moving to Security as a Service benefits both providers and end users

The way we purchase services and products is changing. The traditional concept of buying and owning a product is giving way to the idea that it is possible to purchase the services it offers instead. This approach has come from the consumer realisation that it is the outcome that is important rather than the tools to achieve it. For example, this approach is evident with the rise of music streaming services as opposed to downloads or physical products.   With the physical security industry becoming ever more integrated – and truly open systems now a reality – there is every reason to assume this service-lead trend will come to dominate the way our industry interacts with its clients as well. Interest in service-based security There is a significant change of mindset that the security industry needs to embrace before a large-scale move to Security as a Service can take place. Like many technology sectors in the past, security providers have focussed on ‘shifting boxes’ as their definitive sales model. This approach was especially prevalent when proprietary systems were the mainstay of the security industry. Essentially, if the customer wanted more services they simply bought a new product. This was a straightforward and economic sales approach for manufacturers and installers alike.The security industry needs to embrace a change of mindset before a move to SaaS can take place The flexibility of integrated and open technology has changed the way consumers view their purchase, so it shouldn’t be any surprise that there is increased interest in a service-based approach. Customer choice equates to a change of focus and interest, with physical products being eclipsed by the benefits of the overall solution. We have already seen these changes in other technology areas, notably with smart devices and general IT systems. Cloud-based services put the onus on the result rather than which device the user chooses. This approach is even starting to manifest in areas that couldn’t have been predicted in the past, such as the car industry for example. Consumers are focusing more on the overall costs and convenience of buying a car over the specific specification of the vehicle. Equally, urban dwellers don’t necessarily want the hassle and expense of owning and parking their own vehicle anymore. If you don’t use it every day, it can make more sense to rent a vehicle only when you travel beyond public transport. For these consumers the car has become a service item for a specific journey. Benefits for end users At the heart of this approach is the simple equation that consumers have a need and suppliers need to provide the most cost-effective, and easiest, solution. At the same time, the security operator may not necessarily want to know (or care) what specification the system has, they just want it to perform the task as required.   By discussing with consumers, we can ensure we work even more closely with them to provide the expert support they need and deserve Most security buyers will identify the specific business needs and their budget to achieve this. This is where a service approach really comes into its own. Customers need expert advice on a solution for their requirements which takes away the stress of finding the right products/systems. In the past there was always a risk of purchasing an unsuitable solution, which could potentially be disastrous. The other issue was having to budget for a big capital expenditure for a large installation and then having to find further resources once an upgrade was due when systems went end of life. Most businesses find it far easier to pay a sensible monthly or annual fee that is predictable and can easily be budgeted for. A service model makes this far easier to achieve. Benefits of a service sales model As well as the benefits for end users, there are considerable benefits for security providers too. Rather than simply ‘shifting boxes’ and enduring the inevitable sales peaks and toughs this creates; a service sales model allows manufacturers and installers to enjoy a more stable business model. You don’t have to win new business with every product, but rather sell ongoing services for a set period. Its highly likely that the whole security industry will start to take this approach over the next few years. Manufacturers are already well aware of this shift in customer expectations and are changing their approach to meet demands.There are major opportunities on offer in return for a change of perspective in the security industry With the service and leasing approach already firmly entrenched in other industries, this is well proven in a consumer market. The airline industry is a great example. Manufacturers understand that airlines need flexibility to upscale and downscale operations and therefore whole aircraft and even individual key components (such as engines or seating) can be leased as required. Using this approach, airlines can concentrate on what customers demand and not worry about the logistics of doing this. Manufacturers and leasing businesses provide assurances and guarantees of service time for aircraft and engines, taking care of the servicing and maintenance to ensure this delivery. This approach is just as well suited for the provision of security systems. Servicing the future security market Undoubtedly there are major opportunities on offer in return for a change of perspective in the security industry. However, this will involve substantial changes in some quarters to ensure the business model is aligned with the market. Overall, the security industry needs to not only develop the right systems for the market, but also to deliver them in the right way as well. This will ensure we work even more closely with customers to provide the expert support they need and deserve.