Measuring Success in Cybersecurity

These are legitimate questions. They deserve an answer. But, so does the question: “How do I know I'm healthy?”

Cybersecurity concerns are akin to health concerns. You can practice a healthy lifestyle; you can get regular medical check-ups; you can be vigilant in monitoring your own body. Still, the question remains, “How do I know I'm healthy?”

The answer is, “You don't.”

You do your best to stay healthy; you have tests performed, such as blood tests to monitor certain factors; but you can never know that you are healthy. Dozens of hidden diseases may lurk in your body. In some cases, the body's immune system eliminates an illness without your even knowing it was there, and some … well, some conditions prove fatal. In the worst case, a fatal disease appears with no warning.

Perhaps I've ruined your mood? Let's see if a dose of cybersecurity helps. I'll start with a quote from my favorite management consultant, Peter Drucker:

“There is nothing so useless as doing efficiently that which should not be done at all.”

So, let's make absolutely sure we identify the most important things we should be doing in cybersecurity

To start, make sure you have thought out your cybersecurity program and understand your assets, the threats you face, and your environment's vulnerabilities. Next, you'll want to make sure that you have developed and practiced your incident-response plan; that you are providing cybersecurity awareness training, engaging everyone from the CEO on down; and finally, that you've established good cybersecurity governance by separating IT (which creates value) from cybersecurity (which protects value).

Tolerance Thresholds

As you work through the list above, you'll immediately identify some of the most important tolerance thresholds for your business.

Maximum Tolerable Downtime (MTD)

This is the point where if you have not recovered, the impact becomes severe, or even intolerable. There are many examples of MTD and its variability. Consider a brokerage firm. If the firm is engaged in real-time transactions, then, the maximum tolerable downtime may very well be zero! How do you achieve zero MTD? Typically, via multiple, parallel running, real-time mirrored systems.

Alternatively, consider a real estate practice. They may well decide that they can afford to be down for — say — 24 hours before needing access to their documents.

Recovery Point Objective (RPO)

This is the point in time where you need to recover to. Examples of RPO will depend on the frequency and criticality of data updates. Consider a publishing archive. The data hardly ever changes, it is only added to. Their recovery point objective may well be the time of the last update. If that is quarterly, they may well be satisfied with a three-month RPO, accepting the risk that they may have to reload one quarter's worth of data.

Recovery Time Objective (RTO)

This is how long you can afford to wait before getting back to Business-As-Usual (BAU) or, at least, some acceptable version of BAU. Recovery Time Objectives can get tricky, because they may depend at different levels of acceptable business functionality. For example, a manufacturing plant may have a recovery time objective of 24 hours before needed to accept orders, but a recovery time objective of 1 hour on their industrial control systems. Ultimately, there needs to be an appropriate RTO definition for returning to 100% of BAU. This is always a unique number to each particular business.

You'll know that your cybersecurity program is well conceived and working if, following any incident, your MTD, RPO, and RTO were met.

Cybersecurity Program Development

During the time you and your team spend developing your cybersecurity program, you will be answering the following key questions, either business unit by business unit, or at the company level, depending on your firm's size, geography, even business sector.

Asset Review

Assets, for our purposes, are defined as “anything of value.” Asset valuations answer the question: “What are you protecting, exactly?” in very specific details.

Assets include your company's data and the supporting (curating and controlling) systems, processes, workflows, and people. I call all these “cyber assets.” Other examples of cyber assets include all intellectual property (designs, product information, research, etc.), personally identifiable information, personally identifiable health information, strategic plans, financial plans, merger and acquisition plans, tactics, etc.

How do you know if a cyber asset is of value? Simply ask: “What happens if this asset is destroyed? If it is corrupted? Or, if it becomes unavailable?” Continue with: “What happens if it becomes public? Or, falls into the wrong hands?” If the answer is “Nothing,” move on. Although, you do have to wonder why you have a worthless asset to begin with.

Your cyber asset catalog is more than a list of assets. It must also include metadata about each one of them, including (at a minimum): The asset's owner, custodian, physical location, confidentiality classification, criticality classification, impact classification, and the values for MTD, RPO, and RTO discussed above.

Finally, you will need to maintain an accurate list of asset-specific resources: These are the people who will be needed to bring the asset back to life within the RTO and at RPO. Remember, be very specific.

Business Impact Analysis (BIA)

BIA's answer the question “What happens when things go south on you?” Again, this a business-unit-by-business-unit exercise, and it must be led by the unit head, always in partnership with the cybersecurity and IT teams.

Guiding you through the BIA is the asset classification work you did, and your focus will be on how the impact classification per asset combines into a business impact for each business unit.

Threat Assessment

Threat assessments answer the question “Who's out to get you?” Broadly speaking, there are two types of threats: Internal threats, and external ones.

Internal threats are, by nature, extremely dangerous. All threats have motives, but insider threats have easier means and opportunity to wreak havoc. They take the form of employees, vendors, freelancers, even third party systems connected to your environment. Their motives are as varied as they are: Greed, politics, despair, vengeance, you name it. The fact that they are already inside your perimeter facilitates their choice of means. Is it a picture on their phone? A USB drive? Data corruption? Once inside the castle walls, an insider can work quietly for a very long time and unless proper controls are in place, they can remain undetected until it is far too late.

External threats include all the usual suspects you have been reading about: Script kiddies, Cybercriminals, Social hackers, Cyberspies, Hactivists, Cyberterrorists, and Cyberwarriors serving their country. Their motives range from activism and terrorism, to espionage and patriotism, to money.

Depending on the size and scope of your particular firm, threat assessments can go from relatively simple analysis, to having dedicated staff, tools, and subscriptions to threat intelligence feeds. How do you know where you fall? If anyone in your firm is worried about attacks from other countries, terrorists, or espionage, a then threat assessment is already a top priority.

Vulnerability Assessment

Finally, vulnerability assessments ask the question “How easy is it to get to you?” They focus on identifying weaknesses on information systems, security procedures, and control implementations that can be exploited to gain access. You test against vulnerabilities by hiring a third party to perform vulnerability and penetration testing on your behalf. Hopefully, the results of these tests can lead to remediation of the vulnerabilities, although this is by no means guaranteed. Some systems are too old and have no fixes for their vulnerabilities, or the fixes themselves can “break” other systems upstream or downstream.

In the process of doing all this work, you will also develop your very own Risk Register, and Risk Assessment. Those, along with the rest of your results, will guide your decisions in selecting a set of controls (preventative, detective, corrective, and compensating controls), all layered using a defense-in-depth strategy that places the controls in such a way that when one fails, the next one down steps-up and saves the day.

Best Practice Metrics

As a best-practice, controls are layered to achieve a defense-in-depth strategy so that if one control fails, the next one down step's up and saves the day.

The good news is as you deploy your controls you will have access to a whole host of useful metrics. In combination with your assessment, they become your “cybersecurity health check report.” They include:

• Vulnerability Metrics: How much vulnerability exists in my systems? What kind? How old? How many systems that are patched as per the manufacturer, and at what level? How timely is the systems/asset inventory? How effective is my physical security (stolen/lost devices reporting)?
• Detective Metrics: How soon was I alerted of a (possible) breach? What's is the mean time to detection? What is the mean time to response? What about with regard to my third party vendor incidents?
• System Information and Event Management (SIEM) Metrics: Who's reading them? How are they interpreted? How many times was action taken based on SIEM data? What is the number of systems events tracked? Degree of SIEM comprehensiveness? Number of alerts issued?
• Payload Metrics: What kind of nasty critters (viruses, malware, worms) did we thwart? Where did they come from? How were they delivered (This is called “vector analysis.”)
• Compliance Metrics: How many of my systems are compliant to external regulation standards?
• Cybersecurity Awareness Metrics: What are my employee training statistics? What are the Help Desk statistics before and after cybersecurity awareness training?

Conclusion

As you can see, there is no shortage of metrics. All of them contain valuable information that gauge the well-being of your cybersecurity program. And, as any hypochondriac out there will tell you — health tests and methods of wellness monitoring definitely ease the mind.

But, just like with your body, do not allow the illusion of a calm sea of metrics to lull you into complacency. With apologies to the hypochondriacs out there, the only thing a metric tells you is what happened in the past up to the moment of the test. They cannot, and do not, predict your cybersecurity (or health) future!

Because metrics are so numerous, use them critically. Pick the ones that are most meaningful to your firm, and most importantly, the ones that you and your teams can understand and act on. Then, stay vigilant; stay aware; remain prepared!

I'll close with another quote from my favorite management consultant, Peter Drucker: “Management is doing things right; leadership is doing the right things.”

Do the right thing for your firm. “Living cybersecure” is analogous to “living healthy.” It's not a once-and-done. It's an every-day endeavor involving adjusting, thinking, reacting, and planning for an unknown future.

*****

Chris Moschovitis is the CEO of the Information Technology Management Group, a New York based company focused on providing independent technology and cybersecurity managed services. He is both cybersecurity (CSX, CISM) and Enterprise IT governance (CGEIT) certified. Chris is the co-author of “History of the Internet: 1843 to the Present” as well as a contributor to the “Encyclopedia of Computers and Computer History” and the “Encyclopedia of New Media.” Chris' latest book “Cybersecurity Program Development for Business: The Essential Planning Guide” was published by Wiley in 2018.