Did you accept the risk? Dynamic risk metrics in your environment.
A presentation at SnykCon by Daniel "phrawzty" Maher
Did you accept the risk? Dynamic risk metrics in your environment. SnykCon 2020
a.k.a. The Safety Dance You can dance if you want to—but you can’t leave risk behind. SnykCon 2020
Your Safety Dancers for today Andrew Krug Daniel Maher
The dance card – Risk management (a classic!) – Real-world risk analysis (explosions!) – Risk in the context information technology (cyber!) – Virtual-world risk analysis (routing tables!) – How to shift-left to avoid specific risks ( !) – Walk through a sample application (dance party!)
Crash Course in Risk Risk 101
What is risk? What is safety science? Classic calculation R=f(s,p,c) Risk = f (scenario, probability, consequence)
What is risk? What is safety science? Kaplan and Garrick (1989) What can go wrong? What is the likelihood? (probability) How bad could it be?
Qualitative vs Quantitative Qualitative reasoning ranks likelihood using a scale score metric. 👍 👎 Speedy Light data gathering? Easy Low precision Light data gathering!
Qualitative vs Quantitative Quantitative reasoning uses data to reason about probabilities and consequences. 👍 👎 Accuracy Time Data Relative risks (adjacent)
Hybrid Models T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832019302649?via=ihub. [Accessed: 15-Oct-2020].
What is risk? What is safety science? Hybrid calculation R=f(s,p,c,k) Risk = f(scenario, probability, consequence, knowledge) knowledge = (mix of both quantitative and qualitative data)
Risk analysis in the real world
Real-world risk analysis Industrial Modeling T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S095183201 9302649?via=ihub. [Accessed: 15-Oct-2020].
Real-world risk analysis Oil Drilling 0. Initiating Event “Drive-off” 5. Outcome 1. Stop drive-off No damage 2. Manual EDS Potential wellhead damage 3. Automatic EDS 4. Drive-off Some wellhead damage Wellhead damage
Real-world risk analysis Oil Drilling CONSEQUENCES 0. Initiating Event Drive-off CONTROLS 5. Outcome
- Stop drive-off No damage
- Manual EDS Potential wellhead damage
- Automatic EDS
- Drive-off Some wellhead damage Wellhead damage
Pictures = 1000x words S. E. Services, Inc., “Offshore Drilling Design & Analysis: Stress Engineering,” Offshore Drilling Analysis Testing, 2020. [Online]. Available: https://www.stress.com/capabilities/upstream/offshore-drilling [Accessed: 15-Oct-2020].
Bowtie Model (Hybrid) Purpose Developed to reason about known and unknown risks where the impact is extremely high (ex. loss of life). What’s new? Describes control impacts on risk over time.
Risk in operations, development, and cyber security
Risk as it relates to operations Risk Impact Likelihood The site is down No sales Maybe? Front page of the orange site Strong maybe because metrics? Wake. Up. EVERYONE. 97p based on uptime over the past two years…
Risk as it relates to development Risk Impact Likelihood Failed feature release Time to rollback Reputation Lost revenues How often? Dropped sessions Lost revenue Troubleshooting Bugfix How much test coverage? UI Bugs Reputation Bugfix How much frontend testing?
Risk as it relates to cyber security Risk Impact Likelihood Supply chain vulnerability Time to patch Firefighting Panic ? SQL injection Damage to reputation ?? XSS User data theft Fines Lawsuits ???
Parallels to oil drilling “[A]ccount for issues of human/machine interface, stress, shortage of time, and rare-event experience, which may lead the operator into failure and increase the probability of an accident.” — Nicola Paltrinieri (1996)
Examples of risk in the context of information technology
Risk in the age of tech Triforce of fear! DDoS Failed Deploy Certificate Issues Load Issues Reputation Information Security Tampering SQL Injection Impersonation Authentication Bypass Productivity Data Exfiltration SQL Injection XSS Financial
Risk in the age of tech Triad of consequences! Headline news? Brand damage Customer trust Reputation Information Security How long to restore? How much forensics? Time to rollback? Employee Safety Doxxing Productivity Fines! Legal damages Identity insurance Financial
Risk in the age of tech Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Risk in the age of tech, in tech itself G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Shifting-left in order to deal with specific risk
Where can we shift left? Or get some tooling maybe? ● Web Applications ○ Mostly vulnerabilities ● Miscellaneous Errors ○ Mostly misconfigs and misfires ● Everything Else ○ Mostly phishing G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Where can we shift left? Or get some tooling maybe? ● Web Applications ○ Mostly vulnerabilities ● Miscellaneous Errors ○ Mostly misconfigs and misfires ● Everything Else ○ Mostly phishing G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].
Shifting-left with regards to risk Risk budgets for some? Tools Signal Visibility & Detection Signal Time-based risk
Shifting-left with regards to risk Risk budgets for most?
- Qualitative risk = Time-based risk Comprehensive Visibility https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html
Shifting left with regards to risk What is your risk budget based on?
Shifting left with regards to risk Base your budget on your data!
It’s demo time!
Demo time! event_probability = ( recommendations * max_impact ) a.k.a the more things are wrong the more likely an incident will occur
Demo time!
Demo time!
Show some tools that add risk SEV-1 SEV-2 SEV-3 SEV-4 SEV-5 INFO WARN ERROR CRITICAL Low 1 Medium 2 LOW MED HIGH High 3 Maximum 4
Example : CFN_Nag
Dynamic Risk A method for calculating : Assign a weight to the number of findings Low 1 Medium 2 High 3 WARNING = 1 point CRITICAL = 20 points likelihood = ( points * 0.25 ) Since we’re using a 4 point scale “id”: “W41”, “type”: “WARN”, “message”: “S3 Bucket should have encryption option set”, Maximum 4
Ship that to an aggregator
R=f(s,p,c,k)
So in summary…
In summary… a.k.a. the tl;dr – The best risk assessments incorporate qualitative and quantitative metrics – Your environment is already giving you risk signals – Risk is a tool that software development teams can add to their toolbox in order to help the business go fast and stay safe. – It’s not always wrong to accept SOME risk depending on your risk budget.
In summary… Keep the party going! https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html We’re going to have a free risk assessment training with one of the creators (@kangsterizer) of Mozilla RRA. Email or follow us on Twitter for more info! andrew.krug@datadog.com // @andrewkrug daniel.maher@datadog.com // @phrawzty
Risk management is relatively new to the security industry but in reality insurance teams, government, and finance have been using risk assessments to make decisions for years. In this talk we’ll demonstrate how to apply classical risk management concepts to modern DevOps practices. You’ll learn how to communicate across your organisation using a standard vocabulary to calculate risk at a service level, then see a demonstration of how to dynamically calculate and increase risk levels using Security Scores.
Resources
The following resources were mentioned during the presentation or are useful additional information.
-
Event listing
-
RRA assessment workshop
This training shows how to walk through a risk assessment for an individual service using the Mozilla Rapid Risk Assessment Framework developed by Guillaume Destuynder and Julien Vehent.
Buzz and feedback
Here’s what was said about this presentation on social media.
