C ONTROLLED C HAOS The Inevitable Marriage of DevOps & Security Kelly Shortridge (@swagitda_) S4x20

Hi, I’m Kelly 2 @swagitda_

“Chaos isn’t a pit. Chaos is a ladder.” ― Petyr Baelish, Game of Thrones 3 @swagitda_

Software is eating the world. It’s on the amuse-bouche course in ICS. 4 @swagitda_

Infosec has a choice: marry DevOps or be rendered impotent & irrelevant 5 @swagitda_

Denying the future & the benefits of modern systems will only hurt ICS 6 @swagitda_

How should infosec control chaos & make a marriage to DevOps last? 7 @swagitda_

  1. DevOps Dominion 2. The Metamorphosis
  2. Time to D.I.E. 4. A Phoenix Rises 8 @swagitda_

DevOps Dominion

DevOps is not automation or “agile” 10 @swagitda_

DevOps is a mindset that unifies responsibility and accountability. 11 @swagitda_

Infosec can join DevOps or take a back seat to the future of systems 12 @swagitda_

Chaos & resilience is infosec’s future 13 @swagitda_

What are DevOps’s priorities?

Optimization of software delivery performance so tech delivers value 15 @swagitda_

Stability & speed don’t conflict – resilience & innovation are bffs 16 @swagitda_

Security drives stronger DevOps results. Now ICS security must evolve. 17 @swagitda_

The Metamorphosis

Partitioning of responsibility & accountability engenders conflict 19 @swagitda_

After this evolution, DevOps will be held accountable for security fixes 20 @swagitda_

What goals should infosec pursue in this evolution? 21 @swagitda_

And… why should infosec goals diverge from DevOps goals? 22 @swagitda_

Infosec has arguably failed, so “this is how we’ve always done it” is invalid 23 @swagitda_

The Security of Chaos

“Things will fail” naturally extends into “things will be pwned” 25 @swagitda_

Security failure is when security controls don’t operate as intended 26 @swagitda_

What are the principles of chaotic security engineering? 27 @swagitda_

  1. Expect that security controls will fail & prepare accordingly 28 @swagitda_

  1. Don’t try to avoid incidents – hone your ability to respond to them 29 @swagitda_

What are the benefits of the chaos / resilience approach? 30 @swagitda_

Benefits: lowers remediation costs & stress levels during real incidents 31 @swagitda_

Benefits: minimizes service disruption & improves confidence 32 @swagitda_

Benefits: creates feedback loops to foster understanding of systemic risk 33 @swagitda_

What other ways can infosec become more strategic? 34 @swagitda_

Time to D.I.E.

We need a model promoting qualities that make systems more secure 36 @swagitda_

Enter the D.I.E. model: Distributed, Immutable, Ephemeral 37 @swagitda_

Distributed: multiple systems supporting the same overarching goal 38 @swagitda_

Distributed infrastructure reduces risk of DoS attacks by design 39 @swagitda_

Immutable: infrastructure that doesn’t change after it’s deployed 40 @swagitda_

Servers are now disposable “cattle” rather than cherished “pets” 41 @swagitda_

Immutable infra is more secure by design – ban shell access entirely 42 @swagitda_

Unlimited lives is better for security than game over upon death 43 @swagitda_

Ephemeral: infrastructure with a very short lifespan (dies after a task) 44 @swagitda_

Ephemerality creates uncertainty for attackers (persistence = nightmare) 45 @swagitda_

Installing a rootkit on a resource that dies in minutes is a waste of effort 46 @swagitda_

ICS attacks take months to plan; ephemerality constantly disrupts it 47 @swagitda_

Optimizing for D.I.E. reduces risk by design & supports resilience 48 @swagitda_

A Phoenix Rises

Harness failure as a tool to help you prepare for the inevitable 50 @swagitda_

Game days: practice risky scenarios 51 @swagitda_

Prioritize game days based on potential business impacts 52 @swagitda_

Decision trees: start at target asset, work back to easiest attacker paths 53 @swagitda_

Determine the attacker’s least-cost path (hint: it doesn’t involve 0day) 54 @swagitda_

Architecting chaos

Begin with “dumb” testing before moving to “fancy” testing 56 @swagitda_

Think digital twins, analytics services, or O365… not field-level SCADA 57 @swagitda_

Controlling Chaos: Distributed 58 @swagitda_

Distributed mostly overlaps with availability in modern infra contexts 59 @swagitda_

Chaos Monkey: inject random instances failures to test resilience 60 @swagitda_

Infosec teams can use these tools but make attackers the source of failure 61 @swagitda_

Multi-region services present a fun opportunity to mess with attackers 62 @swagitda_

Shuffle IP blocks regularly to change attackers’ lateral movement game 63 @swagitda_

Controlling Chaos: Immutable 64 @swagitda_

Volatile environments with continually moving parts raise the cost of attack 65 @swagitda_

Create rules like, “If there’s ever a write to disk, crash the node” 66 @swagitda_

Attackers must stay in-memory, which hopefully makes them cry 67 @swagitda_

Metasploit Meterpreter + webshell: Touch passwords.txt & kaboom 68 @swagitda_

Infosec teams can build Docker images with a “bamboozle layer” 69 @swagitda_

Mark garbage files as “unreadable” to craft enticing bait for attackers 70 @swagitda_

Potential goal: self-healing edge devices with immediate reversion 71 @swagitda_

Test: inject attempts at writing to disk to ensure detection & reversion 72 @swagitda_

Controlling Chaos: Ephemeral 73 @swagitda_

Most infosec bugs are stated-related – get rid of state, get rid of bugs 74 @swagitda_

Reverse uptime: longer host uptime adds greater security risk 75 @swagitda_

Test: retrograde libraries, containers, other resources in CI/CD pipelines 76 @swagitda_

Leverage lessons from toll fraud – cloud billing becomes security signal 77 @swagitda_

Test: exfil TBs or run a cryptominer to inform billing spike detection 78 @swagitda_

Conclusion

Security cannot gatekeep DevOps. It must marry it. 80 @swagitda_

Chaos/resilience are natural homes for infosec & represent its future. 81 @swagitda_

Infosec must now evolve to unify responsibility & accountability. 82 @swagitda_

ICS is already cloudy – get ready now before OT migrates as well. 83 @swagitda_

Giving up control isn’t a harbinger of doom. Resilience is a beacon of hope. 84 @swagitda_

“You must have chaos within you to give birth to a dancing star.” ― Friedrich Nietzsche 85 @swagitda_

@swagitda_ /in/kellyshortridge kelly@greywire.net 86 @swagitda_