Controlled Chaos: the Inevitable Marriage of DevOps & Security (Keynote)

A presentation at S4x20 in January 2020 in Miami Beach, FL, USA by Kelly Shortridge

Slide 1

C ONTROLLED C HAOS The Inevitable Marriage of DevOps & Security Kelly Shortridge (@swagitda_) S4x20

Slide 2

Hi, I’m Kelly 2 @swagitda_

Slide 3

“Chaos isn’t a pit. Chaos is a ladder.” ― Petyr Baelish, Game of Thrones 3 @swagitda_

Slide 4

Software is eating the world. It’s on the amuse-bouche course in ICS. 4 @swagitda_

Slide 5

Infosec has a choice: marry DevOps or be rendered impotent & irrelevant 5 @swagitda_

Slide 6

Denying the future & the benefits of modern systems will only hurt ICS 6 @swagitda_

Slide 7

How should infosec control chaos & make a marriage to DevOps last? 7 @swagitda_

Slide 8

DevOps Dominion 2. The Metamorphosis
Time to D.I.E. 4. A Phoenix Rises 8 @swagitda_

Slide 9

DevOps Dominion

Slide 10

DevOps is not automation or “agile” 10 @swagitda_

Slide 11

DevOps is a mindset that unifies responsibility and accountability. 11 @swagitda_

Slide 12

Infosec can join DevOps or take a back seat to the future of systems 12 @swagitda_

Slide 13

Chaos & resilience is infosec’s future 13 @swagitda_

Slide 14

What are DevOps’s priorities?

Slide 15

Optimization of software delivery performance so tech delivers value 15 @swagitda_

Slide 16

Stability & speed don’t conflict – resilience & innovation are bffs 16 @swagitda_

Slide 17

Security drives stronger DevOps results. Now ICS security must evolve. 17 @swagitda_

Slide 18

The Metamorphosis

Slide 19

Partitioning of responsibility & accountability engenders conflict 19 @swagitda_

Slide 20

After this evolution, DevOps will be held accountable for security fixes 20 @swagitda_

Slide 21

What goals should infosec pursue in this evolution? 21 @swagitda_

Slide 22

And… why should infosec goals diverge from DevOps goals? 22 @swagitda_

Slide 23

Infosec has arguably failed, so “this is how we’ve always done it” is invalid 23 @swagitda_

Slide 24

The Security of Chaos

Slide 25

“Things will fail” naturally extends into “things will be pwned” 25 @swagitda_

Slide 26

Security failure is when security controls don’t operate as intended 26 @swagitda_

Slide 27

What are the principles of chaotic security engineering? 27 @swagitda_

Slide 28

Expect that security controls will fail & prepare accordingly 28 @swagitda_

Slide 29

Don’t try to avoid incidents – hone your ability to respond to them 29 @swagitda_

Slide 30

What are the benefits of the chaos / resilience approach? 30 @swagitda_

Slide 31

Benefits: lowers remediation costs & stress levels during real incidents 31 @swagitda_

Slide 32

Benefits: minimizes service disruption & improves confidence 32 @swagitda_

Slide 33

Benefits: creates feedback loops to foster understanding of systemic risk 33 @swagitda_

Slide 34

What other ways can infosec become more strategic? 34 @swagitda_

Slide 35

Time to D.I.E.

Slide 36

We need a model promoting qualities that make systems more secure 36 @swagitda_

Slide 37

Enter the D.I.E. model: Distributed, Immutable, Ephemeral 37 @swagitda_

Slide 38

Distributed: multiple systems supporting the same overarching goal 38 @swagitda_

Slide 39

Distributed infrastructure reduces risk of DoS attacks by design 39 @swagitda_

Slide 40

Immutable: infrastructure that doesn’t change after it’s deployed 40 @swagitda_

Slide 41

Servers are now disposable “cattle” rather than cherished “pets” 41 @swagitda_

Slide 42

Immutable infra is more secure by design – ban shell access entirely 42 @swagitda_

Slide 43

Unlimited lives is better for security than game over upon death 43 @swagitda_

Slide 44

Ephemeral: infrastructure with a very short lifespan (dies after a task) 44 @swagitda_

Slide 45

Ephemerality creates uncertainty for attackers (persistence = nightmare) 45 @swagitda_

Slide 46

Installing a rootkit on a resource that dies in minutes is a waste of effort 46 @swagitda_

Slide 47

ICS attacks take months to plan; ephemerality constantly disrupts it 47 @swagitda_

Slide 48

Optimizing for D.I.E. reduces risk by design & supports resilience 48 @swagitda_

Slide 49

A Phoenix Rises

Slide 50

Harness failure as a tool to help you prepare for the inevitable 50 @swagitda_

Slide 51

Game days: practice risky scenarios 51 @swagitda_

Slide 52

Prioritize game days based on potential business impacts 52 @swagitda_

Slide 53

Decision trees: start at target asset, work back to easiest attacker paths 53 @swagitda_

Slide 54

Determine the attacker’s least-cost path (hint: it doesn’t involve 0day) 54 @swagitda_

Slide 55

Architecting chaos

Slide 56

Begin with “dumb” testing before moving to “fancy” testing 56 @swagitda_

Slide 57

Think digital twins, analytics services, or O365… not field-level SCADA 57 @swagitda_

Slide 58

Controlling Chaos: Distributed 58 @swagitda_

Slide 59

Distributed mostly overlaps with availability in modern infra contexts 59 @swagitda_

Slide 60

Chaos Monkey: inject random instances failures to test resilience 60 @swagitda_

Slide 61

Infosec teams can use these tools but make attackers the source of failure 61 @swagitda_

Slide 62

Multi-region services present a fun opportunity to mess with attackers 62 @swagitda_

Slide 63

Shuffle IP blocks regularly to change attackers’ lateral movement game 63 @swagitda_

Slide 64

Controlling Chaos: Immutable 64 @swagitda_

Slide 65

Volatile environments with continually moving parts raise the cost of attack 65 @swagitda_

Slide 66

Create rules like, “If there’s ever a write to disk, crash the node” 66 @swagitda_

Slide 67

Attackers must stay in-memory, which hopefully makes them cry 67 @swagitda_

Slide 68

Metasploit Meterpreter + webshell: Touch passwords.txt & kaboom 68 @swagitda_

Slide 69

Infosec teams can build Docker images with a “bamboozle layer” 69 @swagitda_

Slide 70

Mark garbage files as “unreadable” to craft enticing bait for attackers 70 @swagitda_

Slide 71

Potential goal: self-healing edge devices with immediate reversion 71 @swagitda_

Slide 72

Test: inject attempts at writing to disk to ensure detection & reversion 72 @swagitda_

Slide 73

Controlling Chaos: Ephemeral 73 @swagitda_

Slide 74

Most infosec bugs are stated-related – get rid of state, get rid of bugs 74 @swagitda_

Slide 75

Reverse uptime: longer host uptime adds greater security risk 75 @swagitda_

Slide 76

Test: retrograde libraries, containers, other resources in CI/CD pipelines 76 @swagitda_

Slide 77

Leverage lessons from toll fraud – cloud billing becomes security signal 77 @swagitda_

Slide 78

Test: exfil TBs or run a cryptominer to inform billing spike detection 78 @swagitda_

Slide 79

Conclusion

Slide 80

Security cannot gatekeep DevOps. It must marry it. 80 @swagitda_

Slide 81

Chaos/resilience are natural homes for infosec & represent its future. 81 @swagitda_

Slide 82

Infosec must now evolve to unify responsibility & accountability. 82 @swagitda_

Slide 83

ICS is already cloudy – get ready now before OT migrates as well. 83 @swagitda_

Slide 84

Giving up control isn’t a harbinger of doom. Resilience is a beacon of hope. 84 @swagitda_

Slide 85

“You must have chaos within you to give birth to a dancing star.” ― Friedrich Nietzsche 85 @swagitda_

Slide 86

@swagitda_ /in/kellyshortridge kelly@greywire.net 86 @swagitda_