Controlled chaos: The inevitable marriage of DevOps and security (Keynote)

A presentation at Velocity Conference in November 2019 in Berlin, Germany by Kelly Shortridge

Slide 1

C ONTROLLED C HAOS The Inevitable Marriage of DevOps & Security Kelly Shortridge (@swagitda_) Velocity Berlin 2019

Slide 2

Hi, I’m Kelly 2 @swagitda_

Slide 3

“Chaos isn’t a pit. Chaos is a ladder.” ― Petyr Baelish, Game of Thrones 3 @swagitda_

Slide 4

Infosec has a choice: marry DevOps or be rendered impotent & irrelevant 4 @swagitda_

Slide 5

Infosec won’t survive in a silo. It must be embedded in software delivery. 5 @swagitda_

Slide 6

DevOps can learn to carve its own path to secure software delivery @swagitda_

Slide 7

How can controlling chaos create a marriage of infosec and DevOps? 7 @swagitda_

Slide 8

Chaos Theory 2. Time to D.I.E. 3. A Phoenix Rises 8 @swagitda_

Slide 9

Chaos Theory

Slide 10

Chaos engineering = continual experimentation to test resilience @swagitda_

Slide 11

“Things will fail” naturally extends into “things will be pwned” 11 @swagitda_

Slide 12

Security failure is when security controls don’t operate as intended 12 @swagitda_

Slide 13

What are the principles of chaotic security engineering? 13 @swagitda_

Slide 14

Expect that security controls will fail & prepare accordingly 14 @swagitda_

Slide 15

Don’t try to avoid incidents – hone your ability to respond to them 15 @swagitda_

Slide 16

Game days: like planned firedrills 16 @swagitda_

Slide 17

Prioritize security game days based on potential business impacts 17 @swagitda_

Slide 18

Decision trees: start at target asset, work back to easiest attacker paths 18 @swagitda_

Slide 19

Determine the attacker’s least-cost path (hint: it doesn’t involve 0day) 19 @swagitda_

Slide 20

Your goal is to raise the cost of attack, ideally beginning at design 20 @swagitda_

Slide 21

Time to D.I.E.

Slide 22

We need a model promoting qualities that make systems more secure 22 @swagitda_

Slide 23

Enter the D.I.E. model by Sounil Yu: Distributed, Immutable, Ephemeral 23 @swagitda_

Slide 24

Distributed: multiple systems supporting the same overarching goal 24 @swagitda_

Slide 25

Distributed infrastructure reduces risk of DoS attacks by design 25 @swagitda_

Slide 26

A service mesh is like an on-demand VPN at the application level 26 @swagitda_

Slide 27

Attackers are forced to escalate privileges to access the iptables layer 27 @swagitda_

Slide 28

Immutable: infrastructure that doesn’t change after it’s deployed 28 @swagitda_

Slide 29

Immutable infra is more secure by design – ban shell access entirely 29 @swagitda_

Slide 30

Patching is no longer a nightmare with version-controlled images @swagitda_

Slide 31

Ephemeral: infrastructure with a very short lifespan (dies after a task) 31 @swagitda_

Slide 32

Ephemerality creates uncertainty for attackers (persistence = nightmare) 32 @swagitda_

Slide 33

Installing a rootkit on a resource that dies in minutes is a waste of effort 33 @swagitda_

Slide 34

Optimizing for D.I.E. reduces risk by design & supports resilience 34 @swagitda_

Slide 35

A Phoenix Rises

Slide 36

Begin with “dumb” testing before moving to “fancy” testing 36 @swagitda_

Slide 37

D.I.E.ing is an art, like everything else @swagitda_

Slide 38

Controlling Chaos: Distributed 38 @swagitda_

Slide 39

Distributed is mostly covered by the existing repertoire of chaos eng tools 39 @swagitda_

Slide 40

Repurpose these tools, but make attackers the source of failure 40 @swagitda_

Slide 41

Multi-region services present a fun opportunity to mess with attackers 41 @swagitda_

Slide 42

Shuffle IP blocks regularly to change attackers’ lateral movement game 42 @swagitda_

Slide 43

Test: inject failure into your service mesh to test authentication controls 43 @swagitda_

Slide 44

Controlling Chaos: Immutable 44 @swagitda_

Slide 45

Immutable infra is like a phoenix – it disappears & comes back a lot 45 @swagitda_

Slide 46

Volatile environments with continually moving parts raise the cost of attack 46 @swagitda_

Slide 47

Create rules like, “If there’s ever a write to disk, crash the node” 47 @swagitda_

Slide 48

Attackers must stay in-memory, which hopefully makes them cry 48 @swagitda_

Slide 49

Bonus: disallowing all local IO improves service reliability 49 @swagitda_

Slide 50

Metasploit Meterpreter + webshell: Touch passwords.txt & kaboom 50 @swagitda_

Slide 51

Build your Docker images with a garbage-filled “bamboozle layer” 51 @swagitda_

Slide 52

Mark garbage files as “unreadable” to craft enticing bait for attackers 52 @swagitda_

Slide 53

A potential goal: architect immutability turtles all the way down 53 @swagitda_

Slide 54

Test: inject attempts at writing to disk to ensure detection & reversion 54 @swagitda_

Slide 55

Treat changes to disk by adversaries similarly to failing disks: mercy kill 55 @swagitda_

Slide 56

Controlling Chaos: Ephemeral 56 @swagitda_

Slide 57

Most infosec bugs are stated-related – get rid of state, get rid of bugs 57 @swagitda_

Slide 58

Reverse uptime: longer host uptime adds greater security risk 58 @swagitda_

Slide 59

Test: change API tokens & test if services still accept old tokens 59 @swagitda_

Slide 60

Test: retrograde libraries, containers, other resources in CI/CD pipelines 60 @swagitda_

Slide 61

Test: inject hashes of old pieces of data to ensure no data persistence 61 @swagitda_

Slide 62

Leverage lessons from toll fraud – cloud billing becomes security signal 62 @swagitda_

Slide 63

Test: exfil TBs or run a cryptominer to inform billing spike detection 63 @swagitda_

Slide 64

Conclusion

Slide 65

Chaos/resilience are natural homes for infosec & represent its future. 65 @swagitda_

Slide 66

The future of infosec involves unified responsibility & accountability. 66 @swagitda_

Slide 67

Security can be innovative and fuel the engine of business as well. 67 @swagitda_

Slide 68

“You must have chaos within you to give birth to a dancing star.” ― Friedrich Nietzsche 68 @swagitda_

Slide 69

@swagitda_ /in/kellyshortridge kelly@greywire.net 69 @swagitda_