Sensory Friendly Monitoring: Keeping the Noise Down

A presentation at DevOps Days PDX in September 2021 in Portland, OR, USA by Quintessence Anx

Slide 1

Slide 1

Sensory Friendly Monitoring Keep the Noise Down Quintessence Anx Developer Advocate

Slide 2

Slide 2

Head’s up: Mute your Echo devices in listening range, or use 🎧 QuintessenceAnx

Slide 3

Slide 3

When we try to know everything… QuintessenceAnx

Slide 4

Slide 4

QuintessenceAnx

Slide 5

Slide 5

Too much noise can QuintessenceAnx

Slide 6

Slide 6

Bury important / high SEV alerts in a sea of low priority notifications QuintessenceAnx

Slide 7

Slide 7

…causing teams to mute individual notifications or their sources. QuintessenceAnx

Slide 8

Slide 8

The result? QuintessenceAnx

Slide 9

Slide 9

The people who needed to be notified, won’t be. QuintessenceAnx

Slide 10

Slide 10

Meanwhile, when we turn the dial back too far… QuintessenceAnx

Slide 11

Slide 11

QuintessenceAnx

Slide 12

Slide 12

Let’s find a happy medium. All alerts are fictional. QuintessenceAnx

Slide 13

Slide 13

Consider: the cost of noise QuintessenceAnx

Slide 14

Slide 14

Your brain on alerts QuintessenceAnx

Slide 15

Slide 15

Time cost: ~25 minutes QuintessenceAnx

Slide 16

Slide 16

Quality Cost QuintessenceAnx

Slide 17

Slide 17

Multitasking Cost QuintessenceAnx

Slide 18

Slide 18

How to reduce the noise? QuintessenceAnx

Slide 19

Slide 19

Be aware, not overwhelmed. QuintessenceAnx

Slide 20

Slide 20

Determine the sources of noise QuintessenceAnx

Slide 21

Slide 21

Categorize the types of noise QuintessenceAnx

Slide 22

Slide 22

Channel the noise into a productive flow QuintessenceAnx

Slide 23

Slide 23

Create a routine to clear the clutter. QuintessenceAnx

Slide 24

Slide 24

Be aware, not overwhelmed • Determine the sources of noise • Categorize the types of noise • Channel the noise into a productive workflow • Create a routine to clear the clutter QuintessenceAnx

Slide 25

Slide 25

Sources of noise QuintessenceAnx

Slide 26

Slide 26

Wait, I need to be aware of myself? All alerts are fictional. QuintessenceAnx

Slide 27

Slide 27

How often do you… • …check your email? • …check your social media? • …check your text messages? • …check your Apple/Google messages? • …the list goes on. All alerts are fictional. QuintessenceAnx

Slide 28

Slide 28

Head’s up: Mute your Echo devices in listening range, or use 🎧 QuintessenceAnx

Slide 29

Slide 29

Communication & Boundaries QuintessenceAnx

Slide 30

Slide 30

Set times to focus on work and mute non-critical alerts. QuintessenceAnx

Slide 31

Slide 31

This includes messages from friends & family QuintessenceAnx

Slide 32

Slide 32

Define “relevant emergency” QuintessenceAnx

Slide 33

Slide 33

Set reasonable expectations for yourself and others. QuintessenceAnx

Slide 34

Slide 34

Communication & Boundaries • Set times to focus on work and mute non-critical alerts. • This includes messages from friends & family • Define “relevant emergency” • Set reasonable expectations for yourself and others. QuintessenceAnx

Slide 35

Slide 35

And what about external sources of noise? QuintessenceAnx

Slide 36

Slide 36

Categorize your noise QuintessenceAnx

Slide 37

Slide 37

False positives QuintessenceAnx

Slide 38

Slide 38

False negatives QuintessenceAnx

Slide 39

Slide 39

Fragility QuintessenceAnx

Slide 40

Slide 40

Frequency (fix it) QuintessenceAnx

Slide 41

Slide 41

Noise Categories • False positives • False negatives • Fragility • Frequency QuintessenceAnx

Slide 42

Slide 42

Save time: create your noise flow QuintessenceAnx

Slide 43

Slide 43

What needs to be known QuintessenceAnx

Slide 44

Slide 44

Who needs to know it QuintessenceAnx

Slide 45

Slide 45

How soon should they know QuintessenceAnx

Slide 46

Slide 46

How should they be notified QuintessenceAnx

Slide 47

Slide 47

Noise Flow • What needs to be known • Who needs to know it • How soon should they know • How should they be notified QuintessenceAnx

Slide 48

Slide 48

Re-Evaluate Redundancy How to add a little complexity to stop a vacuum a.k.a. A bad day in ChatOps QuintessenceAnx

Slide 49

Slide 49

Resilient noise builds trust QuintessenceAnx

Slide 50

Slide 50

How reliable are your tools and services? QuintessenceAnx

Slide 51

Slide 51

How much notification duplication is needed? QuintessenceAnx

Slide 52

Slide 52

Can you switch alert endpoints in the event of a service outage? QuintessenceAnx

Slide 53

Slide 53

Do you regularly evaluate the reliability of your services (internal and external)? QuintessenceAnx

Slide 54

Slide 54

Reliability & Redundancy • How reliable are your tools and services? • How much notification duplication is needed? • Can you switch alert endpoints in the event of a service outage? • Do you regularly evaluate the reliability of your services (internal and external)? QuintessenceAnx

Slide 55

Slide 55

Keep alerts relevant: Sprint Cleaning QuintessenceAnx

Slide 56

Slide 56

For every alert triggered, ask: QuintessenceAnx

Slide 57

Slide 57

Was the notification needed? QuintessenceAnx

Slide 58

Slide 58

How was the incident resolved? QuintessenceAnx

Slide 59

Slide 59

Can the solution be automated? QuintessenceAnx

Slide 60

Slide 60

Is the solution permanent? QuintessenceAnx

Slide 61

Slide 61

How urgently was a solution needed? QuintessenceAnx

Slide 62

Slide 62

For every alert triggered, ask • Was the notification needed? • How was the incident resolved? • Can the solution be automated? • Is the solution permanent? • How urgently is the solution needed? QuintessenceAnx

Slide 63

Slide 63

Slides & Additional Resources on Notist https://noti.st/quintessence QuintessenceAnx

Slide 64

Slide 64

Thank you! Quintessence Anx Developer Advocate 🥑 @ https://noti.st/quintessence