A Whole Team Approach to Testing in Continuous Delivery

A Whole Team Approach to Testing in Continuous Delivery XA 2020 Lisa Crispin and Areti Panou @lisacrispin Copyright 2020 Lisa Crispin and Areti Panou @UnremarkableQA

About us Test Automation U Lisa Crispin Lisa@agiletester.ca Helping to build observability at @lisacrispin Areti Panou areti.panou@gmail.com Product Owner of internal tools at @UnremarkableQA

Learning intentions • Get your team engaged to deploy frequently, with • • • • confidence Learn the language of continuous delivery Exercises you can do with your team to shorten feedback loops Ways to build a quality culture We’ll talk and demonstrate. Then live Q&A! @lisacrispin @UnremarkableQA

We’ve encountered a few obstacles… We’ll each share some ideas & materials to get your team talking Write down your questions as we go! Live Q&A follows! Photo by Zé Ferrari Careto on Unsplash

Building quality in @lisacrispin @UnremarkableQA

Whole team ownership of quality @lisacrispin Graphic by Janet Gregory, inspired by Dan Ashby @UnremarkableQA

Share common goals Whole team ownership of quality Build relationships across roles, teams Continuously learn and improve Take advantage of multiple skill sets, perspectives Continuous, holistic testing

Guidelines for continual improvement • Small slices, learning releases • Iterative improvement • Build towards your goal @lisacrispin @UnremarkableQA

Prioritize! • How does it align with the business? • Is it valuable to the customer? • Are the stories small and testable? Note: see resources slide for more on this @lisacrispin @UnremarkableQA

Continuous improvement… Let’s look at some tools, techniques and frameworks that can help @lisacrispin @UnremarkableQA

Infrastructure …… for fast feedback and frequent delivery @lisacrispin @UnremarkableQA

DevOps isn’t only about infrastructure • • • • • • Collaboration Continuous improvement Continuous learning Build quality in Small batches Everyone is responsible © Libby VanderPloeg @lisacrispin @UnremarkableQA

Let’s agree on some common terminology @lisacrispin @UnremarkableQA

Metrics for success – per State of DevOps Survey • Lead time for changes • Re-work slows us down • Shared understanding speeds us up • Change failure rate • Time to restore service (MTTR) • Deployment frequency @lisacrispin @UnremarkableQA

Continuous Integration (CI) Multiple times per day: • Integrate code into a shared repository • All code is integrated into the same branch @lisacrispin @UnremarkableQA

Validating each change • Typically the start of a pipeline • Each check-in can be verified by an automated build with automated regression tests @lisacrispin @UnremarkableQA

Deployment pipeline • • • • Break the build into stages to speed up feedback Early stages can find most problems -> faster feedback Later stages probe more thoroughly Automated deployment pipelines are central to continuous delivery From A Practical Guide to Testing in DevOps, Katrina Clokie @lisacrispin @UnremarkableQA

Continuous Delivery (CD) @lisacrispin @UnremarkableQA

Continuous Deployment (also CD) @lisacrispin @UnremarkableQA

Deployment Pipeline Developer Commit Stage Stage Stage Stage Stage Stage Deployed to Production

Deployment Pipeline Developer Commit Code is merged Stage (master/trunk) Continuous Integration (CI) Stage Stage Stage Stage Deployed to Production

Deployment Pipeline Developer Commit Code is merged Compilation/ unit testing (master/trunk) Create deployment artifact Testing stage 1 Continuous Integration (CI) Continuous Delivery (CD) Testing stage 2 Business approval Deployed to Production

Deployment Pipeline Developer Commit Code is merged Compilation/ unit testing (master/trunk) Create deployment artifact Testing stage 1 Testing stage 2 Continuous Integration (CI) Continuous Delivery (CD) Continuous Deployment (confusingly, this is also CD) Busines s approva l Deployed to Production

Demo: Visualizing your current pipeline @lisacrispin @UnremarkableQA

Try it later with your team Make your own copy of the “Visualize Pipeline Template” from https://tinyurl.com/XA202CD into a folder shared with your team • Together in a video meeting, use the stage cards to create a model of your pipeline • • • • Think about how to shorten feedback loops You don’t have to use all the virtual step cards Create new step cards as needed Discuss! @lisacrispin @UnremarkableQA

So far we’ve talked about: • The whole team approach to testing in CD • Some terminology to help communication • Ideas for visualizing your pipelines • Start conversations to shorten feedback loops Now, Areti will share more ideas to get your team engaged & improving! @lisacrispin @UnremarkableQA

About Areti Tester @lisacrispin Quality Responsible Quality Coach Product Ower @UnremarkableQA

Look into your pipeline steps • Value • Team acceptance • Identify constraints @lisacrispin @UnremarkableQA

Analyze each pipeline step Purpose Alerting Importance Automation Trustworthiness Dependencies Duration Expansions @lisacrispin Maintainability Frequency @UnremarkableQA

Demo: Analyze your pipeline steps @lisacrispin @UnremarkableQA

Try it later with your team • Make a copy of the Analyze a Pipeline Step Exercise in • • • • • https://tinyurl.com/XA202CD into a folder shared with your team Choose a pipeline step you want to discuss as a team. Each team member notes down answers to the questions in each dimension in a virtual sticky note. Everyone puts their answers in the dimensions. Discuss the results gathered in each dimension. Get an overview on the points that the team agrees on and the ones that can be improved. @lisacrispin @UnremarkableQA

Sharing pipeline steps knowledge Identify pipeline steps Visualize all the steps in your deployment pipeline @lisacrispin @UnremarkableQA

Sharing pipeline steps knowledge Identify pipeline steps Visualize all the steps in your deployment pipeline @lisacrispin Discuss pipeline steps Optimize the steps by engaging the knowledge of the team @UnremarkableQA

Sharing pipeline steps knowledge Identify pipeline steps Visualize all the steps in your deployment pipeline @lisacrispin Discuss pipeline steps Optimize the steps by engaging the knowledge of the team Share ownership of pipeline steps At least 2 people know each step equally well @UnremarkableQA

Rinse & Repeat Identify Share @lisacrispin Discuss @UnremarkableQA

“Continuous” without automation Like driving at night without your headlights, it’s possible… but headlights reduce the risk. Automated regression tests are one headlight. Operability – monitoring, observability, testing in prod – is the other. — from Ashley Hunsberger @lisacrispin @UnremarkableQA

DevOps is about building a quality culture @lisacrispin @UnremarkableQA

Build relationships • • • • • @lisacrispin DevOps culture includes all roles Get to know people in and outside of your team Ask for help, offer help Do food! Build trust so you can learn @UnremarkableQA

Show the benefits of engagement in Quality @lisacrispin @UnremarkableQA

Getting management support • • • • @lisacrispin Speak their language – what’s the bottom line for the company? Help them learn why quality matters Help them learn the value of building quality in Make testing activities and their value visible @UnremarkableQA

Building a quality culture • • • • @lisacrispin Culture change is hard! Transforming leadership is needed Trust and psychological safety are prerequisites to high-performing teams Production ownership: the delivery team cares for the product in production Focus on quality, not speed, to go faster in the long run @UnremarkableQA

Resources for more learning ● Testing in DevOps community site, https://testingInDevOps.org ● Agile Testing Condensed, Agile Testing and More Agile Testing, Lisa Crispin and ● ● ● ● ● ● ● @lisacrispin Janet Gregory, https://agiletester.ca Continuous Delivery by Jez Humble and David Farley, https://continuousdelivery.com A Practical Guide to Testing in DevOps by Katrina Clokie Accelerate by Nicole Forsgren, Jez Humble, Gene Kim “Value streams are made of people”, Liz Keogh, (Also see her work on BDD, Cynefin ™ & Real Options) https://lizkeogh.com/value-streams-are-made-ofpeople/ More Fearless Change by MaryLynn Manns and Linda Rising Learn about behavior-driven development, example mapping: https://docs.cucumber.io/bdd/ Trunk-based development, Paul Hammant, https://trunkbaseddevelopment.com/ @UnremarkableQA

Download the exercise templates & more to share with your team! https://tinyurl.com/XA202CD @lisacrispin @UnremarkableQA

Let’s keep the conversation going… @lisacrispin Gratitude to Abby Bangser, Ashley Hunsberger, Elizabeth Zagroba, Janet Gregory for their contributions @UnremarkableQA

Pictures credits • Slide 28 • • • • • • @lisacrispin Slide 29a, Slide 29b, Slide 29c, Slide 29d, Slide 29e, Slide 29f, Slide 29g, Slide 29h, Slide 29i, Slide 29j Slide 32 Slide 33 Slide 34 Slide 38 Slide 39 @UnremarkableQA