A presentation at eXperience Agile 2020 in in Lisbon, Portugal by Areti Panou
A Whole Team Approach to Testing in Continuous Delivery XA 2020 Lisa Crispin and Areti Panou @lisacrispin Copyright 2020 Lisa Crispin and Areti Panou @UnremarkableQA
About us Test Automation U Lisa Crispin Lisa@agiletester.ca Helping to build observability at @lisacrispin Areti Panou areti.panou@gmail.com Product Owner of internal tools at @UnremarkableQA
Learning intentions • Get your team engaged to deploy frequently, with • • • • confidence Learn the language of continuous delivery Exercises you can do with your team to shorten feedback loops Ways to build a quality culture We’ll talk and demonstrate. Then live Q&A! @lisacrispin @UnremarkableQA
We’ve encountered a few obstacles… We’ll each share some ideas & materials to get your team talking Write down your questions as we go! Live Q&A follows! Photo by Zé Ferrari Careto on Unsplash
Building quality in @lisacrispin @UnremarkableQA
Whole team ownership of quality @lisacrispin Graphic by Janet Gregory, inspired by Dan Ashby @UnremarkableQA
Share common goals Whole team ownership of quality Build relationships across roles, teams Continuously learn and improve Take advantage of multiple skill sets, perspectives Continuous, holistic testing
Guidelines for continual improvement • Small slices, learning releases • Iterative improvement • Build towards your goal @lisacrispin @UnremarkableQA
Prioritize! • How does it align with the business? • Is it valuable to the customer? • Are the stories small and testable? Note: see resources slide for more on this @lisacrispin @UnremarkableQA
Continuous improvement… Let’s look at some tools, techniques and frameworks that can help @lisacrispin @UnremarkableQA
Infrastructure …… for fast feedback and frequent delivery @lisacrispin @UnremarkableQA
DevOps isn’t only about infrastructure • • • • • • Collaboration Continuous improvement Continuous learning Build quality in Small batches Everyone is responsible © Libby VanderPloeg @lisacrispin @UnremarkableQA
Let’s agree on some common terminology @lisacrispin @UnremarkableQA
Metrics for success – per State of DevOps Survey • Lead time for changes • Re-work slows us down • Shared understanding speeds us up • Change failure rate • Time to restore service (MTTR) • Deployment frequency @lisacrispin @UnremarkableQA
Continuous Integration (CI) Multiple times per day: • Integrate code into a shared repository • All code is integrated into the same branch @lisacrispin @UnremarkableQA
Validating each change • Typically the start of a pipeline • Each check-in can be verified by an automated build with automated regression tests @lisacrispin @UnremarkableQA
Deployment pipeline • • • • Break the build into stages to speed up feedback Early stages can find most problems -> faster feedback Later stages probe more thoroughly Automated deployment pipelines are central to continuous delivery From A Practical Guide to Testing in DevOps, Katrina Clokie @lisacrispin @UnremarkableQA
Continuous Delivery (CD) @lisacrispin @UnremarkableQA
Continuous Deployment (also CD) @lisacrispin @UnremarkableQA
Deployment Pipeline Developer Commit Stage Stage Stage Stage Stage Stage Deployed to Production
Deployment Pipeline Developer Commit Code is merged Stage (master/trunk) Continuous Integration (CI) Stage Stage Stage Stage Deployed to Production
Deployment Pipeline Developer Commit Code is merged Compilation/ unit testing (master/trunk) Create deployment artifact Testing stage 1 Continuous Integration (CI) Continuous Delivery (CD) Testing stage 2 Business approval Deployed to Production
Deployment Pipeline Developer Commit Code is merged Compilation/ unit testing (master/trunk) Create deployment artifact Testing stage 1 Testing stage 2 Continuous Integration (CI) Continuous Delivery (CD) Continuous Deployment (confusingly, this is also CD) Busines s approva l Deployed to Production
Demo: Visualizing your current pipeline @lisacrispin @UnremarkableQA
Try it later with your team Make your own copy of the “Visualize Pipeline Template” from https://tinyurl.com/XA202CD into a folder shared with your team • Together in a video meeting, use the stage cards to create a model of your pipeline • • • • Think about how to shorten feedback loops You don’t have to use all the virtual step cards Create new step cards as needed Discuss! @lisacrispin @UnremarkableQA
So far we’ve talked about: • The whole team approach to testing in CD • Some terminology to help communication • Ideas for visualizing your pipelines • Start conversations to shorten feedback loops Now, Areti will share more ideas to get your team engaged & improving! @lisacrispin @UnremarkableQA
About Areti Tester @lisacrispin Quality Responsible Quality Coach Product Ower @UnremarkableQA
Look into your pipeline steps • Value • Team acceptance • Identify constraints @lisacrispin @UnremarkableQA
Analyze each pipeline step Purpose Alerting Importance Automation Trustworthiness Dependencies Duration Expansions @lisacrispin Maintainability Frequency @UnremarkableQA
Demo: Analyze your pipeline steps @lisacrispin @UnremarkableQA
Try it later with your team • Make a copy of the Analyze a Pipeline Step Exercise in • • • • • https://tinyurl.com/XA202CD into a folder shared with your team Choose a pipeline step you want to discuss as a team. Each team member notes down answers to the questions in each dimension in a virtual sticky note. Everyone puts their answers in the dimensions. Discuss the results gathered in each dimension. Get an overview on the points that the team agrees on and the ones that can be improved. @lisacrispin @UnremarkableQA
Sharing pipeline steps knowledge Identify pipeline steps Visualize all the steps in your deployment pipeline @lisacrispin @UnremarkableQA
Sharing pipeline steps knowledge Identify pipeline steps Visualize all the steps in your deployment pipeline @lisacrispin Discuss pipeline steps Optimize the steps by engaging the knowledge of the team @UnremarkableQA
Sharing pipeline steps knowledge Identify pipeline steps Visualize all the steps in your deployment pipeline @lisacrispin Discuss pipeline steps Optimize the steps by engaging the knowledge of the team Share ownership of pipeline steps At least 2 people know each step equally well @UnremarkableQA
Rinse & Repeat Identify Share @lisacrispin Discuss @UnremarkableQA
“Continuous” without automation Like driving at night without your headlights, it’s possible… but headlights reduce the risk. Automated regression tests are one headlight. Operability – monitoring, observability, testing in prod – is the other. — from Ashley Hunsberger @lisacrispin @UnremarkableQA
DevOps is about building a quality culture @lisacrispin @UnremarkableQA
Build relationships • • • • • @lisacrispin DevOps culture includes all roles Get to know people in and outside of your team Ask for help, offer help Do food! Build trust so you can learn @UnremarkableQA
Show the benefits of engagement in Quality @lisacrispin @UnremarkableQA
Getting management support • • • • @lisacrispin Speak their language – what’s the bottom line for the company? Help them learn why quality matters Help them learn the value of building quality in Make testing activities and their value visible @UnremarkableQA
Building a quality culture • • • • @lisacrispin Culture change is hard! Transforming leadership is needed Trust and psychological safety are prerequisites to high-performing teams Production ownership: the delivery team cares for the product in production Focus on quality, not speed, to go faster in the long run @UnremarkableQA
Resources for more learning ● Testing in DevOps community site, https://testingInDevOps.org ● Agile Testing Condensed, Agile Testing and More Agile Testing, Lisa Crispin and ● ● ● ● ● ● ● @lisacrispin Janet Gregory, https://agiletester.ca Continuous Delivery by Jez Humble and David Farley, https://continuousdelivery.com A Practical Guide to Testing in DevOps by Katrina Clokie Accelerate by Nicole Forsgren, Jez Humble, Gene Kim “Value streams are made of people”, Liz Keogh, (Also see her work on BDD, Cynefin ™ & Real Options) https://lizkeogh.com/value-streams-are-made-ofpeople/ More Fearless Change by MaryLynn Manns and Linda Rising Learn about behavior-driven development, example mapping: https://docs.cucumber.io/bdd/ Trunk-based development, Paul Hammant, https://trunkbaseddevelopment.com/ @UnremarkableQA
Download the exercise templates & more to share with your team! https://tinyurl.com/XA202CD @lisacrispin @UnremarkableQA
Let’s keep the conversation going… @lisacrispin Gratitude to Abby Bangser, Ashley Hunsberger, Elizabeth Zagroba, Janet Gregory for their contributions @UnremarkableQA
Pictures credits • Slide 28 • • • • • • @lisacrispin Slide 29a, Slide 29b, Slide 29c, Slide 29d, Slide 29e, Slide 29f, Slide 29g, Slide 29h, Slide 29i, Slide 29j Slide 32 Slide 33 Slide 34 Slide 38 Slide 39 @UnremarkableQA
By Lisa Crispin & Areti Panou
Deploying small changes to production, frequently, at a sustainable pace, is a great way to mitigate risks associated with software delivery. Yet many teams struggle to feel confident to move to continuous delivery. If a team is struggling to deploy, or at least have a production-ready release candidate, every one or two weeks, how can they hope to deploy daily or multiple times per day? How does human-centric testing activities such as exploratory testing fit in with such frequent deployments to production?
The following resources were mentioned during the presentation or are useful additional information.
Exercises used during the presentation
The official site of the event
for free. You
can too.