A presentation at CAST2019 in in United States by Trisha Chetani
Creating Test Stability to achieve Continuous Delivery Trisha Chetani @trisha_1212 @trisha_1212
ME ● Introduction ● Networking - Conferences, Meetups ● Recently moved to Spain ● 125 awesome software testers by Agile Testing Days ● 1st International talk ● Twitter: @trisha_1212 @trisha_1212
WHAT IS THIS TALK ALL ABOUT ● How automated tests were slowing us down ● Increase in flaky tests ● A long journey of investigation and tackling different problems ● Lessons learned ● How I transformed myself from being a bottleneck to an enabler ● Maintained to avoid the fall back. @trisha_1212
THAT’S ME @ VISENZE Years : 2016 ● ● About Visenze My Role @trisha_1212
JOURNEY AT VISENZE - THE PROBLEM SERVICE LEVEL AGREEMENT NOT MAINTAINED FLAKY TESTS Definition:sometimes passes or fails* @trisha_1212 TEAM WERE PUTTING EXTRA EFFORT
THE LONG JOURNEY AHEAD OF THE TEAM FIX FLAKINESS 30:00 MIN CURRENT TIME TAKEN FOR EXECUTION(TEST FAILED MULTIPLY THE TIME) @trisha_1212 COLLABORATE STABILITY AND MUCH MORE… 10:00 MIN GOAL FOR EXECUTION
JOURNEY OF HOW I MADE TEST STABLE @trisha_1212
ADAPTING TO CURRENT SYSTEM ● Exploring about system and application ● Exploring existing tests, at that time being maintained by developers ● Writing very first test ● Deciding the tests flows with the team @trisha_1212
LEARNING TOOLS/TECHNOLOGY ● Existing tool ○ Protractor ○ Docker ○ Jenkins ○ Internal tools and many more ● Tools I thought could help ○ Selenium Grid @trisha_1212
DIGGED DEEPER DEAD CODE @trisha_1212 DUPLICATE TEST OUTDATED TEST MISSING TESTS
CURRENT STATUS - AFTER 5 MONTHS ADAPT TO SINGAPORE ABLE TO FIX THE TEST ALONE @trisha_1212 WORKING ON MULTIPLE PROJECTS ME ADD SO IONSS REGRE S CHECK
BUILD TIME IS… STILL 30 MINUTES @trisha_1212
PROBLEMS ENDLESS EFFORT ONGOING CHANGES @trisha_1212 WORK LIFE BALANCE RED AND GREEN VICIOUS CYCLE
STEP TOWARDS THE SOLUTION ROOT CAUSE ANALYSIS @trisha_1212
UNDERLYING ISSUES FOUND IN ROOT CAUSE ANALYSIS COUPLE OF WEEKS LATER WHILE FOLLOWING RED GREEN CYCLE ● Locators were changing ● Basic flows were changing ● Unexpected changes ● Application slowness ● Automate all tests ● No time to make significant improvement for tests @trisha_1212
FEW WEEKS LATER - I WENT ON LEAVE @trisha_1212
TEAM @trisha_1212 ME - NOT SUSTAINABLE
RETROSPECTIVE DISCUSSION @trisha_1212
RETROSPECTIVE ACTIONS ITEMS ● Stop developing new feature ● Focus on test improvement ● Team decides that developer increase unit and integration tests ● Automate test cases based on risk, priority, business impact* ● Focused on operational excellence tasks @trisha_1212
MY ACTION ITEM - 7 MONTHS ● Digging deeper - focused not only on testing but improve on overall quality aspects. ● Open task - support on operational excellence task ● Not automating all test cases - focus on end to end user flow ● Fixing the existing flaky test cases. @trisha_1212
CODING DEBUGGING ● Locators(id> name> css> xpath) ● Logs ● Assertion ● Screenshots ● Waits ● Retry before failing test ● Small & Independent tests ● JS console errors ● Integrated with library test reports @trisha_1212
FRAMEWORK ENVIRONMENT ● Ease of configuration to test ● Docker ● Test data management ● Selenium grid ● Community support ● Clean test environment ● Test environment and production environment @trisha_1212
IMPROVED PRODUCT TESTABILITY ● Developer help to optimize page load, reduce downtime ● Reduce uncertainty inside application and infrastructure issues ● Reduce the technical debts and refactoring of code ● Seperate the backend and frontend code base and pipeline @trisha_1212
IMPROVED PROCESS TESTABILITY ● Root Cause Analysis for the actual failure of automated code ● Rotation task in maintaining the test code among team member ● Code review of test code ● Priority in maintaining and writing new test code @trisha_1212
IMPROVED PEOPLE TESTABILITY ● Everyone in team started contributing to automated checks ● Increase tech stack understanding ● Improved AWS/Google cloud understanding ● Developed our knowledge on containers @trisha_1212
FINALLY BUILD TIME IS… 5 MINUTES @trisha_1212
MAINTAIN THE PEOPLE TESTABILITY ● Increased cross team system knowledge by evolving more people in system deployments ● Team knowledge asking for feedback about tests and product. ● Pairing with new joiners and making them up to the speed @trisha_1212
MAINTAIN THE PROCESS TESTABILITY ● Update our automated tests continuously ● Maintain pipeline always green ● Having operational day in a sprints @trisha_1212
MAINTAIN THE PRODUCT TESTABILITY ● Monitoring the APIs, ● Continuous Refactoring of code. ● Infrastructure improvement ● Added additional logs and implemented Kibana @trisha_1212
SUMMARY IF YOU WANT TO STABILISE YOUR TESTS YOUR NEED TO… ● ● ● ● ● ● @trisha_1212 IMPLEMENT GOOD CODING PRACTICES ENCOURAGE TEAM RESPONSIBILITY FOR TEST IMPROVEMENT FOCUS ON HAVING STABLE TEST ENVIRONMENTS HAVE PEOPLE WITH THE RIGHT SKILLS AND KNOWLEDGE HAVE GOOD PRODUCT TESTABILITY CREATE AN ENVIRONMENT OF CONTINUOUS IMPROVEMENT AND LEARNING
THANK YOU @ajay184f @ajay184f @trisha_1212 @friendlytester @friendlytester @aahunsberger @mariakedemo @mariakedemo @kriscorbus @kriscorbus
Questions? @trisha_1212 @trisha_1212
The following resources were mentioned during the presentation or are useful additional information.
Here’s what was said about this presentation on social media.
for free. You
can too.