Creating Test Stability to Create Continuous Delivery

Creating Test Stability to achieve Continuous Delivery Trisha Chetani @trisha_1212 @trisha_1212

ME ● Introduction ● Networking - Conferences, Meetups ● Recently moved to Spain ● 125 awesome software testers by Agile Testing Days ● 1st International talk ● Twitter: @trisha_1212 @trisha_1212

WHAT IS THIS TALK ALL ABOUT ● How automated tests were slowing us down ● Increase in flaky tests ● A long journey of investigation and tackling different problems ● Lessons learned ● How I transformed myself from being a bottleneck to an enabler ● Maintained to avoid the fall back. @trisha_1212

THAT’S ME @ VISENZE Years : 2016 ● ● About Visenze My Role @trisha_1212

JOURNEY AT VISENZE - THE PROBLEM SERVICE LEVEL AGREEMENT NOT MAINTAINED FLAKY TESTS Definition:sometimes passes or fails* @trisha_1212 TEAM WERE PUTTING EXTRA EFFORT

THE LONG JOURNEY AHEAD OF THE TEAM FIX FLAKINESS 30:00 MIN CURRENT TIME TAKEN FOR EXECUTION(TEST FAILED MULTIPLY THE TIME) @trisha_1212 COLLABORATE STABILITY AND MUCH MORE… 10:00 MIN GOAL FOR EXECUTION

JOURNEY OF HOW I MADE TEST STABLE @trisha_1212

ADAPTING TO CURRENT SYSTEM ● Exploring about system and application ● Exploring existing tests, at that time being maintained by developers ● Writing very first test ● Deciding the tests flows with the team @trisha_1212

LEARNING TOOLS/TECHNOLOGY ● Existing tool ○ Protractor ○ Docker ○ Jenkins ○ Internal tools and many more ● Tools I thought could help ○ Selenium Grid @trisha_1212

DIGGED DEEPER DEAD CODE @trisha_1212 DUPLICATE TEST OUTDATED TEST MISSING TESTS

CURRENT STATUS - AFTER 5 MONTHS ADAPT TO SINGAPORE ABLE TO FIX THE TEST ALONE @trisha_1212 WORKING ON MULTIPLE PROJECTS ME ADD SO IONSS REGRE S CHECK

BUILD TIME IS… STILL 30 MINUTES @trisha_1212

PROBLEMS ENDLESS EFFORT ONGOING CHANGES @trisha_1212 WORK LIFE BALANCE RED AND GREEN VICIOUS CYCLE

STEP TOWARDS THE SOLUTION ROOT CAUSE ANALYSIS @trisha_1212

UNDERLYING ISSUES FOUND IN ROOT CAUSE ANALYSIS COUPLE OF WEEKS LATER WHILE FOLLOWING RED GREEN CYCLE ● Locators were changing ● Basic flows were changing ● Unexpected changes ● Application slowness ● Automate all tests ● No time to make significant improvement for tests @trisha_1212

FEW WEEKS LATER - I WENT ON LEAVE @trisha_1212

TEAM @trisha_1212 ME - NOT SUSTAINABLE

RETROSPECTIVE DISCUSSION @trisha_1212

RETROSPECTIVE ACTIONS ITEMS ● Stop developing new feature ● Focus on test improvement ● Team decides that developer increase unit and integration tests ● Automate test cases based on risk, priority, business impact* ● Focused on operational excellence tasks @trisha_1212

MY ACTION ITEM - 7 MONTHS ● Digging deeper - focused not only on testing but improve on overall quality aspects. ● Open task - support on operational excellence task ● Not automating all test cases - focus on end to end user flow ● Fixing the existing flaky test cases. @trisha_1212

CODING DEBUGGING ● Locators(id> name> css> xpath) ● Logs ● Assertion ● Screenshots ● Waits ● Retry before failing test ● Small & Independent tests ● JS console errors ● Integrated with library test reports @trisha_1212

FRAMEWORK ENVIRONMENT ● Ease of configuration to test ● Docker ● Test data management ● Selenium grid ● Community support ● Clean test environment ● Test environment and production environment @trisha_1212

IMPROVED PRODUCT TESTABILITY ● Developer help to optimize page load, reduce downtime ● Reduce uncertainty inside application and infrastructure issues ● Reduce the technical debts and refactoring of code ● Seperate the backend and frontend code base and pipeline @trisha_1212

IMPROVED PROCESS TESTABILITY ● Root Cause Analysis for the actual failure of automated code ● Rotation task in maintaining the test code among team member ● Code review of test code ● Priority in maintaining and writing new test code @trisha_1212

IMPROVED PEOPLE TESTABILITY ● Everyone in team started contributing to automated checks ● Increase tech stack understanding ● Improved AWS/Google cloud understanding ● Developed our knowledge on containers @trisha_1212

FINALLY BUILD TIME IS… 5 MINUTES @trisha_1212

MAINTAIN THE PEOPLE TESTABILITY ● Increased cross team system knowledge by evolving more people in system deployments ● Team knowledge asking for feedback about tests and product. ● Pairing with new joiners and making them up to the speed @trisha_1212

MAINTAIN THE PROCESS TESTABILITY ● Update our automated tests continuously ● Maintain pipeline always green ● Having operational day in a sprints @trisha_1212

MAINTAIN THE PRODUCT TESTABILITY ● Monitoring the APIs, ● Continuous Refactoring of code. ● Infrastructure improvement ● Added additional logs and implemented Kibana @trisha_1212

SUMMARY IF YOU WANT TO STABILISE YOUR TESTS YOUR NEED TO… ● ● ● ● ● ● @trisha_1212 IMPLEMENT GOOD CODING PRACTICES ENCOURAGE TEAM RESPONSIBILITY FOR TEST IMPROVEMENT FOCUS ON HAVING STABLE TEST ENVIRONMENTS HAVE PEOPLE WITH THE RIGHT SKILLS AND KNOWLEDGE HAVE GOOD PRODUCT TESTABILITY CREATE AN ENVIRONMENT OF CONTINUOUS IMPROVEMENT AND LEARNING

THANK YOU @ajay184f @ajay184f @trisha_1212 @friendlytester @friendlytester @aahunsberger @mariakedemo @mariakedemo @kriscorbus @kriscorbus

Questions? @trisha_1212 @trisha_1212