Testing as an Engineering Role

Testing , testing, 1, 2, 3 ..

Who is L̶o̶a̶n̶a̶ Ioana Chiorean? I am a PM (former QA TL) & CL Mozillian for more than 9 years now I volunteer - Mozilla & EU Code Week. I worked on devices for more than 10y. Twitter @ioana_cis # lifeofamozillian 2

1,2,3 la perete stop.. ➔ Testing ◆ What is is ◆ Responsibilities ◆ Manual vs Automation ◆ Skills ◆ A day in #lifeofatester ➔ UX Testing ➔ UX @ Mozilla 3

I didn’t choose the thug life, the thug life chose me 😎

2 - 10 .. 2 - 10 .. ➔ Software testing is an investigation conducted to provide stakeholders with information about the quality of the software product or service under test. ➔ Is really about reducing risk and so much more ◆ ◆ ◆ ◆ ◆ ◆ Advocate of user - UX Software Engineer - coding Explorer Scientist - heuristics Writer / Documentarist Jack of all trades, master of ALL!*

• not a bug, it is a feature 5

With great power comes… What are your responsibilities? ➔ ➔ ➔ ➔ ➔ ➔ ➔ ➔ Analyzing users stories and/use cases/requirements for validity and feasibility Collaborate closely with other team members and departments Execute all levels of testing (System, Integration, and Regression) Design and develop automation scripts when needed Detect and track software defects and inconsistencias Provide timely solutions Apply quality engineering principals throughout the Agile product lifecycle Provide support and documentation 6

~= SDLC STLC Requirements Gathering Test team reviews the requirements Design Test planning, analysis and design is done => Test Plan Development Finalize test plan and write test cases Testing Test execution, bug reporting, manual and automated; Regression Deployment Test report is prepared Maintenance Maintenance testing 7

Manual vs Automation

Manual vs Automation Manual ➔ ➔ ➔ ➔ ➔ Black/White/Gray Testing Functional & Non Functional testing ◆ Installation, Compatibility, Smoke and sanity, Regression, Acceptance, Alpha, Beta, Continuous, Destructive, Software performance, Usability, Accessibility, Security, Internationalization and localization, Development, A/B testing Exploratory Time - as major resource Human - important resource 9

Manual vs Automation Automation ➔ ➔ ➔ ➔ ➔ ➔ ➔ User stories coded and ran daily/weekly/per new buildy Can not cover undefine user stories Doesn’t get creative on its on Time - as major resource Storage & machines/devices Set of data Fail test -> ? 10

It’s possible to fly without motors, but not without knowledge and skill Wilbur Wright

What skills do you need for this role? ➔ ➔ ➔ ➔ ➔ ➔ ➔ ➔ ➔ ➔ Attention to details Analytical thinking Independence Proactivity Ownership Logical thinking Creative Continuous learning Efficient communication & so many other… 13

Challenges ➔ ➔ ➔ ➔ ➔ ➔ ➔ ➔ Define clear steps to reproduce for intermittent bugs Be creative enough to catch important bugs via exploratory testing Accommodate working with different timezone Not clear documentation / requirements Unstable environment Lack of feedback loop Testing under time constraints Working on several projects in the same time 14

A day in the life of a tester ➔ ➔ ➔ ➔ ➔ ➔ ➔ Coffee Mails, Sprint Meetings (planning, daily, review, retrospective) Testing ◆ Exploratory & Automated tests runs / Investigate failures ◆ Smokes, Sanity Checks, Sign Offs Features ◆ Discussion with product / devs to get info on existing user stories ◆ Component watch, Test Plan & Health Bug Activities ◆ Log issues, Triage, NI, Debugging, Meetings, meetings, meetings .. more coffee.. or beer.. or wine.. 15

Don’t be the sheep! Be the wolf fox! Photo credit.

http://webcompat.com/ w

UX Testing. Photo credit.

How .. ➔ ➔ ➔ ➔ ➔ ➔ ➔ Attention to details Hallway testing Remote usability testing Expert review Automated expert review A/B testing A11y? 20

A/B ➔ ➔ ➔ ➔ Offering share experience mostly with two (or more) skins ◆ Banners, actions, etc Useful mostly in marketing/sales but not only Looks for user retention/ interaction Analyse correctly ◆ Be sure u can define easily the difference in user data ◆ Bounded in time and one action 21

Alpha/Beta testing ➔ ➔ ➔ ➔ ➔ Set preview builds ex: Nightly/Firefox Preview Hallway testing - give the app to a random person Remote usability testing - give the app/site to people from different areas (geographical, expertise, domain) Expert review - make an audit Gather telemetry data/crashes etc 22

w

Let me be clear.. ➔ ➔ ➔ ➔ ➔ Do research ◆ Who is your user/ diverse user range Share the insights ◆ Kill the presumption that everyone knows Build up specs ◆ Draw, build, showcase Keep in touch ◆ Check progress Teach ◆ Share knowledge to others too 24

A11y

What? ➔ ➔ ➔ ➔ ➔ Vision Disability - Complete Blindness or Color Blindness or Poor Vision ; Visual problems like visual strobe and flashing effect problems Physical Disability - Not able to use the mouse or keyboard with one hand; Poor motor skills like hand movements and muscle slowness Cognitive disability - Learning Difficulties or Poor Memory or not able to understand more complex scenarios Literacy Disability - Reading Problems Hearing Disability - Auditory problems like deafness and hearing impairments; Cannot able to hear or not able to hear clearly 26

Why.. ➔ ➔ ➔ ➔ ➔ ➔ Define corectly A11y Regulations Users - data sets ◆ You do have them, trust me ◆ New ones No ROI No money, honey Doing favors vs doing your job 27

How ➔ ➔ ➔ ➔ ➔ ➔ Color Contrast Patterns & Colors Typography Animations Semantic - elements in order/a11y tree/ Page structure/ Menu nav & list Alt input - links, images, etc 28

Tools ➔ ➔ ➔ ➔ ➔ ➔ Close your eyes and try to navigate JAWS, Orca, Voice Over. Lighthouse Wave Tota11y My ex: ◆ 3 different OS + mobile => 4 ◆ 2 browsers + mobile => 3 ◆ Different Machines / Monitors 29

This should be a nice UX right? Firefox User Experience

https://design.firefox.com w

Values: w

https://design.firefox.com/values w

🐦 @ioana_cis ● ioana@mozilla.com ● http://bit.ly/Testing-SUXM Let’s fix the world one bug at a time!