A presentation at Notts Test in in Nottingham, UK by Graham Smith
The things to remember not to forget when you're testing Android apps
Who are we? Graham Smith @whoisgraham Matthew Herod @matthewherod
JOIN IN Feel free to say “me too!” or to add other ideas in as we go!
What did I forget to test? LET’S TAKE AN SEMI-ANECDOTAL WALK TOGETHER
1 That time my notes App lost its notes... Ah! A Classic!
Using Google Keep, went to test an App feature by enabling the “Dont keep activities setting” returned to Google Keep with all notes lost...
SO WHAT HAPPENED? ▪ Android lifecycle events ▪ Don’t keep activities simulates your app being stopped due to low memory etc ▪ Need to write code to store inputs when this occurs.
https://developer.android.com/guide/components/activities/activity-lifecycle
How can I test this?
Enable developer options on the device.
Go to Developer Settings Enable “Don’t keep activities”
2 That time App crashed when it wasn’t on the screen Yup another classic!
Quite a classic example where something async normally tries to update the view after its gone
▪ Kick off some async process ▪ App no longer has Device focus ▪ After the async process finishes it tries to update the view. ▪ User receives a nasty NPE
▪ How can I test this? ▪ Unit tests for the code! ▫ ▪ Make use of Kotlin’s nullability operators Place your screenshot here Manual testing/UI testing possible but not so easy 12
3 That time App crashed when I rotated the screen But I like landscape...
Did you know? Your Activity gets recreated on rotation! But there are caveats and other options.
SO WHAT HAPPENED?
▪ The screen is rotated ▪ App loses state or get a bit confused ▪ User receives a nasty NPE or the UI doesn’t reflect the right state.
How can I test this? ▪Lock it to portrait ▫ ▫ The “painting over the stain solution”
▪ Look at what state is being held on to ▪ Unit testing - Seriously do it!
▪ UI Automation tests ▫ Espresso has support for this
This is a BIG topic… Be careful if you use configChanges in your manifest https://stackoverflow.com/questions/456211/activity-restart-on-rotation-android
4 That time we shipped a new feature that immediately crashed
We all make mistakes… Time pressures & feature flagging hid an issue in the release build of the App caused by obfuscated code.
SO WHAT HAPPENED? ▪ Feature flagged a feature to “ON” ▪ Users pressed the new menu item ▪ App crashed immediately
Be careful not to forget proguard
What is Proguard? Proguard is used to minify and obfuscate the Java code when creating a release build.
Proguard is a really helpful tool!
How can I fix this? Test the live release! We made a mistake - it was easy to fix. Luckily we have staged rollouts so the impact was minimal to the whole user base.
5 Odds and Evens - Android Fragmentation gotchas... A quick example, one of many...
Android 6 introduced the requirement of a special flag when starting an App from a background process or notification. They removed it in 7. They brought it back in 8+
SO WHAT HAPPENED? ▪ One to remember while in development ▪ Users interacts with a notification that opens an App. ▪ Android 5 & 7 works fine without the flag ▪ Android 6 & 8 crash.
So what do I do about it? Remember to add the flag! Make sure the code has the following OR use another mechanism! intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK); Remember to test on multiple Android versions! But you do this anyway… right?
6 Application stacks, task affinities and the UX Brace yourselves...
How can I fix this? Remember check the launch mode of the Activity! There are lots of different modes that change the way Android creates Activities. In the example before we reuse the existing Activity so to keep memory use down. Learn about Android Tasks Check out the Android documentation: https://developer.android.com/guide/components/activities/ tasks-and-back-stack
7 Migration Testing
Your app probably stores data, like databases and preferences. Make sure your app updates are compatible with data created by previous versions, or implement a migration!
SO WHAT HAPPENED? ▪ Your App stored some data in a database ▪ An update is released with changes to the database schema ▪ The App updates and now the app doesn’t understand it’s own database Schema Change ▪ Either an error or data loss!
▪ How can I test this? ▪ Before publishing a new version run your old version and then install the new version over the top. Does everything still work? ▪ Consider that installing version v7 of your app over version v2 or your app may also produce the same issue. ▪ It’s boring but it’s important!
Thanks! ANY QUESTIONS?
So you've completed that latest feature but what did you, potentially, forget to test? This presentation tries to demonstrate some of the those things you might have forgot to test and what you might be able to do to combat them.
The following resources were mentioned during the presentation or are useful additional information.
A question on Stack Overflow that shows both the complexity and variety of answers, coupled with some decent links to some great blog posts on the topic.
A great article on staged rollouts for Android via the Google Play Store
Useful for migration testing as described in the slides amongst other things.
Full depth guide on launch modes, stacks and much more from Google.
for free. You
can too.