Testing of Android Applications: A Practical Guide for QA Teams

Android testing is not iOS testing with different tooling. The platform gives you an open device ecosystem, manufacturer-modified builds of the OS, and users running versions released years apart. That changes what you test, not just how.

This guide covers the decisions that actually affect defect escape rate: which devices to test on, when an emulator is enough, and how to get a reliable instrumented suite running in CI.

The two test types Android gives you

Everything in the Android test stack sits in one of two buckets, and confusing them is the most common cause of a slow, flaky suite.

Local unit tests run on the JVM on your machine. No device, no emulator, milliseconds per test. Anything that does not touch the Android framework belongs here.

Instrumented tests run on a real device or emulator, because they need the actual Android runtime. UI tests, database migrations and anything touching Context belong here.

The rule of thumb that survives contact with a real project: if a test needs a Context, it is instrumented. If it does not, it is a unit test, and putting it on a device is wasting minutes per run for no added confidence.

# Local unit tests, JVM only, fast
./gradlew test

# Instrumented tests, needs a connected device or running emulator
./gradlew connectedAndroidTest

# One variant only, which is what you want in CI
./gradlew connectedDebugAndroidTest

Fragmentation: what actually matters

Android fragmentation is usually described as a device-count problem. In practice it breaks into three separate risks, and only two of them need real devices.

OS version spread

Your minSdk decides how many platform behaviours you must support. Each Android release changes permissions, background execution limits or storage access, and those changes are where version-specific defects live.

Google publishes current version distribution in Android Studio when you create a project, under the API level selector. Use that rather than a blog post, including this one: the numbers move every quarter and a stale figure is worse than no figure.

Manufacturer modifications

Samsung One UI, Xiaomi HyperOS and others modify the OS meaningfully. Background process management is the usual casualty: an app that syncs reliably on a Pixel can have its worker killed aggressively on a device with a vendor battery optimiser.

This risk is invisible on emulators. Emulators run stock Android, so no emulator will ever reproduce a manufacturer-specific background-kill bug.

Screen and hardware variance

Aspect ratios, display cutouts, foldables and density buckets. Mostly catchable with automated screenshot tests across configured device profiles, which is the cheapest of the three risks to cover.

Emulator or real device

This decision drives most of your device budget, so make it deliberately rather than defaulting to whatever is on the desk.

Emulators are correct for the majority of automated runs. They are reproducible, disposable, and they parallelise. Reach for real hardware when the thing under test is physical: camera behaviour, sensors, biometrics, actual network transitions, battery effects, or vendor OS behaviour.

Gradle-managed devices

Rather than maintaining emulator images by hand, declare them in the build file. The Android Gradle plugin then creates, deploys to and tears down those devices as part of the test run, which removes the commonest source of "works on my machine" in Android CI.

// app/build.gradle.kts
android {
    testOptions {
        managedDevices {
            localDevices {
                create("pixel6api33") {
                    device = "Pixel 6"
                    apiLevel = 33
                    systemImageSource = "aosp-atd"
                }
            }
        }
    }
}

# Gradle creates the device, runs the tests, tears it down
./gradlew pixel6api33DebugAndroidTest

Available for API level 27 and higher. The aosp-atd image is an automated-test-optimised build: it strips components an automated run never needs, so it boots faster and uses less memory than a full Google APIs image.

Building a device matrix you can defend

Testing on every device is impossible and testing on one is negligent. A defensible matrix is small and justified by your own analytics, not by a general popularity list.

1. Pull your actual distribution. Play Console gives you installs by device model and by Android version. That is the only list that matters, and it is specific to your app.
2. Cover the version floor and ceiling. Your minSdk, your targetSdk, and anything holding meaningful share between them.
3. Add one vendor-modified device. Usually Samsung, given its share in most markets. This is the device that finds background-execution defects.
4. Add one low-memory device. Performance defects hide on flagships and surface on budget hardware.
5. Add a foldable or tablet only if your analytics justify it. Otherwise this is effort spent on a configuration your users do not have.

Five to eight physical devices covers most consumer apps, with emulators handling the combinatorial spread around them.

Where Android suites usually break

Flakiness from implicit waits

The single largest source of unreliable Android tests is asserting before the UI has settled. Espresso synchronises with the main thread automatically, but it cannot know about your background work.

Register an idling resource for asynchronous operations rather than sleeping. A Thread.sleep that fixes a flaky test today becomes a slow test that is still flaky under CI load.

Permission dialogs

Runtime permission prompts block UI tests and are easy to forget until the suite fails on a fresh emulator.

# Grant permissions before the run rather than handling dialogs mid-test
adb shell pm grant com.example.app android.permission.CAMERA
adb shell pm grant com.example.app android.permission.ACCESS_FINE_LOCATION

Or use GrantPermissionRule in the test itself, which keeps the setup with the test that needs it.

Animations

Animations cause intermittent failures under load. Disable them on test devices:

adb shell settings put global window_animation_scale 0
adb shell settings put global transition_animation_scale 0
adb shell settings put global animator_duration_scale 0

State leaking between tests

Instrumented tests share an app installation, so a test that writes to shared preferences or the database affects the next one. Clear state explicitly in @Before, and prefer clearPackageData in your test runner configuration so each run starts clean.

Play Store pre-launch checks

Uploading to a Play Console testing track triggers an automated crawl of your app across a set of real devices, returning crash reports, screenshots across configurations, and accessibility and performance findings.

It is genuinely useful and it is not a test suite. It explores your app without knowing your business rules, so it finds crashes and layout breakage, not incorrect behaviour. Treat it as a free smoke test on hardware you do not own, then rely on your own suite for correctness.

A sane CI shape

Run the fast things on every commit and the slow things less often. The alternative, running everything on every push, leads to a suite people disable.

• Every commit: local unit tests. Seconds, no device needed.
• Every pull request: instrumented tests on one or two Gradle-managed emulator profiles covering your minSdk and targetSdk.
• Nightly: the wider emulator matrix, plus screenshot comparison.
• Pre-release: the physical device matrix, plus a Play Console pre-launch report.

The goal is that a failed check on a pull request is believable. A suite that fails randomly gets ignored, and an ignored suite is worse than no suite because it still costs CI minutes and still carries the appearance of coverage.

If you would rather hand this to a team that already runs Android device labs, our Android app testing services cover device matrix design, automation and release sign-off.