Quality Criteria
Testing
You Should Also Read
Android users expect high-quality apps. App quality directly influences the long-term success of your app—in terms of installs, user rating and reviews, engagement, and user retention.
This document helps you assess basic aspects of quality in your app through a compact set of core app quality criteria and associated tests. All Android apps should meet these criteria.
Before publishing your apps, test them against these criteria to ensure that they function well on many devices, meets Android standards for navigation and design, and are prepared for promotional opportunities in the Google Play store. Your testing will go well beyond what's described here—the purpose of this document is to specify the essential quality characteristics all apps should display, so that you can cover them in your test plans.
If you're creating apps for tablets and or Google Play for Education there are additional quality criteria you should consider, which are defined in the Tablet App Quality guidelines and Education Guidelines.
Visual Design and User Interaction
These criteria ensure that your app provides standard Android visual design and interaction patterns where appropriate, for a consistent and intuitive user experience.
Area | ID | Description | Tests |
---|---|---|---|
Standard design | UX-B1 |
App follows Android Design guidelines and uses common UI patterns and icons:
|
CR-all |
Navigation | UX-N1 |
App supports standard system Back button navigation and does not make use of any custom, on-screen "Back button" prompts. |
CR-3 |
UX-N2 |
All dialogs are dismissable using the Back button. |
CR-3 | |
UX-N3 | Pressing the Home button at any point navigates to the Home screen of the device. | CR-1 | |
Notifications | UX-S1 |
Notifications follow Android Design guidelines. In particular:
|
CR-11 |
UX-S2 |
App uses notifications only to:
|
CR-11 |
Related resources
Functionality
These criteria ensure that your app provides expected functional behavior, with the appropriate level of permissions.
Area | ID | Description | Tests |
---|---|---|---|
Permissions | FN-P1 | App requests only the absolute minimum permissions that it needs to support core functionality. | CR-11 |
FN-P2 |
App does not request permissions to access sensitive data (such as Contacts or the System Log) or services that can cost the user money (such as the Dialer or SMS), unless related to a core capability of the app. |
||
Install location | FN-L1 |
App functions normally when installed on SD card (if supported by app). Supporting installation to SD card is recommended for most large apps (10MB+). See the App Install Location developer guide for information about which types of apps should support installation to SD card. |
SD-1 |
Audio | FN-A1 | Audio does not play when the screen is off, unless this is a core feature (for example, the app is a music player). | CR-7 |
FN-A2 | Audio does not play behind the lock screen, unless this is a core feature. | CR-8 | |
FN-A3 | Audio does not play on the home screen or over another app, unless this is a core feature. |
CR-1, CR-2 |
|
FN-A4 | Audio resumes when the app returns to the foreground, or indicates to the user that playback is in a paused state. | CR-1, CR-8 | |
UI and Graphics | FN-U1 |
App supports both landscape and portrait orientations (if possible). Orientations expose largely the same features and actions and preserve functional parity. Minor changes in content or views are acceptable. |
CR-5 |
FN-U2 |
App uses the whole screen in both orientations and does not letterbox to account for orientation changes. Minor letterboxing to compensate for small variations in screen geometry is acceptable. |
CR-5 | |
FN-U3 |
App correctly handles rapid transitions between display orientations without rendering problems. |
CR-5 | |
User/app state | FN-S1 |
App should not leave any services running when the app is in the background, unless related to a core capability of the app. For example, the app should not leave services running to maintain a network connection for notifications, to maintain a Bluetooth connection, or to keep the GPS powered-on. |
CR-6 |
FN-S2 |
App correctly preserves and restores user or app state. App preserves user or app state when leaving the foreground and prevents accidental data loss due to back-navigation and other state changes. When returning to the foreground, the app must restore the preserved state and any significant stateful transaction that was pending, such as changes to editable fields, game progress, menus, videos, and other sections of the app or game.
|
CR-1, CR-3, CR-5 |
Related resources
Performance and Stability
These criteria ensure that apps provide the performance, stability, and responsiveness expected by users.
Area | ID | Description | Tests |
---|---|---|---|
Stability | PS-S1 | App does not crash, force close, freeze, or otherwise function abnormally on any targeted device. | CR-all, SD-1, HA-1 |
Performance | PS-P1 | App loads quickly or provides onscreen feedback to the user (a progress indicator or similar cue) if the app takes longer than two seconds to load. | CR-all, SD-1 |
PS-P2 | With StrictMode enabled (see StrictMode Testing, below), no red flashes (performance warnings from StrictMode) are visible when exercising the app, including during game play, animations and UI transitions, and any other part of the app. | PM-1 | |
Media | PS-M1 | Music and video playback is smooth, without crackle, stutter, or other artifacts, during normal app usage and load. | CR-all, SD-1, HA-1 |
Visual quality | PS-V1 |
App displays graphics, text, images, and other UI elements without noticeable distortion, blurring, or pixelation.
|
CR-all |
PS-V2 |
App displays text and text blocks in an acceptable manner.
|
Related resources
Google Play
These criteria ensure that your apps are ready to publish on Google Play.
Area | ID | Description | Tests |
---|---|---|---|
Policies | GP-P1 | App strictly adheres to the terms of the Google Play Developer Content Policy and does not offer inappropriate content, does not use intellectual property or brand of others, and so on. | GP-all |
GP-P2 |
App maturity level is set appropriately, based on the Content Rating Guidelines. Especially, note that apps that request permission to use the device location cannot be given the maturity level "Everyone". |
GP-1 | |
App Details Page | GP-D1 |
App feature graphic follows the guidelines outlined in this blog post. Make sure that:
|
GP-1, GP-2 |
GP-D2 | App screenshots and videos do not show or reference non-Android devices. | GP-1 | |
GP-D3 | App screenshots or videos do not represent the content and experience of your app in a misleading way. | ||
User Support | GP-X1 | Common user-reported bugs in the Reviews tab of the Google Play page are addressed if they are reproducible and occur on many different devices. If a bug occurs on only a few devices, you should still address it if those devices are particularly popular or new. | GP-1 |
Related resources
Setting Up a Test Environment
To assess the quality of your app, you need to set up a suitable hardware or emulator environment for testing.
The ideal test environment would include a small number of actual hardware devices that represent key form factors and hardware/software combinations currently available to consumers. It's not necessary to test on every device that's on the market — rather, you should focus on a small number of representative devices, even using one or two devices per form factor.
If you are not able to obtain actual hardware devices for testing, you should set up emulated devices (AVDs) to represent the most common form factors and hardware/software combinations.
To go beyond basic testing, you can add more devices, more form factors, or new hardware/software combinations to your test environment. You can also increase the number or complexity of tests and quality criteria.
Test Procedures
These test procedures help you discover various types of quality issues in your app. You can combine the tests or integrate groups of tests together in your own test plans. See the sections above for references that associate specific criteria with specific tests.
Type | Test | Description |
---|---|---|
Core Suite | CR-0 |
Navigate to all parts of the app — all screens, dialogs, settings, and all user flows.
|
CR-1 | From each app screen, press the device's Home key, then re-launch the app from the All Apps screen. | |
CR-2 | From each app screen, switch to another running app and then return to the app under test using the Recents app switcher. | |
CR-3 | From each app screen (and dialogs), press the Back button. | |
CR-5 | From each app screen, rotate the device between landscape and portrait orientation at least three times. | |
CR-6 | Switch to another app to send the test app into the background. Go to Settings and check whether the test app has any services running while in the background. In Android 4.0 and higher, go to the Apps screen and find the app in the "Running" tab. In earlier versions, use "Manage Applications" to check for running services. | |
CR-7 | Press the power button to put the device to sleep, then press the power button again to awaken the screen. | |
CR-8 | Set the device to lock when the power button is pressed. Press the power button to put the device to sleep, then press the power button again to awaken the screen, then unlock the device. | |
CR-9 | For devices that have slide-out keyboards, slide the keyboard in and out at least once. For devices that have keyboard docks, attach the device to the keyboard dock. | |
CR-10 | For devices that have an external display port, plug-in the external display. | |
CR-11 | Trigger and observe in the notications drawer all types of notifications that the app can display. Expand notifications where applicable (Android 4.1 and higher), and tap all actions offered. | |
CR-12 | Examine the permissions requested by the app by going to Settings > App Info. | |
Install on SD Card | SD-1 |
Repeat Core Suite with app installed to device SD card (if supported by app). To move the app to SD card, you can use Settings > App Info > Move to SD Card. |
Hardware acceleration | HA-1 |
Repeat Core Suite with hardware acceleration enabled.
To force-enable hardware acceleration (where supported by device), add
|
Performance Monitoring | PM-1 |
Repeat Core Suite with StrictMode profiling enabled as described below. Pay close attention to garbage collection and its impact on the user experience. |
Google Play | GP-1 | Sign into the Developer Console to review your developer profile, app description, screenshots, feature graphic, maturity settings, and user feedback. |
GP-2 | Download your feature graphic and screenshots and scale them down to match the display sizes on the devices and form factors you are targeting. | |
GP-3 | Review all graphical assets, media, text, code libraries, and other content packaged in the app or expansion file download. | |
Payments | GP-4 | Navigate to all screens of your app and enter all in-app purchase flows. |
Testing with StrictMode
For performance testing, we recommend enabling StrictMode
in your app and using it to catch operations on the main thread and other
threads that could affect performance, network accesses, file reads/writes,
and so on.
You can set up a monitoring policy per thread using StrictMode.ThreadPolicy.Builder
and enable all supported
monitoring in the ThreadPolicy
using detectAll()
.
Make sure to enable visual notification of policy violations
for the ThreadPolicy
using penaltyFlashScreen()
.