This document describes how to create and run tests directly from the command line. You can use the techniques described here if you are developing in an IDE other than Eclipse or if you prefer to work from the command line. This document assumes that you already know how to create a Android application in your programming environment. Before you start this document, you should read the document Testing and Instrumentation, which provides an overview of Android testing.
If you are developing in Eclipse with ADT, you can set up and run your tests directly in Eclipse. For more information, please read Testing in Eclipse, with ADT.
You use the android
tool to create test projects.
You also use android
to convert existing test code into an Android test project,
or to add the run-tests
Ant target to an existing Android test project.
These operations are described in more detail in the section Updating a test project.
The run-tests
target is described in Quick build and run with Ant.
To create a test project with the android
tool, enter:
android create test-project -m <main_path> -n <project_name> -p <test_path>
You must supply all the flags. The following table explains them in detail:
Flag | Value | Description |
---|---|---|
-m, --main |
Path to the project of the application under test, relative to the test application directory. |
For example, if the application under test is in source/HelloAndroid , and you
want to create the test project in source/HelloAndroidTest , then the value of
--main should be ../HelloAndroid .
|
-n, --name |
Name that you want to give the test project. | |
-p, --path |
Directory in which you want to create the new test project. |
The android tool creates the test project files and directory structure in this
directory. If the directory does not exist, android creates it.
|
If the operation is successful, android
lists to STDOUT the names of the files
and directories it has created.
This creates a new test project with the appropriate directories and build files. The directory structure and build file contents are identical to those in a regular Android application project. They are described in detail in the topic Developing In Other IDEs.
The operation also creates an AndroidManifest.xml
file with instrumentation
information. When you run the test, Android uses this information to load the application you
are testing and control it with instrumentation.
For example, suppose you create the Hello, World tutorial application
in the directory ~/source/HelloAndroid
. In the tutorial, this application uses the
package name com.example.helloandroid
and the activity name
HelloAndroid
. You can to create the test for this in
~/source/HelloAndroidTest
. To do so, you enter:
$ cd ~/source $ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest
This creates a directory called ~/src/HelloAndroidTest
. In the new directory you
see the file AndroidManifest.xml
. This file contains the following
instrumentation-related elements and attributes:
<application>
: to contain the
<uses-library>
element.
<uses-library android:name="android.test.runner"
:
specifies this testing application uses the android.test.runner
library.
<instrumentation>
: contains attributes that control Android
instrumentation. The attributes are:
android:name="android.test.InstrumentationTestRunner"
:
InstrumentationTestRunner
runs test cases. It extends both
JUnit test case runner classes and Android instrumentation classes.
android:targetPackage="com.example.helloandroid"
: specifies
that the tests in HelloAndroidTest should be run against the application with the
Android package name com.example.helloandroid
. This is the
package name of the Hello, World
tutorial application.
android:label="Tests for .HelloAndroid"
: specifies a
user-readable label for the instrumentation class. By default,
the android
tool gives it the value "Tests for " plus
the name of the main Activity of the application under test.
You use the android
tool when you need to change the path to the
project of the application under test. If you are changing an existing test project created in
Eclipse with ADT so that you can also build and run it from the command line, you must use the
"create" operation. See the section Creating a test project.
Note: If you change the Android package name of the application under test,
you must manually change the value of the <android:targetPackage>
attribute within the AndroidManifest.xml
file of the test application.
Running android update test-project
does not do this.
To update a test project with the android
tool, enter:
android update-test-project -m <main_path> -p <test_path>
Flag | Value | Description |
---|---|---|
-m, --main |
The path to the project of the application under test, relative to the test project |
For example, if the application under test is in source/HelloAndroid , and
the test project is in source/HelloAndroidTest , then the value for
--main is ../HelloAndroid .
|
-p, --path |
The of the test project. |
For example, if the test project is in source/HelloAndroidTest , then the
value for --path is HelloAndroidTest .
|
If the operation is successful, android
lists to STDOUT the names of the files
and directories it has created.
Once you have created a test project, you populate it with a test application.
The application does not require an Activity
,
although you can define one if you wish. Although your test application can
combine Activities, Android test class extensions, JUnit extensions, or
ordinary classes, you should extend one of the Android test classes or JUnit classes,
because these provide the best testing features.
If you run your tests with InstrumentationTestRunner
(or a related test runner), then it will run all the methods in each class. You can modify
this behavior by using the TestSuite
class.
To create a test application, start with one of Android's test classes in the Java package
android.test
. These extend the JUnit
TestCase
class. With a few exceptions, the Android test
classes also provide instrumentation for testing.
For test classes that extend TestCase
, you probably want to
override the setUp()
and tearDown()
methods:
setUp()
: This method is invoked before any of the test methods in the class.
Use it to set up the environment for the test. You can use setUp()
to instantiate a new Intent
object with the action ACTION_MAIN
.
You can then use this intent to start the Activity under test.
Note: If you override this method, call
super.setUp()
as the first statement in your code.
tearDown()
: This method is invoked after all the test methods in the class. Use
it to do garbage collection and re-setting before moving on to the next set of tests.
Note: If you override this method, you must call
super.tearDown()
as the last statement in your code.
Another useful convention is to add the method testPreConditions()
to your test
class. Use this method to test that the application under test is initialized correctly. If this
test fails, you know that that the initial conditions were in error. When this happens, further
test results are suspect, regardless of whether or not the tests succeeded.
To learn more about creating test applications, see the topic Testing and Instrumentation, which provides an overview of Android testing. If you prefer to follow a tutorial, try the Activity Testing tutorial, which leads you through the creation of tests for an actual Android application.
If you are not developing in Eclipse with ADT, you need to run tests from the command line.
You can do this either with Ant or with the ActivityManager
command line interface.
You can also run tests from the command line even if you are using Eclipse with ADT to develop
them. To do this, you need to create the proper files and directory structure in the test
project, using the android
tool with the option create test-project
.
This is described in the section Working with Test Projects.
You can use Ant to run all the tests in your test project, using the target
run-tests
, which is created automatically when you create a test project with
the android
tool.
This target re-builds your main project and test project if necessary, installs the test
application to the current AVD or device, and then runs all the test classes in the test
application. The results are directed to STDOUT
.
You can update an existing test project to use this feature. To do this, use the
android
tool with the update test-project
option. This is described
in the section Updating a test project.
When you run tests from the command line with the ActivityManager (am
)
command-line tool, you get more options for choosing the tests to run than with any other
method. You can select individual test methods, filter tests according to their annotation, or
specify testing options. Since the test run is controlled entirely from a command line, you can
customize your testing with shell scripts in various ways.
You run the am
tool on an Android device or emulator using the
Android Debug Bridge
(adb
) shell. When you do this, you use the ActivityManager
instrument
option to run your test application using an Android test runner
(usually InstrumentationTestRunner
). You set am
options with command-line flags.
To run a test with am
:
.apk
files) to your current Android device or emulator$ adb shell am instrument -w <test_package_name>/<runner_class>
where <test_package_name>
is the Android package name of your test
application, and <runner_class>
is the name of the Android test runner
class you are using. The Android package name is the value of the package
attribute of the manifest
element in the manifest file
(AndroidManifest.xml
) of your test application. The Android test runner
class is usually InstrumentationTestRunner
.
Your test results appear in STDOUT
.
This operation starts an adb
shell, then runs am instrument
in it
with the specified parameters. This particular form of the command will run all of the tests
in your test application. You can control this behavior with flags that you pass to
am instrument
. These flags are described in the next section.
The general syntax of the am instrument
command is:
am instrument [flags] <test_package>/<runner_class>
The main input parameters to am instrument
are described in the following table:
Parameter | Value | Description |
---|---|---|
<test_package>
|
The Android package name of the test application. |
The value of the package attribute of the manifest
element in the test application's manifest file.
|
<runner_class>
|
The class name of the instrumented test runner you are using. |
This is usually InstrumentationTestRunner .
|
The flags for am instrument
are described in the following table:
Flag | Value | Description |
---|---|---|
-w
|
(none) |
Forces am instrument to wait until the instrumentation terminates
before terminating itself. The net effect is to keep the shell open until the tests
have finished. This flag is not required, but if you do not use it, you will not
see the results of your tests.
|
-r
|
(none) |
Outputs results in raw format. Use this flag when you want to collect
performance measurements, so that they are not formatted as test results. This flag is
designed for use with the flag -e perf true (documented in the section
Instrument options).
|
-e
|
<test_options> |
Provides testing options , in the form of key-value pairs. The
am instrument tool passes these to the specified instrumentation class
via its onCreate() method. You can specify multiple occurrences of
-e <test_options . The keys and values are described in the next table.
The only instrumentation class that understands these key-value pairs is
|
The am instrument
tool passes testing options to
InstrumentationTestRunner
or a subclass in the form of key-value pairs,
using the -e
flag, with this syntax:
-e <key> <value>
Where applicable, a <key> may have multiple values separated by a comma (,).
For example, this invocation of InstrumentationTestRunner
provides multiple
values for the package
key:
$ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 com.android.test/android.test.InstrumentationTestRunner
The following table describes the key-value pairs and their result. Please review the Usage Notes following the table.
Key | Value | Description |
---|---|---|
package
|
<Java_package_name> | The fully-qualified Java package name for one of the packages in the test application. Any test case class that uses this package name is executed. Notice that this is not an Android package name; a test application has a single Android package name but may have several Java packages within it. |
class |
<class_name> | The fully-qualified Java class name for one of the test case classes. Only this test case class is executed. |
<class_name>#method name | A fully-qualified test case class name, and one of its methods. Only this method is executed. Note the hash mark (#) between the class name and the method name. | |
func |
true |
Runs all test classes that extend InstrumentationTestCase .
|
unit |
true |
Runs all test classes that do not extend either
InstrumentationTestCase or PerformanceTestCase .
|
size |
[small | medium | large ]
|
Runs a test method annotated by size. The annotations are @SmallTest ,
@MediumTest , and @LargeTest .
|
perf |
true |
Runs all test classes that implement PerformanceTestCase .
When you use this option, also specify the -r flag for
am instrument , so that the output is kept in raw format and not
re-formatted as test results.
|
debug |
true |
Runs tests in debug mode. |
log |
true |
Loads and logs all specified tests, but does not run them. The test
information appears in STDOUT . Use this to verify combinations of other filters
and test specifications.
|
emma |
true |
Runs an EMMA code coverage analysis and writes the output to /data//coverage.ec
on the device. To override the file location, use the coverageFile key that
is described in the following entry.
Note: This option requires an EMMA-instrumented build of the test
application, which you can generate with the |
coverageFile |
<filename> |
Overrides the default location of the EMMA coverage file on the device. Specify this
value as a path and filename in UNIX format. The default filename is described in the
entry for the emma key.
|
-e
Flag Usage Notes
am instrument
invokes
onCreate(Bundle)
with a Bundle
containing the key-value pairs.
package
key takes precedence over the class
key. If you
specifiy a package, and then separately specify a class within that package, Android
will run all the tests in the package and ignore the class
key.
func
key and unit
key are mutually exclusive.
Here are some examples of using am instrument
to run tests. They are based on
the following structure:
com.android.demo.app.tests
UnitTests
, which contains the methods
testPermissions
and testSaveState
.
FunctionTests
, which contains the methods
testCamera
, testXVGA
, and testHardKeyboard
.
IntegrationTests
,
which contains the method testActivityProvider
.
InstrumentationTestRunner
.
To run all of the test classes in the test application, enter:
$ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner
To run all of the tests in the class UnitTests
, enter:
$ adb shell am instrument -w \ -e class com.android.demo.app.tests.UnitTests \ com.android.demo.app.tests/android.test.InstrumentationTestRunner
am instrument
gets the value of the -e
flag, detects the
class
keyword, and runs all the methods in the UnitTests
class.
To run all of the tests in UnitTests
, and the testCamera
method in
FunctionTests
, enter:
$ adb shell am instrument -w \ -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \ com.android.demo.app.tests/android.test.InstrumentationTestRunner
You can find more examples of the command in the documentation for
InstrumentationTestRunner
.