In this document
- Overview
- TrackRenderer
- SampleSource
- Player Events
- Sending messages to components
- Customizing ExoPlayer
- Digital Rights Management
Key Classes
Related Samples
Playing videos and music is a popular activity on Android devices. The Android framework
provides MediaPlayer
as a quick solution for playing media with minimal
code, and the MediaCodec
and MediaExtractor
classes
are provided for building custom media players. The open source project, ExoPlayer, is a
solution between these two options, providing a pre-built player that you can extend.
ExoPlayer supports features not currently provided by
MediaPlayer
, including Dynamic adaptive streaming
over HTTP (DASH), SmoothStreaming, and persistent caching. ExoPlayer can be extended
to handle additional media formats, and because you include it as part of your app code,
you can update it along with your app.
This guide describes how to use ExoPlayer for playing Android supported media formats, as well as DASH and SmoothStreaming playback. This guide also discusses ExoPlayer events, messages, DRM support and guidelines for customizing the player.
Note: ExoPlayer is an open source project that is not part of the Android framework and is distributed separately from the Android SDK. The project contains a library and a demo app that shows both simple and more advanced use of ExoPlayer:
- ExoPlayer Library — This part of the project contains the core library classes.
- Simple Demo — This part of the app demonstrates a basic use of ExoPlayer.
- Full Demo — This part of the app demonstrates more advanced features, including the ability to select between multiple audio tracks, a background audio mode, event logging and DRM protected playback.
Overview
ExoPlayer is a media player built on top of the MediaExtractor
and
MediaCodec
APIs released in Android 4.1 (API level 16). At the core of this
library is the ExoPlayer
class. This class maintains the player’s global state, but makes few
assumptions about the nature of the media being played, such as how the media data is obtained,
how it is buffered or its format. You inject this functionality through ExoPlayer’s prepare()
method in the form of TrackRenderer
objects.
ExoPlayer provides default TrackRenderer
implementations for audio and
video, which make use of the MediaCodec
and AudioTrack
classes in the Android framework. Both renderers require a SampleSource
object, from which
they obtain individual media samples for playback. Figure 1 shows the high level object model for
an ExoPlayer implementation configured to play audio and video using these components.
![](../../../images/exoplayer/object-model.png)
Figure 1. High level object model for an ExoPlayer configured to play audio
and video using TrackRenderer
objects
TrackRenderer
A TrackRenderer
processes a component of media for playback, such as
video, audio or text. The ExoPlayer class invokes methods on its TrackRenderer
instances from a
single playback thread, and by doing so causes each media component to be rendered as the global
playback position is advanced. The ExoPlayer library provides MediaCodecVideoTrackRenderer
as
the default implementations rendering video and MediaCodecAudioTrackRenderer
for audio.
Both implementations make use of MediaCodec
to decode individual media
samples. They can handle all audio and video formats supported by a given Android device
(see Supported Media
Formats for details). The ExoPlayer library also provides an implementation for rendering
text called TextTrackRenderer
.
The code example below outlines the main steps required to instantiate an ExoPlayer to play video
and audio using the standard TrackRenderer
implementations.
// 1. Instantiate the player. player = ExoPlayer.Factory.newInstance(RENDERER_COUNT); // 2. Construct renderers. MediaCodecVideoTrackRenderer videoRenderer = … MediaCodecAudioTrackRenderer audioRenderer = ... // 3. Inject the renderers through prepare. player.prepare(videoRenderer, audioRenderer); // 4. Pass the surface to the video renderer. player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface); // 5. Start playback. player.setPlayWhenReady(true); ... player.release(); // Don’t forget to release when done!
For a complete example, see the SimplePlayerActivity
in the ExoPlayer demo app, which
correctly manages an ExoPlayer instance with respect to both the Activity
and
Surface
lifecycles.
SampleSource
A standard TrackRenderer
implementation requires a SampleSource
to
be provided in its constructor. A SampleSource
object provides format information and
media samples to be rendered. The ExoPlayer library provides FrameworkSampleSource
and
ChunkSampleSource
. The FrameworkSampleSource
class uses MediaExtractor
to request, buffer and extract the media samples. The ChunkSampleSource
class provides adaptive playback using DASH or SmoothStreaming, and
implements networking, buffering and media extraction within the ExoPlayer library.
Providing media using MediaExtractor
In order to render media formats supported by the Android framework, the FrameworkSampleSource
class uses MediaExtractor
for networking,
buffering and sample extraction functionality. By doing so, it supports any media container format
supported by the version of Android where it is running. For more information about media formats
supported by Android, see Supported
Media Formats.
The diagram in Figure 2 shows the object model for an ExoPlayer implementation using
FrameworkSampleSource
.
![](../../../images/exoplayer/frameworksamplesource.png)
Figure 2. Object model for an implementation of ExoPlayer that renders
media formats supported by Android using FrameworkSampleSource
The following code example outlines how the video and audio renderers are constructed to load the video from a specified URI.
FrameworkSampleSource sampleSource = new FrameworkSampleSource( activity, uri, null, 2); MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer( sampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 0, mainHandler, playerActivity, 50); MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer( sampleSource, null, true);
The ExoPlayer demo app provides a complete implementation of this code in
DefaultRendererBuilder
. The SimplePlaybackActivity
class uses it to play one
of the videos available in the demo app. Note that in the example, video and audio
are muxed, meaning they are streamed together from a single URI. The FrameworkSampleSource
instance provides video samples to the videoRenderer
object and audio samples to the
audioRenderer
object as they are extracted from the media container format. It is also
possible to play demuxed media, where video and audio are streamed separately from different URIs.
This functionality can be achieved by having two FrameworkSampleSource
instances instead
of one.
Providing media for adaptive playback
ExoPlayer supports adaptive streaming, which allows the quality of the media data to be adjusted during playback based on the network conditions. DASH and SmoothStreaming are examples of adaptive streaming technologies. Both these approaches load media in small chunks (typically 2 to 10 seconds in duration). Whenever a chunk of media is requested, the client selects from a number of possible formats. For example, a client may select a high quality format if network conditions are good, or a low quality format if network conditions are bad. In both techniques, video and audio are streamed separately.
ExoPlayer supports adaptive playback through use of the ChunkSampleSource
class,
which loads chunks of media data from which individual samples can be extracted. Each ChunkSampleSource
requires a ChunkSource
object to be injected through its constructor,
which is responsible for providing media chunks from which to load and read samples. The DashMp4ChunkSource
and SmoothStreamingChunkSource
classes provide DASH and SmoothStreaming
playback using the FMP4 container format. The DashWebMChunkSource
class uses the WebM
container format to provide DASH playback.
All of the standard ChunkSource
implementations require a FormatEvaluator
and
a DataSource
to be injected through their constructors. The FormatEvaluator
objects select from the available formats before each chunk is loaded. The DataSource
objects are responsible for actually loading the data. Finally, the ChunkSampleSources
require a LoadControl
object that controls the chunk buffering policy.
The object model of an ExoPlayer configured for a DASH adaptive playback is shown in the
diagram below. This example uses an HttpDataSource
object to stream the media over the
network. The video quality is varied at runtime using the adaptive implementation of FormatEvaluator
, while audio is played at a fixed quality level.
![](../../../images/exoplayer/adaptive-streaming.png)
Figure 3. Object model for a DASH adaptive playback using ExoPlayer
The following code example outlines how the video and audio renderers are constructed.
Handler mainHandler = playerActivity.getMainHandler(); LoadControl loadControl = new DefaultLoadControl( new BufferPool(BUFFER_SEGMENT_SIZE)); BandwidthMeter bandwidthMeter = new BandwidthMeter(); // Build the video renderer. DataSource videoDataSource = new HttpDataSource(userAgent, HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter); ChunkSource videoChunkSource = new DashMp4ChunkSource(videoDataSource, new AdaptiveEvaluator(bandwidthMeter), videoRepresentations); ChunkSampleSource videoSampleSource = new ChunkSampleSource(videoChunkSource, loadControl, VIDEO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE, true); MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer( videoSampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 0, mainHandler, playerActivity, 50); // Build the audio renderer. DataSource audioDataSource = new HttpDataSource(userAgent, HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter); ChunkSource audioChunkSource = new DashMp4ChunkSource(audioDataSource, new FormatEvaluator.FixedEvaluator(), audioRepresentation); SampleSource audioSampleSource = new ChunkSampleSource(audioChunkSource, loadControl, AUDIO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE, true); MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer( audioSampleSource, null, true);
In this code, videoRepresentations
and audioRepresentation
are Representation
objects, each of which describes one of the available media streams. In the DASH
model, these streams are parsed from a media presentation description (MPD) file. The ExoPlayer
library provides a MediaPresentationDescriptionParser
class to obtain Representation
objects from MPD files.
Note: Building Representation objects from MPD files is not required. You can build Representation objects from other data sources if necessary.
The ExoPlayer demo app provides complete implementation of this code in
DashVodRendererBuilder
. The SimplePlaybackActivity
class uses this builder to
construct renderers for playing DASH sample videos in the demo app. It asynchronously fetches a
specified MPD file in order to construct the required Representation
objects. For an
equivalent SmoothStreaming example, see the SmoothStreamingRendererBuilder
class in the
demo app.
Format selection for adaptive playback
For DASH and SmoothStreaming playback, consider both static format selection at the start of playback and dynamic format selection during playback. Static format selection should be used to filter out formats that should not be used throughout the playback, for example formats with resolutions higher than the maximum supported by the playback device. Dynamic selection varies the selected format during playback, typically to adapt video quality in response to changes in network conditions.
Static format selection
When preparing a player, you should consider filtering out some of the available formats if they are not useable for playback. Static format selection allows you to filter out formats that cannot be used on a particular device or are not compatible with your player. For audio playback, this often means picking a single format to play and discarding the others.
For video playback, filtering formats can be more complicated. Apps should first
eliminate any streams that whose resolution is too high to be played by the device. For H.264,
which is normally used for DASH and SmoothStreaming playback, ExoPlayer’s MediaCodecUtil
class provides a maxH264DecodableFrameSize()
method that can be used to determine what
resolution streams the device is able to handle, as shown in the following code example:
int maxDecodableFrameSize = MediaCodecUtil.maxH264DecodableFrameSize(); Format format = representation.format; if (format.width * format.height <= maxDecodableFrameSize) { // The device can play this stream. videoRepresentations.add(representation); } else { // The device isn't capable of playing this stream. }
This approach is used to filter Representations
in the DashVodRendererBuilder
class of the ExoPlayer demo app, and similarly to filter track indices in SmoothStreamingRendererBuilder
.
In addition to eliminating unsupported formats, it should be noted that the ability to
seamlessly switch between H.264 streams of different resolution is an optional decoder feature
available in Android 4.3 (API level 16) and higher, and so is not supported by all devices. The
availability of an adaptive H.264 decoder can be queried using MediaCodecUtil
, as shown in
the following code example:
boolean isAdaptive = MediaCodecUtil.getDecoderInfo(MimeTypes.VIDEO_H264).adaptive;
The MediaCodecVideoTrackRenderer
class is still able to handle resolution changes on
devices that do not have adaptive decoders, however the switch is not seamless. Typically, the
switch creates a small discontinuity in visual output lasting around 50-100ms. For devices that
do not provide an adaptive decoder, app developers may choose to adapt between formats at
a single fixed resolution so as to avoid discontinuities. The ExoPlayer demo app
implementation does not pick a fixed resolution.
Dynamic format selection
During playback, you can use a FormatEvaluator
to dynamically select from the
available video formats. The ExoPlayer library provides a FormatEvaluator.Adaptive
implementation for dynamically selecting between video formats based on the current network
conditions.
This class provides a simple, general purpose reference implementation, however you are
encouraged to write your own FormatEvaluator
implementation to best suit your particular
needs.
Player Events
During playback, your app can listen for events generated by the ExoPlayer that indicate the overall state of the player. These events are useful as triggers for updating the app user interface such as playback controls. Many ExoPlayer components also report their own component specific low level events, which can be useful for performance monitoring.
High level events
ExoPlayer allows instances of ExoPlayer.Listener
to be added and removed using its
addListener()
and removeListener()
methods. Registered listeners are notified of
changes in playback state, as well as when errors occur that cause playback to fail. For more
information about the valid playback states and the possible transitions between them, see the
ExoPlayer source code.
Developers who implement custom playback controls should register a listener and use it to update their controls as the player’s state changes. An app should also show an appropriate error to the user if playback fails.
Low level events
In addition to high level listeners, many of the individual components provided by the
ExoPlayer library allow their own event listeners. For example, MediaCodecVideoTrackRenderer
has constructors that take a MediaCodecVideoTrackRenderer.EventListener
. In the ExoPlayer demo app, SimplePlayerActivity
acts as a listener so that it can adjust the dimensions of the target surface to have the correct
height and width ratio for the video being played:
@Override public void onVideoSizeChanged(int width, int height) { surfaceView.setVideoWidthHeightRatio(height == 0 ? 1 : (float) width / height); }
The RendererBuilder
classes in the ExoPlayer demo app inject the activity as the
listener, for example in the DashVodRendererBuilder
class:
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer( videoSampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 0, mainHandler, playerActivity, 50);
Note that you must pass a Handler
object to the renderer, which determines
the thread on which the listener’s methods are invoked. In most cases, you should use a
Handler
associated with the app’s main thread, as is the case in this example.
Listening to individual components can be useful for adjusting UI based on player events, as
in the example above. Listening to component events can also be helpful for logging performance
metrics. For example, MediaCodecVideoTrackRenderer
notifies its listener of dropped video
frames. A developer may wish to log such metrics to track playback performance in their
app.
Many components also notify their listeners when errors occur. Such errors may or may not cause playback to fail. If an error does not cause playback to fail, it may still result in degraded performance, and so you may wish to log all errors in order to track playback performance. Note that an ExoPlayer instance always notifies its high level listeners of errors that cause playback to fail, in addition to the listener of the individual component from which the error originated. Hence, you should display error messages to users only from high level listeners. Within individual component listeners, you should use error notifications only for informational purposes.
Sending messages to components
Some ExoPlayer components allow changes in configuration during playback. By convention, you make these changes by passing asynchronous messages through the ExoPlayer to the component. This approach ensures both thread safety and that the configuration change is executed in order with any other operations being performed on the player.
The most common use of messaging is passing a target surface to
MediaCodecVideoTrackRenderer
:
player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, surface);
Note that if the surface needs to be cleared because
SurfaceHolder.Callback.surfaceDestroyed()
has been invoked, then you must send this
message using the blocking variant of sendMessage()
:
player.blockingSendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, null);
You must use a blocking message because the contract of surfaceDestroyed()
requires that the
app does not attempt to access the surface after the method returns. The SimplePlayerActivity
class in the demo app demonstrates how the surface should be set and
cleared.
Customizing ExoPlayer
One of the main benefits of ExoPlayer over MediaPlayer
is the ability to
customize and extend the player to better suit the developer’s use case. The ExoPlayer library
is designed specifically with this in mind, defining a number of abstract base classes and
interfaces that make it possible for app developers to easily replace the default implementations
provided by the library. Here are some use cases for building custom components:
TrackRenderer
- You may want to implement a customTrackRenderer
to handle media types other than audio and video. TheTextTrackRenderer
class within the ExoPlayer library is an example of how to implement a custom renderer. You could use the approach it demonstrates to render custom overlays or annotations. Implementing this kind of functionality as aTrackRenderer
makes it easy to keep the overlays or annotations in sync with the other media being played.SampleSource
- If you need to support a container format not already handled byMediaExtractor
or ExoPlayer, consider implementing a customSampleSource
class.FormatEvaluator
- The ExoPlayer library providesFormatEvaluator.Adaptive
as a simple reference implementation that switches between different quality video formats based on the available bandwidth. App developers are encouraged to develop their own adaptiveFormatEvaluator
implementations, which can be designed to suit their use specific needs.DataSource
- ExoPlayer’s upstream package already contains a number ofDataSource
implementations for different use cases, such as writing and reading to and from a persistent media cache. You may want to implement you ownDataSource
class to load data in another way, such as a custom protocol or HTTP stack for data input.
Custom component guidelines
If a custom component needs to report events back to the app, we recommend that you
do so using the same model as existing ExoPlayer components, where an event listener is passed
together with a Handler
to the constructor of the component.
We recommended that custom components use the same model as existing ExoPlayer components to
allow reconfiguration by the app during playback, as described in
Sending messages to components.
To do this, you should implement a ExoPlayerComponent
and receive
configuration changes in its handleMessage()
method. Your app should pass
configuration changes by calling ExoPlayer’s sendMessage()
and blockingSendMessage()
methods.
Digital Rights Management
On Android 4.3 (API level 18) and higher, ExoPlayer supports Digital Rights Managment (DRM)
protected playback. In order to play DRM protected content with ExoPlayer, your app must
inject a DrmSessionManager
into the MediaCodecVideoTrackRenderer
and MediaCodecAudioTrackRenderer
constructors. A DrmSessionManager
object is responsible for
providing the MediaCrypto
object required for decryption, as well as ensuring that the
required decryption keys are available to the underlying DRM module being used.
The ExoPlayer library provides a default implementation of DrmSessionManager
, called
StreamingDrmSessionManager
, which uses MediaDrm
. The session
manager supports any DRM scheme for which a modular DRM component exists on the device. All
Android devices are required to support Widevine modular DRM (with L3 security, although many
devices also support L1). Some devices may support additional schemes such as PlayReady.
The StreamingDrmSessionManager
class requires a MediaDrmCallback
to be
injected into its constructor, which is responsible for actually making provisioning and key
requests. You should implement this interface to make network requests to your license
server and obtain the required keys. The WidevineTestMediaDrmCallback
class in the
ExoPlayer demo app sends requests to a Widevine test server.