Akka HTTP

Version 10.0.7

Marshalling

TODO overhaul for Java

“Marshalling” is the process of converting a higher-level (object) structure into some kind of lower-level representation, often a “wire format”. Other popular names for it are “Serialization” or “Pickling”.

In Akka HTTP “Marshalling” means the conversion of an object of type T into a lower-level target type, e.g. a MessageEntity (which forms the “entity body” of an HTTP request or response) or a full HttpRequest or HttpResponse.

Basic Design

Marshalling of instances of type A into instances of type B is performed by a Marshaller[A, B]. Akka HTTP also predefines a number of helpful aliases for the types of marshallers that you’ll likely work with most:

type ToEntityMarshaller[T] = Marshaller[T, MessageEntity]
type ToByteStringMarshaller[T] = Marshaller[T, ByteString]
type ToHeadersAndEntityMarshaller[T] = Marshaller[T, (immutable.Seq[HttpHeader], MessageEntity)]
type ToResponseMarshaller[T] = Marshaller[T, HttpResponse]
type ToRequestMarshaller[T] = Marshaller[T, HttpRequest]

Contrary to what you might initially expect Marshaller[A, B] is not a plain function A => B but rather essentially a function A => Future[List[Marshalling[B]]]. Let’s dissect this rather complicated looking signature piece by piece to understand what marshallers are designed this way. Given an instance of type A a Marshaller[A, B] produces:

  1. A Future: This is probably quite clear. Marshallers are not required to synchronously produce a result, so instead they return a future, which allows for asynchronicity in the marshalling process.

  2. of List: Rather than only a single target representation for A marshallers can offer several ones. Which one will be rendered onto the wire in the end is decided by content negotiation. For example, the ToEntityMarshaller[OrderConfirmation] might offer a JSON as well as an XML representation. The client can decide through the addition of an Accept request header which one is preferred. If the client doesn’t express a preference the first representation is picked.

  3. of Marshalling[B]: Rather than returning an instance of B directly marshallers first produce a Marshalling[B]. This allows for querying the MediaType and potentially the HttpCharset that the marshaller will produce before the actual marshalling is triggered. Apart from enabling content negotiation this design allows for delaying the actual construction of the marshalling target instance to the very last moment when it is really needed.

This is how Marshalling is defined:

/**
 * Describes one possible option for marshalling a given value.
 */
sealed trait Marshalling[+A] {
  def map[B](f: A ⇒ B): Marshalling[B]

  /**
   * Converts this marshalling to an opaque marshalling, i.e. a marshalling result that
   * does not take part in content type negotiation. The given charset is used if this
   * instance is a `WithOpenCharset` marshalling.
   */
  def toOpaque(charset: HttpCharset): Marshalling[A]
}

object Marshalling {

  /**
   * A Marshalling to a specific [[akka.http.scaladsl.model.ContentType]].
   */
  final case class WithFixedContentType[A](
    contentType: ContentType,
    marshal:     () ⇒ A) extends Marshalling[A] {
    def map[B](f: A ⇒ B): WithFixedContentType[B] = copy(marshal = () ⇒ f(marshal()))
    def toOpaque(charset: HttpCharset): Marshalling[A] = Opaque(marshal)
  }

  /**
   * A Marshalling to a specific [[akka.http.scaladsl.model.MediaType]] with a flexible charset.
   */
  final case class WithOpenCharset[A](
    mediaType: MediaType.WithOpenCharset,
    marshal:   HttpCharset ⇒ A) extends Marshalling[A] {
    def map[B](f: A ⇒ B): WithOpenCharset[B] = copy(marshal = cs ⇒ f(marshal(cs)))
    def toOpaque(charset: HttpCharset): Marshalling[A] = Opaque(() ⇒ marshal(charset))
  }

  /**
   * A Marshalling to an unknown MediaType and charset.
   * Circumvents content negotiation.
   */
  final case class Opaque[A](marshal: () ⇒ A) extends Marshalling[A] {
    def map[B](f: A ⇒ B): Opaque[B] = copy(marshal = () ⇒ f(marshal()))
    def toOpaque(charset: HttpCharset): Marshalling[A] = this
  }
}

Predefined Marshallers

Akka HTTP already predefines a number of marshallers for the most common types. Specifically these are:

  • PredefinedToEntityMarshallers
    • Array[Byte]
    • ByteString
    • Array[Char]
    • String
    • akka.http.scaladsl.model.FormData
    • akka.http.scaladsl.model.MessageEntity
    • T <: akka.http.scaladsl.model.Multipart
  • PredefinedToResponseMarshallers
    • T, if a ToEntityMarshaller[T] is available
    • HttpResponse
    • StatusCode
    • (StatusCode, T), if a ToEntityMarshaller[T] is available
    • (Int, T), if a ToEntityMarshaller[T] is available
    • (StatusCode, immutable.Seq[HttpHeader], T), if a ToEntityMarshaller[T] is available
    • (Int, immutable.Seq[HttpHeader], T), if a ToEntityMarshaller[T] is available
  • PredefinedToRequestMarshallers
    • HttpRequest
    • Uri
    • (HttpMethod, Uri, T), if a ToEntityMarshaller[T] is available
    • (HttpMethod, Uri, immutable.Seq[HttpHeader], T), if a ToEntityMarshaller[T] is available
  • GenericMarshallers
    • Marshaller[Throwable, T]
    • Marshaller[Option[A], B], if a Marshaller[A, B] and an EmptyValue[B] is available
    • Marshaller[Either[A1, A2], B], if a Marshaller[A1, B] and a Marshaller[A2, B] is available
    • Marshaller[Future[A], B], if a Marshaller[A, B] is available
    • Marshaller[Try[A], B], if a Marshaller[A, B] is available

Implicit Resolution

The marshalling infrastructure of Akka HTTP relies on a type-class based approach, which means that Marshaller instances from a certain type A to a certain type B have to be available implicitly.

The implicits for most of the predefined marshallers in Akka HTTP are provided through the companion object of the Marshaller trait. This means that they are always available and never need to be explicitly imported. Additionally, you can simply “override” them by bringing your own custom version into local scope.

Custom Marshallers

Akka HTTP gives you a few convenience tools for constructing marshallers for your own types. Before you do that you need to think about what kind of marshaller you want to create. If all your marshaller needs to produce is a MessageEntity then you should probably provide a ToEntityMarshaller[T]. The advantage here is that it will work on both the client- as well as the server-side since a ToResponseMarshaller[T] as well as a ToRequestMarshaller[T] can automatically be created if a ToEntityMarshaller[T] is available.

If, however, your marshaller also needs to set things like the response status code, the request method, the request URI or any headers then a ToEntityMarshaller[T] won’t work. You’ll need to fall down to providing a ToResponseMarshaller[T] or a ToRequestMarshaller[T] directly.

For writing your own marshallers you won’t have to “manually” implement the Marshaller trait directly. Rather, it should be possible to use one of the convenience construction helpers defined on the Marshaller companion:

object Marshaller
  extends GenericMarshallers
  with PredefinedToEntityMarshallers
  with PredefinedToResponseMarshallers
  with PredefinedToRequestMarshallers {

  /**
   * Creates a [[Marshaller]] from the given function.
   */
  def apply[A, B](f: ExecutionContext ⇒ A ⇒ Future[List[Marshalling[B]]]): Marshaller[A, B] =
    new Marshaller[A, B] {
      def apply(value: A)(implicit ec: ExecutionContext) =
        try f(ec)(value)
        catch { case NonFatal(e) ⇒ FastFuture.failed(e) }
    }

  /**
   * Helper for creating a [[Marshaller]] using the given function.
   */
  def strict[A, B](f: A ⇒ Marshalling[B]): Marshaller[A, B] =
    Marshaller { _ ⇒ a ⇒ FastFuture.successful(f(a) :: Nil) }

  /**
   * Helper for creating a "super-marshaller" from a number of "sub-marshallers".
   * Content-negotiation determines, which "sub-marshaller" eventually gets to do the job.
   *
   * Please note that all marshallers will actualy be invoked in order to get the Marshalling object
   * out of them, and later decide which of the marshallings should be returned. This is by-design,
   * however in ticket as discussed in ticket https://github.com/akka/akka-http/issues/243 it MAY be
   * changed in later versions of Akka HTTP.
   */
  def oneOf[A, B](marshallers: Marshaller[A, B]*): Marshaller[A, B] =
    Marshaller { implicit ec ⇒ a ⇒ FastFuture.sequence(marshallers.map(_(a))).fast.map(_.flatten.toList) }

  /**
   * Helper for creating a "super-marshaller" from a number of values and a function producing "sub-marshallers"
   * from these values. Content-negotiation determines, which "sub-marshaller" eventually gets to do the job.
   *
   * Please note that all marshallers will actualy be invoked in order to get the Marshalling object
   * out of them, and later decide which of the marshallings should be returned. This is by-design,
   * however in ticket as discussed in ticket https://github.com/akka/akka-http/issues/243 it MAY be
   * changed in later versions of Akka HTTP.
   */
  def oneOf[T, A, B](values: T*)(f: T ⇒ Marshaller[A, B]): Marshaller[A, B] =
    oneOf(values map f: _*)

  /**
   * Helper for creating a synchronous [[Marshaller]] to content with a fixed charset from the given function.
   */
  def withFixedContentType[A, B](contentType: ContentType)(marshal: A ⇒ B): Marshaller[A, B] =
    new Marshaller[A, B] {
      def apply(value: A)(implicit ec: ExecutionContext) =
        try FastFuture.successful {
          Marshalling.WithFixedContentType(contentType, () ⇒ marshal(value)) :: Nil
        } catch {
          case NonFatal(e) ⇒ FastFuture.failed(e)
        }

      override def compose[C](f: C ⇒ A): Marshaller[C, B] =
        Marshaller.withFixedContentType(contentType)(marshal compose f)
    }

  /**
   * Helper for creating a synchronous [[Marshaller]] to content with a negotiable charset from the given function.
   */
  def withOpenCharset[A, B](mediaType: MediaType.WithOpenCharset)(marshal: (A, HttpCharset) ⇒ B): Marshaller[A, B] =
    new Marshaller[A, B] {
      def apply(value: A)(implicit ec: ExecutionContext) =
        try FastFuture.successful {
          Marshalling.WithOpenCharset(mediaType, charset ⇒ marshal(value, charset)) :: Nil
        } catch {
          case NonFatal(e) ⇒ FastFuture.failed(e)
        }

      override def compose[C](f: C ⇒ A): Marshaller[C, B] =
        Marshaller.withOpenCharset(mediaType)((c: C, hc: HttpCharset) ⇒ marshal(f(c), hc))
    }

  /**
   * Helper for creating a synchronous [[Marshaller]] to non-negotiable content from the given function.
   */
  def opaque[A, B](marshal: A ⇒ B): Marshaller[A, B] =
    strict { value ⇒ Marshalling.Opaque(() ⇒ marshal(value)) }

  /**
   * Helper for creating a [[Marshaller]] combined of the provided `marshal` function
   * and an implicit Marshaller which is able to produce the required final type.
   */
  def combined[A, B, C](marshal: A ⇒ B)(implicit m2: Marshaller[B, C]): Marshaller[A, C] =
    Marshaller[A, C] { ec ⇒ a ⇒ m2.compose(marshal).apply(a)(ec) }
}

Deriving Marshallers

Sometimes you can save yourself some work by reusing existing marshallers for your custom ones. The idea is to “wrap” an existing marshaller with some logic to “re-target” it to your type.

In this regard wrapping a marshaller can mean one or both of the following two things:

  • Transform the input before it reaches the wrapped marshaller
  • Transform the output of the wrapped marshaller

For the latter (transforming the output) you can use baseMarshaller.map, which works exactly as it does for functions. For the former (transforming the input) you have four alternatives:

  • baseMarshaller.compose
  • baseMarshaller.composeWithEC
  • baseMarshaller.wrap
  • baseMarshaller.wrapWithEC

compose works just like it does for functions. wrap is a compose that allows you to also change the ContentType that the marshaller marshals to. The ...WithEC variants allow you to receive an ExecutionContext internally if you need one, without having to depend on one being available implicitly at the usage site.

Using Marshallers

In many places throughput Akka HTTP marshallers are used implicitly, e.g. when you define how to complete a request using the Routing DSL.

However, you can also use the marshalling infrastructure directly if you wish, which can be useful for example in tests. The best entry point for this is the akka.http.scaladsl.marshalling.Marshal object, which you can use like this:

The URI model
Unmarshalling
The source code for this page can be found here.