HTTP Interface for Batch Requests

Clients normally send individual operations to ArangoDB in individual HTTP requests. This is straightforward and simple, but has the disadvantage that the network overhead can be significant if many small requests are issued in a row.

To mitigate this problem, ArangoDB offers a batch request API that clients can use to send multiple operations in one batch to ArangoDB. This method is especially useful when the client has to send many HTTP requests with a small body/payload and the individual request results do not depend on each other.

Clients can use ArangoDB's batch API by issuing a multipart HTTP POST request to the URL /_api/batch handler. The handler will accept the request if the Content-type is multipart/form-data and a boundary string is specified. ArangoDB will then decompose the batch request into its individual parts using this boundary. This also means that the boundary string itself must not be contained in any of the parts. When ArangoDB has split the multipart request into its individual parts, it will process all parts sequentially as if it were a standalone request. When all parts are processed, ArangoDB will generate a multipart HTTP response that contains one part for each part operation result. For example, if you send a multipart request with 5 parts, ArangoDB will send back a multipart response with 5 parts as well.

The server expects each part message to start with exactly the following "header": 

Content-type: application/x-arango-batchpart

You can optionally specify a Content-Id "header" to uniquely identify each part message. The server will return the Content-Id in its response if it is specified. Otherwise, the server will not send a Content-Id "header" back. The server will not validate the uniqueness of the Content-Id. After the mandatory Content-type and the optional Content-Id header, two Windows line breaks (i.e. \r\n\r\n) must follow. Any deviation of this structure might lead to the part being rejected or incorrectly interpreted. The part request payload, formatted as a regular HTTP request, must follow the two Windows line breaks literal directly.

Note that the literal Content-type: application/x-arango-batchpart technically is the header of the MIME part, and the HTTP request (including its headers) is the body part of the MIME part.

An actual part request should start with the HTTP method, the called URL, and the HTTP protocol version as usual, followed by arbitrary HTTP headers. Its body should follow after the usual \r\n\r\n literal. Part requests are therefore regular HTTP requests, only embedded inside a multipart message.

The following example will send a batch with 3 individual document creation operations. The boundary used in this example is XXXsubpartXXX.

Examples

> curl -X POST --data-binary @- --header "Content-type: multipart/form-data; boundary=XXXsubpartXXX" http://localhost:8529/_api/batch
--XXXsubpartXXX
Content-type: application/x-arango-batchpart
Content-Id: 1

POST /_api/document?collection=xyz HTTP/1.1

{"a":1,"b":2,"c":3}
--XXXsubpartXXX
Content-type: application/x-arango-batchpart
Content-Id: 2

POST /_api/document?collection=xyz HTTP/1.1

{"a":1,"b":2,"c":3,"d":4}
--XXXsubpartXXX
Content-type: application/x-arango-batchpart
Content-Id: 3

POST /_api/document?collection=xyz HTTP/1.1

{"a":1,"b":2,"c":3,"d":4,"e":5}
--XXXsubpartXXX--

The server will then respond with one multipart message, containing the overall status and the individual results for the part operations. The overall status should be 200 except there was an error while inspecting and processing the multipart message. The overall status therefore does not indicate the success of each part operation, but only indicates whether the multipart message could be handled successfully.

Each part operation will return its own status value. As the part operation results are regular HTTP responses (just included in one multipart response), the part operation status is returned as a HTTP status code. The status codes of the part operations are exactly the same as if you called the individual operations standalone. Each part operation might also return arbitrary HTTP headers and a body/payload:

Examples

HTTP/1.1 200 OK
Connection: Keep-Alive
Content-type: multipart/form-data; boundary=XXXsubpartXXX
Content-length: 1055

--XXXsubpartXXX
Content-type: application/x-arango-batchpart
Content-Id: 1

HTTP/1.1 202 Accepted
Content-type: application/json; charset=utf-8
Etag: "9514299"
Content-length: 53

{"error":false,"_id":"xyz/9514299","_key":"9514299","_rev":"9514299"}
--XXXsubpartXXX
Content-type: application/x-arango-batchpart
Content-Id: 2

HTTP/1.1 202 Accepted
Content-type: application/json; charset=utf-8
Etag: "9579835"
Content-length: 53

{"error":false,"_id":"xyz/9579835","_key":"9579835","_rev":"9579835"}
--XXXsubpartXXX
Content-type: application/x-arango-batchpart
Content-Id: 3

HTTP/1.1 202 Accepted
Content-type: application/json; charset=utf-8
Etag: "9645371"
Content-length: 53

{"error":false,"_id":"xyz/9645371","_key":"9645371","_rev":"9645371"}
--XXXsubpartXXX--

In the above example, the server returned an overall status code of 200, and each part response contains its own status value (202 in the example):

When constructing the multipart HTTP response, the server will use the same boundary that the client supplied. If any of the part responses has a status code of 400 or greater, the server will also return an HTTP header x-arango-errors containing the overall number of part requests that produced errors:

Examples

> curl -X POST --data-binary @- --header "Content-type: multipart/form-data; boundary=XXXsubpartXXX" http://localhost:8529/_api/batch
--XXXsubpartXXX
Content-type: application/x-arango-batchpart

POST /_api/document?collection=nonexisting

{"a":1,"b":2,"c":3}
--XXXsubpartXXX
Content-type: application/x-arango-batchpart

POST /_api/document?collection=xyz

{"a":1,"b":2,"c":3,"d":4}
--XXXsubpartXXX--

In this example, the overall response code is 200, but as some of the part request failed (with status code 404), the x-arango-errors header of the overall response is 1:

Examples

HTTP/1.1 200 OK
x-arango-errors: 1
Content-type: multipart/form-data; boundary=XXXsubpartXXX
Content-length: 711

--XXXsubpartXXX
Content-type: application/x-arango-batchpart

HTTP/1.1 404 Not Found
Content-type: application/json; charset=utf-8
Content-length: 111

{"error":true,"code":404,"errorNum":1203,"errorMessage":"collection \/_api\/collection\/nonexisting not found"}
--XXXsubpartXXX
Content-type: application/x-arango-batchpart

HTTP/1.1 202 Accepted
Content-type: application/json; charset=utf-8
Etag: "9841979"
Content-length: 53

{"error":false,"_id":"xyz/9841979","_key":"9841979","_rev":"9841979"}
--XXXsubpartXXX--

Please note that the database used for all part operations of a batch request is determined by scanning the original URL (the URL that contains /_api/batch). It is not possible to override the database name in part operations of a batch. When doing so, any other database name used in a batch part will be ignored.

executes a batch request

executes a batch request

POST /_api/batch

Request Body (required)

The multipart batch request, consisting of the envelope and the individual batch parts.

Executes a batch request. A batch request can contain any number of other requests that can be sent to ArangoDB in isolation. The benefit of using batch requests is that batching requests requires less client/server roundtrips than when sending isolated requests.

All parts of a batch request are executed serially on the server. The server will return the results of all parts in a single response when all parts are finished.

Technically, a batch request is a multipart HTTP request, with content-type multipart/form-data. A batch request consists of an envelope and the individual batch part actions. Batch part actions are "regular" HTTP requests, including full header and an optional body. Multiple batch parts are separated by a boundary identifier. The boundary identifier is declared in the batch envelope. The MIME content-type for each individual batch part must be application/x-arango-batchpart.

Please note that when constructing the individual batch parts, you must use CRLF (\ \) as the line terminator as in regular HTTP messages.

The response sent by the server will be an HTTP 200 response, with an optional error summary header x-arango-errors. This header contains the number of batch part operations that failed with an HTTP error code of at least 400. This header is only present in the response if the number of errors is greater than zero.

The response sent by the server is a multipart response, too. It contains the individual HTTP responses for all batch parts, including the full HTTP result header (with status code and other potential headers) and an optional result body. The individual batch parts in the result are seperated using the same boundary value as specified in the request.

The order of batch parts in the response will be the same as in the original client request. Client can additionally use the Content-Id MIME header in a batch part to define an individual id for each batch part. The server will return this id is the batch part responses, too.

Example:

Sending a batch request with five batch parts:

  • GET /_api/version

  • DELETE /_api/collection/products

  • POST /_api/collection/products

  • GET /_api/collection/products/figures

  • DELETE /_api/collection/products

The boundary (SomeBoundaryValue) is passed to the server in the HTTP Content-Type HTTP header. Please note the reply is not displayed all accurate.

shell> curl -X POST --header 'Content-Type: multipart/form-data; boundary=SomeBoundaryValue' --data-binary @- --dump - http://localhost:8529/_api/batch <<EOF
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: myId1

GET /_api/version HTTP/1.1

--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: myId2

DELETE /_api/collection/products HTTP/1.1

--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: someId

POST /_api/collection/products HTTP/1.1

{"name": "products" }

--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: nextId

GET /_api/collection/products/figures HTTP/1.1

--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: otherId

DELETE /_api/collection/products HTTP/1.1
--SomeBoundaryValue--

EOF

HTTP/1.1 200 OK
content-type: multipart/form-data; boundary=SomeBoundaryValue
x-arango-errors: 1
x-content-type-options: nosniff

"--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: myId1

HTTP/1.1 200 OK
Server: 
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 59

{\"server\":\"arango\",\"version\":\"3.2.0\",\"license\":\"community\"}
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: myId2

HTTP/1.1 404 Not Found
Server: 
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 88

{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'products'\"}
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: someId

HTTP/1.1 200 OK
Server: 
Location: /_db/_system/_api/collection/products
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 132

{\"id\":\"9152\",\"name\":\"products\",\"waitForSync\":false,\"isVolatile\":false,\"isSystem\":false,\"status\":3,\"type\":2,\"error\":false,\"code\":200}
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: nextId

HTTP/1.1 200 OK
Server: 
Location: /_db/_system/_api/collection/products/figures
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 708

{\"id\":\"9152\",\"name\":\"products\",\"isSystem\":false,\"doCompact\":true,\"isVolatile\":false,\"journalSize\":33554432,\"keyOptions\":{\"type\":\"traditional\",\"allowUserKeys\":true,\"lastValue\":0},\"waitForSync\":false,\"indexBuckets\":8,\"count\":0,\"figures\":{\"indexes\":{\"count\":1,\"size\":32128},\"compactionStatus\":{\"message\":\"compaction not yet started\",\"time\":\"2017-07-19T21:50:48Z\"},\"documentReferences\":0,\"waitingFor\":\"-\",\"alive\":{\"count\":0,\"size\":0},\"dead\":{\"count\":0,\"size\":0,\"deletion\":0},\"datafiles\":{\"count\":0,\"fileSize\":0},\"journals\":{\"count\":0,\"fileSize\":0},\"compactors\":{\"count\":0,\"fileSize\":0},\"revisions\":{\"count\":0,\"size\":48192},\"lastTick\":0,\"uncollectedLogfileEntries\":0},\"status\":3,\"type\":2,\"error\":false,\"code\":200}
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: otherId

HTTP/1.1 200 OK
Server: 
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 38

{\"id\":\"9152\",\"error\":false,\"code\":200}
--SomeBoundaryValue--"

Example:

Sending a batch request, setting the boundary implicitly (the server will in this case try to find the boundary at the beginning of the request body).

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/batch <<EOF
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart

DELETE /_api/collection/notexisting1 HTTP/1.1

--SomeBoundaryValue
Content-Type: application/x-arango-batchpart

DELETE _api/collection/notexisting2 HTTP/1.1
--SomeBoundaryValue--

EOF

HTTP/1.1 200 OK
content-type: 
x-arango-errors: 2
x-content-type-options: nosniff

"--SomeBoundaryValue
Content-Type: application/x-arango-batchpart

HTTP/1.1 404 Not Found
Server: 
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 92

{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'notexisting1'\"}
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart

HTTP/1.1 404 Not Found
Server: 
Connection: 
Content-Type: application/json; charset=utf-8
Content-Length: 101

{\"error\":true,\"code\":404,\"errorNum\":404,\"errorMessage\":\"unknown path '_api/collection/notexisting2'\"}
--SomeBoundaryValue--"

Return Codes

  • 200: is returned if the batch was received successfully. HTTP 200 is returned even if one or multiple batch part actions failed.
  • 400: is returned if the batch envelope is malformed or incorrectly formatted. This code will also be returned if the content-type of the overall batch request or the individual MIME parts is not as expected.
  • 405: is returned when an invalid HTTP method is used.

Examples

Sending a batch request with five batch parts:

  • GET /_api/version
  • DELETE /_api/collection/products
  • POST /_api/collection/products
  • GET /_api/collection/products/figures
  • DELETE /_api/collection/products The boundary (SomeBoundaryValue) is passed to the server in the HTTP Content-Type HTTP header. Please note the reply is not displayed all accurate.
shell> curl -X POST --header 'Content-Type: multipart/form-data; boundary=SomeBoundaryValue' --data-binary @- --dump - http://localhost:8529/_api/batch <<EOF
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: myId1

GET /_api/version HTTP/1.1
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: myId2

DELETE /_api/collection/products HTTP/1.1
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: someId

POST /_api/collection/products HTTP/1.1

{"name": "products" }
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: nextId

GET /_api/collection/products/figures HTTP/1.1
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart
Content-Id: otherId

DELETE /_api/collection/products HTTP/1.1
--SomeBoundaryValue--

EOF

HTTP/1.1 200 OK
content-type: multipart/form-data; boundary=SomeBoundaryValue
x-arango-errors: 1
x-content-type-options: nosniff

"--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\nContent-Id: myId1\r\n\r\nHTTP/1.1 200 OK\r\nServer: \r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 59\r\n\r\n{\"server\":\"arango\",\"version\":\"3.2.0\",\"license\":\"community\"}\r\n--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\nContent-Id: myId2\r\n\r\nHTTP/1.1 404 Not Found\r\nServer: \r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 88\r\n\r\n{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'products'\"}\r\n--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\nContent-Id: someId\r\n\r\nHTTP/1.1 200 OK\r\nServer: \r\nLocation: /_db/_system/_api/collection/products\r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 132\r\n\r\n{\"id\":\"9152\",\"name\":\"products\",\"waitForSync\":false,\"isVolatile\":false,\"isSystem\":false,\"status\":3,\"type\":2,\"error\":false,\"code\":200}\r\n--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\nContent-Id: nextId\r\n\r\nHTTP/1.1 200 OK\r\nServer: \r\nLocation: /_db/_system/_api/collection/products/figures\r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 708\r\n\r\n{\"id\":\"9152\",\"name\":\"products\",\"isSystem\":false,\"doCompact\":true,\"isVolatile\":false,\"journalSize\":33554432,\"keyOptions\":{\"type\":\"traditional\",\"allowUserKeys\":true,\"lastValue\":0},\"waitForSync\":false,\"indexBuckets\":8,\"count\":0,\"figures\":{\"indexes\":{\"count\":1,\"size\":32128},\"compactionStatus\":{\"message\":\"compaction not yet started\",\"time\":\"2017-07-19T21:50:48Z\"},\"documentReferences\":0,\"waitingFor\":\"-\",\"alive\":{\"count\":0,\"size\":0},\"dead\":{\"count\":0,\"size\":0,\"deletion\":0},\"datafiles\":{\"count\":0,\"fileSize\":0},\"journals\":{\"count\":0,\"fileSize\":0},\"compactors\":{\"count\":0,\"fileSize\":0},\"revisions\":{\"count\":0,\"size\":48192},\"lastTick\":0,\"uncollectedLogfileEntries\":0},\"status\":3,\"type\":2,\"error\":false,\"code\":200}\r\n--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\nContent-Id: otherId\r\n\r\nHTTP/1.1 200 OK\r\nServer: \r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 38\r\n\r\n{\"id\":\"9152\",\"error\":false,\"code\":200}\r\n--SomeBoundaryValue--"

Sending a batch request, setting the boundary implicitly (the server will in this case try to find the boundary at the beginning of the request body).

shell> curl -X POST --data-binary @- --dump - http://localhost:8529/_api/batch <<EOF
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart

DELETE /_api/collection/notexisting1 HTTP/1.1
--SomeBoundaryValue
Content-Type: application/x-arango-batchpart

DELETE _api/collection/notexisting2 HTTP/1.1
--SomeBoundaryValue--

EOF

HTTP/1.1 200 OK
content-type: 
x-arango-errors: 2
x-content-type-options: nosniff

"--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\n\r\nHTTP/1.1 404 Not Found\r\nServer: \r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 92\r\n\r\n{\"error\":true,\"code\":404,\"errorNum\":1203,\"errorMessage\":\"unknown collection 'notexisting1'\"}\r\n--SomeBoundaryValue\r\nContent-Type: application/x-arango-batchpart\r\n\r\nHTTP/1.1 404 Not Found\r\nServer: \r\nConnection: \r\nContent-Type: application/json; charset=utf-8\r\nContent-Length: 101\r\n\r\n{\"error\":true,\"code\":404,\"errorNum\":404,\"errorMessage\":\"unknown path '_api/collection/notexisting2'\"}\r\n--SomeBoundaryValue--"

Deletes an existing dump batch

handle a dump batch command

DELETE /_api/replication/batch/{id} Note: These calls are uninteresting to users.

Path Parameters

  • id (required): The id of the batch.

Deletes the existing dump batch, allowing compaction and cleanup to resume.

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Return Codes

  • 204: is returned if the batch was deleted successfully.
  • 400: is returned if the batch was not found.
  • 405: is returned when an invalid HTTP method is used.

Create new dump batch

handle a dump batch command

POST /_api/replication/batch Note: These calls are uninteresting to users.

A JSON object with these properties is required:

  • ttl: the time-to-live for the new batch (in seconds) A JSON object with the batch configuration.

Creates a new dump batch and returns the batch's id.

The response is a JSON object with the following attributes:

  • id: the id of the batch

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Return Codes

  • 204: is returned if the batch was created successfully.
  • 400: is returned if the ttl value is invalid or if DBserver attribute is not specified or illegal on a coordinator.
  • 405: is returned when an invalid HTTP method is used.

Prolong existing dump batch

handle a dump batch command

PUT /_api/replication/batch/{id} Note: These calls are uninteresting to users.

A JSON object with these properties is required:

  • ttl: the time-to-live for the new batch (in seconds)

Extends the ttl of an existing dump batch, using the batch's id and the provided ttl value.

If the batch's ttl can be extended successfully, the response is empty.

Note: on a coordinator, this request must have the query parameter DBserver which must be an ID of a DBserver. The very same request is forwarded synchronously to that DBserver. It is an error if this attribute is not bound in the coordinator case.

Path Parameters

  • id (required): The id of the batch.

Return Codes

  • 204: is returned if the batch's ttl was extended successfully.
  • 400: is returned if the ttl value is invalid or the batch was not found.
  • 405: is returned when an invalid HTTP method is used.

© ArangoDB - the native multi-model NoSQL database