Chapter 3. Object Storage API examples

To request an XML response, append the format=xml query parameter to the request.

This example cURL command shows account information and list containers in the account, and asks for the response in XML:

# curl -i $publicURL?format=xml \
         -X GET -H "X-Auth-Token: $token"

HTTP/1.1 200 OK
Content-Length: 262
X-Account-Object-Count: 1
X-Timestamp: 1389453423.35964
X-Account-Meta-Subject: Literature
X-Account-Bytes-Used: 14
X-Account-Container-Count: 2
Content-Type: application/xml; charset=utf-8
Accept-Ranges: bytes
X-Trans-Id: tx69f60bc9f7634a01988e6-0052d9544b
Date: Fri, 17 Jan 2014 16:03:23 GMT

<?xml version="1.0" encoding="UTF-8"?>
<account name="my_account">
    <container>
        <name>janeausten</name>
        <count>0</count>
        <bytes>0</bytes>
    </container>
    <container>
        <name>marktwain</name>
        <count>1</count>
        <bytes>14</bytes>
    </container>
</account>

This section provides an overview of the authentication process. For request and response details, see Authenticate in the OpenStack Identity Service API v2.0 Reference.

To show how much data you have stored in the system and the number of containers that you are using, send a HEAD request to the Object Storage service.

Use the -X switch to specify the HEAD method.

Use the -i switch to send the HTTP response to terminal output.

Include the authentication token in the X-Auth-Token header.

# curl -i $publicURL -X HEAD -H "X-Auth-Token: $token"

HTTP/1.1 204 No Content
Content-Length: 0
X-Account-Object-Count: 1
X-Account-Meta-Book: MobyDick
X-Timestamp: 1389453423.35964
X-Account-Bytes-Used: 14
X-Account-Container-Count: 2
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
X-Trans-Id: txafb3504870144b8ca40f7-0052d955d4
Date: Fri, 17 Jan 2014 16:09:56 GMT

The X-Account-Bytes-Used response header shows the total bytes stored for the entire account.

The X-Account-Container-Count response header shows the number of containers in this storage account.

This example shows account details, lists containers, and asks for a JSON response:

# curl -i $publicURL?format=json -X GET -H "X-Auth-Token: $token"

HTTP/1.1 200 OK
Content-Length: 96
X-Account-Object-Count: 1
X-Timestamp: 1389453423.35964
X-Account-Meta-Subject: Literature
X-Account-Bytes-Used: 14
X-Account-Container-Count: 2
Content-Type: application/json; charset=utf-8
Accept-Ranges: bytes
X-Trans-Id: tx274a77a8975c4a66aeb24-0052d95365
Date: Fri, 17 Jan 2014 15:59:33 GMT 

[
   {
      "count":0,
      "bytes":0,
      "name":"janeausten"
   },
   {
      "count":1,
      "bytes":14,
      "name":"marktwain"
   }
]

This example shows account details, lists containers, and asks for an XML response:

# curl -i $publicURL?format=xml -X GET -H "X-Auth-Token: $token"

HTTP/1.1 200 OK
Content-Length: 262
X-Account-Object-Count: 1
X-Timestamp: 1389453423.35964
X-Account-Meta-Subject: Literature
X-Account-Bytes-Used: 14
X-Account-Container-Count: 2
Content-Type: application/xml; charset=utf-8
Accept-Ranges: bytes
X-Trans-Id: tx69f60bc9f7634a01988e6-0052d9544b
Date: Fri, 17 Jan 2014 16:03:23 GMT 

<?xml version="1.0" encoding="UTF-8"?>
<account name="my_account">
    <container>
        <name>janeausten</name>
        <count>0</count>
        <bytes>0</bytes>
    </container>
    <container>
        <name>marktwain</name>
        <count>1</count>
        <bytes>14</bytes>
    </container>
</account>

The X-Container-Read metadata header defines the access control list (ACL) permissions for who can read objects in a container. Before you set this header, only users with a valid authentication token for the account can read objects in that container.

List containers to show the absence of the X-Container-Read header:

# curl –X GET -i -H "X-Auth-Token: $token" $publicURL/jerry

HTTP/1.1 204 No Content
X-Container-Object-Count: 0
X-Container-Bytes-Used: 0
Accept-Ranges: bytes
X-Trans-Id: tx3aa52e951fc64b63bc1fda27902b9bd3
Content-Length: 0
Date: Tue, 15 Nov 2011 03:29:22 GMT

Set the X-Container-Read header to enable read and list access to everyone:

# curl –X PUT -i \
-H "X-Auth-Token: $token" \
-H "X-Container-Read: .r:*,.rlistings" \
$publicURL/jerry

HTTP/1.1 202 Accepted
Content-Length: 58
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txf2befb56b1854a50995f710f2db48089
Date: Tue, 15 Nov 2011 03:33:16 GMT

202 Accepted

The request is accepted for processing.

For a list of valid X-Container-Read header values, see ACLs.

To see the metadata change:

# curl –X GET -i -H "X-Auth-Token: $token" $publicURL/jerry

HTTP/1.1 204 No Content
X-Container-Object-Count: 0
X-Container-Read: .r:*,.rlistings
X-Container-Bytes-Used: 0
Accept-Ranges: bytes
X-Trans-Id: txb40eb86d949345f7bc66b01e8b63c3a5
Content-Length: 0
Date: Tue, 15 Nov 2011 03:33:36 GMT

After you give everyone read access, anyone can access any object in the container from a browser. To do so, a user appends the object name to the X-Storage-URL header value used in the session. For example:

$publicURL/jerry/cereal.jpg

To create a container, issue a PUT request. You do not need to check if a container already exists before you issue a PUT request. The operation creates a container or updates an existing container, as appropriate.

Example requests and responses:

For a complete description of HTTP 1.1 header definitions, see Header Field Definitions.

	Note
	You can also use this technique to page through large lists of objects.

For more information about how to page through large lists of containers and objects, see the section called “Page through large lists of containers or objects”.

For a list of five container names, if you specify a limit of two, two items are returned. You can assume there are more names to list, so make another request with a marker of the last item returned.

For example, assume the following list of container names:

apples
bananas
kiwis
oranges
pears

Now, retrieve an object that you previously uploaded. First, remove the local copy:

# ls -l

total 504
-rw-r--r--@ 1 petecj2  staff   44765 Nov  7 14:49 JingleRocky.jpg
-rw-r--r--@ 1 petecj2  staff  100864 Nov  7 14:47 RockyAndBuster.jpg
-rw-r--r--@ 1 petecj2  staff  107103 Nov  7 14:47 SittingBuster.jpg

# rm JingleRocky.jpg
# ls -l

total 416
-rw-r--r--@ 1 petecj2  staff  100864 Nov  7 14:47 RockyAndBuster.jpg
-rw-r--r--@ 1 petecj2  staff  107103 Nov  7 14:47 SittingBuster.jpg      

Be sure not to use -i switch here because you want the raw data, which you pipe to a file:

# curl –X GET -H "X-Auth-Token: $token" $publicURL/dogs/JingleRocky.jpg > JingleRocky.jpg

# ls -l

total 504
-rw-r--r--  1 petecj2  staff   44765 Nov  7 15:11 JingleRocky.jpg
-rw-r--r--@ 1 petecj2  staff  100864 Nov  7 14:47 RockyAndBuster.jpg
-rw-r--r--@ 1 petecj2  staff  107103 Nov  7 14:47 SittingBuster.jpg

Next, Object Storage provides a facility to copy objects from one container to another entirely on the server side. To do this, you do a PUT with the destination container and new object name while passing a special X-Copy-From header and a Content-Length of zero:

# curl –X PUT -i -H "X-Auth-Token: $token" -H "X-Copy-From: /dogs/JingleRocky.jpg" -H "Content-Length: 0" $publicURL/elaine/JingleRocky.jpg

HTTP/1.1 201 Created
Content-Length: 118
Content-Type: text/html; charset=UTF-8
Etag: f7d40eceffdd9c2ecab226105737b2a6
X-Copied-From: dogs/JingleRocky.jpg
Last-Modified: Mon, 07 Nov 2011 23:23:53 GMT
X-Trans-Id: tx244cd14df1b94d8c91ec5dcf8c5f9da4
Date: Mon, 07 Nov 2011 23:23:54 GMT

<html><head><title>201 Created</title></head><body><h1>201 Created</h1><br /><br /></body></html>

You can then confirm the new location of the object. Issue a GET request with the destination container:

# curl –X GET -i -H "X-Auth-Token: $token" $publicURL/elaine/

HTTP/1.1 200 OK
X-Container-Object-Count: 1
X-Container-Bytes-Used: 44765
Accept-Ranges: bytes
Content-Length: 16
Content-Type: text/plain; charset=utf-8
X-Trans-Id: tx46986b4a09b34790924fd43842b2b0dd
Date: Mon, 07 Nov 2011 23:24:05 GMT

JingleRocky.jpg

To delete an object from its container:

# curl –X DELETE -i -H "X-Auth-Token: $token" $publicURL/elaine/JingleRocky.jpg

HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txd45f04422b034e6f8447de400b78cbf3
Date: Mon, 07 Nov 2011 23:32:39 GMT

List containers to confirm the deletion:

# curl –X GET -i -H "X-Auth-Token: $token" $publicURL/elaine/

HTTP/1.1 204 No Content
X-Container-Object-Count: 0
X-Container-Bytes-Used: 0
Accept-Ranges: bytes
X-Trans-Id: txc9b43bf4d896405eb9a88ca468bf7b2d
Content-Length: 0
Date: Mon, 07 Nov 2011 23:32:41 GMT

You can get at container metadata directly simply by appending the name of the container to a HEAD request:

# curl –X HEAD -i \
    -H "X-Auth-Token: $token" \
    $publicURL/dogs
HTTP/1.1 204 No Content
X-Container-Object-Count: 0
X-Container-Bytes-Used: 0
Accept-Ranges: bytes
X-Trans-Id: tx3dd984f9482341dd97546e9d49d65e90
Content-Length: 0
Date: Mon, 07 Nov 2011 20:39:41 GMT

To delete a container:

# curl –X DELETE -i \
    -H "X-Auth-Token: $token" \
    $publicURL/george
HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
X-Trans-Id: tx3fa3857f266f44319d9b8f4bf7ce7fc8
Date: Mon, 07 Nov 2011 20:42:58 GMT

Then let's confirm the delete by listing the containers again:

# curl –X GET -i \
    -H "X-Auth-Token: $token" \
    $publicURL
HTTP/1.1 200 OK
X-Account-Object-Count: 0
X-Account-Bytes-Used: 0
X-Account-Container-Count: 4
Accept-Ranges: bytes
Content-Length: 24
Content-Type: text/plain; charset=utf-8
X-Trans-Id: tx2475741852b849ce9403e382fe3f8015
Date: Mon, 07 Nov 2011 20:43:08 GMT

cosmo
dogs
elaine
jerry

To create a static large object:

This example places the segment objects into the segments container and the manifest object into the images container. Using a dedicated container for segment objects is convenient.

Assuming you have already split the image into three files, you can upload them. You have removed non-essential response headers so you can see the important details.

# curl –X PUT -i -H "X-Auth-Token: $token" -T ./piece1 $publicURL/segments/terrier-jpg-one

HTTP/1.1 201 Created
Content-Length: 4000000
Etag: f7365c1419b4f349592c00bd0cfb9b9a

# curl –X PUT -i -H "X-Auth-Token: $token" -T ./piece2 $publicURL/segments/terrier-jpg-two

HTTP/1.1 201 Created
Content-Length: 2000000
Etag: ad81e97b10e870613aecb5ced52adbaa

# curl –X PUT -i -H "X-Auth-Token: $token" -T ./piece3 $publicURL/segments/terrier-jpg-three

HTTP/1.1 201 Created
Content-Length: 1000
Etag: 00b046c9d74c3e8f93b320c5e5fdc2c3

At this stage, you can create the manifest listing. Notice that the size and ETag are copied from the previous uploads. Create a file called manifest.json with the following content:

[
   {
      "path":"segments/terrier-jpg-one",
      "etag":"f7365c1419b4f349592c00bd0cfb9b9a",
      "size_bytes":4000000
   },
   {
      "path":"segments/terrier-jpg-two",
      "etag":"ad81e97b10e870613aecb5ced52adbaa",
      "size_bytes":2000000
   },
   {
      "path":"segments/terrier-jpg-three",
      "etag":"00b046c9d74c3e8f93b320c5e5fdc2c3",
      "size_bytes":1000
   }
]

The final operation is to upload this content into a manifest object. To indicate that this is a manifest object, you must specify the multipart-manifest=put query parameter.

# curl –X PUT -i -H "X-Auth-Token: $token" -T ./manifest.json  $publicURL/images/terrier-jpg?multipart-manifest=put

Examine the static large object. Notice that its size is the total size of all the segments:

# curl –X HEAD -i -H "X-Auth-Token: $token" $publicURL/images/terrier-jpg

HTTP/1.1 200 OK
Content-Length: 6001000
Etag: "0c922c37f915efb1c9b97e6328b3e660"