Make it durable¶

Use Object Storage to store fractals¶

The Fractals app currently uses the local file system on the instance to store the images that it generates. For a number of reasons, this approach is not scalable or durable.

Because the local file system is ephemeral storage, the fractal images are lost along with the instance when the instance is terminated. Block-based storage, which the Block Storage section discusses, avoids that problem, but like local file systems, it requires administration to ensure that it does not fill up, and immediate attention if disks fail.

The Object Storage service manages many of the tasks normally managed by the application owner. The Object Storage service provides a scalable and durable API that you can use for the fractals app, eliminating the need to be aware of the low level details of how objects are stored and replicated, and how to grow the storage pool. Object Storage handles replication for you. It stores multiple copies of each object. You can use the Object Storage API to return an object, on demand.

First, learn how to connect to the Object Storage endpoint:

from __future__ import print_function
import hashlib
from shade import *

conn = openstack_cloud(cloud='myfavoriteopenstack')

To begin to store objects, we must first make a container. Call yours fractals:

container_name = 'fractals'
container = conn.create_container(container_name)
print(container)

You should see output such as:

Munch({u'content-length': u'0', u'x-container-object-count': u'0',
u'accept-ranges': u'bytes', u'x-container-bytes-used': u'0',
u'x-timestamp': u'1463950178.11674', u'x-trans-id':
u'txc6262b9c2bc1445b9dfe3-00574277ff', u'date': u'Mon, 23 May 2016
03:24:47 GMT', u'content-type': u'text/plain; charset=utf-8'})

You should now be able to see this container appear in a listing of all containers in your account:

print(conn.list_containers())

[Munch({u'count': 0, u'bytes': 0, u'name': u'fractals'}),
Munch({u'count': 0, u'bytes': 0, u'name': u'fractals_segments'})]

The next logical step is to upload an object. Find a photo of a goat online, name it goat.jpg, and upload it to your fractals container:

file_path = 'goat.jpg'
object_name = 'an amazing goat'
container = conn.get_container(container_name)
object = conn.create_object(container=container_name, name=object_name, filename=file_path)

List objects in your fractals container to see if the upload was successful. Then, download the file to verify that the md5sum is the same:

print(conn.list_objects(container_name))

[Munch({u'hash': u'd1408b5bf6510426db6e2bafc2f90854', u'last_modified':
u'2016-05-23T03:34:59.353480', u'bytes': 63654, u'name': u'an amazing
goat', u'content_type': u'application/octet-stream'})]

object = conn.get_object(container_name, object_name)
print(object)

print(hashlib.md5(open('goat.jpg', 'rb').read()).hexdigest())

d1408b5bf6510426db6e2bafc2f90854

Finally, clean up by deleting the test object:

conn.delete_object(container_name, object_name)

Munch({u'content-length': u'0', u'x-container-object-count': u'0',
u'accept-ranges': u'bytes', u'x-container-bytes-used': u'0',
u'x-timestamp': u'1463950178.11674', u'x-trans-id':
u'tx46c83fa41030422493110-0057427af3', u'date': u'Mon, 23 May 2016
03:37:23 GMT', u'content-type': u'text/plain; charset=utf-8'})

Now, no more objects are available in the fractals container.

print(conn.list_objects(container_name))

[]

container_name = 'fractals'
print(conn.get_container(container_name))

import base64
import cStringIO
import json
import requests

endpoint = 'http://IP_API_1'
params = { 'results_per_page': '-1' }
response = requests.get('%s/v1/fractal' % endpoint, params=params)
data = json.loads(response.text)
for fractal in data['objects']:
    r = requests.get('%s/fractal/%s' % (endpoint, fractal['uuid']), stream=True)
    with open(fractal['uuid'], 'wb') as f:
        for chunk in r.iter_content(chunk_size=1024):
            if chunk:
                f.write(chunk)
    conn.create_object(container=container_name, name=fractal['uuid'])

for object in conn.list_objects(container_name):
    print(object)

Extra features¶

for object in conn.list_objects(container_name):
    conn.delete_object(container_name, object['name'])
conn.delete_container(container_name)

metadata = {'foo': 'bar'}
conn.create_object(container=container_name, name=fractal['uuid'],
    metadata=metadata
)

Next steps¶

You should now be fairly confident working with Object Storage. You can find more information about the Object Storage SDK calls at:

Or, try one of these tutorial steps:

• Block Storage: Migrate the database to block storage, or use the database-as-a-service component.
• Orchestration: Automatically orchestrate your application.
• Networking: Learn about complex networking.
• Advice for developers new to operations: Get advice about operations.
• Going crazy: Learn some crazy things that you might not think to do ;)