Scaling out¶

Create more tasks¶

Use SSH with the existing SSH keypair to log in to the app-controller controller instance.

$ ssh -i ~/.ssh/id_rsa USERNAME@IP_CONTROLLER

Call the faafo command-line interface to request the generation of five large fractals.

$ faafo create --height 9999 --width 9999 --tasks 5

If you check the load on the worker, you can see that the instance is not doing well. On the single CPU flavor instance, a load average greater than 1 means that the server is at capacity.

$ ssh -i ~/.ssh/id_rsa USERNAME@IP_WORKER uptime
10:37:39 up  1:44,  2 users,  load average: 1.24, 1.40, 1.36

Create more API service requests¶

API load is a slightly different problem than the previous one regarding capacity to work. We can simulate many requests to the API, as follows:

Use SSH with the existing SSH keypair to log in to the app-controller controller instance.

$ ssh -i ~/.ssh/id_rsa USERNAME@IP_CONTROLLER

Use a for loop to call the faafo command-line interface to request a random set of fractals 500 times:

$ for i in $(seq 1 500); do faafo --endpoint-url http://IP_CONTROLLER create & done

If you check the load on the app-controller API service instance, you see that the instance is not doing well. On your single CPU flavor instance, a load average greater than 1 means that the server is at capacity.

$ uptime
10:37:39 up  1:44,  2 users,  load average: 1.24, 1.40, 1.36

The sheer number of requests means that some requests for fractals might not make it to the message queue for processing. To ensure that you can cope with demand, you must also scale out the API capability of the Fractals application.

Remove the existing app¶

Go ahead and delete the existing instances and security groups that you created in previous sections. Remember, when instances in the cloud are no longer working, remove them and re-create something new.

for instance in conn.list_nodes():
    if instance.name in ['all-in-one','app-worker-1', 'app-worker-2', 'app-controller']:
        print('Destroying Instance: %s' % instance.name)
        conn.destroy_node(instance)


for group in conn.ex_list_security_groups():
    if group.name in ['control', 'worker', 'api', 'services']:
        print('Deleting security group: %s' % group.name)
        conn.ex_delete_security_group(group)

Extra security groups¶

As you change the topology of your applications, you must update or create security groups. Here, you re-create the required security groups.

api_group = conn.ex_create_security_group('api', 'for API services only')
conn.ex_create_security_group_rule(api_group, 'TCP', 80, 80)
conn.ex_create_security_group_rule(api_group, 'TCP', 22, 22)

worker_group = conn.ex_create_security_group('worker', 'for services that run on a worker node')
conn.ex_create_security_group_rule(worker_group, 'TCP', 22, 22)

controller_group = conn.ex_create_security_group('control', 'for services that run on a control node')
conn.ex_create_security_group_rule(controller_group, 'TCP', 22, 22)
conn.ex_create_security_group_rule(controller_group, 'TCP', 80, 80)
conn.ex_create_security_group_rule(controller_group, 'TCP', 5672, 5672, source_security_group=worker_group)

services_group = conn.ex_create_security_group('services', 'for DB and AMQP services only')
conn.ex_create_security_group_rule(services_group, 'TCP', 22, 22)
conn.ex_create_security_group_rule(services_group, 'TCP', 3306, 3306, source_security_group=api_group)
conn.ex_create_security_group_rule(services_group, 'TCP', 5672, 5672, source_security_group=worker_group)
conn.ex_create_security_group_rule(services_group, 'TCP', 5672, 5672, source_security_group=api_group)

A floating IP helper function¶

Define a short function to locate unused or allocate floating IPs. This saves a few lines of code and prevents you from reaching your floating IP quota too quickly.

def get_floating_ip(conn):
    '''A helper function to re-use available Floating IPs'''
    unused_floating_ip = None
    for floating_ip in conn.ex_list_floating_ips():
        if not floating_ip.node_id:
            unused_floating_ip = floating_ip
            break
    if not unused_floating_ip:
        pool = conn.ex_list_floating_ip_pools()[0]
        unused_floating_ip = pool.create_floating_ip()
    return unused_floating_ip

Split the database and message queue¶

Before you scale out your application services, like the API service or the workers, you must add a central database and an app-services messaging instance. The database and messaging queue will be used to track the state of fractals and to coordinate the communication between the services.

userdata = '''#!/usr/bin/env bash
curl -L -s http://git.openstack.org/cgit/openstack/faafo/plain/contrib/install.sh | bash -s -- \
    -i database -i messaging
'''

instance_services = conn.create_node(name='app-services',
                                     image=image,
                                     size=flavor,
                                     ex_keyname='demokey',
                                     ex_userdata=userdata,
                                     ex_security_groups=[services_group])
instance_services = conn.wait_until_running([instance_services])[0][0]
services_ip = instance_services.private_ips[0]

Scale the API service¶

With multiple workers producing fractals as fast as they can, the system must be able to receive the requests for fractals as quickly as possible. If our application becomes popular, many thousands of users might connect to our API to generate fractals.

Armed with a security group, image, and flavor size, you can add multiple API services:

userdata = '''#!/usr/bin/env bash
curl -L -s http://git.openstack.org/cgit/openstack/faafo/plain/contrib/install.sh | bash -s -- \
    -i faafo -r api -m 'amqp://guest:guest@%(services_ip)s:5672/' \
    -d 'mysql+pymysql://faafo:password@%(services_ip)s:3306/faafo'
''' % { 'services_ip': services_ip }
instance_api_1 = conn.create_node(name='app-api-1',
                                  image=image,
                                  size=flavor,
                                  ex_keyname='demokey',
                                  ex_userdata=userdata,
                                  ex_security_groups=[api_group])
instance_api_2 = conn.create_node(name='app-api-2',
                                  image=image,
                                  size=flavor,
                                  ex_keyname='demokey',
                                  ex_userdata=userdata,
                                  ex_security_groups=[api_group])
instance_api_1 = conn.wait_until_running([instance_api_1])[0][0]
api_1_ip = instance_api_1.private_ips[0]
instance_api_2 = conn.wait_until_running([instance_api_2])[0][0]
api_2_ip = instance_api_2.private_ips[0]

for instance in [instance_api_1,  instance_api_2]:
    floating_ip = get_floating_ip(conn)
    conn.ex_attach_floating_ip_to_node(instance, floating_ip)
    print('allocated %(ip)s to %(host)s' % {'ip': floating_ip.ip_address, 'host': instance.name})

These services are client-facing, so unlike the workers they do not use a message queue to distribute tasks. Instead, you must introduce some kind of load balancing mechanism to share incoming requests between the different API services.

A simple solution is to give half of your friends one address and half the other, but that solution is not sustainable. Instead, you can use a DNS round robin to do that automatically. However, OpenStack networking can provide Load Balancing as a Service, which Networking explains.

Scale the workers¶

To increase the overall capacity, add three workers:

userdata = '''#!/usr/bin/env bash
curl -L -s http://git.openstack.org/cgit/openstack/faafo/plain/contrib/install.sh | bash -s -- \
    -i faafo -r worker -e 'http://%(api_1_ip)s' -m 'amqp://guest:guest@%(services_ip)s:5672/'
''' % {'api_1_ip': api_1_ip, 'services_ip': services_ip}
instance_worker_1 = conn.create_node(name='app-worker-1',
                                     image=image, size=flavor,
                                     ex_keyname='demokey',
                                     ex_userdata=userdata,
                                     ex_security_groups=[worker_group])
instance_worker_2 = conn.create_node(name='app-worker-2',
                                     image=image, size=flavor,
                                     ex_keyname='demokey',
                                     ex_userdata=userdata,
                                     ex_security_groups=[worker_group])
instance_worker_3 = conn.create_node(name='app-worker-3',
                                     image=image, size=flavor,
                                     ex_keyname='demokey',
                                     ex_userdata=userdata,
                                     ex_security_groups=[worker_group])

Adding this capacity enables you to deal with a higher number of requests for fractals. As soon as these worker instances start, they begin checking the message queue for requests, reducing the overall backlog like a new register opening in the supermarket.

This process was obviously a very manual one. Figuring out that we needed more workers and then starting new ones required some effort. Ideally the system would do this itself. If you build your application to detect these situations, you can have it automatically request and remove resources, which saves you the effort of doing this work yourself. Instead, the OpenStack Orchestration service can monitor load and start instances, as appropriate. To find out how to set that up, see Orchestration.