Proxy

Proxy Controllers

Base

class swift.proxy.controllers.base.Controller(app)

Bases: object

Base WSGI controller class for the proxy

GET(*a, **kw)

Handler for HTTP GET requests.

Parameters:req – The client request Returns:the response to the client

GETorHEAD_base(req, server_type, ring, partition, path)

Base handler for HTTP GET or HEAD requests.

Parameters:

• req – swob.Request object
• server_type – server type used in logging
• ring – the ring to obtain nodes from
• partition – partition
• path – path for the request

Returns:

swob.Response object

HEAD(*a, **kw)

Handler for HTTP HEAD requests.

Parameters:req – The client request Returns:the response to the client

OPTIONS(*a, **kw)

Base handler for OPTIONS requests

Parameters:req – swob.Request object Returns:swob.Response object

account_info(account, req=None)

Get account information, and also verify that the account exists.

Parameters:

• account – name of the account to get the info for
• req – caller’s HTTP request context object (optional)

Returns:

tuple of (account partition, account nodes, container_count) or (None, None, None) if it does not exist

allowed_methods

autocreate_account(env, account)

Autocreate an account

Parameters:

• env – the environment of the request leading to this autocreate
• account – the unquoted account name

best_response(req, statuses, reasons, bodies, server_type, etag=None, headers=None)

Given a list of responses from several servers, choose the best to return to the API.

Parameters:

• req – swob.Request object
• statuses – list of statuses returned
• reasons – list of reasons for each status
• bodies – bodies of each response
• server_type – type of server the responses came from
• etag – etag
• headers – headers of each response

Returns:

swob.Response object with the correct status, body, etc. set

container_info(account, container, req=None)

Get container information and thusly verify container existence. This will also verify account existence.

Parameters:

• account – account name for the container
• container – container name to look up
• req – caller’s HTTP request context object (optional)

Returns:

dict containing at least container partition (‘partition’), container nodes (‘containers’), container read acl (‘read_acl’), container write acl (‘write_acl’), and container sync key (‘sync_key’). Values are set to None if the container does not exist.

generate_request_headers(orig_req=None, additional=None, transfer=False)

Create a list of headers to be used in backend requets

Parameters:

• orig_req – the original request sent by the client to the proxy
• additional – additional headers to send to the backend
• transfer – If True, transfer headers from original client request

Returns:

a dictionary of headers

have_quorum(statuses, node_count)

Given a list of statuses from several requests, determine if a quorum response can already be decided.

Parameters:

• statuses – list of statuses returned
• node_count – number of nodes being queried (basically ring count)

Returns:

True or False, depending on if quorum is established

is_origin_allowed(cors_info, origin)

Is the given Origin allowed to make requests to this resource

Parameters:

• cors_info – the resource’s CORS related metadata headers
• origin – the origin making the request

Returns:

True or False

make_requests(req, ring, part, method, path, headers, query_string='')

Sends an HTTP request to multiple nodes and aggregates the results. It attempts the primary nodes concurrently, then iterates over the handoff nodes as needed.

Parameters:

• req – a request sent by the client
• ring – the ring used for finding backend servers
• part – the partition number
• method – the method to send to the backend
• path – the path to send to the backend (full path ends up being /<$device>/<$part>/<$path>)
• headers – a list of dicts, where each dict represents one backend request that should be made.
• query_string – optional query string to send to the backend

Returns:

a swob.Response object

pass_through_headers = []

server_type = 'Base'

transfer_headers(src_headers, dst_headers)

Transfer legal headers from an original client request to dictionary that will be used as headers by the backend request

Parameters:

• src_headers – A dictionary of the original client request headers
• dst_headers – A dictionary of the backend request headers

class swift.proxy.controllers.base.GetOrHeadHandler(app, req, server_type, ring, partition, path, backend_headers)

Bases: object

fast_forward(num_bytes)

Will skip num_bytes into the current ranges.

Params num_bytes:  

the number of bytes that have already been read on this request. This will change the Range header so that the next req will start where it left off.

Raises:

• NotImplementedError – if this is a multirange request
• ValueError – if invalid range header
• HTTPRequestedRangeNotSatisfiable – if begin + num_bytes > end of range

get_working_response(req)

is_good_source(src)

Indicates whether or not the request made to the backend found what it was looking for.

Parameters:src – the response from the backend Returns:True if found, False if not

swift.proxy.controllers.base.clear_info_cache(app, env, account, container=None)

Clear the cached info in both memcache and env

Parameters:

• app – the application object
• account – the account name
• container – the containr name or None if setting info for containers

swift.proxy.controllers.base.close_swift_conn(src)

Force close the http connection to the backend.

Parameters:src – the response from the backend

swift.proxy.controllers.base.cors_validation(func)

Decorator to check if the request is a CORS request and if so, if it’s valid.

Parameters:func – function to check

swift.proxy.controllers.base.delay_denial(func)

Decorator to declare which methods should have any swift.authorize call delayed. This is so the method can load the Request object up with additional information that may be needed by the authorization system.

Parameters:func – function for which authorization will be delayed

swift.proxy.controllers.base.get_account_info(env, app, swift_source=None)

Get the info structure for an account, based on env and app. This is useful to middlewares.

Note

This call bypasses auth. Success does not imply that the request has authorization to the account.

swift.proxy.controllers.base.get_account_memcache_key(account)

swift.proxy.controllers.base.get_container_info(env, app, swift_source=None)

Get the info structure for a container, based on env and app. This is useful to middlewares.

Note

This call bypasses auth. Success does not imply that the request has authorization to the container.

swift.proxy.controllers.base.get_container_memcache_key(account, container)

swift.proxy.controllers.base.get_info(app, env, account, container=None, ret_not_found=False, swift_source=None)

Get the info about accounts or containers

Note: This call bypasses auth. Success does not imply that the

request has authorization to the info.

Parameters:

• app – the application object
• env – the environment used by the current request
• account – The unquoted name of the account
• container – The unquoted name of the container (or None if account)

Returns:

the cached info or None if cannot be retrieved

swift.proxy.controllers.base.get_object_env_key(account, container, obj)

Get the keys for env (env_key) where info about object is cached :param account: The name of the account :param container: The name of the container :param obj: The name of the object :returns a string env_key

swift.proxy.controllers.base.get_object_info(env, app, path=None, swift_source=None)

Get the info structure for an object, based on env and app. This is useful to middlewares.

Note

This call bypasses auth. Success does not imply that the request has authorization to the object.

swift.proxy.controllers.base.headers_to_account_info(headers, status_int=200)

Construct a cacheable dict of account info based on response headers.

swift.proxy.controllers.base.headers_to_container_info(headers, status_int=200)

Construct a cacheable dict of container info based on response headers.

swift.proxy.controllers.base.headers_to_object_info(headers, status_int=200)

Construct a cacheable dict of object info based on response headers.

swift.proxy.controllers.base.source_key(resp)

Provide the timestamp of the swift http response as a floating point value. Used as a sort key.

Parameters:resp – bufferedhttp response object

swift.proxy.controllers.base.update_headers(response, headers)

Helper function to update headers in the response.

Parameters:

• response – swob.Response object
• headers – dictionary headers

Account

class swift.proxy.controllers.account.AccountController(app, account_name, **kwargs)

Bases: swift.proxy.controllers.base.Controller

WSGI controller for account requests

DELETE(*a, **kw)

HTTP DELETE request handler.

GETorHEAD(req)

Handler for HTTP GET/HEAD requests.

POST(*a, **kw)

HTTP POST request handler.

PUT(*a, **kw)

HTTP PUT request handler.

add_acls_from_sys_metadata(resp)

server_type = 'Account'

Container

class swift.proxy.controllers.container.ContainerController(app, account_name, container_name, **kwargs)

Bases: swift.proxy.controllers.base.Controller

WSGI controller for container requests

DELETE(*a, **kw)

HTTP DELETE request handler.

GET(*a, **kw)

Handler for HTTP GET requests.

GETorHEAD(req)

Handler for HTTP GET/HEAD requests.

HEAD(*a, **kw)

Handler for HTTP HEAD requests.

POST(*a, **kw)

HTTP POST request handler.

PUT(*a, **kw)

HTTP PUT request handler.

clean_acls(req)

pass_through_headers = ['x-container-read', 'x-container-write', 'x-container-sync-key', 'x-container-sync-to', 'x-versions-location']

server_type = 'Container'

Object

class swift.proxy.controllers.obj.ObjectController(app, account_name, container_name, object_name, **kwargs)

Bases: swift.proxy.controllers.base.Controller

WSGI controller for object requests.

COPY(*a, **kw)

HTTP COPY request handler.

DELETE(*a, **kw)

HTTP DELETE request handler.

GET(*a, **kw)

Handler for HTTP GET requests.

GETorHEAD(req)

Handle HTTP GET or HEAD requests.

HEAD(*a, **kw)

Handler for HTTP HEAD requests.

POST(*a, **kw)

HTTP POST request handler.

PUT(*a, **kw)

HTTP PUT request handler.

iter_nodes_local_first(ring, partition)

Yields nodes for a ring partition.

If the ‘write_affinity’ setting is non-empty, then this will yield N local nodes (as defined by the write_affinity setting) first, then the rest of the nodes as normal. It is a re-ordering of the nodes such that the local ones come first; no node is omitted. The effect is that the request will be serviced by local object servers first, but nonlocal ones will be employed if not enough local ones are available.

Parameters:

• ring – ring to get nodes from
• partition – ring partition to yield nodes for

server_type = 'Object'

swift.proxy.controllers.obj.check_content_type(req)

swift.proxy.controllers.obj.copy_headers_into(from_r, to_r)

Will copy desired headers from from_r to to_r :params from_r: a swob Request or Response :params to_r: a swob Request or Response

Proxy Server

class swift.proxy.server.Application(conf, memcache=None, logger=None, account_ring=None, container_ring=None, object_ring=None)

Bases: object

WSGI application for the proxy server.

check_config()

Check the configuration for possible errors

error_limit(node, msg)

Mark a node as error limited. This immediately pretends the node received enough errors to trigger error suppression. Use this for errors like Insufficient Storage. For other errors use error_occurred().

Parameters:

• node – dictionary of node to error limit
• msg – error message

error_limited(node)

Check if the node is currently error limited.

Parameters:node – dictionary of node to check Returns:True if error limited, False otherwise

error_occurred(node, msg)

Handle logging, and handling of errors.

Parameters:

• node – dictionary of node to handle errors for
• msg – error message

exception_occurred(node, typ, additional_info)

Handle logging of generic exceptions.

Parameters:

• node – dictionary of node to log the error for
• typ – server type
• additional_info – additional information to log

get_controller(path)

Get the controller to handle a request.

Parameters:path – path from request Returns:tuple of (controller class, path dictionary) Raises :ValueError (thrown by split_path) if given invalid path

handle_request(req)

Entry point for proxy server. Should return a WSGI-style callable (such as swob.Response).

Parameters:req – swob.Request object

iter_nodes(ring, partition, node_iter=None)

Yields nodes for a ring partition, skipping over error limited nodes and stopping at the configurable number of nodes. If a node yielded subsequently gets error limited, an extra node will be yielded to take its place.

Note that if you’re going to iterate over this concurrently from multiple greenthreads, you’ll want to use a swift.common.utils.GreenthreadSafeIterator to serialize access. Otherwise, you may get ValueErrors from concurrent access. (You also may not, depending on how logging is configured, the vagaries of socket IO and eventlet, and the phase of the moon.)

Parameters:

• ring – ring to get yield nodes from
• partition – ring partition to yield nodes for
• node_iter – optional iterable of nodes to try. Useful if you want to filter or reorder the nodes.

modify_wsgi_pipeline(pipe)

Called during WSGI pipeline creation. Modifies the WSGI pipeline context to ensure that mandatory middleware is present in the pipeline.

Parameters:pipe – A PipelineWrapper object

set_node_timing(node, timing)

sort_nodes(nodes)

Sorts nodes in-place (and returns the sorted list) according to the configured strategy. The default “sorting” is to randomly shuffle the nodes. If the “timing” strategy is chosen, the nodes are sorted according to the stored timing data.

update_request(req)

swift.proxy.server.app_factory(global_conf, **local_conf)

paste.deploy app factory for creating WSGI proxy apps.