Contents
OpenStack Object Storage (code-named Swift) is open source software for creating redundant, scalable data storage using clusters of standardized servers to store petabytes of accessible data. It is a long-term storage system for large amounts of static data that can be retrieved, leveraged, and updated. Object Storage uses a distributed architecture with no central point of control, providing greater scalability, redundancy and permanence. Objects are written to multiple hardware devices, with the OpenStack software responsible for ensuring data replication and integrity across the cluster. Storage clusters scale horizontally by adding new nodes. Should a node fail, OpenStack works to replicate its content from other active nodes. Because OpenStack uses software logic to ensure data replication and distribution across different devices, inexpensive commodity hard drives and servers can be used in lieu of more expensive equipment.
Object Storage is ideal for cost effective, scale-out storage. It provides a fully distributed, API-accessible storage platform that can be integrated directly into applications or used for backup, archiving and data retention. Block Storage allows block devices to be exposed and connected to compute instances for expanded storage, better performance and integration with enterprise storage platforms, such as NetApp, Nexenta and SolidFire.
| Features | Benefits |
|---|---|
| Leverages commodity hardware | No lock-in, lower price/GB |
| HDD/node failure agnostic | Self healingReliability, data redundancy protecting from failures |
| Unlimited storage | Huge & flat namespace, highly scalable read/write accessAbility to serve content directly from storage system |
| Multi-dimensional scalability (scale out architecture)Scale vertically and horizontally-distributed storage | Backup and archive large amounts of data with linear performance |
| Account/Container/Object structureNo nesting, not a traditional file system | Optimized for scaleScales to multiple petabytes, billions of objects |
| Built-in replication3x+ data redundancy compared to 2x on RAID | Configurable number of accounts, container and object copies for high availability |
| Easily add capacity unlike RAID resize | Elastic data scaling with ease |
| No central database | Higher performance, no bottlenecks |
| RAID not required | Handle lots of small, random reads and writes efficiently |
| Built-in management utilities | Account Management: Create, add, verify, delete usersContainer Management: Upload, download, verifyMonitoring: Capacity, host, network, log trawling, cluster health |
| Drive auditing | Detect drive failures preempting data corruption |
| Expiring objects | Users can set an expiration time or a TTL on an object to control access |
| Direct object access | Enable direct browser access to content, such as for a control panel |
| Realtime visibility into client requests | Know what users are requesting |
| Supports S3 API | Utilize tools that were designed for the popular S3 API |
| Restrict containers per account | Limit access to control usage by user |
| Support for NetApp, Nexenta, SolidFire | Unified support for block volumes using a variety of storage systems |
| Snapshot and backup API for block volumes | Data protection and recovery for VM data |
| Standalone volume API available | Separate endpoint and API for integration with other compute systems |
| Integration with Compute | Fully integrated to Compute for attaching block volumes and reporting on usage |
The swift client is the command-line interface (CLI) for the OpenStack Object Storage API and its extensions. This chapter documents swift version 2.0.3.
For help on a specific swift command, enter:
$ swift help COMMAND [--debug] [--info] [--quiet] [--auth <auth_url>]
[--auth-version <auth_version>] [--user <username>]
[--key <api_key>] [--retries <num_retries>]
[--os-username <auth-user-name>] [--os-password <auth-password>]
[--os-tenant-id <auth-tenant-id>]
[--os-tenant-name <auth-tenant-name>]
[--os-auth-url <auth-url>] [--os-auth-token <auth-token>]
[--os-storage-url <storage-url>] [--os-region-name <region-name>]
[--os-service-type <service-type>]
[--os-endpoint-type <endpoint-type>]
[--os-cacert <ca-certificate>] [--insecure]
[--no-ssl-compression]
<subcommand> ...
Subcommands
- delete
Delete a container or objects within a container
- download
Download objects from containers
- list
Lists the containers for the account or the objects for a container
- post
Updates meta information for the account, container, or object; creates containers if not present
- stat
Displays information for the account, container, or object
- upload
Uploads files or directories to the given container
- capabilities
List cluster capabilities
swift -A https://auth.api.rackspacecloud.com/v1.0 -U user -K api_key stat -v
swift --os-auth-url https://api.example.com/v2.0 --os-tenant-name tenant \
--os-username user --os-password password list
swift --os-auth-token 6ee5eb33efad4e45ab46806eac010566 \
--os-storage-url https://10.1.5.2:8080/v1/AUTH_ced809b6a4baea7aeab61a \
list
swift list --lh
- --version
show program's version number and exit
- -h, --help
show this help message and exit
- -s, --snet
Use SERVICENET internal network
- -v, --verbose
Print more info
- --debug
Show the curl commands and results of all http queries regardless of result status.
- --info
Show the curl commands and results of all http queries which return an error.
- -q, --quiet
Suppress status output
- -A AUTH, --auth=AUTH URL
for obtaining an auth token
- -V AUTH_VERSION, --auth-version=AUTH_VERSION
Specify a version for authentication. Defaults to 1.0.
- -U USER, --user=USER
User name for obtaining an auth token.
- -K KEY, --key=KEY
Key for obtaining an auth token.
- -R RETRIES, --retries=RETRIES
The number of times to retry a failed connection.
- --os-username=<auth-user-name>
OpenStack username. Defaults to
env[OS_USERNAME].- --os-password=<auth-password>
OpenStack password. Defaults to
env[OS_PASSWORD].- --os-tenant-id=<auth-tenant-id>
OpenStack tenant ID. Defaults to
env[OS_TENANT_ID]- --os-tenant-name=<auth-tenant-name>
OpenStack tenant name. Defaults to
env[OS_TENANT_NAME].- --os-auth-url=<auth-url>
OpenStack auth URL. Defaults to
env[OS_AUTH_URL].- --os-auth-token=<auth-token>
OpenStack token. Defaults to
env[OS_AUTH_TOKEN]. Used with --os-storage-url to bypass the usual username/password authentication.- --os-storage-url=<storage-url>
OpenStack storage URL. Defaults to
env[OS_STORAGE_URL]. Overrides the storage url returned during auth. Will bypass authentication when used with --os-auth-token.- --os-region-name=<region-name>
OpenStack region name. Defaults to
env[OS_REGION_NAME]- --os-service-type=<service-type>
OpenStack Service type. Defaults to
env[OS_SERVICE_TYPE]- --os-endpoint-type=<endpoint-type>
OpenStack Endpoint type. Defaults to
env[OS_ENDPOINT_TYPE]- --os-cacert=<ca-certificate>
Specify a CA bundle file to use in verifying a TLS (https) server certificate. Defaults to
env[OS_CACERT]- --insecure
Allow swiftclient to access servers without having to verify the SSL certificate. Defaults to
env[SWIFTCLIENT_INSECURE](set to 'true' to enable).- --no-ssl-compression
This option is deprecated and not used anymore. SSL compression should be disabled by default by the system SSL library
Usage: Delete a container or objects within a container
Positional arguments
- <container>
Name of container to delete from
- [object]
Name of object to delete. Specify multiple times for multiple objects
Optional arguments
- --all
Delete all containers and objects
- --leave-segments
Do not delete segments of manifest objects
- --object-threads <threads>
Number of threads to use for deleting objects. Default is 10
- --container-threads <threads>
Number of threads to use for deleting containers. Default is 10
Usage: Download objects from containers
Positional arguments
- <container>
Name of container to download from. To download a whole account, omit this and specify --all.
- [object]
Name of object to download. Specify multiple times for multiple objects. Omit this to download all objects from the container.
Optional arguments
- --all
Indicates that you really want to download everything in the account
- --marker
Marker to use when starting a container or account download
- --prefix <prefix>
Only download items beginning with <prefix>
- --output <out_file>
For a single file download, stream the output to <out_file>. Specifying "-" as <out_file> will redirect to stdout
- --object-threads <threads>
Number of threads to use for downloading objects. Default is 10
- --container-threads <threads>
Number of threads to use for downloading containers. Default is 10
- --no-download
Perform download(s), but don't actually write anything to disk
- --header <header_name:header_value>
Adds a customized request header to the query, like "Range" or "If-Match". This argument is repeatable. Example --header "content-type:text/plain"
- --skip-identical
Skip downloading files that are identical on both sides
Usage: Lists the containers for the account or the objects for a container
Positional arguments
- [container]
Name of container to list object in
Optional arguments
- --long
Long listing format, similar to ls -l
- --lh
Report sizes in human readable format similar to ls -lh
- --totals
Used with -l or --lh, only report totals
- --prefix
Only list items beginning with the prefix
- --delimiter
Roll up items with the given delimiter. For containers only. See OpenStack Swift API documentation for what this means.
Usage: Updates meta information for the account, container, or object.
If the container is not found, it will be created automatically.
Positional arguments
- [container]
Name of container to post to
- [object]
Name of object to post. Specify multiple times for multiple objects
Optional arguments
- --read-acl <acl>
Read ACL for containers. Quick summary of ACL syntax: .r:*, .r:-.example.com, .r:www.example.com, account1, account2:user2
- --write-acl <acl>
Write ACL for containers. Quick summary of ACL syntax: account1 account2:user2
- --sync-to <sync-to>
Sync To for containers, for multi-cluster replication
- --sync-key <sync-key>
Sync Key for containers, for multi-cluster replication
- --meta <name:value>
Sets a meta data item. This option may be repeated. Example: -m Color:Blue -m Size:Large
- --header <header>
Set request headers. This option may be repeated. Example -H "content-type:text/plain"
Usage: Displays information for the account, container, or object
Positional arguments
- [container]
Name of container to stat from
- [object]
Name of object to stat. Specify multiple times for multiple objects
Optional arguments
- --lh
Report sizes in human readable format similar to ls -lh
Usage: Uploads specified files and directories to the given container
Positional arguments
- <container>
Name of container to upload to
- <file_or_directory>
Name of file or directory to upload. Specify multiple times for multiple uploads
Optional arguments
- --changed
Only upload files that have changed since the last upload
- --skip-identical
Skip uploading files that are identical on both sides
- --segment-size <size>
Upload files in segments no larger than <size> and then create a "manifest" file that will download all the segments as if it were the original file
- --segment-container <container>
Upload the segments into the specified container. If not specified, the segments will be uploaded to a <container>_segments container so as to not pollute the main <container> listings.
- --leave-segments
Indicates that you want the older segments of manifest objects left alone (in the case of overwrites)
- --object-threads <threads>
Number of threads to use for uploading full objects. Default is 10.
- --segment-threads <threads>
Number of threads to use for uploading object segments. Default is 10.
- --header <header>
Set request headers with the syntax header:value. This option may be repeated. Example -H "content-type:text/plain".
- --use-slo
When used in conjunction with --segment-size will create a Static Large Object instead of the default Dynamic Large Object.
- --object-name <object-name>
Upload file and name object to <object-name> or upload dir and use <object-name> as object prefix instead of folder name
