.. _s3_configuration_options:

Configuration Options
---------------------

Connector
^^^^^^^^^

``format.class``
  The format class to use when writing data to the store.

  * Type: class
  * Importance: high

``flush.size``
  Number of records written to store per partition before invoking file commits.

  * Type: int
  * Importance: high

``retry.backoff.ms``
  The retry backoff in milliseconds. This config is used to notify Kafka connect to retry delivering a message batch or performing recovery in case of transient exceptions.

  * Type: long
  * Default: 5000
  * Importance: low

``filename.offset.zero.pad.width``
  Width to zero pad offsets in store's filenames if offsets are too short in order to provide fixed width filenames that can be ordered by simple lexicographic sorting.

  * Type: int
  * Default: 10
  * Valid Values: [0,...]
  * Importance: low

``schema.cache.size``
  The size of the schema cache used in the Avro converter.

  * Type: int
  * Default: 1000
  * Importance: low

S3
^^

``s3.bucket.name``
  The S3 Bucket.

  * Type: string
  * Importance: high

``s3.region``
  The AWS region to be used the connector.

  * Type: string
  * Default: us-west-2
  * Valid Values: [us-gov-west-1, ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, eu-central-1, eu-west-1, eu-west-2, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2, cn-north-1]
  * Importance: medium

``s3.part.size``
  The Part Size in S3 Multi-part Uploads.

  * Type: int
  * Default: 104857600
  * Valid Values: [5242880,...,2147483647]
  * Importance: high

``s3.credentials.provider.class``
  Credentials provider or provider chain to use for authentication to AWS. By default the  connector uses 'DefaultAWSCredentialsProviderChain'.

  * Type: class
  * Default: com.amazonaws.auth.DefaultAWSCredentialsProviderChain
  * Valid Values: Any class implementing: interface com.amazonaws.auth.AWSCredentialsProvider
  * Importance: low

``s3.ssea.name``
  The S3 Server Side Encryption Algorithm.

  * Type: string
  * Default: ""
  * Importance: low

``s3.wan.mode``
  Use S3 accelerated endpoint.

  * Type: boolean
  * Default: false
  * Importance: medium

Storage
^^^^^^^

``storage.class``
  The underlying storage layer.

  * Type: class
  * Importance: high

``store.url``
  Store's connection URL, if applicable.

  * Type: string
  * Default: null
  * Importance: high

``topics.dir``
  Top level directory to store the data ingested from Kafka.

  * Type: string
  * Default: topics
  * Importance: high

``directory.delim``
  Directory delimiter pattern

  * Type: string
  * Default: /
  * Importance: medium

``file.delim``
  File delimiter pattern

  * Type: string
  * Default: +
  * Importance: medium

Partitioner
^^^^^^^^^^^

``partitioner.class``
  The partitioner to use when writing data to the store. You can use ``DefaultPartitioner``, which preserves the Kafka partitions; ``FieldPartitioner``, which partitions the data to different directories according to the value of the partitioning field specified in ``partition.field.name``; ``TimeBasedPartitioner``, which partitions data according to ingestion time.

  * Type: class
  * Default: io.confluent.connect.storage.partitioner.DefaultPartitioner
  * Importance: high
  * Dependents: ``partition.field.name``, ``partition.duration.ms``, ``path.format``, ``locale``, ``timezone``, ``schema.generator.class``

``partition.field.name``
  The name of the partitioning field when FieldPartitioner is used.

  * Type: string
  * Default: ""
  * Importance: medium

``partition.duration.ms``
  The duration of a partition milliseconds used by ``TimeBasedPartitioner``. The default value -1 means that we are not using ``TimeBasedPartitioner``.

  * Type: long
  * Default: -1
  * Importance: medium

``path.format``
  This configuration is used to set the format of the data directories when partitioning with ``TimeBasedPartitioner``. The format set in this configuration converts the Unix timestamp to proper directories strings. For example, if you set ``path.format='year'=YYYY/'month'=MM/'day'=dd/'hour'=HH/``, the data directories will have the format ``/year=2015/month=12/day=07/hour=15``.

  * Type: string
  * Default: ""
  * Importance: medium

``locale``
  The locale to use when partitioning with ``TimeBasedPartitioner``.

  * Type: string
  * Default: ""
  * Importance: medium

``timezone``
  The timezone to use when partitioning with ``TimeBasedPartitioner``.

  * Type: string
  * Default: ""
  * Importance: medium

``schema.generator.class``
  The schema generator to use with partitioners.

  * Type: class
  * Importance: high