A build input provides source content for builds to operate on. There are several ways to provide source in OpenShift Origin. In order of precedence:
Different inputs can be combined into a single build. As the inline Dockerfile takes precedence, it can overwrite any other file named Dockerfile provided by another input. Binary input and Git repositories are mutually exclusive inputs.
Input secrets are useful for when you do not want certain resources or
credentials used during a build to be available in the final application image
produced by the build, or want to consume a value that is defined in a Secret
resource. External artifacts can be used to pull in additional files that are
not available as one of the other build input types.
Whenever a build is run:
A working directory is constructed and all input content is placed in the working directory. For example, the input Git repository is cloned into the working directory, and files specified from input images are copied into the working directory using the target path.
The build process changes directories into the contextDir, if one is
defined.
The inline Dockerfile, if any, is written to the current directory.
The content from the current directory is provided to the build process
for reference by the
Dockerfile, custom builder logic, or
assemble script. This means any input content that resides outside the
contextDir will be ignored by the build.
The following example of a source definition includes multiple input types and an explanation of how they are combined. For more details on how each input type is defined, see the specific sections for each input type.
source:
git:
uri: https://github.com/openshift/ruby-hello-world.git (1)
images:
- from:
kind: ImageStreamTag
name: myinputimage:latest
namespace: mynamespace
paths:
- destinationDir: app/dir/injected/dir (2)
sourcePath: /usr/lib/somefile.jar
contextDir: "app/dir" (3)
dockerfile: "FROM centos:7\nRUN yum install -y httpd" (4)| 1 | The repository to be cloned into the working directory for the build. |
| 2 | /usr/lib/somefile.jar from myinputimage will be stored in <workingdir>/app/dir/injected/dir. |
| 3 | The working directory for the build will become <original_workingdir>/app/dir. |
| 4 | A Dockerfile with this content will be created in <original_workingdir>/app/dir, overwriting any existing file with that name. |
When a dockerfile value is supplied, the content of this field
will be written to disk as a file named Dockerfile. This is
done after other input sources are processed, so if the input
source repository contains a Dockerfile in the root directory,
it will be overwritten with this content.
The typical use for this field is to provide a Dockerfile to a
Docker
strategy build.
The source definition is part of the spec section in the BuildConfig:
source:
dockerfile: "FROM centos:7\nRUN yum install -y httpd" (1)| 1 | The dockerfile field contains an inline Dockerfile that will be built. |
Additional files can be provided to the build process via images. Input images
are referenced in the same way the From and To image targets are defined.
This means both container images and
image
stream tags can be referenced. In conjunction with the image, you must provide
one or more path pairs to indicate the path of the files or directories to copy
the image and the destination to place them in the build context.
The source path can be any absolute path within the image specified. The destination must be a relative directory path. At build time, the image will be loaded and the indicated files and directories will be copied into the context directory of the build process. This is the same directory into which the source repository content (if any) is cloned. If the source path ends in /. then the content of the directory will be copied, but the directory itself will not be created at the destination.
Image inputs are specified in the source definition of the BuildConfig:
source:
git:
uri: https://github.com/openshift/ruby-hello-world.git
images: (1)
- from: (2)
kind: ImageStreamTag
name: myinputimage:latest
namespace: mynamespace
paths: (3)
- destinationDir: injected/dir (4)
sourcePath: /usr/lib/somefile.jar (5)
- from:
kind: ImageStreamTag
name: myotherinputimage:latest
namespace: myothernamespace
pullSecret: mysecret (6)
paths:
- destinationDir: injected/dir
sourcePath: /usr/lib/somefile.jar| 1 | An array of one or more input images and files. |
| 2 | A reference to the image containing the files to be copied. |
| 3 | An array of source/destination paths. |
| 4 | The directory relative to the build root where the build process can access the file. |
| 5 | The location of the file to be copied out of the referenced image. |
| 6 | An optional secret provided if credentials are needed to access the input image. |
|
This feature is not supported for builds using the Custom Strategy. |
When specified, source code will be fetched from the location supplied.
If an inline Dockerfile is supplied, it will overwrite the Dockerfile
(if any) in the contextDir of the Git repository.
The source definition is part of the spec section in the BuildConfig:
source:
git: (1)
uri: "https://github.com/openshift/ruby-hello-world"
ref: "master"
contextDir: "app/dir" (2)
dockerfile: "FROM openshift/ruby-22-centos7\nUSER example" (3)| 1 | The git field contains the URI to the remote Git repository of the
source code. Optionally, specify the ref field to check out a specific Git
reference. A valid ref can be a SHA1 tag or a branch name. |
| 2 | The contextDir field allows you to override the default location inside
the source code repository where the build looks for the application source
code. If your application exists inside a sub-directory, you can override the
default location (the root folder) using this field. |
| 3 | If the optional dockerfile field is provided, it should be a string
containing a Dockerfile that overwrites any Dockerfile that may exist in the
source repository. |
If the ref field denotes a pull request, the system will use a git fetch operation
and then checkout FETCH_HEAD.
When no ref value is provided, OpenShift Origin performs a shallow clone
(--depth=1). In this case, only the files associated with the most recent
commit on the default branch (typically master) are downloaded. This results
in repositories downloading faster, but without the full commit history. To
perform a full git clone of the default branch of a specified repository, set
ref to the name of the default branch (for example master).
If your Git repository can only be accessed using a proxy, you can define the
proxy to use in the source section of the BuildConfig. You can configure
both a HTTP and HTTPS proxy to use. Both fields are optional. Domains for which
no proxying should be performed can also be specified via the NoProxy field.
|
Your source URI must use the HTTP or HTTPS protocol for this to work. |
source:
git:
uri: "https://github.com/openshift/ruby-hello-world"
httpProxy: http://proxy.example.com
httpsProxy: https://proxy.example.com
noProxy: somedomain.com, otherdomain.comCluster administrators can also configure a global proxy for Git cloning using Ansible.
|
For Pipeline strategy builds, given the current restrictions with the Git
plug-in for Jenkins, any Git operations through the Git plug-in will not
leverage the HTTP or HTTPS proxy defined in the |
Builder pods require access to any Git repositories defined as source for a build. Source clone secrets are used to provide the builder pod with access it would not normally have access to, such as private repositories or repositories with self-signed or untrusted SSL certificates.
The following source clone secret configurations are supported.
|
You can also use combinations of these configurations to meet your specific needs. |
Builds are run with the builder service account, which must have access to any source clone secrets used. Access is granted with the following command:
$ oc secrets link builder mysecret
|
Limiting secrets to only the service accounts that reference them is disabled by
default. This means that if |
When a BuildConfig is created, OpenShift Origin can automatically populate its
source clone secret reference. This behaviour allows the resulting Builds to
automatically use the credentials stored in the referenced Secret to
authenticate to a remote Git repository, without requiring further
configuration.
To use this functionality, a Secret containing the Git repository credentials
must exist in the namespace in which the BuildConfig will later be created.
This Secret must additionally include one or more annotations prefixed with
build.openshift.io/source-secret-match-uri-. The value of each of these
annotations is a URI pattern, defined as follows. When a BuildConfig is created
without a source clone secret reference and its Git source URI matches a URI
pattern in a Secret annotation, OpenShift Origin will automatically insert a
reference to that Secret in the BuildConfig.
A URI pattern must consist of:
a valid scheme (*://, git://, http://, https:// or ssh://).
a host (* or a valid hostname or IP address optionally preceded by *.).
a path (/* or / followed by any characters optionally including *
characters).
In all of the above, a * character is interpreted as a wildcard.
|
URI patterns only match Git source URIs which are conformant to RFC3986. For example, https://github.com/openshift/origin.git. They do not match the alternate SSH style that Git also uses. For example, [email protected]:openshift/origin.git. It is not valid to attempt to express a URI pattern in the alternate style, or to include a username/password component in a URI pattern. |
If multiple Secrets match the Git URI of a particular BuildConfig,
OpenShift Origin will select the secret with the longest match. This allows for
basic overriding, as in the following example.
The following fragment shows two partial source clone secrets, the first
matching any server in the domain mycorp.com accessed by HTTPS, and the second
overriding access to servers mydev1.mycorp.com and mydev2.mycorp.com:
kind: Secret
apiVersion: v1
metadata:
name: matches-all-corporate-servers-https-only
annotations:
build.openshift.io/source-secret-match-uri-1: https://*.mycorp.com/*
data:
...
kind: Secret
apiVersion: v1
metadata:
name: override-for-my-dev-servers-https-only
annotations:
build.openshift.io/source-secret-match-uri-1: https://mydev1.mycorp.com/*
build.openshift.io/source-secret-match-uri-2: https://mydev2.mycorp.com/*
data:
...Add a build.openshift.io/source-secret-match-uri- annotation to a pre-existing
secret using:
$ oc annotate secret mysecret \
'build.openshift.io/source-secret-match-uri-1=https://*.mycorp.com/*'
Source clone secrets can be added manually to a build configuration by adding a
sourceSecret field to the source section inside the BuildConfig and
setting it to the name of the secret that you created (basicsecret, in this
example).
apiVersion: "v1"
kind: "BuildConfig"
metadata:
name: "sample-build"
spec:
output:
to:
kind: "ImageStreamTag"
name: "sample-image:latest"
source:
git:
uri: "https://github.com/user/app.git"
sourceSecret:
name: "basicsecret"
strategy:
sourceStrategy:
from:
kind: "ImageStreamTag"
name: "python-33-centos7:latest"|
You can also use the $ oc set build-secret --source bc/sample-build basicsecret |
Defining Secrets in the BuildConfig provides more information on this topic.
If the cloning of your application is dependent on a .gitconfig file,
then you can create a secret that contains it, and then add
it to the builder service account, and then your BuildConfig.
To create a secret from a .gitconfig file:
$ oc create secret generic <secret_name> --from-file=<path/to/.gitconfig>
|
SSL verification can be turned off if [http]
sslVerify=false
|
Basic authentication requires either a combination of --username and
--password, or a token to authenticate against the SCM server.
Create the secret first before using the user name and password to access the
private repository:
$ oc create secret generic <secret_name> \
--from-literal=username=<user_name> \
--from-literal=password=<password> \
--type=kubernetes.io/basic-auth
To create a basic authentication secret with a token:
$ oc create secret generic <secret_name> \
--from-literal=password=<token> \
--type=kubernetes.io/basic-auth
SSH key based authentication requires a private SSH key.
The repository keys are usually located in the $HOME/.ssh/ directory, and
are named id_dsa.pub, id_ecdsa.pub, id_ed25519.pub, or id_rsa.pub by
default. Generate SSH key credentials with the following command:
$ ssh-keygen -t rsa -C "[email protected]"
|
Creating a passphrase for the SSH key prevents OpenShift Origin from building. When prompted for a passphrase, leave it blank. |
Two files are created: the public key and a corresponding private key (one of
id_dsa, id_ecdsa, id_ed25519, or id_rsa). With both of these in place,
consult your source control management (SCM) system’s manual on how to upload
the public key. The private key is used to access your private repository.
Before using the SSH key to access the private repository, create the secret first:
$ oc create secret generic <secret_name> \
--from-file=ssh-privatekey=<path/to/ssh/private/key> \
--type=kubernetes.io/ssh-auth
The set of TLS certificate authorities that are trusted during a git clone
operation are built into the OpenShift Origin infrastructure images. If your Git
server uses a self-signed certificate or one signed by an authority not trusted
by the image, you have several options.
Create a secret with a CA certificate file (recommended).
A secret containing a CA certificate in a key named ca.crt will
automatically be used by Git to trust your self-signed or otherwise un-trusted
TLS certificate during the git clone operation. Using this method is
significantly more secure than disabling Git’s SSL verification, which accepts
any TLS certificate that is presented.
$ oc create secret generic mycert --from-file=ca.crt=</path/to/file> (1)
| 1 | The key name ca.crt must be used. |
Disable Git TLS verification.
You can disable Git’s TLS verification by setting the GIT_SSL_NO_VERIFY
environment variable to true in the appropriate strategy section of your build
configuration. You can use the
oc set env command to
manage BuildConfig environment variables.
Below are several examples of how you can combine the above methods for creating source clone secrets for your specific needs.
To create an SSH-based authentication secret with a .gitconfig file:
$ oc create secret generic <secret_name> \
--from-file=ssh-privatekey=<path/to/ssh/private/key> \
--from-file=<path/to/.gitconfig> \
--type=kubernetes.io/ssh-auth
To create a secret that combines a .gitconfig file and CA certificate:
$ oc create secret generic <secret_name> \
--from-file=ca.crt=<path/to/certificate> \
--from-file=<path/to/.gitconfig>
To create a basic authentication secret with a CA certificate file:
$ oc create secret generic <secret_name> \
--from-literal=username=<user_name> \
--from-literal=password=<password> \
--from-file=ca-cert=</path/to/file> \
--type=kubernetes.io/basic-auth
To create a basic authentication secret with a .gitconfig file:
$ oc create secret generic <secret_name> \
--from-literal=username=<user_name> \
--from-literal=password=<password> \
--from-file=</path/to/.gitconfig> \
--type=kubernetes.io/basic-auth
To create a basic authentication secret with a .gitconfig file and CA certificate file:
$ oc create secret generic <secret_name> \
--from-literal=username=<user_name> \
--from-literal=password=<password> \
--from-file=</path/to/.gitconfig> \
--from-file=ca-cert=</path/to/file> \
--type=kubernetes.io/basic-auth
Streaming content in binary format from a local file system to the builder is
called a Binary type build. The corresponding value of
BuildConfig.spec.source.type is Binary for such builds.
This source type is unique in that it is leveraged solely based on your use of
the oc start-build.
|
Binary type builds require content to be streamed from the local file system, so automatically triggering a binary type build (e.g. via an image change trigger) is not possible, because the binary files cannot be provided. Similarly, you cannot launch binary type builds from the web console. |
To utilize binary builds, invoke oc start-build with one of these options:
--from-file: The contents of the file you specify are sent as a binary stream
to the builder. You can also specify a URL to a file. Then, the builder stores
the data in a file with the same name at the top of the build context.
--from-dir and --from-repo: The contents are archived and sent as a binary
stream to the builder. Then, the builder extracts the contents of the archive
within the build context directory. With --from-dir, you can also specify
a URL to an archive, which will be extracted.
--from-archive: The archive you specify is sent to the builder, where it is
extracted within the build context directory. This option
behaves the same as --from-dir; an archive is created on your host first,
whenever the argument to these options is a directory.
In each of the above cases:
If your BuildConfig already has a Binary source type defined, it will
effectively be ignored and replaced by what the client sends.
If your BuildConfig has a Git source type defined, it is dynamically
disabled, since Binary and Git are mutually exclusive, and the data in
the binary stream provided to the builder takes precedence.
Instead of a file name, you can pass a URL with HTTP or HTTPS schema to
--from-file and --from-archive. When using --from-file with a URL, the
name of the file in the builder image is determined by the Content-Disposition
header sent by the web server, or the last component of the URL path if the
header is not present. No form of authentication is supported and it is not
possible to use custom TLS certificate or disable certificate validation.
When using oc new-build --binary=true, the command ensures that the
restrictions associated with binary builds are enforced. The resulting
BuildConfig will have a source type of Binary, meaning that the only
valid way to run a build for this BuildConfig is to use oc
start-build with one of the --from options to provide the requisite binary
data.
The dockerfile and contextDir source options have
special meaning with binary builds.
dockerfile can be used with any binary build source. If dockerfile is
used and the binary stream is an archive, its contents serve as a replacement
Dockerfile to any Dockerfile in the archive. If dockerfile is used with the
--from-file argument, and the file argument is named dockerfile, the value
from dockerfile replaces the value from the binary stream.
In the case of the binary stream encapsulating extracted archive content, the
value of the contextDir field is interpreted as a subdirectory within the
archive, and, if valid, the builder changes into that subdirectory before
executing the build.
In some scenarios, build operations require credentials to access dependent resources, but it is undesirable for those credentials to be available in the final application image produced by the build. You can define input secrets for this purpose.
For example, when building a Node.js application, you can set up your private mirror for Node.js modules. In order to download modules from that private mirror, you have to supply a custom .npmrc file for the build that contains a URL, user name, and password. For security reasons, you do not want to expose your credentials in the application image.
This example describes Node.js, but you can use the same approach for adding SSL certificates into the /etc/ssl/certs directory, API keys or tokens, license files, and more.
To add an input secret to an existing BuildConfig:
Create the secret, if it does not exist:
$ oc create secret generic secret-npmrc \
--from-file=.npmrc=<path/to/.npmrc>
This creates a new secret named secret-npmrc, which contains the base64 encoded content of the ~/.npmrc file.
Add the secret to the source section in the existing BuildConfig:
source:
git:
uri: https://github.com/openshift/nodejs-ex.git
secrets:
- secret:
name: secret-npmrcTo include the secret in a new BuildConfig, run the following command:
$ oc new-build \
openshift/nodejs-010-centos7~https://github.com/openshift/nodejs-ex.git \
--build-secret secret-npmrc
During the build, the .npmrc file is copied into the directory where the
source code is located. In OpenShift Origin S2I builder images, this
is the image working directory, which is set using the WORKDIR instruction
in the Dockerfile. If you want to specify another directory, add a
destinationDir to the secret definition:
source:
git:
uri: https://github.com/openshift/nodejs-ex.git
secrets:
- secret:
name: secret-npmrc
destinationDir: /etcYou can also specify the destination directory when creating a new
BuildConfig:
$ oc new-build \
openshift/nodejs-010-centos7~https://github.com/openshift/nodejs-ex.git \
--build-secret “secret-npmrc:/etc”
In both cases, the .npmrc file is added to the /etc directory of the build environment. Note that for a Docker strategy the destination directory must be a relative path.
When using a Source strategy, all defined input secrets are copied to their
respective destinationDir. If you left destinationDir empty, then the
secrets are placed in the working directory of the builder image.
The same rule is used when a destinationDir is a relative path; the secrets
are placed in the paths that are relative to the image’s working directory. The
destinationDir must exist or an error will occur. No directory paths are
created during the copy process.
|
Currently, any files with these secrets are world-writable (have |
When using a Docker strategy, you can add all defined input secrets into your
container image using the
ADD and
COPY instructions in
your Dockerfile.
If you do not specify the destinationDir for a secret, then the files will be
copied into the same directory in which the Dockerfile is located. If you
specify a relative path as destinationDir, then the secrets will be copied
into that directory, relative to your Dockerfile location. This makes the
secret files available to the Docker build operation as part of the context
directory used during the build.
FROM centos/ruby-22-centos7 USER root ADD ./secret-dir /secrets COPY ./secret2 / # Create a shell script that will output secrets when the image is run RUN echo '#!/bin/sh' > /secret_report.sh RUN echo '(test -f /secrets/secret1 && echo -n "secret1=" && cat /secrets/secret1)' >> /secret_report.sh RUN echo '(test -f /secret2 && echo -n "relative-secret2=" && cat /secret2)' >> /secret_report.sh RUN chmod 755 /secret_report.sh CMD ["/bin/sh", "-c", "/secret_report.sh"]
|
Users should normally remove their input secrets from the final application image so that the secrets are not present in the container running from that image. However, the secrets will still exist in the image itself in the layer where they were added. This removal should be part of the Dockerfile itself. |
When using a Custom strategy, all the defined input secrets are available
inside the builder container in the /var/run/secrets/openshift.io/build
directory. The custom build image is responsible for using these secrets
appropriately. The Custom strategy also allows secrets to be defined as
described in Custom Strategy Options.
There is no technical difference between existing strategy secrets and the input secrets. However, your builder image might distinguish between them and use them differently, based on your build use case.
The input secrets are always mounted into the
/var/run/secrets/openshift.io/build directory or your builder can parse the
$BUILD environment variable, which includes the full build object.
It is not recommended to store binary files in a source repository. Therefore, you may find it necessary to define a build which pulls additional files (such as Java .jar dependencies) during the build process. How this is done depends on the build strategy you are using.
For a Source build strategy, you must put appropriate shell commands into
the assemble script:
#!/bin/sh
APP_VERSION=1.0
wget http://repository.example.com/app/app-$APP_VERSION.jar -O app.jar#!/bin/sh
exec java -jar app.jar|
For more information on how to control which assemble and run script is used by a Source build, see Overriding Builder Image Scripts. |
For a Docker build strategy, you must modify the Dockerfile and invoke
shell commands with the
RUN instruction:
FROM jboss/base-jdk:8 ENV APP_VERSION 1.0 RUN wget http://repository.example.com/app/app-$APP_VERSION.jar -O app.jar EXPOSE 8080 CMD [ "java", "-jar", "app.jar" ]
In practice, you may want to use an environment variable for the file location
so that the specific file to be downloaded can be customized using an
environment variable defined on the BuildConfig, rather than updating the
Dockerfile or
assemble script.
You can choose between different methods of defining environment variables:
Using the .s2i/environment file (only for a Source build strategy)
Providing
explicitly using oc start-build --env (only for builds that are triggered
manually)
You can supply builds with a .docker/config.json file with valid credentials for private Docker registries. This allows you to push the output image into a private Docker registry or pull a builder image from the private Docker registry that requires authentication.
|
For the OpenShift Origin Docker registry, this is not required because secrets are generated automatically for you by OpenShift Origin. |
The .docker/config.json file is found in your home directory by default and has the following format:
auths:
https://index.docker.io/v1/: (1)
auth: "YWRfbGzhcGU6R2labnRib21ifTE=" (2)
email: "[email protected]" (3)| 1 | URL of the registry. |
| 2 | Encrypted password. |
| 3 | Email address for the login. |
You can define multiple Docker registry entries in this file. Alternatively, you
can also add authentication entries to this file by running the docker login
command. The file will be created if it does not exist.
Kubernetes provides
Secret objects, which can
be used to store configuration and passwords.
Create the secret from your local .docker/config.json file:
$ oc create secret generic dockerhub \
--from-file=.dockerconfigjson=<path/to/.docker/config.json> \
--type=kubernetes.io/dockerconfigjson
This generates a JSON specification of the secret named dockerhub and
creates the object.
Once the secret is created, add it to the builder service account. Each build is
run with the builder role, so you must give it access your secret with the
following command:
$ oc secrets link builder dockerhub
Add a pushSecret field into the output section of the BuildConfig and
set it to the name of the secret that you created, which in the above example
is dockerhub:
spec:
output:
to:
kind: "DockerImage"
name: "private.registry.com/org/private-image:latest"
pushSecret:
name: "dockerhub"You can also use the oc set build-secret command to set the push secret on
the build configuration:
$ oc set build-secret --push bc/sample-build dockerhub
Pull the builder container image from a private Docker registry by specifying the
pullSecret field, which is part of the build strategy definition:
strategy:
sourceStrategy:
from:
kind: "DockerImage"
name: "docker.io/user/private_repository"
pullSecret:
name: "dockerhub"You can also use the oc set build-secret command to set the pull secret on
the build configuration:
$ oc set build-secret --pull bc/sample-build dockerhub
|
This example uses |