Estimated Reading Time: 15 mins
To follow this tutorial on your own computer, please install the jet CLI locally first.
Please see the example in the codeship-tool examples repository for how to run a registry during the build process and push a new image to this registry.
Get the AES encryption key from the General settings page of your Codeship project and save it to your repository as codeship.aes (adding it to the .gitignore file is a good idea).
Login to the Docker Hub locally and save the encrypted credentials file to your repository.
docker login
# follow the onscreen instructions
# ...
jet encrypt ${HOME}/.docker/config.json dockercfg.encrypted
git add dockercfg.encrypted
git commit -m "Adding encrypted credentials for docker push"
codeship-services.yml file. It will probably look similar to the following:app:
build:
image: username/repository_name
dockerfile: Dockerfile
Configure your codeship-steps.yml file. Your service image_name can differ from the repository defined in your steps file. Your image will be tagged and pushed based on the push step.
If you don’t want to push the image for each build, add a tag entry to the below step and it will only be run on that specific branch or git tag.
- service: app
type: push
image_name: username/repository_name
registry: https://index.docker.io/v1/
encrypted_dockercfg_path: dockercfg.encrypted
Note that if you are using Mac OSX, the newer versions of Docker have changed to store credentials in the OSX keychain rather than in a configuration file.
To get the appropriate authentication file on OSX, you will need to remove the credsStore line from Docker’s config.json to disable Keychain storage, re-run docker login and then use the values it then generates in your updated dockercfg as shown above.
Prerequisites: You will need to have a robot account for your Quay repository. Please see the documentation on Robot Accounts for Quay.io on how to set it up for your repository.
Note that permissions can be set per robot account, so if you are seeing authentication failures you should check that the individual robot account being used is configured with appropriate access.
Once you have configured the robot account, download the .dockercfg file for this account, by heading over to the Robots Account tab in your settings. From there, either credit a new robot account or click on an existing robot account. In the pop-up window, the Docker Configuration tab will have an option to download an auth.json file that you will download and use in the next steps.
Get the AES encryption key from the General settings page of your Codeship project and save it to your repository as codeship.aes (again, adding it to the .gitignore file is a good idea).
Encrypt the credentials for accessing Quay.io by running the following command and commit the encrypted file to your repository.
jet encrypt dockercfg dockercfg.encrypted
git add dockercfg.encrypted
git commit -m "Adding encrypted credentials for docker push"
Add the robot user to the Quay.io repository with the appropriate permissions (at least Write).
Configure your codeship-services.yml file. It will probably look similar to the following:
app:
build:
image: quay.io/username/repository_name
dockerfile: Dockerfile
Configure your codeship-steps.yml file. Your service image_name can differ from the repository defined in your steps file. Your image will be tagged and pushed based on the push step.
If you don’t want to push the image for each build, add a tag entry to the below step and it will only be run on that specific branch or git tag.
- service: app
type: push
image_name: quay.io/username/repository_name
registry: quay.io
encrypted_dockercfg_path: dockercfg.encrypted
Along with being able to push to private registries, you can also push to tags other than latest. To do so, simply add the tag as part of your push step using the image_tag declaration.
- service: app
type: push
image_name: quay.io/username/repository_name
image_tag: dev
registry: quay.io
encrypted_dockercfg_path: dockercfg.encrypted
This image_tag field can contain a simple string, or be part of a Go template. You can compose your image tag from a variety of provided values. Note that because we use Go for our Regex support, negative regexes and conditional regexes are not supported.
ProjectID (the Codeship defined project ID)BuildID (the Codeship defined build ID)RepoName (the name of the repository according to the SCM)Branch (the name of the current branch)CommitID (the commit hash or ID)CommitMessage (the commit message)CommitDescription (the commit description, see footnote)CommitterName (the name of the person who committed the change)CommitterEmail (the email of the person who committed the change)CommitterUsername (the username of the person who committed the change)Time (a golang Time object of the build time)Timestamp (a unix timestamp of the build time)StringTime (a readable version of the build time)StepName (the user defined name for the push step)ServiceName (the user defined name for the service)ImageName (the user defined name for the image)Ci (defaults to true)CiName (defaults to codeship)To tag your image based on the Commit ID, use the string "{{ .CommitID }}". You can template together multiple keys into a tag by simply concatenating the strings: "{{ .CiName }}-{{ .Branch }}". Be careful about using raw values, however, since the resulting string will be stripped of any invalid tag characters.
Using Codeship Pro, you can easily use private Docker images as base images for your containers.
Similar to pushing images, you need to save your encrypted dockercfg file in the registry and reference it for any step using private base images (or for groups of steps). You also need to specify in your Dockerfile and your codeship-services.yml file which images from your registry you want to use.
You can also pull images from multiple registries within the same build, as long as you provide all necessary credentials.
To download a private base image, you’ll need to provide your account credentials so that your Docker builds can authenticate with the registry and access your image. You’ll probably want to encrypt these credentials to keep them secure.
To encrypt your image registry credentials:
Get the AES encryption key from the General settings page of your Codeship project and save it to your registry as codeship.aes (adding it to the .gitignore file is a good idea).
Run the jet encrypt command against your image registry dockercfg file. See below for guidance on creating this file depending on which image registry you’re using.
To use an image from Dockerhub, you’ll first need to login locally and mv the credentials file to your registry.
docker login
# follow the onscreen instructions
# ...
jet encrypt ${HOME}/.docker/config.json dockercfg.encrypted
# or (depending if you are on an older version of Docker)
jet encrypt ${HOME}/dockercfg dockercfg.encrypted
Next, you’ll want to configure your Dockerfile. It will probably look similar to the following:
FROM username/registry_name
# ...
You will also need to configure your codeship-steps.yml file to provide your account credentials on every step that uses an image from your Dockerhub account.
- service: app
command: /bin/true
encrypted_dockercfg_path: dockercfg.encrypted
Quay.io has slightly different permissions and specification requirement than Dockerhub, so make sure to provide the necessary specifications for accessing your Quay repos.
First configure a robot account
Once you have configured the robot account, download the dockercfg file for this account, by heading over to the Robots Account tab in your settings, clicking the gear icon, selecting View Credentials and hitting the download button.
Save the file as dockercfg in your registry, encrypt it and add the unencrypted version to
.gitignore
echo "dockercfg" >> .gitignore
jet encrypt dockercfg dockercfg.encrypted
dockercfg.encrypted as well as the .gitignore file:git add dockercfg.encrypted .gitignore
git commit -m "Adding encrypted credentials for docker push"
With your permissions defined for your Quay.io account, you’ll now want to make sure your image definitions specify the Quay.io registry.
In your Dockerfile:
FROM quay.io/username/registry_name
# ...
You will also need to configure your codeship-steps.yml file to provide your account credentials on every step that using an image from your Quay.io registry.
- service: app
command: /bin/true
encrypted_dockercfg_path: dockercfg.encrypted
You can also access images from privately or self-hosted registries.
In your Dockerfile:
FROM registryURL/username/registry_name
# ...
You will also need to configure your codeship-steps.yml file to provide your account credentials on every step that using a private base image.
- service: app
command: /bin/true
encrypted_dockercfg_path: dockercfg.encrypted
You might see an error like this when pulling a private base image using your encrypted dockercfg file:
Failed to parse dockercfg: invalid character '___' after top-level value
This means that either your dockercfg has a syntax problem or that it was encrypted with an incorrect or incomplete AES key, or an AES key from another project.
Try deleting your dockercfg and your AES key, then re-downloading the AES key and re-encrypting the dockercfg file.
This error means that the encrypted file was unable to be decrypted locally. This is because the AES key is missing.
See the instructions above for downloading your AES key locally to address this issue.
If you need a key regenerated, you can submit a ticket to the help desk from your account. Keep in mind that this will leave current encrypted credentials and environmental variables invalid for future builds on Codeship until they are re-encrypted using the new key.
Codeship supports using private registries for pulling and pushing images by allowing static dockercfg credentials to be encrypted as part of your codebase. Due to an increasing number of container registry vendors using different methods to generate Docker temporary credentials, we have added support for custom dockercfg credential generation. By using a custom service within your list of Codeship services, you can integrate with a standard dockercfg generation container for your desired provider. Codeship will provide a basic set of images supporting common providers, however you will also be able to use custom images to integrate with custom registries.
Taking advantage of this feature is fairly simple. First off, add a service using the image for your desired registry provider to your codeship-services.yml file. You can add any links, environment variables or volumes you need, just like with a regular service.
# codeship-services.yml
app:
build:
dockerfile: ./Dockerfile
image: myservice.com/myuser/myapp
myservice_generator:
image: codeship/myservice-dockercfg-generator
encrypted_env: creds.encrypted
To use this generator service, simply reference it using the dockercfg_service field in lieu of an encryped_dockercfg in your steps or services file.
# codeship-steps.yml
- type: push
service: app
registry: myservice.com
image_name: myservice.com/myuser/myapp
dockercfg_service: myservice_generator
Codeship will run the service to generate a dockercfg as needed. Under the hood, Codeship will launch a container with the specified image, mount a volume and request a dockercfg be written to a temporary file on that volume. As soon as the dockercfg is read, it is deleted from the filesystem. The container logs from this generator service will be visible on the command line and in the Codeship interface.
Keep in mind that different generator images may have different requirements. If your generator image, for example, performs a docker login, you may need to set add_docker: true in order to use it. Be sure to read the documentation for your specific provider when implementing these generator services.
The majority of container registries use standard, static credentials, and even using a custom authentication proxy, most of the time you can generate a compatible static dockercfg and encrypt it for use within your pipeline. Should you need to use dynamic credentials, or some other method of securely retrieving static credentials during the CI/CD process, you’ll need to use a generator service. Luckily creating your own image is simple, the only requirement is that the image writes the dockercfg to a path provided via a CMD argument.
$ docker run -it -v /tmp/:/opt/data mygenerator /opt/data/dockercfg
$ cat /tmp/dockercfg # read generated dockercfg
The container must be provided with any credentials or configuration needs to generate a dockercfg via environment variables. When the image is being used in a build, this information is provided via the service definition in the codeship-services.yml file. Codeship will run the container any time it needs to generate the dockercfg, however the cost of this can be mitigated by locally caching credentials using a host volume mount defined in the service definition. The container image would be responsible for managing this cached folder, and checking the presence and validitity of credentials in the cache before returning them.
Here is a list of the standard dockercfg generators we support. If you don’t see your desired provider on this list, please reach out to support, or create it yourself.
Get in touch if you need more help, or post on Stack Overflow using the tag #Codeship.