You will need roughly 23 minutes to read this article.
If you need to reset your AES key you can do so by visiting Project Settings > General and clicking Reset project AES key.
If you are using private images, you will need to authenticate with the image registries to pull and push from your account.
On Docker Hub, you can use public images without authentication. Note that as of November 1, 2020, Docker introduced rate limits. Based on your usage you may need to upgrade your Docker plan.
The most common way authenticate with image registries is to provide your account credentials via an encrypted dockercfg file. This keeps your credentials secure while allowing you to push and pull from private registry accounts.
This encryption happens with our Jet CLI tool, similar to using encrypted environment variables. To encrypt your image registry credentials:
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "your_auth_string",
"email": "your_email"
}
}
}
Get your AES encryption key from the General settings page of your CodeShip project and save it to your registry as codeship.aes (adding it to the .gitignore file is a good idea so that it does not end up in your repository).
Run the jet encrypt command against your image registry dockercfg file. This typically looks like jet encrypt dockercfg dockercfg.encrypted. but you can name it whatever you’d like.
The newly encrypted dockercfg file will be committed to your repo and used in your codeship-services.yml and codeship-steps.yml files to authenticate with your registry on pull and push.
If you are using Apple macOS, the newer versions of Docker store credentials in the macOS keychain rather than in a configuration file.
We created a workaround to help you get the appropriate authentication file created on macOS:
credentials.env:DOCKER_USERNAME=...
DOCKER_PASSWORD=...
DOCKER_REGISTRY=https://index.docker.io/v1/
Note that if your Docker Hub account has 2FA enabled, you will need to create a personal access token and use that token for DOCKER_PASSWORD above.
dockercfg file:docker run -it --rm \
--env-file=credentials.env \
-v "$(pwd):/opt/data/" \
-v "/var/run/docker.sock:/var/run/docker.sock" \
codeship/dockercfg-generator /opt/data/dockercfg
Note that the DOCKER_REGISTRY endpoint can be changed to reference a registry other than Docker Hub, such as Quay.io, as long as the registry authenticates with the docker login command.
Due to an increasing number of container registry vendors using different methods to generate Docker temporary credentials, we also have support for custom dockercfg credential generation at runtime. By using a custom service within your list of CodeShip services, you can integrate with a standard dockercfg generation container for your desired provider.
Taking advantage of this feature is fairly simple. First off, add a service using the image for your desired registry provider to your codeship-services.yml file. You can add any links, encrypted environment variables or volumes you need, just like with a regular service.
# codeship-services.yml
app:
build:
dockerfile: Dockerfile
image: myservice/myuser/myapp
myservice_generator:
image: codeship/myservice-dockercfg-generator
encrypted_env_file:
- creds.encrypted
To use this generator service, simply reference it using the dockercfg_service field in lieu of an encryped_dockercfg in your steps or services file.
# codeship-steps.yml
- type: push
service: app
registry: myservice.com
image_name: myservice.com/myuser/myapp
dockercfg_service: myservice_generator
CodeShip will run the service to generate a dockercfg as needed.
Note that in these examples we show the registry credential directives used on both Services and Steps at different points. We allow for either configuration in the case of pulling an image from a private registry. In the case of pushing an image to a private registry the registry credential directive must be included on the push step, though.
After setting up your registry authentication using the encrypted dockercfg file method shown above, you will want to configure your codeship-services.yml:
app:
build:
image: username/repository_name
dockerfile: Dockerfile
The image defined above will be tagged and pushed based on the push step in your codeship-steps.yml file:
- service: app
type: push
image_name: username/repository_name
encrypted_dockercfg_path: dockercfg.encrypted
After setting up your registry authentication using the encrypted dockercfg file method shown above, you will want to configure your codeship-services.yml or your Dockerfile to reference the image you are pulling:
FROM username/registry_name
# ...
You will also need to configure your codeship-steps.yml file to provide your account credentials via the encrypted dockercfg file on every step that uses an image from your Docker Hub account.
- service: app
command: /bin/true
encrypted_dockercfg_path: dockercfg.encrypted
To use the encrypted dockercfg file authentication method with Quay.io, you will first need to have create robot account with the requires permissions for your Quay repository. Please see the documentation on Robot Accounts for Quay.io on how to set it up for your repository.
Note that permissions can be set per robot account, so if you are seeing authentication failures you should check that the individual robot account being used is configured with appropriate access.
Next, you will need to download the .dockercfg file for this account by heading over to the Robots Account tab in your settings. From there, either credit a new robot account or click on an existing robot account. In the pop-up window, the Docker Configuration tab will have an option to download an auth.json file.
Once you have encrypted this auth.json file using the encrypted dockercfg method, you will want to configure your codeship-services.yml:
app:
build:
image: quay.io/username/repository_name
dockerfile: Dockerfile
encrypted_dockercfg_path: dockercfg.encrypted
Next, you will need to configure your codeship-steps.yml file.
- service: app
type: push
image_name: quay.io/username/repository_name
registry: quay.io
encrypted_dockercfg_path: dockercfg.encrypted
To pull images from private Quay.io accounts, you will need to configure your Quay robot account permissions and authentication via the encrypted dockercfg file as discussed in the above instructions regarding push steps.
After setting up your registry authentication, you will want to configure your codeship-services.yml or your Dockerfile to reference the image you are pulling:
FROM quay.io/username/registry_name
# ...
You will also need to configure your codeship-steps.yml file to provide your account credentials on every step that using an image from your Quay.io registry.
- service: app
command: /bin/true
encrypted_dockercfg_path: dockercfg.encrypted
Pushing to a custom or self-hosted registry is similar to using Docker Hub or Quay.io.
You will want to specify your registry URL and provide your registry credentials in an encrypted dockercfg file on a push step in your codeship-steps.yml:
- service: app
type: push
image_name: your_registry/your_image
registry: your_registry_url
encrypted_dockercfg_path: dockercfg.encrypted
You can access images from privately or self-hosted registries with non-standard registry locations.
In your Dockerfile:
FROM your_registry_url/username/your_image
# ...
You will also need to configure your codeship-steps.yml file to provide your account credentials on every step that uses a private base image:
- service: app
command: /bin/true
encrypted_dockercfg_path: dockercfg.encrypted
To push to Google GCR in your builds, you will want to make use of our service generator method for registry authentication. This is because Google uses a token-based login system.
We maintain an image you can easily add to your push step to generate these credentials for you.
First, you will need to provide your Google credentials as encrypted environment variables for your Google authentication service. Also note that our image name must include your GCR registry path for your push step to authenticate. Here is an example codeship-services.yml:
myapp:
build:
image: gcr.io/my_org/myapp
dockerfile_path: Dockerfile.test
dockercfg_generator:
image: codeship/gcr-dockercfg-generator
add_docker: true
encrypted_env_file:
- gcr.env.encrypted
Now, you will need a push step in your codeship-steps.yml with the dockercfg_service directive. This directive runs the service specified, when it is pushing, to generate the necessary authentication token.
Note that GCR requires the fully registry path in our image name, and the account you are authenticating with Google must have the necessary account permissions as well. Here is an example codeship-steps.yml:
- name: Push To GCR
service: myapp
type: push
image_name: gcr.io/my_org/my_app
registry: https://gcr.io
dockercfg_service: dockercfg_generator
Learn more about using Google Cloud with CodeShip Pro.
To pull images from Google GCR, you will need to provide the image, including the registry path, as well as use the service generator for authentication in your codeship-services.yml.
For example:
myapp:
image: gcr.io/my_org/my_app
dockercfg_service: dockercfg_generator
dockercfg_generator:
image: codeship/gcr-dockercfg-generator
add_docker: true
encrypted_env_file:
- gcr.env.encrypted
This will use the image we maintain for Google authentication to generate credentials on image pull. Note that you will need to have your AWS credentials set via the encrypted environment variables for the generator service, and that the AWS account you are authenticating with will need appropriate IAM permissions.
Learn more about using Google Cloud with CodeShip Pro.
To push to AWS ECR in your builds, you will want to make use of our service generator method for registry authentication. This is because AWS uses a token-based login system.
We maintain an image you can easily add to your push step to generate these credentials for you.
First, you will need to provide your AWS credentials as encrypted environment variables for your AWS authentication service. Also note that our image name must include your ECR registry path for your push step to authenticate. Here is an example codeship-services.yml:
myapp:
build:
image: 870119404647.dkr.ecr.us-east-1.amazonaws.com/myapp
dockerfile_path: Dockerfile.test
dockercfg_generator:
image: codeship/aws-ecr-dockercfg-generator
add_docker: true
encrypted_env_file:
- aws.env.encrypted
Now, you will need a push step in your codeship-steps.yml with the dockercfg_service directive. This directive runs the service specified, when it is pushing, to generate the necessary authentication token.
ECR requires the fully registry path in our image name, and the account you are authenticating with AWS must have the necessary IAM permissions as well. Here is an example codeship-steps.yml:
- name: Push To ECR
service: myapp
type: push
image_name: 870119404647.dkr.ecr.us-east-1.amazonaws.com/myapp
registry: https://870119404647.dkr.ecr.us-east-1.amazonaws.com
dockercfg_service: dockercfg_generator
Note that to authenticate with ECR, you will need to provide the following environment variables via your encrypted environment variables in order to authenticate with AWS successfully:
AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYAWS_REGIONLearn more about using AWS with CodeShip Pro.
To pull images from ECR, you will need to provide the image, including the registry path, as well as use the service generator for authentication in your codeship-services.yml.
For example:
myapp:
image: 870119404647.dkr.ecr.us-east-1.amazonaws.com/my_image
dockercfg_service: dockercfg_generator
dockercfg_generator:
image: codeship/aws-ecr-dockercfg-generator
add_docker: true
encrypted_env_file:
- aws.env.encrypted
This will use the image we maintain for AWS authentication to generate credentials on image pull. Note that you will need to have your AWS credentials set via the encrypted environment variables for the generator service, and that the AWS account you are authenticating with will need appropriate IAM permissions.
Learn more about using AWS with CodeShip Pro.
To push to IBM Cloud in your builds, you will want to make use of our service generator method for registry authentication. This is because IBM Cloud uses a CLI-based login system.
We maintain an image you can easily add to your push step to generate these credentials for you.
First, you will need to provide your IBM Cloud API key as BLUEMIX_API_KEY via an encrypted environment variables for your IBM Cloud authentication service. Also note that our image name must include your IBM Cloud registry path for your push step to authenticate. Here is an example codeship-services.yml:
app:
build:
image: your_org/your_image
dockerfile_path: ./Dockerfile
bluemix_dockercfg:
image: codeship/ibm-bluemix-dockercfg-generator
add_docker: true
encrypted_env_file:
- bluemix.env.encrypted
Now, you will need a push step in your codeship-steps.yml with the dockercfg_service directive. This directive runs the service specified, when it is pushing, to generate the necessary authentication token.
Note that IBM Cloud requires the fully registry path in our image name, and the account you are authenticating with must have at least one namespace configured with the IBM Cloud Container Registry product:
- name: Push To IBM Cloud
service: app
type: push
image_name: registry.ng.bluemix.net/codeship/codeship-testing
registry: registry.ng.bluemix.net
dockercfg_service: bluemix_dockercfg
To see a full example of using IBM Cloud Container Registry with CodeShip Pro, visit our example repository.
To pull images from a IBM Cloud Container Registry, you will need to provide the image, including the registry path, as well as use the service generator for authentication in your codeship-services.yml.
For example:
base:
build:
image: registry.ng.bluemix.net/your_namespace/image
path: ./base
dockerfile_path: Dockerfile
dockercfg_service: bluemix_dockercfg
bluemix_dockercfg:
image: codeship/ibm-bluemix-dockercfg-generator
add_docker: true
encrypted_env_file:
- bluemix.env.encrypted
This will use the image we maintain for IBM Cloud authentication to generate credentials on image pull. Note that you will need to have the BLUEMIX_API_KEY variable set via encrypted environment variables for the generator service.
To see a full example of using IBM Cloud Container Registry with CodeShip Pro, visit our example repository.
To push to Azure Container Service in your builds, you will want to make use of our service generator method for registry authentication.
We maintain an image you can easily add to your push step to generate these credentials for you.
First, you will need to add the following credentials as encrypted environment variables:
AZURE_USERNAME - Your username of the Admin user of the registryAZURE_PASSWORD - The password associated with the above admin userAZURE_REGISTRY - The URL of the registry you want to access (in the form of NAME.azurecr.io)Note that you must enable the Admin user for your the specific Azure Container Registry, which you can do via the Access keys settings page of the registry you want to push the image to.
Also note that our image name must include your Azure Container Service registry path for your push step to authenticate. Here is an example codeship-services.yml:
app:
build:
image: codeship.azurecr.io/codeship-testing
dockerfile_path: ./Dockerfile
azure_dockercfg:
image: codeship/azure-dockercfg-generator
add_docker: true
encrypted_env_file:
- azure.env.encrypted
Now, you will need a push step in your codeship-steps.yml with the dockercfg_service directive. This directive runs the service specified, when it is pushing, to generate the necessary authentication token.
Note that the Azure Container Service requires the fully registry path in our image name:
- service: app
type: push
tag: master
image_name: codeship.azurecr.io/codeship-testing
registry: codeship.azurecr.io
dockercfg_service: azure_dockercfg
To see a full example of using the Azure Container Service with CodeShip Pro, visit our example repository.
To pull images from Azure Container Service, you will need to provide the image, including the registry path, as well as use the service generator for authentication in your codeship-services.yml.
For example:
app:
build:
image: codeship.azurecr.io/codeship-testing
dockerfile_path: ./Dockerfile
dockercfg_service: azure_dockercfg
azure_dockercfg:
image: codeship/azure-dockercfg-generator
add_docker: true
encrypted_env_file:
- azure.env.encrypted
This will use the image we maintain for Azure Container Service authentication to generate credentials on image pull.
Note that you will need the following credentials set via the encrypted environment variables for the generator service:
AZURE_USERNAME - Your username of the Admin user of the registryAZURE_PASSWORD - The password associated with the above admin userAZURE_REGISTRY - The URL of the registry you want to access (in the form of NAME.azurecr.io)Note that you must enable the Admin user for your the specific Azure Container Registry, which you can do via the Access keys settings page of the registry you want to push the image to.
To see a full example of using the Azure Container Service with CodeShip Pro, visit our example repository.
Along with being able to push to private registries, you can also push to tags other than latest. To do so, simply add the tag as part of your push step using the image_tag declaration.
- service: app
type: push
image_name: quay.io/username/repository_name
image_tag: dev
registry: quay.io
encrypted_dockercfg_path: dockercfg.encrypted
This image_tag field can contain a simple string, or be part of a Go template. You can compose your image tag from a variety of provided values. Note that because we use Go for our Regex support, negative regexes and conditional regexes are not supported.
ProjectID (the CodeShip defined project ID)BuildID (the CodeShip defined build ID)RepoName (the name of the repository according to the SCM)Branch (the name of the current branch)CommitID (the commit hash or ID)CommitMessage (the commit message)CommitDescription (the commit description, see footnote)CommitterName (the name of the person who committed the change)CommitterEmail (the email of the person who committed the change)CommitterUsername (the username of the person who committed the change)Time (a golang Time object of the build time)Timestamp (a unix timestamp of the build time)StringTime (a readable version of the build time)StepName (the user defined name for the push step)ServiceName (the user defined name for the service)ImageName (the user defined name for the image)Ci (defaults to true)CiName (defaults to codeship)To tag your image based on the Commit ID, use the string "{{ .CommitID }}". You can template together multiple keys into a tag by simply concatenating the strings: "{{ .CiName }}-{{ .Branch }}". Be careful about using raw values, however, since the resulting string will be stripped of any invalid tag characters.
dockercfgYou might see an error like this when pulling a private base image using your encrypted dockercfg file:
Failed to parse dockercfg: invalid character '___' after top-level value
This means that either your dockercfg has a syntax problem or that it was encrypted with an incorrect or incomplete AES key, or an AES key from another project.
Try deleting your dockercfg and your AES key, then re-downloading the AES key and re-encrypting the dockercfg file.
This error means that the encrypted file was unable to be decrypted locally. This is because the AES key is missing.
See the instructions above for downloading your AES key locally to address this issue.
If you need a key regenerated, you can submit a ticket to the help desk from your account. Keep in mind that this will leave current encrypted credentials and environmental variables invalid for future builds on CodeShip until they are re-encrypted using the new key.
If you don’t want to push the image for each build, add a tag entry to the below step and it will only be run on that specific branch or git tag.
Contact our support team or post on Stack Overflow using the tag #codeship. Did you check the status page and changelog?
There are also several code examples and sample projects available for you to get started with.
Does this article need improvement? If so, please send feedback or submit a pull request!