You will need roughly 25 minutes to read this article.
Missing versions only improved the integration with our hosted platform and have no user facing changes / bug fixes.
foowill be presented as
foo (2). This kind of counter based de-duplication is only enforced between sibling steps. Two steps name
foowhich are not siblings will remain named
CI_BUILD_APPROVEDboolean environment variable, indicating whether or not manual approval has been given.
identitytokenfield in docker config. Docker config generators which produce config containing an identity token rather than a username and password will authenticate successfully with the target repository. Prior to this change such authentication flows would result in an authentication failure. This reinstates our ability to support platforms such as IBM Cloud (see https://github.com/codeship-library/ibm-bluemix-utilities).
jet completioncommand to generate bash/zsh completions for Jet
jet encryptwith a checksum to verify both file and key integrity.
dockercfg_servicedirective, will be cached for up to 10 minutes. This will ensure we don’t exhaustively make calls to Docker login authentication providers. It also improves our ability to collapse concurrent builds of services to a single build call, when creating services which rely on these directives.
links. For more information please see https://documentation.codeship.com/pro/builds-and-configuration/services/#deprecated-keys-1 for more info.
healthchecksdirective for healthchecks built into a Docker image. For images that contain a healthcheck, we will poll for availability every 1 second for up to 60 minutes before proceeding. You can find the health polling status in your logs.
depends_onservice directive. Now with support for the
depends_onkey in the
codeship-services.yml. This behaves similarly to links, in the sense that it describes the dependency of one service to another. The difference is that it does not support any kind of additional networking changes. The legacy
linksdirective supported network aliasing and “service discovery” environment variables. These are now not necessary as each service is discoverable via DNS using it’s service name as the address.
server: image: python command: "python3 -m http.server" app: build: . command: "curl server:8000" depends_on: - server
The above example is all that is required from a dependency perspective, for a service
app to boot and talk to a dependent service
server over the network via DNS.
jet updatecommand to allow Jet to update itself to the most recent version
jet cleanupcommand for removing resources left behind by previous jet commands. For example, when the
--no-removeflag is used it leaves containers and networks around. The cleanup subcommand will remove these Docker resources.
jet validatecommand to validate codeship-services.yml and codeship-steps.yml files
servicestop-level keys to be present for services files to be seen as versioned.
--linkport mappings are ordered. Such that the mapping
<alias>_PORTalways contains the lowest port mapping described. As per Docker’s legacy links spec.
Updates to jet cli to improve final state messaging The jet command now uses an appropriate color and symbol when a step finishes to represent the steps final state.
Jet CLI now defaults to a colored log driver
--log-driver v1 to use the original driver
Updating to Docker API 1.29 (17.05-ce) You can now use multi-stage builds with the Jet CLI, as well as build arguments in the FROM instruction in your Dockerfile.
Full stops in step and service names.
service and step names can now contain full stops. For example, // in codeship-services.yml golang.service: image: golang
// in codeship-steps.yml step.one: service: golang.service command: go run main.go
This version includes a deprecation warning for mounting volumes that don’t exist. Also, it includes a warning a warning for mounting volumes via absolute paths. Volumes must exist and be mounted via relative path. Check out the docs here: https://documentation.codeship.com/pro/getting-started/docker-volumes/
You can now define a set of steps to be executed when any run, push or group step fails. This will allow users to define recovery options when steps do fail. Any time a regular step fails, the build will fail regardless of whether the resulting failure steps fail or succeed. You can also define recursive failure pipelines for steps that are part of a failure pipeline themselves. For more information see the Codeship Documentaion on Failure Pipelines
Fix for error when running jet run against Docker 1.13. Containers now wont get created with a trailing slash in their name when using jet run (does not affect jet steps).
Adding support for build args In codeship-services.yml, you can now declare build arguments to be consumed at image buildtime. Check out https://documentation.codeship.com/pro/getting-started/build-arguments/
Jet can now handle parsing all services indented in a services
section of the yml/json codeship services file. Configurations
with services indented under a
services section will be treated
the same as configurations where all services are defined on the
context key to the build directive
In your codeship-services.yml file, you can now specify a
context key that
contains a directory with a Dockerfile. This is identical to the existing
key but more in line with Docker’s naming conventions. The
path key will still
be supported. Note:
context must contain a directory, not a URL to a git repository.
Updating step status message for skipped steps
If a step is skipped due to a
on the step, the step status will now be shown as
This release updates how Jet catches circular dependecy issues to account for aliased links and volume_from entries. With this in place Jet will correctly error immediately after detecting a circular service dependency through links or volumes_from when an alias us used for the link or volume reference.
In a recent change we added validation requiring an “=” in all environment variables. We are removing this check but keeping new checks that error if non UTF8 characters are present in environment variables. This should help users detect issues with encrypted environment variables not decrypting as expected.
This change fixes a bug where the environment parsing for Codeship values was duplicating keys.
This change fixes some bugs around how we parse and validate environment. With this change we ensure we correctly parse empy values, strip out comments or blank entries, and error actively on invalid ones.
Jet will now ignore all blank strings added to environment instead of passing them into the container config. This includes those present if env files and those encrypted.
Jet will now validate environment variables before attempting to start containers. This helps protect against triggering errors in the underlying OCI implementation. Environment variables must contain only valid UTF8 characters and at least one ‘=’.
This release changes how we compile Jet to update from Go 1.6 to Go 1.7. Go 1.6 has some known issues on OSX Sierra, this changes fixes some issues with running Jet on OSX Sierra.
Removed remote-cache flag from Jet CLI.
This flag has been marked as deprecated since July. For local builds, rely on the local Docker image cache. Codeship no longer provides a remote cache during local build runs.
Jet log fetching has been updated to retry several times when the docker logs request returns a non error response. This allows us to capture logs for more than ~2 hours.
Adding support for cache fallback branch.
When using cached services, you can now specify which branch to use as a fallback
if no cached image is found for your service. The default fallback branch is
To override this, add
default_cache_branch to your service in the codeship-services.yml
file. If no value is present, the cache will fall back to
master. Note that caching only works
for builds run on our infrastructure and not with the Jet CLI.
Cached image import optimization.
If two services use an identical cached image and attempt to load the service in parallel, the first service is responsible for loading it, and other services will wait. Services will also check to see if the image ID is available on the build machine before downloading.
Updating error handling for S3 head object requests.
A bug in the AWS SDK for golang did not return the correct error code for S3 HeadObject request errors, which prevented new cached services from falling back to the master cache because an empty error was returned. We’ve updated the SDK to include the bugfix, and updated our internal handling of AWS errors.
With 1.9.0 we introduced a bug around how services were built. This version fixes that bug and allows images to be built in parallel.
This change updates how Jet pulls and builds images
to parallelize it where possible. This should mean your
jet steps and
jet run commands are faster, as well as
when you run
jet load with multiple services specified
This change updates the way cached images are stored and retrieved. Cached images are now imported and exported using Docker’s save and load mechanisms, instead of being pushed and pulled from a registry. For local builds using the Jet CLI, remote caching has been removed. Instead, rely on the local Docker image cache. For more information, see our documentation: https://codeship.com/documentation/docker/caching/
cached: trueservice images are created
bugs fixed: If the user specifies a tag for a cached image, Codeship will append our cache tag to the existing image tag using a hyphen as a delimiter.
This change ensures that images which are tagged and pushed for push steps or for caching are provided with a name which Docker considers to be valid.
This change updates the StringTime environment field to use RFC3339 format in order to remove spaces from the env variable. Values in this field will now be in the format “2006-01-02T15:04:05Z07:00”
This release changes the version of Go used to build the jet binary from 1.5.2 to 1.6.0.
This change adds a –remote-cache flag to allow users to enable the remote cache when using the jet tool. The remote cache will now be off by default.
This change disables the cache tag pushing and pulling with –no-cache=true. This means that if that flag is present, no cache docker images will be pulled or pushed.
This change ensures that all GCR and ECR dockercfg are refreshed before they expire.
This change updates what remote docker api we use to manage build containers from 1.18 to 1.22.
This release updates Jet to v1.0. No other changes are included.
Updating docker parameters to pass memory limits and CPU shares correctly for build containers. This will enforce CPU shares and memory limits defined in the services file when steps are being executed
Fixes a bug where links used alongside volumes would cause a race condition. Links can now be used alongside containers.
This change fixes an intermittent issues where jet would fail sometimes when cleaning up containers on exit. Jet will no longer panic and exit when an issue cleaning up occurs, but will handle the issue gracefully and continue.
This change fixes the regex used to parse image tag names and determine authentication using the dockercfg file.
This change updates the fork of fsouza/go-dockerclient codeship uses to be based off of 299d728486342c894e7fafd68e3a4b89623bef1d, pulling in some recent bug fixes from upstream.
Updating method used to transfer credentials to use relative volumes mounted between containers. This makes dockercfg credential transmission valid on the hosted CI service
Switching directories used to store generated dockercfg which was causing an error due to the directory not existing by default. Jet will now use a standard base directory. This should not be a user-visible change
The method jet uses to parse urls from image names is now more standardized, allowing all valid url characters to be used in the registry url. “foo-bar.com/myrepo/myimage” will now reference the registry “foo-bar.com”.
Jet users can now use a service from their codeship-services file to write a custom dockercfg, or use a secondary authentication method to unlock docker credentials.
You can now specify an encrypted dockercfg to use for a specific service in the services file, rather than in all relevant steps of the steps file. This should be particularly useful when using caching, or when using private base images.
With this change when caching pulls error, the build fails. This change turns those pulls into warnings and allow the build to continue.
If a user specifies an image tag template, but the generated image tag string is empty, Jet will now error. If a user does not specify an image_tag, or specifies an empty string Jet will use the default tag, “latest”.
This only affects non-ci jet runs. If the ci-branch flag is not specified by the user, then “master” is assumed as the cache source. This change ensures that if the branch is not provided then the blank tag is not pulled, or pushed once the image has been built.
This change ensures that configured volumes for containers are evaluated as full paths in the volume definition. This allows volumes to be mounted to remote or virtualized docker instances.
This change fixes several bugs around
You can now use a local Docker socket and it will be
mounted into the container. This change also stops converting
“tcp” host URLs to “https” when TLS is being used as
this is no longer supported by Docker.
This change fixes a bug where jet diverged from how docker-compose handled local relative directory mounts. With this change in place, you can mount folders within your build directory.
Jet will now tag images built for services based on the image name specified in the push step. This allows different services to build the images with different names, push to the same repository.
With this change jet will remove link and volume containers upon error, unless
--no-remove flag is specified.
In order to use regex matching, the regex must include one of the regex line matchers $ and ^. This reduces the chances of an intended strict tag match matching a substring of a longer tag.
This change sanitizes the cache tag generated since branches may contain characters which are not valid for tags
Jet will now detect circular dependencies in the services file. This matches the behaviour of docker-compose, which errors when reading a compose.yml which contains a circular loop.
This change ensures that jet honors termination signals during builds and pulls. The relevant build and pull will still continue, however jet will not build, pull or run any further services, and exit.
This change fixes some issues with how jet cleans up containers when shutting down and ignores container not found errors when trying to destroy containers.
Cache pulls which have registry errors will be ignored since they are triggered in cases by a not found response. Since a cache may not exist for a given build, this can cause false negatives.
This change removes the error raised when no credentials are found for a specific registry. This may not actually be an configuration issue.
When using cached jet builds, the cached image will now be pushed regardless of whether a push step was run.
Avoiding duplicate image pulls
This update synchronizes duplicate image pulls to avoid race conditions. Multiple services with the same image will result in a single image pull. Those services will wait for the initial pull to complete.
Fixing duplicate build bug
This update fixes a bug with the synchronization of duplicate image builds. Previously services which depended on an image already being built would immediately continue as soon as a previous, in-progress build was detected
Support regex in jet tags
Steps files can now use regex to determine what steps to run e.g. tag: “test-\w+” matches test-feature1, test-123. Exact string matches take precedent, and invalid regex will not error, only fail to match.
Fixing caching pull bug
This change extends the pull errors we discard during a cache pull. All pull errors relating to the image not being found are discarded. This change makes that selection process more compatible with different registries.
Updating jet to pull image cache before building
Jet will now determine an image cache to pull and use as a build cache during docker image builds. If an image exists with a tag “codeship-cache-$BRANCH”, it will be pulled and provide faster builds. Jet will also automatically push to this cache when an image is pushed.
Fixing AES key panic
Jet will now error when no AES key is found, but an encrypted dockercfg file is used
Fixing bug with private registry base images
This change adds support for Dockerfile image bases from private registries
Change default path for encryption key
We look for a codeship.aes in your project directory instead of $HOME
Fixing service load bug
This change resolve an issue where links and volumes were not loading correctly.
Fixing bug with detecting duplicate image building
This change fixes a bug where jet would give false positives when detecting cases where images are being rebuilt
Fixing bug with image building
This change fixes a bug where a single image was being build multiple times for a specific set of steps. All steps now use the same image which is built once
Fixing image unqiueness bug
This fixes a bug where image uniqueness checks were causing builds to error
Jet run errors correctly
Jet run will now exit if an error occurs while starting services
Already built error
Jet will now return an error if multiple services try to build the same image name
Updating Commit build environment variable to CommitID
This change renames Commit to CommitID to bring jet in line with other CI systems. The Commit ID/SHA can not be accessed via for push tags and via the CI_COMMIT_ID environment variable within containers
Exposing Build Environment Variables
The same variables used for Docker image tagging will now be available as environment variables in any containers running steps. We have also extended what variables are available. Here is a complete list:
For tagging within push steps the following variables will also be available, and the resulting template will be stripped of any invalid characters.
We have enabled a simple retry mechanism when pushing docker images. Often simultaneous image pushes cause problems, so when an image push step receives one of several listed errors from the registry, the push will be attempted again. You may see some duplicate logging where push steps ran several times due to registry errors.
Jet now comes in two flavours, statically linked and dynamically linked. With the dynamic version you can take advantage of local network resources like
.local DNS entries. The statically linked version will have a much better time running in reduced environments like scratch containers.
Enforcing shared services
Each step will now have one instance of a service, shared via the compose specification listed in your services files. Previously this created one container instance for each service using that dependency.
Fixing container naming bug
Invalid characters are now stripped out of auto-generated container names
Capturing more errors
Jet will also pick up link container creation errors
Jet now builds missing images when pushing and you no longer need to run dummy commands to force services to build in order to push them
Jet reads services/compose file more in line with
Previously in relation to service Dockerfile paths and the build declaration
docker-compose would differ.
docker-compose would allow you to specify a context directory for the entire build, not just the location of the Dockerfile, while jet would enforce using the top level directory as the context directory for the entire build.
Now you can specify a context directory for building an image by using the
build: custom_directory tag in your services file, which should match
docker-compose in behavior.
pushsteps. This can be either a hardcoded script or a golang
Templateobject referencing a number of variables. Please see the push steps documentation for the available variables.
jet version subcommand.
Fixed the Docker build context.
Contact our support team or post on Stack Overflow using the tag
#codeship. Did you check the status page and changelog?
There are also several code examples and sample projects available for you to get started with.