This article is about Codeship Pro.

Using Multiple Repositories

You'll need roughly 4 minutes to read this article.

To follow this tutorial on your own computer, please install the jet CLI locally first.

Multi-Repo Build Pipelines

When using microservices, you may have several repos you want to build together or in sequences. There are two primary setups to accomplish this, and for each setup there are a couple of methods you can use.

  • You can create separate CI/CD projects for each repository, allowing them to deploy independently of one another. Then, you can either have each project bring in all of your individual component repos for full-stack testing, or you can create a separate, agnostic project just for the purposes of combining them for full-stack testing.

  • You can create a single project that pulls from all of your repos, deploying all components together whenever the master project is updated.

Once you decide which workflow you like, there are two main ways to combine your repos for full-stack testing.

Using Image Pushes

If you opt to create separate CI/CD projects for each individual component, you can have each component’s pipeline end with an image push to an image registry.

Then, either with a separate project or as a part of the build process for each component, you can pull in the latest images for each component to build and test your combined application.

Cloning Multiple Repos During A Build

In addition to the above method using images in a registry, you can also clone the repos themselves during the build process - either with a separate project or as part of the build for each individual component.

Creating Your SSH Key

The first step of cloning a repo into your build process is to create your SSH key authentication. Another option for cloning from GitHub would be to use OAuth instead of an SSH key.

The following command will create two files in your local repository. keyfile.rsa contains your private key that we will encrypt and put into your repository. This encrypted file will be decrypted on Codeship as part of your build. The second file can be added to services that you want to access.

ssh-keygen -t rsa -b 4096 -C "" -f keyfile.rsa

Now you have to copy the content of keyfile.rsa into a build arguments file, in this case we’ll name it sshkey.args. Make sure to replace newlines with \n so the whole SSH key is in one line. The following is an example of ssh_key.args.


And here is a docker run command that will do the work of replacing your newlines with \n

docker run -it --rm -v ${PATH_TO_PRIVATE_KEY}:/key alpine sed -E ':a;N;$!ba;s/\r?\n/\\n/g' /key

After preparing the sshkey.args file we can encrypt it with the local CLI. Follow the encryption tutorial to turn the sshkey.args file into a sshkey.args.encrypted file.

You can then add it to a service with the encrypted_args_file option. It will be automatically decrypted on Codeship.

  build: .
    encrypted_args_file: sshkey.args.encrypted

Using The Key During The Build

Before running a command that needs SSH available make sure to run the following commands in that container. They will set up the SSH key so you can access external services.

mkdir -p "$HOME/.ssh"
echo -e $PRIVATE_SSH_KEY >> $HOME/.ssh/id_rsa

Cloning Another Repo

Once you have the SSH authentication working, then you’ll just need a Dockerfile that grabs git and clones the repo you want:

FROM ubuntu:latest
RUN apt-get update && apt-get install -y ca-certificates git-core ssh
RUN mkdir -p "$HOME/.ssh"
RUN echo -e $PRIVATE_SSH_KEY >> $HOME/.ssh/id_rsa
RUN git clone repo:repo.git

Note that this is a high-level, directional example and will require additional work to use in a live build.

Building With Repos Via API

Codeship Pro does not yet have an API for triggering builds, but we will be launching an API for this purpose later in 2017.