Migrating From Jenkins To Codeship - Testing

In every project there is a need to test and confirm that functionality is working as planned. This takes the the form of numerous practices to determine: from unit, integration, or user interface testing to smoke testing, black box, or others. Some of these forms of testing are simple while some are complex in their configuration, management, and usage.

Testing with Jenkins Systems

In the Jenkins system, there are a number of ways to initiate tests to run. There are plugins and other respective hacks, scripts, and webhooks to get unit tests or related actions to trigger. Jenkins does a pretty decent job performing unit tests or basic integration tests if you keep it simple.

However, having an environment in which tests are run in a singular pipeline can lead to side effects that require timely troubleshooting and problem-solving to fix. Because of this static server state versus running containers and rebuilding the environment every time, certain things get set, configured, or otherwise altered that do not mimic production environments.

The complexity of managing this in a set server build with Jenkins and the respective tests needed to manage that becomes a difficult task unto itself.

Testing your Applications with Codeship Pro as Compared to Jenkins

With Codeship Pro, the testing options open up massively, without any plugins or extra configuration.

Codeship Pro’s simple configuration files (codeship-services.yml and codeship-steps.yml) can be run on the container instances that are completing the actual build. By association, not only does this provide testing insight into the Codeship build that is run but it also provides the ability to run the build locally using the Codeship Pro Jet CLI.

Simply set up the codeship-steps.yml file as shown.

- name: deluge
  service: deluge_build
  command: echo 'A first build step for whatever.'
- name: deluge_tests
  service: go_tests
  command: go test
- name: deluge_another_test
  service: ruby_tests
  command: ruby tests

Now whenever the build is run, whether locally or via commit to the Codeship build process, the tests will be run and results displayed. For a recent setup, here are some of the results from a Codeship Pro build, when it is run with Codeship. The picture shows Codeship Pro’s web interface for the build output.

Codeship Pro Build Example

As mentioned, with the Jet CLI tool you can run your builds locally before you push to your source code management system or to help you debug. Here you can see the output in the console after using the jet steps command.

Codeship Pro Jet CLI Output Example

With these steps, Codeship provides a means of unit test coverage for the build. This doesn’t provide a local build option, but tests could of course simply be executed manually via the Jet CLI, IDE, or whatever method for the local option.

Testing your Applications with Codeship Basic as Compared to Jenkins

The most common testing mechanism is executing the unit tests within a code base. Let’s take a quick look at the setup of tests in Codeship Basic.

First, set the command within your project’s tests section of the interface.

Codeship Basic Setup Example

Here is a quick code sample test for a Node.js application that we have here.

var assert = require('assert');

describe('Where the important things happen', function () {
  describe('where the functional is good all', function () {
    it('should be good', function () {
      assert.equal(true, true);

GitHub Testing Integration

When migrating to Codeship, once you added test commands in the Project Settings, as shown in the previous section, then any code pushed to your repository on any branch will trigger the source code management system’s interface to identify if the merge will build successfully.

For example, click the Compare & pull request button on GitHub.

Codeship Basic Repo Integration Example

Next you’ll see the pending check running.

Codeship Basic PR Example

Finally the build succeeds, and the green check box displays. The Merge pull request button turns green.

Codeship Basic Repo Status Example


In this document we’ve shown some of the power and options for testing on Codeship Basic and Codeship Pro. It opens the door to add unit, integration, smoke, or other systems testing. Even further testing could be implemented by pushing deployment to utilize remote UI testing or other means.

Now that you know how to migrate your tests from Jenkins to Codeship we suggest looking into our other walk-throughs:

You also might be interested in downloading these migration guides as PDFs. You can do so here.

Need More Help?

Get in touch if you need more help, or post on Stack Overflow using the tag #Codeship.

  • Ask The Helpdesk A Question
  • Code Examples And Sample Projects
    • Was This Article Helpful?