Automated UI testing with Cypress and Shippable

Some background info before my recipe. If you’re in hurry, scroll down directly to Install Instructions.

Most of the time UI automated testing relies on Selenium, and several other node packages. Basically the setup I was using 2 years ago to test a react app had dependencies like this:

• mocha (testing framework)
• chai (for assertions)
• chai-as-promised (cleaner assertions)
• selenium-webdriver (for browser automation)
• phantomjs-prebuilt (headless browser, to run the tests in the command line)
• chromedriver (to run the tests using Chrome, good for debug)
• coffee-script (cleaner sintax)

Not to mention that phantomjs was discontinued in favor of Chrome Headless.

Anyway, the whole shebang was not that stable: as Selenium is an external framework relying on a client/server communication, and phantomjs was not a real browser, several times our working tests will return false negatives.

Then, let’s say “hi!” to cypress and its incredible package dependency:

• cypress

Yeap, that’s it: it’s a self contained node package, using Electron to run our tests with, in a headless way or with a friendly UI:

Install Instructions

Just run in your project folder:

npm install cypress --save-dev

After that, we’ll change some default behaviors:

• we want nice commands to start cypress
• we don’t want to keep cypress in our project root folder
• we don’t want to save videos on test success
• we don’t want cypress assets to be saved on git

Nice commands to start cypress

Open your package.json and add to the scripts section:

"scripts": {
  "cypress:open": "cypress open",
  "cypress": "cypress run"
},

So you’ll be able to call npm run cypress to run the tests in the command line and npm run cypress:open to open the test runner UI.

Change some defaults

By default, cypress will get installed to our project root folder, in a sub-folder named cypress. I moved mine to react/spec/system.

Additionaly I’ve changed my configuration to not save videos on test success and I’ve added a default value for the baseUrl (you can read more configuration values in the configuration documentation).

For all these changes, we need to create a cypress.json file in our project folder with the the following content:

{
  "baseUrl": "http://localhost",
  "fixturesFolder": "react/spec/system/fixtures",
  "integrationFolder": "react/spec/system/integration",
  "pluginsFile": "react/spec/system/plugins/index.js",
  "screenshotsFolder": "react/spec/system/screenshots",
  "supportFile": "react/spec/system/support/index.js",
  "videosFolder": "react/spec/system/videos",
  "video": false
}

Don’t save cypress assets on git

I still kept the cypress default behavior to take screenshots of testing failure when running them in the command line.

But I don’t want to save the screenshots (or even videos, if temporarily enabled) on git. So edit your .gitignore and add these lines:

# Cypress test results assets
react/spec/system/screenshots/*
react/spec/system/videos/*

Write a simple test

If you want more background on this, visit the cypress Write Your First Test page.

Create a file react/spec/system/integration/sample_spec.js with the following content:

describe('My First Test', function() {
  it('Visits Home', function() {
    cy.visit('/')
  })
})

Note that we’re opening the / page, which will visit the baseUrl defined in cypress.json.

As you can guess, visit opens a URL. Check the cypress API commands documentation about all available commands. You can create custom commands in react/spec/system/support/commands.js file.

Now check if our simple sample is working with npm run cypress:open and see the magic happening!

Continuous Integration with Shippable

Now let’s configure our Continuous Integration. I like Shippable for that, but any other CI tool will have a similar setup.

Why I like Shippable? Their Hosted SaaS is easy to configure (just a shippable.yml in your project), it has free or affordable tiers, builtin support for GitHub and BitBucket and works like a charm.

Anyway, like I said, probably any other CI tool will have similar setup. You can check in cypress CI integration page recipes for Jenkins, TrevisCI, CircleCI and several others.

This post is not about to teach how to setup Shippable, but, in a quick overview, for a first setup, you need to follow these steps to enable in your project:

1. Enable a project
2. Configure build triggers
3. Build your shippable.yml

Now, assuming you have Shippable working with a node_js setup, let’s change the ci section of shippable.yml to add our cypress tests:

ci:

  # Redirect the output to `/dev/null` due long outputs freezing the build
  - npm install --quiet >> /dev/null

  # Starting app server
  - rails server -d

  # cypress specs
  - npm run cypress

You can see that I’m using Shippable with Ubuntu in a rails application. If you’re using a different OS with Shippable, change the apt-get line to match your OS package manager. You can replace the line to start the web server too (rails server -d) by your application server.