BlogEducation: Tutorials

Testing deployed GitLab pages using Cypress and GitLab CI

May 5, 2021

•

By Gleb Bahmutov

In this blog post we will deploy a static site to GitLab Pages, and then we will run end-to-end browser tests against the deployed site to make sure it is working as expected. The Cypress Test Runner makes it extremely simple to write full browser end-to-end (E2E) tests, while GitLab CI is a powerful platform for running such tests.

The application

Today's example is a static site built from a Markdown source file using VuePress. You can find the full source code at gitlab.com/bahmutov/gitlab-pages-example.

The example repository

The "docs" folder contains the Markdown files to be converted into the static HTML pages.

The "docs" folder keeps the source Markdown and the VuePress settings files

The VuePress settings in the docs/.vuepress/config.js file contains the main static site settings.

module.exports = {
  title: 'GitLab + Cypress = ❤️',
  description: 'Static site deployed to GitLab pages ...',
  base: '/gitlab-pages-example/',
  dest: 'public'
}

As of this writing, I am using VuePress v1.8 and Cypress v7. In the root package.json file I set up a few utility scripts for building and testing the site locally.

{
  "scripts": {
    "test": "cypress run",
    "start": "vuepress dev docs",
    "docs:build": "vuepress build docs",
    "dev": "start-test http://localhost:8080/gitlab-pages-example/ cy:open",
    "cy:open": "cypress open"
  },
  "dependencies": {
    "vuepress": "1.8.2"
  },
  "devDependencies": {
    "cypress": "7.2.0",
    "start-server-and-test": "1.12.1"
  }
}

When I want to start the site locally, and open the Cypress Test Runner I execute the npm run dev command via start-server-and-test utility.

The tests

We probably do not need a lot of tests in this case, just a smoke test to make sure the site is working. 

// cypress/integration/spec.js
/// <reference types="cypress" />

describe('Static site', () => {
  it('works', () => {
    cy.visit('/')
    cy.contains('h1', 'GitLab + Cypress = ❤️')
  })
})
The local static site tested locally

In our test the URL to visit is taken from the cypress.json file that contains the Cypress test settings

{
  "fixturesFolder": false,
  "supportFile": false,
  "pluginsFile": false,
  "baseUrl": "http://localhost:8080/gitlab-pages-example/"
}

The tests are very lightweight and do not need fixtures, custom commands, or any additional Cypress plugins.

The GitLab CI

Let's run the above test on GitLab CI. We need to repeat the same steps on CI as we do locally: we need to install NPM dependencies, start the server, and run the tests. The .gitlab-ci.yml file can be as simple as:

image: cypress/base:14.16.0

stages:
  - test

local-e2e:
  stage: test

  script:
    - npm ci --prefer-offline
    - npm start &
    - npx cypress run

Tip: read the Cypress on GitLab CI guide here

To speed up the installation of NPM dependencies on every CI run, we should preserve both the downloaded NPM modules folder and the Cypress binary folder. By default, NPM caches its downloaded modules in ~/.npm folder, and Cypress caches its binary in the ~/.cache folder. But the GitLab CI can only cache the folders local to its local working folder. Thus let's tell NPM and Cypress to use the folders relative to the current working folder.

image: cypress/base:14.16.0

# to cache both npm modules and Cypress binary we use environment variables
# to point at the folders we can list as paths in "cache" job settings
# see the Node caching advice for GitLab CI
# https://docs.gitlab.com/ee/ci/caching/#cache-nodejs-dependencies
# see Cypress caching documentation at
# https://on.cypress.io/caching
variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm"
  CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

stages:
  - test

local-e2e:
  stage: test

  cache:
    paths:
      - .npm
      - cache/Cypress

  script:
    - npm ci --prefer-offline
    - npm start &
    - npx cypress run

Every CI run should be faster from now on, as it reuses the previous install.

The GitLab Pages

Let's build the HTML site and deploy it to GitLab Pages. Our GitLab CI file will use another job to build and deploy; this job will run during the "deploy" stage, which automatically executes after the "test" stage successfully finishes.

image: cypress/base:14.16.0

stages:
  - test
  - deploy

# to cache both npm modules and Cypress binary we use environment variables
# to point at the folders we can list as paths in "cache" job settings
# see the Node caching advice for GitLab CI
# https://docs.gitlab.com/ee/ci/caching/#cache-nodejs-dependencies
# see Cypress caching documentation at
# https://on.cypress.io/caching
variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm"
  CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

local-e2e:
  stage: test

  cache:
    paths:
      - .npm
      - cache/Cypress

  script:
    - npm ci --prefer-offline
    - npm start &
    - npx cypress run

pages:
  # this job will deploy the built site to GitLab Pages
  # https://docs.gitlab.com/ee/user/project/pages/
  stage: deploy

  cache:
    paths:
      - .npm
      - cache/Cypress

  script:
    - npm ci --prefer-offline
    # build the static site for publishing
    - npm run docs:build

  artifacts:
    paths:
      # folder to be deployed
      - public

  # NOTE: only deploy from the main branch, otherwise
  # ANY branch will deploy, overwriting the page
  only:
    - main

You can find the deployed site at https://bahmutov.gitlab.io/gitlab-pages-example/.

The deployed site

Testing the deployed site

Every major transformation like building and deploying the site can break it; it could be the domain name, it could be the links, etc. Even though we have tested the site locally, we should also run all tests or some smoke tests against the deployed site. GitLab CI allows to run the tests against the deployed site during the "confidence-check" stage which automatically runs after the "deploy" stage successfully finishes and the site has been deployed.

image: cypress/base:14.16.0

stages:
  - test
  - deploy
  - confidence-check

# to cache both npm modules and Cypress binary we use environment variables
# to point at the folders we can list as paths in "cache" job settings
# see the Node caching advice for GitLab CI
# https://docs.gitlab.com/ee/ci/caching/#cache-nodejs-dependencies
# see Cypress caching documentation at
# https://on.cypress.io/caching
variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm"
  CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

local-e2e:
  stage: test

  cache:
    paths:
      - .npm
      - cache/Cypress

  script:
    - npm ci --prefer-offline
    - npm start &
    - npx cypress run

pages:
  # this job will deploy the built site to GitLab Pages
  # https://docs.gitlab.com/ee/user/project/pages/
  stage: deploy

  cache:
    paths:
      - .npm
      - cache/Cypress

  script:
    - npm ci --prefer-offline
    # build the static site for publishing
    - npm run docs:build

  artifacts:
    paths:
      # folder to be deployed
      - public

  # NOTE: only deploy from the main branch, otherwise
  # ANY branch will deploy, overwriting the page
  only:
    - main

e2e:
  stage: confidence-check

  cache:
    paths:
      - .npm
      - cache/Cypress

  script:
    - npm ci --prefer-offline
    - npx cypress run --config baseUrl=$CI_PAGES_URL

  # NOTE: only the main branch deploys the pages
  # thus we run the E2E tests after deploy on the main branch too
  only:
    - main

Notice how we pass the deployed site URL to the Cypress test runner using --config baseUrl=$CI_PAGES_URL command line argument; it will overwrite the baseUrl set in the cypress.json file.

Tip: to see all available GitLab CI environment variables I use the print-env module like this:

# prints all environment variables that start with CI_
- npx @bahmutov/print-env CI_

Here is the GitLab pipeline as it runs the tests, deploys the built site, and then tests the deploy.

The GitLab CI pipeline runs with test, deploy, and confidence check stages

Recording test runs

If the Cypress test fails, the screenshots and videos would offer tremendous help in debugging the failure. We can store the produced test artifacts on GitLab CI, but recording the test results on the Cypress Dashboard is much, much simpler to set up and use.

To record on the Dashboard we need to set the CYPRESS_RECORD_KEY environment variable (keep it secret!) through the GitLab CI settings page.

Pass the Cypress record key via an environment variable

We also add the --record parameter when calling Cypress from the CI script

local-e2e:
  ...
  - npx cypress run --record --tag local
 
e2e:
  ...
  - npx cypress run --record --config baseUrl=$CI_PAGES_URL --tag production

Every pull request runs the local E2E tests, and once merged to the main branch and deployed, the tests run against the "production" site. You can see the tagged run results on the project's public Dashboard page here.

Project's Cypress Dashboard page shows the latest runs

Tip: to get even more from Cypress tests, add the Cypress GitLab integration to your project.

See also