BlogEducation: Tutorials

Fast Tests, Tiny Docker Image

August 8, 2018

•

By Gleb Bahmutov

If the Cypress Test Runner were a person, its best friend would be a person named Docker. Really, Cypress and Docker work so well together! For example, all our CI builds are using cypress-docker-images to include all necessary dependencies in order to successfully install and run Cypress tests. Just run npm ci and cypress run and you are good to go.

FROM cypress/base:10
RUN npm install
# or even better: RUN npm ci
RUN $(npm bin)/cypress run

We use this approach to run a lot of tests and examples ourselves, mostly on CircleCI, and we have detailed documentation for various other CIs.

We use this approach to run a lot of tests and examples ourselves, mostly on CircleCI, and we have detailed documentation for various other CIs.

In this blog post I will show you how to use Docker multi-stage builds to keep the production image size minimal while still running end-to-end tests using Cypress.

Note: you can find the complete source code for this blog post in this repo.

Multi-stage builds

I have described building several Docker images from a single Dockerfile in the blog post - Making small Docker image. In a nutshell, we are going to describe how to build several images and even copy some files from one image to another image to make sure we are using already tested files in production.

# first image will be our test image and will include Cypress
# and any end-to-end tests
FROM cypress/base:10 as TEST
COPY package.json .
RUN npm install
RUN npm test
...
# this is our output "production" image
# without any test dependencies
FROM busybox as PROD
# for example we can copy some files from TEST image to this production image
COPY --from=TEST /app/public /public
...

By making two images we can avoid installing dev dependencies in the production image, which keeps its size very small.

Caching is the hard problem

When designing a Docker image to run tests, we must carefully consider how to cache files, because this affects how and whether the Docker build will run individual commands. What we really want during the docker build . execution:

  • Only re-install NPM dependencies if the package.json (or package-lock.json) file changes
  • Re-run the Cypress tests only if our spec files or the source files change

Here is the simplest Dockerfile:

FROM cypress/base:10 as TEST
WORKDIR /app
# dependencies will be installed only if the package.json file changes
COPY package.json .
RUN npm install
# copy spec files and website files
COPY cypress cypress
COPY cypress.json .
COPY public public
# rerun E2E tests only if any of the previous files change
RUN npm test

You can play with this setup by changing source files and rerunning docker build .. Depending on the file changed you should see different outputs:

  • The first time you build the image, it will install Cypress and will execute the E2E tests
  • Any time you change package.json it will re-install Cypress and will execute the E2E tests
  • If you only change spec files inside the cypress/integration folder, the npm install command will be skipped because the package.json file has not changed, and the Docker build command is smart to pull a cached layer image. Only RUN npm test will be executed
  • If nothing changes, no commands will be run - and docker build . will finish quickly.

Always running E2E tests

You might want to always run E2E tests, even if the spec files have not changed. This is a very common case if you are testing an external site. In this case you will run into the Docker cache busting problem which only has hacky solutions 😖. I like defining a build argument before the command to bust on demand, and passing a new value whenever I want to rerun it.

ARG BUST=1
# if you run "docker build . --build-arg BUST=foo"
# it will bust this cache and it will rerun all commands from here
RUN npm test

To always rerun the npm test command I need to pass a new value to the BUST argument which I can do by using a timestamp

docker build . --build-arg BUST=$(date +%s)

Imperfect, but works.

Size savings

So how big of a size savings are we talking about? Is the complexity of multi-stage build worth it? We are using a very small busybox image to statically serve the public folder. Compared to cypress/base:10 + NPM dev dependencies (which includes unzipped Cypress) it is tiny.

docker images
REPOSITORY             TAG        IMAGE ID            CREATED             SIZE
none                   none       5271d608f7b2        About an hour ago   1.35GB
cypress/base           10         1613db8573fa        3 months ago        926MB
busybox                latest     22c2dd5ee85d        2 weeks ago         1.16MB

Holy moly!

Not only does cypress/base:10 weigh almost 1000x more than busy box, installing NPM dependencies and the Cypress binary adds another 400MB!

Here are the layers of the built TEST image to see where the megabytes are coming from

$ docker history 5271d608f7b2
IMAGE               CREATED             CREATED BY                                      SIZE
5271d608f7b2        About an hour ago   |1 HOSTNAME=1533380839 /bin/sh -c npm test      5.48MB
e4e87ef3eb74        About an hour ago   /bin/sh -c #(nop)  ARG HOSTNAME=1               0B
2805214ce399        About an hour ago   /bin/sh -c ls -la public                        0B
0f221e552006        About an hour ago   /bin/sh -c ls -la                               0B
aae6148fa25a        About an hour ago   /bin/sh -c #(nop) COPY dir:7b879359721ac4fd1…   70B
15c188cccb9a        About an hour ago   /bin/sh -c #(nop) COPY file:a04afd1a50dad2d5…   3B
a83b62ac2ca6        About an hour ago   /bin/sh -c #(nop) COPY dir:f9f6ce2869336983a…   2.57kB
75ef1af2b83d        About an hour ago   /bin/sh -c npm ci                               416MB
aa88b6688a96        About an hour ago   /bin/sh -c #(nop)  ENV CI=1                     0B
ba417326cec3        About an hour ago   /bin/sh -c #(nop) COPY file:a86fcb846fef19bc…   79.5kB
78e0ee52d5a6        About an hour ago   /bin/sh -c #(nop) COPY file:91da48d89c17264e…   805B
3038f77e74b3        9 days ago          /bin/sh -c #(nop) WORKDIR /app                  0B
1613db8573fa        3 months ago        /bin/sh -c npm -v                               0B
acc9a9e3993d        3 months ago        /bin/sh -c node -v                              0B
6ca05a9fabe0        3 months ago        /bin/sh -c npm i -g npm@6                       37.2MB
fc56dfde691a        3 months ago        /bin/sh -c apt-get update &&   apt-get insta…   214MB
26cbfbc03e3f        3 months ago        /bin/sh -c #(nop)  CMD ["node"]                 0B
           3 months ago        /bin/sh -c set -ex   && for key in     6A010…   4.47MB
           3 months ago        /bin/sh -c #(nop)  ENV YARN_VERSION=1.6.0       0B
           3 months ago        /bin/sh -c ARCH= && dpkgArch="$(dpkg --print…   58.9MB
           3 months ago        /bin/sh -c #(nop)  ENV NODE_VERSION=10.0.0      0B
           4 months ago        /bin/sh -c set -ex   && for key in     94AE3…   129kB
           4 months ago        /bin/sh -c groupadd --gid 1000 node   && use…   335kB
           4 months ago        /bin/sh -c set -ex;  apt-get update;  apt-ge…   320MB
           4 months ago        /bin/sh -c apt-get update && apt-get install…   123MB
           4 months ago        /bin/sh -c set -ex;  if ! command -v gpg > /…   0B
           4 months ago        /bin/sh -c apt-get update && apt-get install…   44.6MB
           4 months ago        /bin/sh -c #(nop)  CMD ["bash"]                 0B
           4 months ago        /bin/sh -c #(nop) ADD file:bc844c4763367b5f0…   123MB

Linux dependencies necessary to run Cypress are large, and node_modules is huge too.

Conclusion

If you plan to build a Docker image with Cypress end-to-end tests to serve in production, you must use the Docker multi-stage feature to avoid dragging dev dependencies into production.