Note: This post was written by Benjie Gillam, maintainer of the popular OSS project PostGraphile. He specializes in building tools for GraphQL, Node.js, and PostgreSQL developers.
Getting started with a fullstack JavaScript project these days is exciting, but often overwhelming. There's so much tooling that can make our lives easier, but if we're not careful then we can lose a tonne of time to choosing between similar tools, and making others play nicely together. Over the past few years I've felt this pain too many times, and seen many others fall into the same trap, so at Graphile we've been putting together an advanced starter project for full-stack JavaScript development called "Graphile Starter". We've been adding and removing tools, iterating on the structure of the project, and making everything work smoothly together, trying to reach development nirvana. It's taken a lot of work!
Code commits to Graphile Starter over time
Graphile Starter covers all the bases including popular linting, prettier formatting, pre-configured testing, powerful code generation, productive migrations, performant job queues, passing CI and peaceful deployment. We've even baked in a full account management system with OAuth and username/password authentication, forgot password flow, and multiple email management. It's a fully formed foundation for your next project.
Testing is Key
In open source, as it is everywhere, testing is key - not just so you can trust that your application will work great in production, but also so we can spot issues in pull requests more rapidly, allowing us to spend our open source time more efficiently. The popular Jest test runner was a clear choice for both unit testing and GraphQL testing. After some consideration it emerged that Jest also suited SQL testing. However, end-to-end testing is a different beast, and so for that we employed Cypress.
Testing Should be Convenient
One of the biggest barriers to writing tests is inconvenience. It's too much of a barrier to have to reboot the server in test mode. Running two servers is likely to wear through your laptop's battery faster than you'd like.
In the end, we decided the best way would be to enable you to run your tests directly against your development instance, this way your regular workflow can be maintained, with hot-loading and all that other development-optimised goodness.
Determinism and reproducibility
But non-deterministic tests are the enemy of productivity. Mostly they pass, sometimes they fail, and you're never able to reproduce the failure when you want to, so working out why it failed can take a huge amount of time. Let's avoid that!
Cypress encourages us to clean up state before tests run. This makes complete sense - if you clean up beforehand, then any tests that failed (or didn't run yet, or were skipped) can't cause issues for the current test. Furthermore, if you clean up beforehand and you don't clean up afterwards, then when something goes wrong you can take a look at the data and have a better chance of reproducing the issue. This is even easier to do if you're testing against your development instance, because you can use all of your regular development debugging tools to figure out what went wrong! AWESOME!
Convention for Test Data
For this to work we need to have a way to reset the test data before each test so that our tests remain repeatable. To accomplish this, we reasoned that having a convention for test data would allow us to install and manipulate test data in the development instance without interfering with our manually entered development data.
We decided on a convention: users whose username began with the word test were for test accounts only; thus the test engine could safely delete them and it wouldn't break developer flow. You may need a different convention for your data, or maybe multiple conventions for different data stores, but this pattern can probably apply to your data too.
Since a lot of our state is stored on the server, we need to invoke some kind of server command.
Issuing Server Commands
Cypress enables you to trigger requests from your tests using the request() method. We can use this to send commands to our server to do actions such as clearing the test data, creating a user account, and getting the server into the right state for our text. Getting the URLs and parameters right for these commands could become a chore if we're not careful... Fortunately we can write custom Cypress commands in TypeScript so that our editor's auto-complete can guide us as to what's possible!
Here's what the server commands look like currently in Graphile Starter:
// cypress/support/commands.ts
// Reference the types for TypeScript:
/// <reference types="Cypress" />
/**
* Deletes all users with username starting 'test'.
*/
function serverCommand(
command: "clearTestUsers"
): Chainable<{
success: true;
}>;
/**
* Creates a verified or unverified user, bypassing all safety checks.
* Redirects to `next`.
*
* Default values:
*
* - username: `testuser`
* - email: `${username}@example.com`
* - verified: false
* - name: `${username}`
* - password: `TestUserPassword`
* - next: `/`
*/
function serverCommand(
command: "createUser",
payload: {
username?: string;
email?: string;
verified?: boolean;
name?: string;
password?: string;
next?: string;
}
): Chainable<{
user: User;
userEmailId: number;
verificationToken: string | null;
}>;
/**
* Gets the secrets for the specified email, allowing Cypress to perform email
* validation. If unspecified, email defaults to `[email protected]`.
*/
function serverCommand(
command: "getEmailSecrets",
payload?: { email?: string }
): Chainable<{
user_email_id: number;
verification_token: string | null;
}>;
// The actual implementation of the 'serverCommand' function.
function serverCommand(command: string, payload?: any): any {
const url = `${Cypress.env(
"ROOT_URL"
)}/cypressServerCommand?command=${encodeURIComponent(command)}${
payload ? `&payload=${encodeURIComponent(JSON.stringify(payload))}` : ""
}`;
// GET the url, and return the response body (JSON is parsed automatically)
return cy.request(url).its("body");
}
You may notice that we're actually defining just one command, serverCommand(command, payload), but that we're using TypeScript function overloads to indicate what commands are available, and what the shape of the payload (if any) is for each command. This makes running server commands simple, and means that if we change the shape of these commands on the server we can refactor them easily in our tests without worrying that we missed a spot.
We also have a handy helper command for following the Cypress best practices for selecting elements, i.e. using the data-cy attribute, without having to write [data-cy=...] in every selector:
// More in cypress/support/commands.ts
function getCy(cyName: string): Chainable<JQuery<HTMLElement>> {
return cy.get(`[data-cy=${cyName}]`);
}
And finally we need to extend the Cypress Chainable type with these commands:
// More in cypress/support/commands.ts
export {}; // Make this a module so we can `declare global`
declare global {
namespace Cypress {
interface Chainable {
serverCommand: typeof serverCommand;
getCy: typeof getCy;
}
}
}
We can invoke these commands in our tests in a straight-forward manner:
// cypress/integration/login.spec.ts
it("fails on bad password", () => {
// Setup
cy.serverCommand("createUser", {
username: "testuser",
name: "Test User",
verified: true,
password: PASSWORD,
});
cy.visit("/login");
cy.getCy("loginpage-button-withusername").click();
// Action
cy.getCy("loginpage-input-username").type("testuser");
cy.getCy("loginpage-input-password").type(PASSWORD + "!"); // INCORRECT PASSWORD!
cy.getCy("loginpage-button-submit").click();
// Assertion
cy.contains("Incorrect username or password").should("exist");
});
Handling Commands on the Server
This is an incredible boon for test writing, but writing the test code is only half the battle - for this test to pass we need the server to implement these server commands.
We need to add a route handler for /cypressServerCommand, parse the query string and extract the command (string) and the payload (JSON), and then call the relevant functionality based on the command. Our server can then reply with the data, or trigger a redirect, or do whatever else is necessary.
It's important to ensure that this functionality can only run in test or development environments. For extra safety against XSS/CSRF/etc attacks we require that the developer opts into this functionality with the ENABLE_CYPRESS_COMMANDS=1 environmental variable.
We're using the Express framework within Node.js in the Graphile Starter project; here's what our server command handling looks like:
// @app/server/src/middleware/installCypressServerCommand.ts
// Only enable this in test/development mode
if (["test", "development"].includes(process.env.NODE_ENV || "")) {
/*
* Furthermore we require the `ENABLE_CYPRESS_COMMANDS` environmental variable
* to be set; this gives us extra protection against accidental XSS/CSRF
* attacks.
*/
const safeToRun = process.env.ENABLE_CYPRESS_COMMANDS === "1";
/*
* This function is invoked for the /cypressServerCommand route and is
* responsible for parsing the request and handing it off to the relevant
* function.
*/
const handleCypressServerCommand: RequestHandler = async (req, res, next) => {
/*
* If we didn't set ENABLE_CYPRESS_COMMANDS, output a warning to the server
* log, and then pretend the /cypressServerCommand route doesn't exist.
*/
if (!safeToRun) {
console.error("ENABLE_CYPRESS_COMMANDS is not set.");
return next();
}
try {
// Try to read and parse the commands from the request.
const { query } = req;
if (!query) {
throw new Error("Query not specified");
}
const { command: rawCommand, payload: rawPayload } = query;
if (!rawCommand) {
throw new Error("Command not specified");
}
const command = String(rawCommand);
const payload = rawPayload ? JSON.parse(rawPayload) : {};
// Now run the actual command:
const result = await runCommand(req, res, command, payload);
/*
* For an example `runCommand` function, see:
*
* https://github.com/graphile/starter/blob/master/@app/server/src/middleware/installCypressServerCommand.ts
*
*/
if (result === null) {
/*
* When a command returns null, we assume they've handled sending the
* response. This allows commands to do things like redirect to new
* pages when they're done.
*/
} else {
/*
* The command returned a result, send it back to the test suite.
*/
res.json(result);
}
} catch (e) {
/*
* If anything goes wrong, let the test runner know so that it can fail
* the test.
*/
res.status(500).json({
error: {
message: e.message,
stack: e.stack,
},
});
}
};
// Add the route handler:
app.get(
"/cypressServerCommand",
urlencoded({ extended: false }), // Helps us to parse query strings.
handleCypressServerCommand
);
}
Thanks for reading, and here's to pain-free end-to-end testing for everyone! 🥂