Testing apps hosted on Codepen.io

Lots of people love to quickly throw web applications on Codepen.io. Who can refuse a quick and powerful way to start writing code immediately; sharing results is almost instant. In this blog post I will show how to write end-to-end tests for web apps embedded in a Codepen.

Warning: no longer working

Warning ⚠️  Codepen.io has added a CAPTCHA to prevent scrapers and 3rd party tools from accessing the hosted web apps. The testing approach shown in this blog post no longer works, but we will keep it here for reference.

HyperApp example

My example is a tiny counter application from HyperApp. It is one of their demo apps linked from their home page. You can find the original Codepen here. The entire application is extremely small, easy to follow and very functionally pure. I cannot stop raving about it.

const { h, app } = hyperapp
/** @jsx h */

app({
  state: {
    count: 0
  },
  view: (state, actions) =>
    <main>
      <h1>{state.count}</h1>
      <button
        onclick={actions.down}
        disabled={state.count <= 0}>ー</button>
      <button onclick={actions.up}>＋</button>
    </main>,
  actions: {
    down: state => ({ count: state.count - 1 }),
    up: state => ({ count: state.count + 1 })
  }
})

That’s the entire application. When it runs, the app shows a counter and two buttons. when clicking on the buttons - the counter should increment and decrement (and  be restricted to non-negative numbers).

Great little application. Let’s see how we can write some end-to-end tests?

One thing to notice is that the function call app(...) returns the actions object. We can use this function to change the application’s internal state, bypassing having to click the DOM buttons.

Will our application handle negative count for example?

First attempt

I created a demo project in our cypress-io/cypress-example-recipes with our HyperApp application. Using Cypress v1.1.1 I wrote the first test - I just want to load the Codepen with the application (I am using my fork codepen.io/bahmutov/full/ZaMxgz/ to make sure it stays unchanged in this example).

// cypress/integration/spec.js
// our Codepen has top level URL
const url = 'https://codepen.io/bahmutov/full/ZaMxgz/'

it('loads', () => {
  cy.visit(url)
})

Great, the Codepen loads, and I can even interact with it by clicking the counter buttons.

Now let’s inspect the Codepen to grab the <main> element of the HyperApp application.

The Cypress docs give advice on how to reach into nested iframes and we plan to make it even easier.

But there is a problem. The top level Codepen loads from codepen.io, but the web application loads from the domain s.codepen.io (you can see it if you open the DevTools and inspect the source code / Network requests).

Working inside an iframe from a different domain is prevented by the browser’s security model. We can disable it in Cypress and work with the elements inside, but the code gets ugly very very fast.

const url = 'https://codepen.io/bahmutov/full/ZaMxgz/'

it('loads', () => {
  cy.visit(url)
  cy.get('iframe#result')
    .then($iframe => {
      const $body = $iframe.contents().find('body')
      return cy.wrap($body).find('main')
    })
    .then(main => {
      cy.wrap(main).should('be.visible')
      cy.wrap(main).contains('button', '＋').click()
    })
})

While it starts to look like pretty ugly, the test works, our click registers, and the action updates the DOM.

But if we hover or .click() or any other step in the Test Runner, we see a blank page in the Codepen’s iframe in the application preview 😠.

So what can we do in this case? Can we test a Codepen with the full power of Cypress?

Writing HTML

Yes, we can! We are going to use a nice little trick that utilizes the cy.request() method. First, like I said there is the top level Codepen domain, and there is an internal domain where the web application is actually rendered. In our case the internal url is https://s.codepen.io/bahmutov/fullpage/ZaMxgz. But when we try to visit this url directly from our tests, we see a warning.

Browsers do not let us control the Referer header directly, but servers can! Cypress comes with a built-in proxy we can use to make almost any request. So let’s grab the internal iframe HTML source. Yes, the HTML source.

// our Codepen has top level URL
const url = 'https://codepen.io/bahmutov/full/ZaMxgz/'
// that loads app from this URL
const iframeUrl = 'https://s.codepen.io/bahmutov/fullpage/ZaMxgz'

it('loads HTML source', () => {
  // see docs at https://on.cypress.io/api/request
  cy.request({
      method: 'GET',
      url: iframeUrl,
      headers: {
        referer: url,
        accept: 'text/html'
      }
    })
    .its('body')
})

We are requesting the HTML document from the Codepen using referer and content-type headers in the cy.request() call. We are yielded HTML text.

Great, we got the embeddable HTML page of the web application. How does it help us inside the Test Runner? We have not even loaded a URL to test!

Here is the trick: we have the application’s preview iframe inside Cypress, and it already has a document object - it just does not have any HTML yet, because it has not visited a website yet. So we can just write the HTML text we have received straight into the test document using the infamous document.write method.

And that is our little trick - by “brute-forcing” HTML code into the test iframe, we bypass the iframe sandbox restrictions AND can conveniently work with the DOM, because now the web application is at the top level in the test frame. Here is my complete callback function that runs before each test.

// our Codepen has top level URL
const url = 'https://codepen.io/bahmutov/full/ZaMxgz/'
// that loads app from this URL
const iframeUrl = 'https://s.codepen.io/bahmutov/fullpage/ZaMxgz'

beforeEach(function loadAppIFrameAndSetAsOurTestDocument () {
  cy.request({
      method: 'GET',
      url: iframeUrl,
      headers: {
        Referer: url,
        accept: 'text/html'
      }
    })
    .its('body')
    .then(html => {
      cy.document().then(document => {
        document.write(html)
        document.close()
      })
    })
  cy.get('main').should('be.visible')
})

End-to-end tests

We can begin our end-to-end tests with a simple use case. Our application starts with the counter at zero. Let us confirm that this is the case.

I have gone ahead and factored out getting the counter DOM element because I expect to use it in several tests.

const getCount = () => cy.get('main').find('h1')

it('starts with zero', () => {
  getCount().contains('0')
})

Beautiful! We can hover / click on each step in the Test Runner’s Command Log to highlight the relevant DOM elements - the snapshots are working perfectly.

We can click on the “plus” and “minus” buttons and confirm that they do indeed increment the counter.

// NOTE: while it looks like buttons have regular ASCII "+" and "-"
// in reality these are Unicode symbols ＋ and ー
const getPlus = () => cy.get('main').contains('button', '＋')
const getMinus = () => cy.get('main').contains('button', 'ー')

it('increments and decrements via UI', () => {
  getPlus().click()
  getPlus().click()
  getPlus().click()
  getMinus().click()
  getCount().contains(2).should('be.visible')
})

We can confirm that the “minus” button gets disabled when the count reaches zero, and we cannot get a negative number.

it('has decrement button disabled initially', () => {
  getMinus().should('be.disabled')
})

it('cannot decrement by clicking on disabled minus button', () => {
  getCount().contains(0).should('be.visible')
  getMinus().click({ force: true }) // because button is disabled
  getCount().contains(0).should('be.visible')
})

it('enables decrement button for positive numbers', () => {
  getPlus().click()
  getMinus().should('not.be.disabled')

  getMinus().click()
  getMinus().should('be.disabled')
})

We do not have to always interact with the application through its DOM. Our application saved the reference returned by the framework’s app(...) call. This reference is the “actions” object!

// save the reference so we can dispatch
// actions on the web app
window._app = app({
  // ...
  actions: {
    down: state => ({ count: state.count - 1 }),
    up: state => ({ count: state.count + 1 })
  }
})
// window._app is the actions object {down, up}
// window._app.down()
// window._app.up()

By extracting the app reference from the test iframe as shown below we can directly dispatch actions and observe the DOM behavior. For example, we can get the counter display to show negative numbers - something we could not do through the “minus” button.

// returns window._app = app(...) reference
// created in the Codepen
const getApp = () => cy.window().its('_app')

it('returns actions object', () => {
  getApp().should('have.all.keys', 'down', 'up')
})

it('can drive DOM via App actions', () => {
  getApp().then(actions => {
    actions.up()
    actions.up()
    actions.up()
    actions.down()
    getCount().contains(2).should('be.visible')
  })
})

it('can even drive App into invalid state', () => {
  getApp().then(actions => {
    actions.down()
    actions.down()
    getCount().contains(-2).should('be.visible')
    getMinus().should('be.disabled')
  })
})

Watching these tests run is pure pleasure.

Links

• example source code
• Cypress.io documentation
• https://github.com/cypress-io/cypress