In this blog post we continue exploring flaky tests and how to fix them. We will look at the situation when a page submits a form and then continues to interact with the page, which causes a problem.
Tip: you can find all existing similar posts under the tag Flake. You can find the source code for this blog post in Cypress example recipes repository.
The error
The original error was submitted by Cypress user Marc Guenther. In the example, the HTML form is submitted while the test keeps interacting with the page.
<html>
<body>
<h1>Form</h1>
<form action="">
<select onchange="document.forms[0].submit()">
<option value="1">First</option>
<option value="2">Second</option>
</select>
<br />
<input />
</form>
</body>
</html>describe('Form submit', () => {
it('no explicit wait, this will fail but shouldn\'t', () => {
cy.visit('')
cy.get('select').select('Second')
cy.get('input').type('Hallo')
})
})The test fails with "Element is detached from DOM" error. Meaning Cypress is trying to click / type / interact with the elements on the page, but for some reason these elements are no longer on the page.
When this error happens, we must look at the application. We need to understand what the application is doing in response to the test runner's actions. The first place to look is the Command Log. A red flag to me is the "page loaded" message in the Command Log.
Step through the test
A good way to slow down the test is to use cy.pause and step through the Cypress test. You can also keep the DevTools open to see network activity. Since there is a page load, let's look at the "document" resources loaded during the test.
it('no explicit wait, this will fail but shouldn\'t', () => {
cy.visit('')
cy.get('select').pause().select('Second')
cy.get('input').type('Hallo')
})
We step through the test command by command. The Network tab shows that selecting the value using cy.select submits the form in this case, causing the new page to be requested from the server and loaded. Since the Test Runner is paused after selecting the element, we get the page to fully load before the next command cy.get runs. Since the page no longer suddenly reloads, the cy.get('input').type('Hallo') commands finish correctly.
Stepping through the test is one of my favorite ways to debug a flaky test.
Waiting for page load
The solution to the flaky test is clear: we must tell the Test Runner to "wait" until the submitted form loads the new page. Only after that can we continue with other test steps. We can achieve the wait in multiple ways.
Hardcoded sleep
The simplest way is perhaps adding cy.wait to the test and sleep for a short period of time.
it('explicit wait after submit, this will succeed', () => {
cy.visit('')
cy.get('select').select('Second')
cy.wait(1000)
cy.get('input').type('Hallo')
})
While this is a simple solution, it has two shortcomings:
- The delay slows down the test.
- The delay might not be enough in situations where the server is slower.
The test might still remain flaky and occasionally fail on continuous integration server just because the page submission might take longer than one second. Of course Test Retries could help, but first let's explore better solutions in this particular case.
Wait for URL change
Notice that the Command Log shows "page load" followed by a "new url" message. The form submission loads the page with form fields encoded in the URL. In our case, the submission still goes to the URL previous url/? Let's look at the cy.location object after form submission.
it('wait for URL change', () => {
cy.visit('')
cy.get('select').select('Second')
cy.wait(1000)
cy.location()
})In the above test I force the test to wait for one second (temporarily) and then run cy.location() command. I run this command only to be able to click on it after the test finishes. With the DevTools console open, the command shows all fields in the yielded location object.
Great, you can see that after submitting the form, the location object includes the search='?' property. Let's use this as assertion and only continue the test once the page has reloaded.
it('wait for URL change', () => {
cy.visit('')
cy.get('select').select('Second')
cy.location('search').should('equal', '?')
cy.get('input').type('Hallo')
})
The test reliably passes. The "page load" message is now always before the "get" command message. Thus our test will no longer run faster than the application. Even if the server on CI takes slightly longer than usual to respond, the assertion will retry until the submitted page is loaded.
Wait for the document resource
We can wait for the page in several other ways. For example, we can spy on the network call fetching the `/?` resource and wait for this request to finish.
it('wait for document network call', () => {
cy.visit('')
cy.route2('/?').as('doc')
cy.get('select').select('Second')
cy.wait('@doc')
cy.get('input').type('Hallo')
})The test above uses the cy.route2 command that can spy on any network request made by the application, and other amazing things. In our case it prevents the test from working with the old DOM after the form is submitted.
Look out of at the window
In yet another test we can detect when the page reloads before continuing by looking at the application's window object. When we visit the page the first time, we can set our property on that object from the test. When the form is submitted and the new document is loaded, the browser will recreate the window object again. Our test can detect when that custom property is gone from the window object - that means the new page is ready.
it('wait for window object to be refreshed', () => {
cy.visit('')
cy.window().then((w) => w.initial = true)
cy.get('select').select('Second')
cy.window().its('initial').should('be.undefined')
cy.get('input').type('Hallo')
})
If you look really closely at the GIF (pronounced like "git" with an "f"), you can kind of see the assertion showing in blue "expected true to be undefined" before switching to the green "expected undefined to be undefined" when the new window object is created.
Final Thoughts
The End-to-End test does not need to "know" the implementation details, but it needs to checkpoint itself to the application's behavior. Just like a real user needs to wait for the submitted form to fully load before typing more fields, our test needed to wait too. Luckily, Cypress makes observing the external effects of the page load easy: you can wait for the URL to update, or for the network call to finish, or even inspect the window object to know when it i safe to proceed.
Happy Testing!