How to test a single-page application?

Among the various things to test within your application, you would typically want to make sure that a user must know their password to log in. This would be the following test (more about it here):

Testing a regular website

Frameworks such as Django make testing very easy. To write the above test, a developer would write this:

• Data
  • create user Alice with email “[email protected]” and password “Pass*%20”
• Fake-user action
  • go to “/login/” page
  • enter username “[email protected]”
  • enter password “alice”
  • submit the form
• Expected “correct” result
  • user remains on the page “/login/”
  • user is not logged in
  • user sees “incorrect password”

For that test to work, the developers do not need to do much. They would obviously have to write it in real code (Python for example) but everything would work out of the box as expected with Django. For example, Django will take care of using a different database for the tests. In the test database, there is no existing user, so you can add an account for Alice. That simple test would take a lot more work in a single-page application.

Testing a single-page application

Most frontend libraries come with tools to write tests but these do not guarantee that the whole stack is working well together, they just test the frontend code. In other words, for an application written in Django & React; writing React tests only does not allow you to check that the full app is working properly. Furthermore, as single-page applications allow users to interact in various ways, it is harder for developers to decide on what to test. Finally, frontend development tends to attract more junior developers who have limited experience writing tests. For these reasons, many developers would not even bother.

To me, the most valuable tests for a single-page application are end-to-end tests (from the frontend to the backend) that check the frontend (React), the API (Django) and their interactions. For example, I want to know that when a user tries to login (frontend & backend), a spinner shows up (frontend) while the email / password are checked (backend) and that the user is then redirected to their dashboard where their name appears.

Such a test allows me to check that:

• the app calls the correct API path to check the login credentials (username / password couple tin the database)
• the app can interpret the login API result
• the loading spinner component is working
• the login form component is working
• the dashboard component is working
• the app gets the logged-in details from the API properly
• the app shows the name of the currently logged-in user

This would be a very useful end-to-end test. Ideally, you want one of these for every single “user-action” and you can be reassured that your application is (still) working as planned.

The solution (in 2020):
End-to-end tests with Cypress

Cypress makes writing end-to-end tests really easy. In Cypress, the tests contain user actions (click here, type that text, press enter, etc.) and some visual assertions (this text shows up, that form is opened or closed, that button is disabled, etc.). In the following capture, the login page is checked with a total of 8 tests.

You can see their description on the left (“displays password incorrect error” was the example mentioned before). On the right, developers can see what is happening in the browser during the test’s run.

This example comes from the official Cypress introduction video:

Developer experience

Tests in Cypress are fairly easy to write and developers can visualise them running in a browser, pause and go back to see step-by-step what is happening. This is extremely valuable for debugging. On top of that, tests can also be executed without opening a browser which is useful to run them automatically as a regular (and mandatory) part of building and deploying the website. This catches bugs before the system can even be deployed.

Cypress will take some initial effort to set up but it takes less and less effort to add more tests. In comparison, when you are testing manually, every test takes the same amount of work.

Within days, Cypress tests will make development faster and therefore cheaper. In the medium and long run; it will enable the desired virtuous circle (link to first article).

Client experience

At the Interaction Consortium, our clients do not write or run any tests themselves, they do not check them and do not contribute to them. They do not even see them running. However they trust us so they are aware that random bugs they run into are for the most part (no one is perfect) out of our control - an embedded form stopped working; an external API stopped responding, etc. I cannot remember a single client asking us “Are you sure?” or “Could you check again?” when running into an issue. That sort of relationship would not be possible without a good test suite.

In some cases, we develop the frontend part only. In that case, the client is tech savvy and the test suite is sometimes discussed. We trialled Cypress for a client in this situation recently and they loved the way we developed their test-suite for the frontend.

Cypress is not perfect

Cypress tests are fantastic for debugging but they take more work than the usual Django tests we write. Like the app, end-to-end tests require an API. It could be the “real” API running in “test-mode” or a separate fake (simulated) API. That API must include features to allow the frontend to define data (see above table) such as “there is no user”, “this user is deactivated” “this user has no history” “this user has bought 10 products” etc. Building and maintaining this API is extra work.

Tests must be written and maintained. This is true for any type of tests but in the case of Cypress, it takes more time. The syntax is fairly strange - Cypress uses Javascript with Mocha and Chai. Developers must also be a bit careful. For example, testing if an element was properly removed might give a false positive (the test passes when it should not – see below).

Cypress tests run in Chrome. If you are after multi-browser testing, Cypress will not be helpful. Cypress is great to test all the functionalities, not so much for the visual aspect (CSS).

Conclusion

Testing single-page applications is both challenging and crucial. Luckily, Cypress makes that task easier. Currently, in 2020, Cypress remains my favourite tool for frontend and/or end-to-end testing. It takes more work than backend testing (a traditional website or an API) but it is one hundred percent worth it for everyone involved. I highly recommend it.

How to write a false-positive test in Django

(Don't try this at home!)

1. User loads his TODO tasks
2. Assert that the task “Read IC blog” is present on the page
3. User clicks on the button “Done” for the task “Read IC blog”
4. Assert that the task “Read IC blog” is not present anymore
   1. The spinner shows up
   2. Cypress is happy with that: “Read the IC blog” is not showing up

That test might pass even if the code does not work (false-positive). To understand why, you must think about what is actually happening on the page after clicking on the button “Done”. What is likely to happen is that a spinner is showing up while the application retrieves the updated TODO list. To avoid this timing issue, you could check that another task is still visible (Cypress will wait for the task to appear). The test would become:

1. User loads his TODO tasks
2. Assert that the task “Read IC blog” is present on the page
3. Assert that the task “Call my parents” is present on the page
4. User clicks on the button “Done” for the task “Read IC blog”
5. Assert that the task “Call my parents” is still present
   1. The spinner shows up
   2. Cypress waits for a while until the task appears; all the TODO tasks are loaded
6. Assert that the task “Read IC blog” is not present anymore