A suite you can trust

A test can pass and still be a problem. Let me show you the two ways that happens, because you will meet both.

Problem 1: the test races the app.

This is the fixed sleep from the waiting lecture. A test with driver.sleep(400) passes on your fast laptop and fails on the slower build server, maybe once in ten runs.

What makes this dangerous is not the failure. It is what the team learns. They see red, they re-run the build, it goes green. After a month, everyone re-runs red builds without looking at them.

I have seen how that story ends. A payment bug reached production on a Friday afternoon. The build had caught it. Three people re-ran that build without opening the report, because everyone had learned that red usually meant nothing.

The fix is the one you already know: use driver.wait with a real condition.

Problem 2: the test depends on another test.

This one is sneakier. Look:

let email; // shared between tests

it('signs in', async function () {
  email = 'demo@nimbus.app';
  // ... uses email ...
});

it('rejects a wrong password', async function () {
  // ... also uses email ...
});

Run them in order and both pass. Run the second one alone and it fails, because email was never set.

The second test was never really passing. It was borrowing something the first test happened to leave behind.

In Selenium there is a second version of this problem, and it is even more common: sharing one browser between tests. If test 1 logs in and test 2 assumes it is still logged in, test 2 cannot run on its own.

The fix for both is the same idea. Give every test its own setup:

beforeEach(async function () {
  driver = await new Builder().forBrowser('chrome').build();
  await driver.get('/');
});

afterEach(async function () {
  await driver.quit();
});

A fresh browser for each test is slower, but it means no test can depend on another one's state.

Here is a simple check you can run on any suite. Run one test on its own. It must pass. If it only passes when other tests run first, it is not finished.

Remember this: a test must pass alone, in any order, every time.

Reference: Driver sessions.