lecture 9 of 160 completed

Checks that actually catch bugs

A weak assertion passes on a broken app. Here are the three weak ones you will write without noticing.

Selenium does not give you assertions. You bring your own library, and you decide what to check. That freedom means you can write a test that passes on a completely broken app without realising it.

Here are the three weak checks I see most often in code review.

Weak check 1: you check something exists, not what it says.

const subtotal = await driver.findElement(By.css('[data-testid="cart-subtotal"]'));
assert.ok(await subtotal.isDisplayed());

The subtotal is on screen. Is it correct? This test does not know. If the price calculation breaks and shows $0.00, it still passes.

Check the number your customer will pay:

assert.strictEqual(await subtotal.getText(), '$78.00');

Weak check 2: you check one item, not the whole list.

You remove one product from the cart:

const found = await driver.findElements(By.xpath("//*[contains(text(),'Trail Runner Backpack')]"));
assert.strictEqual(found.length, 0);

That product is gone. But what if the remove button emptied the entire cart? This test still passes. Add the count:

const lines = await driver.findElements(By.css('[data-testid="cart-line"]'));
assert.strictEqual(lines.length, 2);

Weak check 3: you only test when things go well.

Most real bugs live in what an app should refuse:

await driver.findElement(By.css('[data-testid="checkout-card"]')).sendKeys('1111');
await driver.findElement(By.css('[data-testid="place-order-button"]')).click();

const error = await driver.wait(
  until.elementLocated(By.css('[data-testid="card-error"]')),
  5000
);
assert.ok((await error.getText()).includes('valid card number'));

const confirmed = await driver.findElements(By.css('[data-testid="order-confirmed"]'));
assert.strictEqual(confirmed.length, 0);

Look at those last two lines. You check the error appeared and that the order did not go through. Note the use of findElements for the absence check: it returns an empty list instead of throwing, so a length of zero is your proof.

Do not skip that check. I have seen a checkout that showed "Invalid card number" in red and created the order anyway. The test only checked the error message, so nobody noticed for two weeks.

Before you finish any test, ask yourself:

If this feature were broken, would my test still pass?

If the answer is yes, make the check stricter.

Remember this: check the value, check the whole list, and check what should not happen.

Reference: Finding elements.

now prove it

Reading is the setup. Solve these problems in the editor. We break the app on purpose to check your test would catch it.