Start software QA with the failures that would matter to users, then define observable evidence for each release decision. Give every critical journey an expected outcome, a test condition and an owner. Keep first-run failures visible when retries succeed, and record the risks that the chosen checks leave unresolved before approving a release.
Which risks should the tests cover?
Begin with a customer or staff task rather than a list of screens. For a quotation application, the important outcome might be that a revised quote reaches the correct account with the correct version. A page appearing successfully addresses only part of that task. Write down the consequence of an incorrect outcome so the team can prioritise its checks.
Build a small risk register covering the proposed change. Include the happy path, rejected input, interrupted work and access by the wrong account. Give each risk a verification method appropriate to it: an automated check, a manual examination or a controlled user task. Some risks need several kinds of evidence.
Choose the expected result before executing the test. Playwright provides assertions through an expect function and a chosen matcher. Documentation. A test named submit quote is incomplete as a specification unless it says what submission means. Identify the resulting record, visible confirmation and any downstream handover that the test is expected to establish.
What should an automated assertion actually establish?
Playwright web-specific asynchronous matchers wait for the expected condition. Documentation. For a web journey, use a condition tied to the outcome you need: the expected status, the intended destination or the record displayed for the correct account. Ask whether the selected condition could also become true after an incorrect action. A generic success message may be too weak for the intended decision.
Playwright retrying assertions must be awaited. Documentation. Playwright generic non-retrying assertions do not automatically retry. Documentation. Keep assertion behaviour visible in the test description. A check that waits for a status to appear has a different meaning from a one-off comparison of a value already read. Reviewers need to understand that difference when diagnosing intermittent failures.
A failed Playwright assertion normally terminates test execution. Documentation. Decide which checks must stop a scenario and which observations can be collected together. For example, a failure to establish the intended account may invalidate later assertions about its records. Explain dependencies in the test design rather than letting a long report imply every later step was meaningfully exercised.
An original risk-to-release evidence matrix
Use this practical checklist during change review. Fill it with the actual change, environment and evidence references. The suggested rows show how to organise a decision; they are not a report of executed testing.
| Risk under review | Proposed check | Evidence to retain | Release question |
|---|---|---|---|
| Wrong account receives a quote | Use distinct synthetic accounts and attempt the handover | Account identifiers and observed record | Was the intended recipient established? |
| Old revision is submitted | Edit a draft and compare the submitted version | Version and resulting contents | Does the result match the approved revision? |
| Required data is missing | Submit an incomplete record | Response and persisted state | Is the rejection understandable and safe? |
| Work is interrupted | Stop at a defined boundary and resume | State before and after interruption | Can the next action be determined? |
| Test is intermittent | Preserve initial failure and retry result | Attempt sequence and environment | Is the cause understood sufficiently? |
| Related behaviour changes | Exercise the adjacent customer task | Regression evidence reference | What existing behaviour was checked? |
Add columns for risk owner and unresolved follow-up in the working copy. A risk without an assigned decision-maker tends to remain a comment in a report. Name the person who can accept the consequence, require more evidence or stop the release.
Avoid reducing the matrix to a count of green checks. A large suite focused on low-impact presentation details can leave a consequential ownership or data-integrity question unanswered. Review the relationship between risks and checks, including the risks for which no useful evidence has yet been collected.
Does a successful retry erase an earlier failure?
Playwright does not retry failed tests by default. Documentation. Playwright labels a test flaky when it fails initially but passes on a retry. Documentation. Keep those categories visible in the release report. Ask the team to explain the initial failure instead of presenting the final attempt alone. A repeated run may help diagnosis, but the change in outcome is itself something the reviewer needs to see.
Playwright exposes the retry number through testInfo.retry. Documentation. Use the attempt information to attach logs and observations to the relevant run. Where a retry deliberately resets state, explain what was reset and why. Otherwise a successful second attempt may be interpreted as evidence about conditions that no longer existed.
In Playwright serial mode, subsequent tests are skipped after a test fails. Documentation. Playwright recommends isolated tests so they can run and retry independently. Documentation. Document whether scenarios are independent or share a sequence. If an early setup failure prevents later tasks from running, show those tasks as unexecuted. Keep that distinction separate from a failure of the application behaviour they were intended to examine.
Hypothetical example: a quote appears twice
Imagine a QA rehearsal in which a user submits a quote, sees a delayed confirmation and tries again. The proposed risk is that the system could create duplicate work. Give the test a specific expected outcome and identify which record or event would demonstrate it. This hypothetical example is a test-design exercise.
Observe the first submission boundary and preserve the state before repeating the action. If the application reports success, compare the relevant records rather than relying only on the page message. Record the action sequence and any deliberate delays introduced by the test arrangement.
Suppose the initial test fails and a fresh run succeeds. Keep both outcomes in the evidence bundle. Investigate whether the change relates to application state, test data, the environment or the test itself. The release owner can then decide whether the evidence is adequate for the intended use and what must happen next.
How should the release decision be recorded?
Write the conclusion against the tested version and environment. Summarise the customer tasks checked, the unresolved failures, the unexecuted checks and the remaining assumptions. Include a recovery or rollback decision appropriate to the actual deployment process, with a named owner.
A useful QA handover lets another person locate the evidence behind each consequential conclusion. Store the change reference, test conditions, observations and decision together. When the software changes again, use those references to decide which checks need to be repeated rather than treating an old result as permanent approval.
Keep commercial scoping and implementation responsibilities in the web development brief. For adjacent production care, see website maintenance. Get started with the customer journey, proposed change and acceptance question you want to discuss.
By Spout Team. AI-assisted drafting by Codex using the linked primary documentation.