Automation is not a moral good. It is an economic decision. The right question is not “Can we automate this?” but “Should we?” The following heuristics help teams choose targets that reduce risk and cycle time without bloating the suite.
Automate High-Risk, High-Repeat Scenarios
Focus first on critical user journeys such as authentication, checkout, or money movement, high-frequency tasks that consume manual time each release, and areas with a history of defects. Doing so maximizes the defect-finding value per test run. For context on layering and scope, see the Google Testing Blog on the testing pyramid.
Prefer Stable Interfaces
When a feature or interface is still changing rapidly, defer UI automation and validate behavior at API or contract levels. Add UI coverage once the interaction model stabilizes. For contract testing approaches, see Pact.
Use the Three-Run Rule
Before promoting a new test into the main suite, require it to pass three consecutive runs in a CI environment. Tests that flake under load are not ready for prime time. For background, the Google Testing Blog discusses flaky tests at scale.
Apply the 5x Maintenance Test
Estimate how many times a test will need maintenance in a quarter. If a test is likely to require updates more than five times, move that check down the stack or drop it. Volatile UI details are better validated via integration or component tests.
Keep Assertions Specific and Cheap
Make assertions narrow and meaningful, for example a status code, a checksum, or a visible role, rather than broad and brittle, such as full page text comparisons. Fast, focused assertions isolate defects and speed feedback.
Track ROI With a Simple Score
Score candidates on Business Impact, Failure Likelihood, Run Frequency, and Maintenance Cost on a 1-5 scale. Automate items with a high composite score and reasonable maintenance. Keep the rest manual or validate below the UI.
Prune Aggressively
Every quarter, remove tests that fail to catch real defects or that consume outsized maintenance. Healthy suites are edited suites.
The Heuristic Mindset
These rules do not replace judgment. They make judgment visible. By framing automation as a constrained optimization problem, coverage versus cost, you build a suite that stays useful as products evolve.
Published 2025-10-09 · Updated 2025-10-29