Flaky Test Investigation Audit
Find causes of tests that sometimes fail.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Investigate flaky tests. Paste failing test code, failure logs from several runs and the environment description: [FLAKY_DATA]. How tests run in CI: [CI_SETUP]. Frequency of failure: [FREQUENCY]. Look for time dependence, shared state, ordering assumptions, network calls, race conditions, resource limits and random data without seeds. Rank likely causes by evidence and suggest how to reproduce each locally. Return a table with Test, Likely cause, Evidence and Fix, a quarantine policy that does not hide problems, and a plan to track flake rate over time. Also suggest how to capture more evidence on the next failure, such as seeds, timestamps and environment details, and how to run the test repeatedly under load to reproduce the problem. Add an owner and deadline for each flaky test, and a rule that a quarantined test must be fixed or deleted within a set time. Finish with the single most likely cause and the cheapest experiment that would confirm it. Then create a stability programme for the whole suite. Define how to measure flake rate per test and per area using the last fifty runs, how to rank tests by cost of failure, and how to track the trend weekly. Propose a triage rota so that every flaky failure is owned, with a service level for investigation and a rule on quarantine that has an expiry. Write a short guide to writing stable tests: avoiding sleeps, controlling time and randomness, isolating data, waiting on conditions, and cleaning up. Suggest tooling to run tests in random order and in parallel on a schedule to expose hidden coupling. Add a communication template for telling the team when a fix has landed, and a dashboard layout with the key numbers. Finish with an experiment to confirm that the main fix worked by running the suite repeatedly under load, and a decision on whether to keep, rewrite or delete each remaining flaky test.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.