Create intermediate Works with: Any AI

A/B Test Results Summary for Marketing Decisions

Table output Has example

Decide whether to roll out a winning variant.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

Summarise this test. Variants and what changed: [VARIANTS]. Visitors and conversions per variant: [RESULTS]. Test dates: [DATES]. Business impact of a conversion: [CONVERSION_VALUE].

Calculate conversion rate per variant as conversions divided by visitors, the absolute and relative difference, and an approximate margin of error for each rate using 1.96 times the square root of p times 1 minus p divided by n. Say whether the intervals overlap and what that implies.

Recommend roll out, extend or stop, with the reason. Note risks such as short duration, a holiday during the test, or multiple changes. Return a summary table and one paragraph for stakeholders. Do not call a result proven from a small sample.

Show an invented example: variant A 400 visitors, 20 conversions = 5.0 percent; variant B 400 visitors, 30 conversions = 7.5 percent. Difference = 2.5 points, relative lift 50 percent. Margin of error for A = 1.96 x sqrt(0.05 x 0.95 / 400) = 1.96 x 0.0109 = about 2.1 points; for B = 1.96 x sqrt(0.075 x 0.925 / 400) = 1.96 x 0.0132 = about 2.6 points. The intervals (2.9 to 7.1 and 4.9 to 10.1) overlap, so the result is suggestive, not conclusive. Apply the same steps to my data and recommend extending the test if intervals overlap.

Add a decision matrix with three outcomes, each with its trigger, action and owner. Include a section on what we do if the test result contradicts our expectations: record it, check for errors in setup, and avoid explaining it away without evidence. Add a paragraph on external factors that may have affected the period, such as promotions, outages or seasonality, and how to note them in the record. Suggest the date for a follow-up check after rollout to confirm the gain persists, with the same measure.

Close by recording the decision, the date and the evidence in a shared test log, so the same idea is not retested without new reasons.

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

Invented test. A: 400 visitors, 20 conversions = 5.0 percent. B: 400 visitors, 30 conversions = 7.5 percent.
Difference 2.5 points; relative lift 50 percent.
Margin of error A: 1.96 x sqrt(0.05 x 0.95 / 400) = about 2.1 points, interval 2.9 to 7.1. B: 1.96 x sqrt(0.075 x 0.925 / 400) = about 2.6 points, interval 4.9 to 10.1.
The intervals overlap, so the result is suggestive but not conclusive.
Recommendation: extend the test until each variant has about 1,000 visitors. Risk: the test ran across a holiday week.

• testing