Create advanced Works with: Any AI

Experiment Design Document

Structured brief Has example

Plan a test with a clear decision rule.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

Design an experiment. Hypothesis idea: [HYPOTHESIS_IDEA]. Product area and traffic: [AREA_AND_TRAFFIC]. Baseline metric values: [BASELINE]. Decision it supports: [DECISION].

Produce: a hypothesis in the form "If we [change], then [metric] will [direction and size], because [reason]"; primary metric and guardrails; population and assignment (random split, unit of randomization); minimum detectable effect and sample size guidance (state the formula approach and that I should confirm with a statistician or calculator: n per group is roughly 16 x variance / delta squared for 80 percent power at 5 percent significance); duration (full weekly cycles); risks of interference; decision rules written in advance (ship, iterate, stop); and analysis plan.

Add a pre-launch checklist (instrumentation tested, no overlap with other tests, QA of variants) and a results template. Warn against peeking and multiple comparisons. If traffic is too low, propose alternatives (qualitative tests).

Include the experiment brief as a one-page summary with hypothesis, variants, metrics, duration, sample size, decision rule, and risks, and state what will be logged so results can be audited after the test.

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

Hypothesis: If we show the delivery date on the product page, then add-to-basket rate will rise by 5 percent relative, because customers feel more certain.
Primary metric: add-to-basket rate, baseline 8.0 percent. Guardrails: bounce rate, page load time, support contacts.
Sample size: detecting 0.4 points (8.0 to 8.4) with n per group about 16 x p(1-p) / delta squared = 16 x 0.08 x 0.92 / 0.004 squared = 73,600. That is large, so run for 3 weeks at 25,000 visitors a week per group or accept a larger lift.
Assignment: random by visitor. Duration: full weeks.
Decision rule: ship if lift above 3 percent relative and no guardrail breach; extend if inconclusive; stop if negative.
Do not peek; log exposure events.

• experiment-design
• structured-brief