Audit intermediate Works with: Any AI

Sample Size and Method Fit Check

Table output Has example

Check whether the number of participants suits the claim.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

[VERIFY] is an editorial flag, not a blank to fill. Wherever the prompt shows [VERIFY], check that item against a reliable source before you use it.

Check whether my sample supports my claims. Study description, method and number of participants: [STUDY_DESCRIPTION]. Claims I want to make: [CLAIMS].

For each claim state whether the sample and method can support it: qualitative studies show problems and reasons, not exact prevalence; small quantitative samples give wide intervals. Where percentages are quoted, compute an approximate margin of error using 1.96 times the square root of p times 1 minus p divided by n and show it.

Return a table with Claim, Support level and Safer wording, and recommendations for additional data. Do not overstate certainty.

Add a worked margin of error: if 6 of 10 participants completed a task, the success rate is 60 percent and the approximate margin of error is 1.96 times the square root of 0.6 times 0.4 divided by 10, which is 1.96 times 0.155, about 30 points; so the plausible range is roughly 30 to 90 percent, too wide to compare with another version. With 30 participants the margin would be about 17 points. Show this for my numbers and propose careful wording such as 6 of 10 completed the task and 4 did not. Add rules for reporting small samples.

Add a worked claim table with invented studies. Claim: most users prefer the new layout; evidence: 5 of 8 interviewees preferred it; support level: weak for most, because a qualitative sample cannot estimate prevalence; safer wording: 5 of 8 participants preferred the new layout; the reasons were speed and clarity. Claim: the new flow is faster; evidence: median time fell from 90 to 70 seconds in 12 participants; support level: moderate, with an approximate interval; safer wording: in this small sample the median time fell by 20 seconds; a larger test would confirm. Claim: 80 percent of customers want feature X; evidence: a survey of 40 volunteers; support: weak, since volunteers differ from customers; safer wording: 32 of 40 survey respondents wanted X. Add guidance on when to run a larger quantitative follow-up and how many participants might be needed, using an approximate formula with a calculator [VERIFY].

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

Invented. Task success 6 of 10 = 60 percent.
Margin of error = 1.96 x sqrt(0.6 x 0.4 / 10) = 1.96 x 0.155 = about 30 points: the range is roughly 30 to 90 percent.
With 30 participants: 1.96 x sqrt(0.24 / 30) = 1.96 x 0.089 = about 17 points.
Safer wording: "6 of 10 participants completed the task; 4 did not; the sample is too small to compare versions."
Claim "most users prefer the new layout": 5 of 8 interviewees preferred it; weak support for a claim about most users.

• audit