Reproducibility Audit of an Analysis
Check whether someone else can rerun your work.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Audit my analysis for reproducibility. Project structure and how I run it: [PROJECT_RUN]. Environment details: [ENVIRONMENT]. Test from a fresh start: list the steps a colleague would take, and note each place where it would fail (absolute paths, missing data, unpinned versions, hidden state, random seeds, undocumented manual steps). Present findings in a table with Issue | Impact | Fix | Effort. Provide a target setup: pinned environment, one command to run, seeds set, outputs checked against expected hashes or summary statistics. Write a README outline and a final checklist for handover. Mention how to record the hardware and library versions for results that depend on them. Add a scoring rubric of five levels from not reproducible to fully reproducible, with the criteria for each, and a plan to move up one level in a week. Include a record of the audit date and findings so that progress can be tracked, and a final test by a colleague who has not seen the project. Add a worked audit record with three invented findings in the Issue, Impact, Fix, Effort format: an absolute path in a script (breaks on any other machine; replace with a relative path from the project root; ten minutes), unpinned library versions (results differ after an update; export a lock file; thirty minutes), and a manual spreadsheet step (not repeatable; script it or document it with a checklist; two hours). Label it an illustration. Add the exact fresh-start test: new clone, new environment, one documented command, and a comparison of key output numbers against a saved reference file with a tolerance I set. Provide the handover README outline in twelve headings and a one-week plan for moving one level up the maturity scale. Finish with a table that records the audit date, the level reached, the issues fixed, and the issues deferred with a reason, so repeated audits show real progress.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.
See an example result (fictional data)
Fresh start test: clone repo, create environment, run script. Issue | Impact | Fix | Effort Absolute path /Users/sam/data | Fails on any other machine | Config file with relative path | 15 min Unpinned library versions | Results changed after update | Lock file | 30 min Random split without seed | Different test set each run | Set seed, record it | 5 min Manual step: rename column in Excel | Not repeatable | Do it in code | 20 min Level: 2 of 5 (runs with help). Target: one command, seeds set, output hashes match. README: purpose, setup, run, expected outputs, data source.
Illustrative only, made-up data. Your results will differ and must be reviewed.