Rating Scale and Metric Validity Review
Check scales, anchors, and what the numbers mean.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Check what my rating scales and metrics really measure. Survey or metric definitions: [PASTE_SCALES_AND_METRICS]. How results are used: [DECISIONS]. Sample results: [SAMPLE_RESULTS]. Work through the checklist for each scale or metric and mark Pass, Weak, or Fail with an example and a fix: [ ] The question measures one idea (not double-barreled) [ ] The wording is neutral, with no leading or loaded terms [ ] The scale has a sensible number of points with clear anchors (labels) at each end, and a neutral midpoint where appropriate [ ] Direction is consistent (for example higher always means better) across items [ ] The scale suits the question (agreement scales on factual questions are weak) [ ] Respondents can answer from experience (not hypothetical futures) [ ] "Not applicable" and "prefer not to say" options exist when needed [ ] Averages are not used where distributions matter; report percentages by category [ ] Index scores combine items that belong together and are checked for consistency [ ] Changes in wording between waves are flagged because trends may break [ ] Targets are justified, not arbitrary [ ] The metric can be gamed, and incentives are checked Return a table of problems, rewritten questions, and a recommendation on what to stop reporting. Explain in plain language what each number can and cannot tell us.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.