Duplicate Detection and Record Linkage Plan
Find duplicates safely without deleting good rows.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Plan how to find duplicate records in my table. Columns and meaning: [COLUMNS_AND_MEANING]. Sample rows including suspected duplicates: [SAMPLE_ROWS]. Cost of a false merge versus a missed duplicate: [COST_OF_ERRORS]. Produce a plan in stages: 1. Define "duplicate" for this table in one sentence (same person, same transaction, same event) and list columns that should and should not count. 2. Exact duplicates: code with duplicated(keep=False) and a count of groups. 3. Near duplicates: blocking keys to avoid comparing every pair (for example first three letters of surname plus postcode), a comparison function per field (exact, normalized, fuzzy, numeric tolerance), and a weighted score with thresholds for auto-merge, manual review, and no match. 4. A survivorship rule for picking the surviving record (most complete, most recent, trusted source) written as an ordered list. 5. A review sample: draw 50 pairs at random from each band, label them, and compute precision for the auto-merge band. If precision is below your target, say what to change. 6. An audit trail: never delete, flag with a group id and keep the originals. Privacy: note that linking records about people may be regulated [VERIFY in [JURISDICTION]].
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.