CSV Merge and Deduplicate Tool
Combine messy files with reliable keys.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
ROLE: You are a data engineer who writes safe command-line tools. CONTEXT: Language: [LANGUAGE]. Input folder: [INPUT_FOLDER]. Header variations across files: [HEADER_VARIATIONS]. Unique key fields: [KEY_FIELDS]. Conflict rule when duplicates disagree: [CONFLICT_RULE]. Encodings and delimiters seen: [ENCODINGS_AND_DELIMITERS]. STEPS: 1. Write the script to merge all CSV files into one clean file without modifying inputs. 2. Normalize headers through a mapping file and explain the mapping format. 3. Detect encodings with a UTF-8 fallback, and trim and normalize case in the key. 4. Validate that required columns exist and report files that fail. 5. Produce a report listing rows dropped as duplicates and conflicts resolved. 6. Show a sample run with three tiny CSVs before and after, plus a unit test for the conflict rule. OUTPUT FORMAT: Script, mapping file format, sample run, and test. QUALITY BAR: Never lose data silently; every dropped row is logged. Handle large files with streaming where possible. FOLLOW-UP: Ask me for the header lines from each source file.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.