Pipeline from Raw Export to Analysis-Ready Table
Design a reproducible cleaning pipeline.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Design a reproducible pipeline that turns a raw export into an analysis-ready table. Raw sources and refresh frequency: [RAW_SOURCES]. Target table and its grain: [TARGET_TABLE]. Team skill level and tools: [TEAM_AND_TOOLS]. Produce: 1. A stage diagram in text: raw (never edited) -> staged (typed, renamed) -> cleaned (rules applied) -> analysis-ready (joined, derived columns), with the artifact stored at each stage. 2. For each stage the functions it needs, written as small pure functions with a docstring and an input and output contract. 3. A configuration file layout for column mappings, thresholds, and paths, so no value is hard-coded. 4. Logging: row counts in and out of each step, with an alert if rows drop by more than a set percent. 5. Quality gates between stages (schema check, null checks, uniqueness) that stop the run. 6. Idempotency: rerunning gives the same output; show how to write outputs atomically. 7. A run book: how to run, how to backfill a month, how to debug a failed run. 8. A table of decisions I must make and the default you suggest for each. Keep it simple enough for a team of two.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.