API-to-Database ETL Job
Move data from an API into a warehouse reliably.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Move data from an API into a warehouse reliably. API and its limits: [API_DETAILS]. Target database and schema: [TARGET_DATABASE]. Data volume and update frequency: [VOLUME_AND_FREQUENCY]. Language and scheduler: [LANGUAGE_AND_SCHEDULER]. Write the job and verify it against this checklist: [ ] Authentication secrets come from environment variables or a secret store, never from code [ ] Incremental extraction using a cursor, updated-since timestamp, or ID, stored safely; a full backfill mode exists [ ] Pagination handled until the end, with a maximum page guard [ ] Rate limits respected; retries with exponential backoff and jitter for temporary errors; clear failure for permanent errors [ ] Raw responses are landed or logged (without secrets) so a transformation bug can be replayed [ ] Transformations are explicit: types, null handling, time zones, and field renames [ ] Loads are idempotent (upsert on a natural key or merge), so reruns do not duplicate rows [ ] Schema changes are detected and reported before they break the load [ ] Row counts and checksums are compared between extract and load, with an alert on mismatch [ ] Deleted or archived records are handled [ ] Run metadata (start, end, rows, status) is stored [ ] Alerts on failure and on silence (no run happened) [ ] Personal data is minimized and access-controlled Provide the code, the table definitions, the scheduling entry, tests with a mocked API, and a runbook for failure and backfill.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.