Batch Job and ETL Performance Review
Speed up scheduled data processing.
The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.
Speed up a scheduled data processing job. Job description and schedule: [JOB_DESCRIPTION]. Language and tools: [LANGUAGE_AND_TOOLS]. Data volumes and run time now versus the target: [VOLUMES_AND_TIMES]. Code or SQL of the slow stage: [PASTE_CODE]. Logs or timings per stage: [TIMINGS]. Return tables: 1. Stage timing: Stage | Time | Share of total | Rows | Throughput | Waiting (I/O, CPU, lock, network). 2. Bottleneck hypotheses: Hypothesis | Evidence | How to test | Expected gain. 3. Fix options: row-by-row to set-based operations, batch size, indexes or partitions, incremental loading instead of full reload, parallelism with limits, avoiding repeated lookups, compression, and moving heavy work closer to the data. 4. Risk table: Fix | Correctness risk | Resource risk | Rollback. 5. Test plan: equal output comparison (row counts and checksums), timing on a production-sized copy, and monitoring after release. Recommend the order of work, starting with measurement, and an alert for jobs that exceed their normal run time. Do not optimize stages that are not on the critical path.
Fill in before use
Optional: type your details and the prompt above updates. Nothing is saved or sent.
Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.