Create intermediate Works with: Any AI

Train-Test Split and Leakage Audit

Table output

Make sure evaluation scores are honest.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

Audit my evaluation setup for data leakage. Problem and target: [PROBLEM_AND_TARGET]. Features: [FEATURES]. How I split the data and preprocess it: [SPLIT_AND_PREPROCESSING]. Code: [PASTE_CODE].

Check each leakage route and mark ok, suspect, or leak, quoting my code:
1. Target leakage: features created after the outcome, or proxies of it (a refund flag when predicting churn).
2. Preprocessing leakage: scaling, imputation, encoding, or feature selection fitted on all data before splitting. Show the correct pipeline using Pipeline and fit on train only.
3. Duplicate or near-duplicate rows across train and test.
4. Group leakage: the same customer or patient in both sets (use GroupKFold).
5. Time leakage: random split on time-ordered data (use a forward-chaining split).
6. Hyperparameter tuning on the test set; show nested cross-validation or a separate validation set.

Output a table, then corrected code and a sanity test: a model trained on shuffled labels should score near chance. Say what scores would be too good to be true for this problem. List the questions to ask the data owners.

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

• leakage-audit