The Feature That Was 100% Correlated With the Answer — Because It Was the Answer - Devoured

Back to Devoured - August 31, 2026

DEVOURED

The Feature That Was 100% Correlated With the Answer — Because It Was the Answer

Data machine-learning Level Up Git Connected

Data scientists often accidentally leak target variables into training features through pre-split data processing, rendering models useless in production.

What: Target encoding and rolling averages can unintentionally include information from the future or the target label itself. The fix is to perform data splits before computing any features and to use group-aware transformations only on the training subset.

Why it matters: This underscores the need for rigorous feature engineering pipelines, as modern automated tools often ignore temporal or leakage constraints, leading to models that appear high-performing during validation but fail entirely on live data.

Takeaway: Before training, ensure all feature transformations—especially target-based encodings—are calculated using only the training split to prevent leakage.

Decoder

Target encoding: Replacing a categorical value with the mean of the target variable for that category.
Out of fold (OOF): Generating predictions or features for a training instance using a model trained on other parts of the data, ensuring the model never sees the instance it is predicting.

Original article

Target encoding, rolling averages, and shifted features can silently leak the answer when computed before a proper split or across mixed entities. The safe workflow is to split first, calculate training-set encodings out of fold, fit other statistics only on training data, use group-aware transforms, and give unseen categories honest fallbacks.

Explore all 65 articles in today's digest