Feature Engineering

A feature is a transformation from raw records to a number the model can use. It is learned from training data, and it has to be reproduced identically at serving time — which is where it usually breaks.

Per-entity counts, sums, rates and recency over windows. They dominate tabular models, and they are the main source of train/serve skew because they depend on a clock, a source and a null policy at once.

Bucket edges, means and standard deviations are fitted on the training fold and shipped with the model. Refit them anywhere else and the model receives inputs from a transformation it never learned.

One-hot, ordinal and embedding encodings turn categories into numbers. Each has a vocabulary that was fitted on training data, an unseen-category policy, and a serving path that must apply both identically.

Replace a category with the mean label for that category. Powerful on high-cardinality features, and a leak unless the rate for each row is computed without that row, out of fold, with a smoothed prior.

Windows, lags, recency, calendar features and "as of" timestamps. Every one is anchored to a clock, and the rule is that the anchor is the prediction time and no window ends after it.

Why a value is missing is information. Imputation is fitted on the training fold and shipped, the missingness indicator is often the better feature, and the null policy must be identical at serving.