Deep learning has automated many forms of representation learning, but tabular machine learning still rewards deliberate feature engineering. In business data, useful predictive structure often appears as recency, frequency, ratios, historical aggregates, and domain-specific states rather than as clean raw columns.

An operational table is usually designed to support transactions, reporting, or administration. It may contain dates, identifiers, mutable status fields, sparse categories, missing values, and historical artifacts. A predictive model receives these columns without understanding the process that produced them.

The central problem is therefore not how to create as many variables as possible. It is how to construct a compact representation of the decision using only information that would have been available when the prediction was made.

This article treats feature engineering as part of model design. The main argument is that a feature is useful only when it improves the decision, survives out-of-sample evaluation, and can be reproduced at prediction time without using future information.

The model sees columns, not the business process

Images contain spatial relationships. Text contains token order and linguistic structure. Tabular business data is less regular.

One dataset may combine customer identifiers, event timestamps, product groups, financial values, binary flags, and categories created by several operational systems. Two columns with the same data type may represent completely different mechanisms.

A model does not know that a date represents the last completed transaction, that a status field is updated after an account closes, or that a missing value means a customer was not eligible for a process. It only sees numerical or encoded inputs.

Feature engineering translates operational history into variables aligned with the prediction problem.

The feature family does not determine whether a variable is valid. The data boundary does.

The correct unit is a prediction made at a specific time

Suppose the system makes a prediction for entity \(i\) at time \(t_i\).

Let \(H_i(t_i^-)\) denote the information recorded before that prediction time. A valid historical feature can be expressed as:

$$x_{i,j} = g_j\left(H_i(t_i^-)\right)$$

Here, \(g_j\) is the transformation used to construct feature \(j\). It may count events, calculate an average, find the most recent timestamp, encode a category, or combine several historical variables.

The important part of the definition is \(t_i^-\). The transformation may use information available before the decision, but not events recorded after it.

A leaked feature instead depends on future information:

$$x_{i,j}^{leak} = g_j\left(H_i(t_i^-), H_i(t_i^+)\right)$$

where \(H_i(t_i^+)\) contains events or updates occurring after prediction.

The model may achieve excellent validation results when the same leaked value is present in both training and test data. Those results do not describe a deployable prediction process because the future component will not exist when the real decision is made.

The strongest feature may be the first one to investigate

A surprisingly large improvement should trigger a leakage investigation before a celebration.

Leakage frequently enters through variables that look operationally legitimate. A final account status may summarize what happened after prediction. A transaction total may have been calculated at the end of the month rather than at the decision date. A categorical encoding may have used target values from the validation rows.

The statistical model does not distinguish legitimate signal from information that would have been unavailable. If the leaked variable predicts the outcome well, the model will use it.

This is why feature validation cannot rely only on importance scores or metric lift. Each strong feature needs a temporal and operational explanation.

A safe historical aggregate

Consider a small transaction table. The model must generate features on day 7.

The valid history contains only events with event_day < 7. Later transactions must be excluded even if they are already present in the analytical dataset when the model is trained.

import pandas as pd


events = pd.DataFrame(
    {
        "customer_id": [1, 1, 1, 2, 2],
        "event_day": [1, 4, 8, 2, 9],
        "amount": [100, 140, 90, 50, 120],
    }
)

prediction_day = 7

history = events.loc[
    events["event_day"] < prediction_day
]

features = (
    history.groupby("customer_id")
    .agg(
        transaction_count=("amount", "size"),
        amount_mean=("amount", "mean"),
        amount_total=("amount", "sum"),
    )
    .round(1)
)

print(features.to_string())

The following output was produced by executing the code:

             transaction_count  amount_mean  amount_total
customer_id
1                            2        120.0           240
2                            1         50.0            50

The transaction on day 8 for customer 1 and the transaction on day 9 for customer 2 are absent from the features. This is not a minor implementation detail. It defines the historical information set represented by the model input.

The same logic must also be applied during validation. If historical aggregates are calculated once using the full table and the data are split afterwards, future records can already be embedded in the features.

A leakage experiment

The following synthetic experiment demonstrates how one unavailable variable can transform a realistic validation result into an implausibly strong one.

The dataset contains 20,000 customers. Five features represent information available before the churn prediction: tenure, recent sessions, support activity, recent spending, and days since the last session.

The variable closed_account_flag is different. It is generated after the churn outcome and closely reflects whether the customer actually closed the account. Including it in the model creates target leakage.

The data are synthetic. The experiment demonstrates the effect of the information boundary and should not be interpreted as evidence about real churn behavior.

import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split


rng = np.random.default_rng(42)
n_samples = 20_000

tenure_months = rng.integers(
    1,
    61,
    size=n_samples,
)
sessions_30d = rng.poisson(
    5 + 0.04 * tenure_months
)
support_tickets_30d = rng.poisson(
    0.8 + 0.2 * (sessions_30d < 3)
)
spend_30d = rng.gamma(
    shape=2.0,
    scale=40.0,
    size=n_samples,
)
days_since_last_session = np.clip(
    rng.exponential(
        scale=8.0,
        size=n_samples,
    ),
    0,
    60,
)

logit = (
    -1.8
    - 0.12 * sessions_30d
    \+ 0.055 * days_since_last_session
    \+ 0.28 * support_tickets_30d
    - 0.006 * spend_30d
    - 0.01 * tenure_months
)

churn_probability = (
    1.0 / (1.0 + np.exp(-logit))
)
churn = rng.binomial(
    1,
    churn_probability,
)

# This field is updated after the outcome.
closed_account_flag = np.where(
    churn == 1,
    rng.binomial(
        1,
        0.95,
        size=n_samples,
    ),
    rng.binomial(
        1,
        0.03,
        size=n_samples,
    ),
)

data = pd.DataFrame(
    {
        "tenure_months": tenure_months,
        "sessions_30d": sessions_30d,
        "support_tickets_30d": (
            support_tickets_30d
        ),
        "spend_30d": spend_30d,
        "days_since_last_session": (
            days_since_last_session
        ),
        "closed_account_flag": (
            closed_account_flag
        ),
        "churn": churn,
    }
)

safe_features = [
    "tenure_months",
    "sessions_30d",
    "support_tickets_30d",
    "spend_30d",
    "days_since_last_session",
]

leaky_features = (
    safe_features
    + ["closed_account_flag"]
)

train, test = train_test_split(
    data,
    test_size=0.30,
    random_state=42,
    stratify=data["churn"],
)

for name, feature_names in [
    ("safe", safe_features),
    ("leaky", leaky_features),
]:
    model = LogisticRegression(
        max_iter=2_000
    )
    model.fit(
        train[feature_names],
        train["churn"],
    )

    probabilities = model.predict_proba(
        test[feature_names]
    )[:, 1]

    auc = roc_auc_score(
        test["churn"],
        probabilities,
    )

    print(f"{name}_auc={auc:.3f}")

print(
    "churn_rate="
    f"{data['churn'].mean():.3f}"
)

The following values are the actual output from that execution:

safe_auc=0.699
leaky_auc=0.976
churn_rate=0.079

The large improvement is evidence of a broken experiment

The model using only prediction-time features achieves a test ROC-AUC of 0.699. This indicates moderate separation in the synthetic dataset.

After adding closed_account_flag, ROC-AUC rises to 0.976. The stronger score does not represent better feature engineering. It results from supplying the model with a field created after the outcome.

A random train-test split does not protect against this type of leakage. The split separates rows, but both partitions contain the same invalid feature definition. Validation faithfully estimates performance for a process in which the future status is already known.

The correct fix is not stronger regularization or a more conservative model. The feature must be removed or redefined so that its value can be reconstructed at prediction time.

Validation must reproduce the information flow

A validation split is credible only when it represents the sequence in which information becomes available.

For a temporal prediction problem, training should normally precede validation in time. This prevents the model from learning directly from later periods and makes changes in data distributions more visible.

However, a time-based split is not sufficient by itself. Feature calculations must also respect time. A rolling aggregate calculated over the entire table can leak future events even when the final model is trained with an earlier time period.

Scikit-learn recommends placing learned preprocessing inside a Pipeline so that it is fitted only on the training portion of each split. For time-ordered observations, TimeSeriesSplit provides expanding training sets and later evaluation sets, although the exact split design must still match the application.

Categorical encodings require particular care

Low-cardinality categories can often be represented with one-hot encoding. High-cardinality variables such as merchant identifiers, product codes, or locations require more deliberate treatment.

Target encoding replaces a category with a statistic derived from the target, such as a smoothed category-level outcome mean. This can create a compact and useful representation, but it also creates a direct leakage path.

If a row contributes its own target to its encoded value, the feature partially reveals the answer. The risk becomes severe for rare categories because one observation can dominate the category statistic.

Safer implementations use out-of-fold or ordered calculations. Scikit-learn's TargetEncoder uses internal cross-fitting when transforming training data. CatBoost uses ordered target statistics and ordered boosting to reduce prediction shift associated with target leakage in categorical processing.

These methods reduce a specific leakage mechanism. They do not prove that the category is stable, available in production, or meaningful for the future population.

Feature engineering is model-dependent

A feature is not universally good or bad. Its value depends partly on the representation already available to the estimator.

A tree can learn that low activity and high recency form a risky combination. It cannot reconstruct historical activity when the raw table contains only individual events and no usable temporal representation.

Likewise, a neural model can learn an embedding for a product identifier. It cannot determine that the identifier changed definition after a system migration unless that process is represented in the data and evaluation.

Ratios and aggregates need semantic definitions

Ratios can normalize customers or products with very different scales, but the denominator defines their meaning.

For example, complaint count divided by order count may distinguish one complaint in two orders from one complaint in two hundred orders. The ratio becomes unstable for customers with very few orders and undefined when the denominator is zero.

A robust definition should specify three elements:

• the numerator and denominator windows,
• the treatment of zero or missing denominators,
• the minimum history required before the value is considered reliable.

Rolling aggregates require similar precision. "Average spending" is incomplete unless the definition specifies the time window, eligible transaction states, currency treatment, refund handling, and prediction-time boundary.

These details are not merely data-cleaning choices. They define the model input.

Feature quality is more than model importance

Feature importance measures how a fitted model used a variable under a particular dataset and estimator. It does not establish that the variable is causally meaningful, stable, or safe for deployment.

A useful feature should be evaluated across three dimensions:

• Predictive contribution: does it improve decision-relevant validation results beyond a baseline?
• Stability: does its value persist across time periods, folds, and important segments?
• Operational validity: can it be calculated consistently, on time, and at acceptable cost?

A feature that dominates only one split may represent a temporary artifact. A feature that improves average quality but fails for a key segment may require modification or removal. A strong offline feature may still be unsuitable if it arrives too late for the decision.

Use ablation rather than collecting features indefinitely

Adding variables can improve training performance while increasing system complexity, missingness, maintenance cost, and exposure to drift.

I would compare a candidate feature set against a deliberately simple baseline and then remove groups of related features. This reveals whether a feature family contributes stable information or merely duplicates existing variables.

The relevant comparison is not only the highest validation score. It also includes variance across splits, inference cost, data dependencies, and the consequences of feature failure.

A slightly weaker model with a smaller, stable feature set can be preferable when its inputs are easier to reproduce and monitor.

A feature contract makes assumptions explicit

A feature definition should describe both its calculation and its availability.

feature_contract = {
    "name": (
        "customer_amount_mean_30d"
    ),
    "entity": "customer_id",
    "source": "transactions",
    "calculation": (
        "mean(amount) over accepted "
        "transactions"
    ),
    "time_window": (
        "[prediction_time - 30 days, "
        "prediction_time)"
    ),
    "freshness_limit": "6 hours",
    "null_policy": (
        "fallback_to_segment_median"
    ),
    "owner": "customer-data-platform",
}

The contract prevents the same feature name from quietly acquiring a different meaning in another pipeline.

At minimum, it should make three properties explicit:

• which records and time interval contribute to the value,
• when the feature is expected to become available,
• what happens when the value is missing, stale, or invalid.

The contract should be versioned and connected to executable tests. Documentation alone cannot detect that the production query no longer matches the definition.

Training and inference must share semantics

Feature logic should live somewhere more stable than a notebook cell.

This does not require a feature store in every project. A shared SQL transformation, versioned Python package, reproducible batch query, or tested transformation pipeline may be sufficient.

The important requirement is semantic equivalence. Training and inference should agree about timestamps, missing values, category mappings, units, aggregation windows, and filtering rules.

For batch systems, this can be tested by reconstructing historical feature values from point-in-time data. For online systems, matched records can compare offline features with values produced by the serving path.

A difference between the two paths should be treated as a model-input defect rather than as harmless implementation variation.

A practical feature-validation sequence

I would organize feature development into three stages:

1. Define the information boundary. Specify the prediction timestamp, available history, target window, and forbidden future information.
2. Evaluate representation. Compare candidate features with a simple baseline using leakage-safe preprocessing and validation aligned with deployment.
3. Validate operation. Test reproducibility, freshness, missingness, segment stability, and equivalence between training and inference.

The stages are iterative. A production limitation may require a different feature, while validation may reveal that an operationally convenient variable does not provide stable signal.

The production interpretation is conservative

A feature should not be promoted because it has high importance or produces one impressive metric improvement.

Before including it in a production model, I would require evidence that it satisfies three conditions:

• it uses only information available at the decision time,
• its contribution survives appropriate out-of-sample and segment evaluation,
• its definition can be reproduced and monitored in the production environment.

These requirements reduce the number of candidate features. That is usually beneficial.

The purpose of feature engineering is not to maximize column count. It is to create a representation that remains meaningful when the training notebook is gone and the system must make predictions on new data.

Key takeaways

• Feature engineering for tabular ML is representation design constrained by the prediction-time information boundary.
• A large validation improvement can indicate leakage when the feature depends on future events or target-derived preprocessing fitted outside the training fold.
• The most useful production features are predictive, stable, reproducible, and available when the decision must be made.

Sources

1. Kuhn, M., and Johnson, K. (2019). Feature Engineering and Selection: A Practical Approach for Predictive Models. Chapman and Hall/CRC.
2. Micci-Barreca, D. (2001). A Preprocessing Scheme for High-Cardinality Categorical Attributes in Classification and Prediction Problems. ACM SIGKDD Explorations Newsletter, 3(1), 27-32.
3. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., and Gulin, A. (2018). CatBoost: Unbiased Boosting with Categorical Features. Advances in Neural Information Processing Systems, 31.
4. Scikit-learn developers. Common Pitfalls and Recommended Practices. Scikit-learn documentation.
5. Scikit-learn developers. TimeSeriesSplit. Scikit-learn documentation.
6. Scikit-learn developers. Target Encoder's Internal Cross-Fitting. Scikit-learn documentation.