Foundry AI Data Leakage: How to Split HPDC and LPDC Data for Trustworthy Validation

Serdar Perçin
21 hours ago
5 min read

Short answer: A random row split is usually not enough for foundry AI. If neighbouring shots, the same die or cavity, one melt batch, or later inspection information appears on both sides of the split, the model can look accurate while failing on the next shift, campaign or tool change. Validation should preserve production groups, respect time, and test the operating condition the model will actually face.

Why a random split can give the wrong answer

Foundry records are not independent lottery balls. Consecutive HPDC shots share die temperature, spray behaviour, lubricant condition, metal history, vacuum performance and machine state. LPDC castings may share furnace batch, pressure programme, cooling circuit state and mould campaign. Parts from one cavity or die can also carry a repeatable signature.

A random 80/20 split can therefore place almost identical production neighbours in both training and test data. The test set is technically unseen, but its conditions are already represented in the training set. This is a form of data leakage: the evaluation receives information that would not be available when the model meets genuinely new production.

Leakage is not limited to accidentally including the quality result as an input. It can also enter through a rolling mean calculated with future shots, scaling fitted on the complete dataset, a rework code recorded after inspection, or image variants of the same casting distributed across different folds.

The foundry question that must be defined first

Before choosing an algorithm, write one sentence that fixes the prediction moment and the deployment target. For example:

At part ejection, use only signals available up to that moment to estimate whether the casting should receive priority inspection; the model must remain valid for the next production campaign on the same approved part and process route.

That sentence defines which columns are legitimate, which future records are prohibited, and what “unseen” must mean. A model intended for the next shot needs a time-respecting test. A model intended to transfer to another die, cavity or machine needs a group holdout that deliberately withholds that condition.

A seven-step validation gate for HPDC and LPDC

Freeze the prediction timestamp. List every feature and confirm that its value exists before the decision is made. Quarantine final inspection, rework, sorting and downstream machining fields unless they are the target.
Build a production-group hierarchy. At minimum preserve part number, process route, machine, die, cavity, melt or furnace batch, alloy lot, shift, tool campaign and maintenance state. The strongest shared cause becomes the grouping key for each validation question.
Remove duplicate representations. Keep repeated images, reprocessed sensor windows and corrected copies of one casting in the same partition. Split by the physical casting or shot identity, not by the number of files.
Run more than one realistic test. Use grouped validation for repeated production families, a forward time split for the next campaign, and—when transfer is claimed—a deliberately withheld die, cavity, machine or alloy condition. GroupKFold and TimeSeriesSplit are implementation patterns, not substitutes for process knowledge.
Fit preprocessing inside training only. Imputation, scaling, feature selection, oversampling, thresholds and hyperparameter tuning must learn only from the training portion of each fold. Lock the complete pipeline before opening the final test set.
Score the decision, not only the classifier. Report confusion matrices and class-specific recall, especially for rare defect modes. Separate false-release risk from false-reject cost. A single accuracy or average cross-validation score can conceal the failure that matters most to production.
Define the refusal boundary. If the test condition lies outside approved part, alloy, die, sensor or process ranges, the model should flag insufficient evidence and route the decision to engineering. Passing a historical test does not authorize automatic parameter changes.

Modelled example: a validation plan, not a customer result

Consider a purely illustrative dataset of 30,000 casting records from two machines, three dies, four cavities and six months of production. The numbers are modelled only to explain the method.

A random split could place shots from the same warm-up sequence, melt batch and cavity on both sides. A more defensible plan would train on months one to four, tune with group folds that keep melt batches and tool campaigns intact, validate on month five, and reserve month six as an untouched forward test that includes the next maintenance state. A separate leave-one-die or leave-one-cavity challenge would be required before claiming transfer.

The important output is not a promised percentage. It is a transparent statement of where the model has evidence, where it has not, and which production change invalidates the claim.

What the evidence supports—and what it does not

The International Journal of Metalcasting has documented the practical foundry challenge of joining material, machine time-series and inspection records at different levels, and also notes the difficulty created by rare failures and small datasets. That supports disciplined data linking and cautious evaluation; it does not prove that one split design or model works for every casting line.

Research on leakage shows that contaminated evaluation can produce overly optimistic findings across many fields. Official scikit-learn guidance likewise recommends separating test data before preprocessing, using non-overlapping groups when samples share a source, and respecting order when future data must not train the past. NIST frames validation as objective evidence for an intended use and calls for testing, evaluation, verification and validation throughout the AI lifecycle.

None of these sources replaces foundry engineering judgement. The correct grouping boundary depends on the causal structure of the line, the intended decision and the cost of error.

My engineering position

My default position is conservative: if a model cannot pass a time-respecting test and a production-group challenge, it is not ready to advise the next campaign. A lower but honest score is more valuable than a high score built on shared shots, shared tools or future information. The purpose of validation is not to defend the model; it is to discover where the model should refuse.

This gate continues Castella’s work on casting quality-label measurement systems, shot-to-defect traceability and model drift and refusal.

Frequently asked questions

Is a random train/test split ever acceptable in a foundry?

Only when records are genuinely exchangeable and no meaningful shot, part, tool, batch or time dependency crosses the boundary. That condition should be demonstrated, not assumed.

Should I use grouped cross-validation or a time split?

Use the split that matches the deployment claim. Grouping tests independence from a shared source; a forward time split tests the next period. Mature validation often needs both.

What is the best grouping key?

The strongest source of shared variation for the question being tested. It may be physical casting ID, shot sequence, melt batch, die, cavity, tool campaign or maintenance state. One universal key does not exist.

Does a passed validation test permit closed-loop parameter control?

No. Predictive validation is only one layer. Automatic control also requires bounded actions, process-safety constraints, change management, monitoring and an engineer-approved fallback.