Which parts will fail quality control?

1.18 million real parts from a Bosch assembly line, 52 stations, 0.58% failing final inspection. The useful question is not a leaderboard metric but an inspection budget: pull the riskiest few percent of parts, and how many failures do you stop?

Code on GitHub Kaggle competition

0.418Kaggle private MCC (winner ~0.52)

61%of failures caught inspecting 5% of later parts

11xfailure rate next to a failed part

40%cost of quality vs no inspection, escape = 100x inspection

01Failures come in lots

Parts produced next to a failed part fail at 6.4%, eleven times the base rate. The strongest features describe where a part sits in its lot: whether the next part has a consecutive Id and the same start time. They use timestamps and Ids only, never a neighbour's label, so nothing leaks.

Features	CV MCC	Later parts MCC
968 measurements	0.228	0.061
+ process (route, stations, time per line)	0.231	0.038
+ sequence (position in the production flow)	0.402	0.207

02Cross-validation flatters the model

Bar chart of MCC by feature set under shuffled CV and a temporal holdout; every set scores much lower on later parts, measurements only falling from 0.228 to 0.061. — Shuffled CV mixes parts from the whole period. On parts produced after the training period, measurements alone drop to 0.061; only the sequence features hold up, and their threshold still transfers (0.207 vs an oracle 0.208).

The Kaggle test set interleaves with training parts in time, so the leaderboard (0.418) tracks CV. A model running on the line would see the temporal number, and that is the one the inspection policy below is built on.

03From a score to an inspection policy

Capture curve: percent of failures caught against percent of parts inspected on a log axis; the sequence model reaches about 74% at 5% inspected, measurement-only models about 35%, random inspection 5%. — Out-of-fold capture curve. On later parts the sequence model catches 22% of failures at 1% inspected, 32% at 2%, 61% at 5% and 77% at 10%.

With inspection costing 1 and an escaped failure costing 100, the cost-minimising policy inspects 6.6% of production and brings the cost of quality to 40% of inspecting nothing.

04Caveats

Measurements are anonymous, so the few that matter (stations S29, S24, S33) cannot be traced to physical quantities.
One model family, default-ish hyperparameters, no ensembling beyond three seeds.
The cost model assumes inspection catches every failure it examines.