AI Visual Inspection and Traceability on Production Lines

AI-based visual inspection replaces or complements human quality control and classic machine vision with models trained on real images of good and defective product. Done well, it detects defects that a rule-based system cannot describe, holds a stable acceptance criterion shift after shift and, when connected to plant traceability, lets you move from "this part is bad" to "these are all the parts in the batch that might be". What follows: what it takes to get there, and where most projects stumble.

What AI-based visual inspection is

Visually inspecting a part means answering a question with the image as the only evidence: does this product conform or not? For decades that question was answered by people at the end of the line and, since the nineties, by machine vision systems applying programmed rules to the image. AI-based inspection changes the decision mechanism: instead of programming what a defect is, you train a model with examples of good and bad parts, and it is the model that learns the boundary between the two.

The change sounds subtle and is not. A rule-based system only detects what someone managed to describe in advance. A learned model detects what appears in the data, including variants of the defect nobody had formulated. In exchange, it demands something classic vision never asked for: a labelled image set representative of real production, with its variations in material, lighting and position.

To ground the concepts we will use one example throughout the article: a plastic injection line producing housings for domestic appliances. The usual defects are sink marks (depressions caused by shrinkage), flash along the parting lines, stains of degraded material and ejector marks. Some are geometric and easy to measure; others, such as a faint stain on a textured surface, are exactly the kind of defect where classic vision struggles.

From classic rule-based vision to learned models

Classic machine vision breaks inspection down into deterministic operations: thresholding, edge detection, template matching, measurement of distances and areas. A vision engineer configures each operation and sets tolerances. This approach is still the right one for many tasks: verifying a cap is present, measuring a diameter, reading a Data Matrix code. When the defect can be described with geometry and contrast, rules are fast, explainable and cheap to validate.

The problem appears with appearance defects. On the housings line of our example, a sink mark is a shallow depression whose visibility depends on the angle of the light; a stain of degraded material can have any shape, size and position. Describing that with rules means chaining dozens of filters with brittle thresholds, and every change of material colour or pellet supplier forces a readjustment. The typical result is a system that either lets defects through or rejects good parts, and that nobody on the shop floor dares to touch.

Learned models, mostly convolutional neural networks, invert the approach. There are three main families:

Approach	What it answers	What it needs	When to use it
Classification	Is the part good or bad? Which type of defect does it have?	Images labelled by class, at part level	When the verdict is enough and the defect's position is not needed
Detection and segmentation	Where is the defect and how large is it?	Images annotated with boxes or masks per defect	When position or area determines acceptance
Anomaly detection	Does this part look like the known good parts?	Mostly images of good parts	When there are too few real defects to train on

The third row deserves attention because it fits the reality of many plants: if the line produces 1% defective parts, gathering a thousand defect images means photographing a hundred thousand parts. Anomaly detection methods learn the distribution of what is normal and flag as suspect whatever strays from it, which allows a start with a short history. The trade-off is a higher false alarm rate, which should be managed as we will see further on.

The practical decision is not "rules or AI" but where to put each. On the housings line, checking the presence of metal inserts and reading the mould code are solved with rules; sink marks and stains, with a model. Mature systems combine both layers and use the rules output as a pre-check: if the part is badly positioned or the image is out of focus, there is no point asking the model for its opinion.

What it takes to work: data, lighting and integration

Labelled images: the asset nobody has on day one

The model is worth what its training data is worth. Three conditions separate a useful dataset from a collection of photos:

Representativeness. The images must cover the real variation of production: every material colour, every mould, every cavity, machine start-ups and steady-state running. A model trained only on parts from the morning shift will discover in production that the natural light in the building changes in the afternoon.
Consistent labelling. If two inspectors disagree on whether a slight sink mark is a defect, the model will learn that ambiguity. Before labelling thousands of images, the acceptance criterion must be closed in writing, with photographed borderline examples, exactly as you would with a physical defect board.
Data traceability. Every image must record which camera, which lighting, which product reference and which date it was captured with. Without those metadata it is impossible to diagnose why the model fails more on one reference than another.

As an order of magnitude, a reasonable binary classifier can start with a few hundred examples per class by leaning on transfer learning, and an anomaly detector with even fewer. The exact figure depends on the problem; what does not change is that the dataset is built continuously, not once. The system must be born with a mechanism to capture and label the doubtful cases that appear in production, because those cases are the retraining material.

Lighting and optics: the half of the project that is not software

No model compensates for an image in which the defect cannot be seen. Lighting is an engineering decision, not an accessory: low-angle grazing light to reveal relief defects such as sink marks, dark field for scratches on glossy surfaces, diffuse dome lighting for curved parts with reflections, backlighting for silhouettes. On the housings line, the sink mark that is invisible under diffuse frontal light shows up sharply under low grazing light; that test, done with the part in hand before buying anything, defines half the architecture of the inspection station.

The golden rule is to stabilise everything you can: an enclosure blocking ambient light, strobe lighting synchronised with the camera trigger to freeze moving parts, and periodic checks of lamp intensity, which degrades over hours of use. Every variation removed at capture is variation the model does not have to learn, and therefore fewer training images and fewer surprises.

In-line integration: cycle time and decision

An inspection system that cannot decide within the cycle time is useless. If the line produces a housing every four seconds, the whole chain (camera trigger, image transfer, inference and command to the reject actuator) must fit inside that margin with room to spare. This usually settles where the compute lives in favour of the edge: inference runs on a device at the line side, and the cloud is left for retraining, monitoring and archiving. The system also needs machine signals: the trigger synchronised with the part position, the result published to the PLC that drives the reject ejector, and a defined behaviour when inspection fails (does the line stop, or do parts pass flagged as uninspected?).

This side of the project is more industrial connectivity than data science: protocols with the PLC, bandwidth to move images, buffering when the network drops. It is the layer we cover in the machine vision connectivity solution, where cameras stop being islands and their results enter the same data layer as the rest of the plant's signals.

False positives and false negatives: the cost of being wrong

Every inspection system errs in two directions, and the two cost money in different ways. A false negative (letting a defective part through) costs complaints, returns and, in regulated sectors, the risk of a product recall. A false positive (rejecting a good part) costs scrap, manual reinspection and something less visible but more corrosive: the shop floor's trust. A system that frequently rejects good parts ends up with an operator going through the reject bin and returning parts to the line, and at that moment the automatic inspection has, in practice, ceased to exist.

The balance between the two errors is governed by the model's decision threshold, and setting it is a business decision, not a technical one. It is worth writing down the numbers of the specific case. On the housings line: if a false negative reaches the appliance customer, the cost includes the contract's PPM penalty and a possible 100% sort of the stock already shipped; if a false positive scraps a good housing, the cost is the material plus the machine time of that part. With those two costs on the table, deciding whether the threshold leans towards severity or leniency stops being a debate of opinions.

Three practices keep errors under control in production:

A grey zone with human review. Instead of a single threshold, two: above it, automatic rejection; below it, acceptance; in between, the image goes to a verification station. The grey zone starts wide and narrows as the model proves its reliability, and every human decision on it is a fresh label for retraining.
Continuous measurement against sampling. Periodic sampling audited by quality (accepted parts reinspected by hand) gives the real false negative rate, which the system alone cannot know.
Drift watching. A change of pellet supplier, a reworked mould or a degraded lamp shifts the image distribution. Monitoring the rejection rate and the distribution of model scores per reference and per shift detects drift before it becomes an entire misclassified batch.

Traceability: from a single defect to the affected batch

Detecting the defect is half the value. The other half appears when every inspection is tied to the part's identity and its manufacturing context: which machine, mould and cavity produced it, with which raw material batch, on which shift and with which process parameters. That link turns inspection into a source of traceability both backwards and forwards.

Backwards: if the customer complains about a stained housing, the history of images and results lets you narrow down when the problem started and what accompanied it. In the example, the correlation between the stains and a specific batch of recycled pellets is found in minutes if every inspection result stores the material batch; without that link, it is found after weeks of mould trials.

Forwards: if a systematic defect is discovered late, the urgent question is which product is affected and where it is. With traced inspection, the answer is a query: all parts from cavity 3 between 14:10 and 16:40 on Tuesday, which correspond to these pallets and these shipments. The recall shrinks from "the whole consignment" to a bounded subset, and the cost difference between the two usually pays for the system.

Technically this demands three pieces: identification of the part or the container (Data Matrix code, batch label or, at minimum, timestamp plus line position), a data model joining inspection results with product genealogy and process parameters, and image retention with a defined policy (images of rejects and grey-zone cases are kept longer than those of good parts, for which a sample may be enough). On that base, crossing inspection results with process signals is a natural case of operational analytics: the defect rate per cavity and per material batch stops being a monthly report and becomes a live signal to act on.

In-line architecture: camera, edge and data layer

A reasonable reference architecture has four levels:

Level	Components	Responsibility
Capture	Industrial camera, optics, lighting, trigger sensor	Produce a stable, repeatable image of every part
Edge	Line-side compute device with the deployed model	Inference within the cycle time and reject command to the PLC
Plant data layer	Data broker or bus, historian, batch identifiers	Join inspection result, genealogy and process
Cloud or data centre	Image store, training, model monitoring	Retrain, version and audit the deployed models

Two decisions make the difference in the medium term. The first is treating the model as a versioned artefact: every verdict on the shop floor must be attributable to a specific version of the model and the threshold, because without that there is no possible audit of a complaint. The second is not leaving the results locked inside the camera vendor's software: published on the plant data layer alongside the rest of the signals, inspection verdicts feed quality analysis, KPI calculation and process improvement, instead of living in a proprietary history that only the integrator ever consults.

How to start without burning the project

AI inspection projects fail through well-known patterns: choosing the plant's hardest defect as the first case, training on laboratory images that do not resemble line images, or deploying without a retraining plan and watching performance degrade with the first material change. The sequence that avoids most of these failures:

Choose a case with a frequent defect, a clear cost of error and an acceptance criterion that two people apply identically. Frequency matters: a defect that appears once a month generates neither data to learn from nor evidence to validate with.
Solve the capture before the model: test lighting set-ups with real parts, fix the station's mechanics and capture a first dataset under production conditions.
Validate the model in shadow mode: the system decides but does not act, and its verdicts are compared for weeks against the current inspection. This phase yields the real error rates with which to negotiate the threshold.
Go live with a wide grey zone and human review, and narrow it with evidence.
Connect the results to batch traceability from day one, even if the cross with process data comes later: the data not stored today will not exist when it is needed.

Frequently asked questions

When does AI pay off compared with classic machine vision?

When the defect is hard to describe with rules: appearance variations, textures, defects with unpredictable shape and position. For component presence or absence, dimensional measurement or code reading, classic vision remains simpler, faster and easier to validate. Mature systems combine both.

How many images does it take to train an inspection model?

It depends on the approach. A classifier using transfer learning can start with a few hundred examples per class; an anomaly detector mainly needs images of good parts, which are plentiful. More important than the initial figure is the continuous improvement mechanism: capturing the doubtful production cases, labelling them and retraining with them.

What if the system rejects too many good parts?

It is the failure mode that kills the most projects, because the shop floor stops trusting the system. It is managed with a double threshold creating a grey zone of human review, with explicit measurement of the false positive rate per reference, and with retraining from the decisions of that review. The threshold is set by comparing the real cost of each type of error, not by factory default.

How is visual inspection connected to batch traceability?

Every inspection result is stored linked to the identity of the part or the container (code, batch, timestamp and position) and to its manufacturing context: machine, mould, cavity, raw material batch and process parameters. With that link, a defect detected late translates into a query that bounds which parts, pallets and shipments are affected, instead of withdrawing entire production as a precaution.

Should inference run at the edge or in the cloud?

The accept-or-reject decision must be taken at the line side, within the cycle time and without depending on the network: that places it at the edge. The cloud contributes what is not in a hurry: image storage, retraining, model versioning and performance monitoring over time.

If you are considering a visual inspection system, the order matters: first the capture and the acceptance criterion, then the model, and from the start the connection to the plant data layer. At Captia we approach that journey through machine vision connectivity and operational analytics, within the industrial artificial intelligence unit.