Over the weekend my girlfriend was busy. So what’s a guy to do? This guy built a CNN image recognition system and trained it on five types of flower. With over 3,000 flower images, it worked. Point it at a sunflower and yes, it predicted a sunflower. Same with roses, tulips, dandelions and daises.

But what flower is my dog? Yes, I pointed my model at an image of my dog sat upright. The model predicted with 49% confidence my dog is a tulip. My model isn’t broken. It only knows 5 answers, so everything in its world is one of five flowers. It answered the question it was built for, not the one I asked. This is the problem Marco has just brought home to Castellan Wines – our fictitious winery.

“It all just works”

Castellan Wines has a good problem. Sales have grown faster than the bottling hall, and the manual inspection that coped with a quieter line no longer does. It is still done by eye. It is slow, it varies by shift, and a bad bottle reaching a retailer costs far more than the bottle is worth.

CEO Marco has seen the answer. He came back from an AI conference with a tote bag and a conviction: you point cameras at the bottling line, the AI learns what a good bottle looks like, and it all just works. He watched it happen on a stand. And he was filled with the promise of AI.

The demo was real. It was also a demo: one bottle shape, perfect lighting, and defects chosen to be found.

So Clare has a one-line brief, “build me a defect detection model”. Her harder job, is to deliver AI Reality. She is Castellan’s data scientist, and she is honest about being new to computer vision and convolutional neural networks (CNNs). So she does the sensible thing. She experiments first and writes the blueprint second, and she organises both around three questions.

Question one: what decision are we supporting?

Marco’s brief is one model, pass or fail. Clare spends a morning on the line with the bottling line inspectors and writes down what they check.

They look for damage: is there a chip or a crack, and where? They check the fill level, which is a measurement. They check the cap is sitting square. And they check the label: the right one, straight and readable, which is reading text and checking position.

Four different jobs. Roll them into one pass/fail and QA gets a red light with no idea why.

Then there’s the camera. Clare props her iPhone beside the line and films a few minutes. Cracks show up reasonably well. The fill line doesn’t. Through dark, reflective glass, reflections can move the apparent liquid line more than any real variation does.

A setup good enough to spot a cracked bottle is not automatically good enough to measure one. I wrote about this in Now You See It, Now You Don’t: a model can only work with what the camera lets it see.

And as for the data, which is Charlie’s department. Castellan has a year of QA logs and almost no labelled images. Defects are rare, which is good news for the winery and awkward for anyone training a computer vision model.

So the first page of the blueprint doesn’t say “quality model”. It says four tasks, each with its own evidence, its own camera requirement, and its own answer.

Question two: do we need machine learning at all?

Here’s the answer nobody puts on a conference slide. For fill level and cap alignment, no machine learning.

Clare gets there by experiment. Her first instinct is Marco’s: collect images, train a network. Then she looks at what she would be asking it. Is the liquid line above or below a known mark? Is the cap sitting square?

Those are questions about where an edge is, and engineers have had reliable ways of finding edges in images for decades.

So she builds a short, old-fashioned pipeline in an afternoon. Capture the frame, remove the noise, boost the contrast, find the edges, compare the liquid line against a calibrated reference, and pass or fail.

The order matters. On her first attempt she boosts the contrast before removing the noise, and amplifies the very speckle she wanted rid of.

No training data. No GPU. Nothing to retrain. And when a bottle is rejected, the reason is a measurement a supervisor can check with a ruler, not a confidence score.

It comes with two honest conditions. It only works once the lighting is fixed, most likely with a light behind the bottle so the liquid line shows as a clean edge. And a fixed reference needs recalibrating whenever the bottle format or the lighting changes. That is a job for a technician with a checklist, not a data scientist with a GPU.

Marco is not thrilled. He was promised AI. Clare’s answer is that you can crack a nut with a sledgehammer, but you don’t get much nut. He’s not impressed.

Where the environment is controlled and the decision can be written as a rule, the classical approach isn’t the budget option. It’s the right approach every time.

Question three: where does deep learning earn its place?

Damage. Chips, cracks and contamination come in every shape, in any position, on glass that reflects everything around it. Clare can’t write a rule for “looks cracked”. This is the kind of uncertainty neural networks are good at, so this is where Marco gets his AI.

Even then, most of the blueprint is restraint.

She doesn’t start from zero. Defects are rare, so there will never be enough examples to train a network from scratch. The blueprint starts with a model that has already learned to see from millions of general images, and adapts it.

It’s like hiring an experienced inspector from another industry. They already know how to look, and you only have to teach them what a bad bottle is. Dark glass is not much like holiday photos, though, so the adapting has to go deeper than swapping the last layer.

She picks speed where the line can afford it. The line doesn’t stop, so the blueprint specifies a fast single-pass detector over a slower, more careful two-stage one. That’s normally a risk with small defects.

The camera sits at a fixed distance, so every bottle arrives the same size in the frame, which helps. A hairline crack is still a hairline crack. So the blueprint also names the evidence that would justify the heavier model: cracks missed in testing.

She decides which mistake is worse. A false alarm sends a good bottle to a human for a second look. A miss sends a cracked bottle to a customer. Those are not equal, so the threshold is set to over-report, and borderline cases go to a person.

Charlie, (our data engineer in this story), spots the catch. Defects are rare, so even a small false-alarm rate means most flagged bottles are perfectly good. Make the system too cautious and the inspectors spend their shift re-checking good bottles. The blueprint has to state both numbers: the misses Castellan will tolerate, and the false alarms the bench can absorb in a shift.

And she respects the boring details, two of which she learns the hard way.

The first is labelling. A hairline crack is thin and diagonal, so a box drawn round it is mostly good glass. Clare and Charlie mark up the same set of images separately and find they have drawn the boxes differently almost every time. Unless the people doing the labelling agree how to draw it, the model learns from an argument.

The second is her proudest experiment. She films the line for an afternoon, splits the frames at random into training and test sets, and gets a score that would look wonderful on a conference slide.

Charlie points out that consecutive frames are nearly identical. Her model was being tested on pictures it had practically seen. She splits by production batch instead, and the score comes back down to earth.

Neither detail came up on the conference stand, and there’s no reason it would in ten minutes. Both decide whether an accuracy figure means anything.

The blueprint doesn’t say whether Castellan buys this or builds it. Mature products exist for damage detection and one may well be the right choice. What Clare’s experiments give Marco is the list of questions to ask whoever supplies it.

One line, three answers

Marco asked for one model. The blueprint gives him three answers.

Damage gets deep learning, adapted from a pretrained model, because it is too variable to write as rules. Fill level and cap alignment get classical image processing, once the lighting is sorted, because they are geometry against a known reference.

Labels wait. Reading text is something you can buy off the shelf. Nobody has yet shown that a camera on this line can see a curved, moving label well enough to read it.

It isn’t what Marco heard at the conference. It is something he can cost, schedule and check.

Computer vision is not black magic. It’s a chain of engineering decisions, and most of them are about the camera and the data, not the model. Use the simplest thing that works, and step up only when the evidence tells you to.

There’s one question Clare hasn’t answered, and Marco has already asked it: how do we know it works, and how do we know it keeps working? Lighting changes, new bottle formats arrive, cameras get knocked. A vision system is not fit and forget. That’s the next post.

Reality has to be built in, not bolted on.