Click here for – Part 1: The Artificial Neuron (Node)

---

Picture a modern manufacturing plant, bristling with IoT sensors. Temperature readings in the hundreds. Vibration readings out to several decimal places. Every few seconds, a live feed of numbers, feeding into a system whose only job is to answer one very important question: “Is a machine about to fail?”

Feed those numbers into a neural network, train it on enough historical data, and it learns to answer that question — reliably enough that plants genuinely use systems like this to schedule maintenance before a breakdown rather than after one. Which raises the obvious question. How does a network built from nothing but weighted sums and simple functions learn something as specific as “these vibration patterns, at this temperature, mean trouble in the next 24 hours”? Nobody wrote that rule down for it. Where does that knowledge actually come from?

To answer that, we need to go back to a puzzle that looks nothing like a factory floor at all.

Back to School

Miss Smith once set the class a problem she knew none of us could solve. Four dots on a grid, two black, two white, arranged so that no single straight line could separate the colours. “Draw me one line,” she said, “that puts all the black dots on one side and all the white dots on the other.” We tried. We failed. She let us fail for a good five minutes before she told us it was impossible — not hard, impossible — and that the only way to solve it was to stop drawing one line and draw two lines instead. I didn’t know it at the time, but I’d just met XOR, decades before I’d hear the name.

XOR — “exclusive or” — is about as simple as logic gets. It’s true when exactly one of two inputs is true, and false otherwise. A child can learn the rule in ten seconds. And yet this is precisely the problem that stopped the Perceptron dead in 1969. Minsky and Papert proved, with unarguable maths, that a single artificial neuron — one weighted sum, one activation function, one decision boundary — could never learn XOR. The four points of the problem simply cannot be separated by one straight line, in exactly the same way Miss Smith’s dots couldn’t be. The field took the result badly. Funding dried up. The first AI winter set in, and it lasted the best part of a decade.

The way out, when someone finally found it, was almost insultingly obvious in hindsight: if one line isn’t enough, use two. So if one neuron isn’t enough, use more than one — and arrange them in layers. That single idea is also the answer to the factory floor question. A vibration-and-temperature failure signature isn’t separable by one straight line any more than XOR is. It needs a network, not a neuron.

From one neuron to a neural network

A single artificial neuron, as we covered last time, takes some inputs, weights them, sums them, and passes the result through an activation function. It draws one boundary. Stack several neurons side by side, and feed all of their outputs forward into another neuron, and something changes qualitatively. Each neuron in that first layer can learn its own boundary — its own “line” — and the neuron receiving all of those outputs can combine them into a shape no single line could draw on its own. Two lines, combined correctly, can box in XOR’s four points perfectly.

This arrangement is called a feedforward neural network. Data enters through an input layer, where each neuron corresponds to one feature — a pixel value, a sensor reading, a temperature. It’s not a computational layer so much as a doorway. From there, the data passes through one or more hidden layers, where the actual work happens: each neuron takes a weighted sum of everything the previous layer handed it, adds a bias, and squashes the result through an activation function, exactly as we walked through with a single neuron last time. “Feedforward” simply describes the traffic rules — information only ever moves forward, layer to layer. Finally, an output layer produces the answer in whatever shape the problem demands: one neuron for a yes/no decision, ten neurons if you’re identifying a handwritten digit.

When every neuron in one layer connects to every neuron in the next, you’ll hear it called a “fully connected” or “dense” layer. Remember that phrase — it matters later in this series, because not every network wires its layers this way.

How it learns

None of this is any use if the weights are wrong, and at the start, they are — set randomly before training begins. The network makes a prediction, a loss function scores how wrong that prediction was, and backpropagation works backward through the network asking, for every single weight, “if I nudged you slightly, would the error go up or down?” Then it nudges every weight a small step in whichever direction reduces the error. Do that across enough examples, enough times, and the weights settle into values that actually mean something.

The choice of loss function matters more than people expect. Predicting a continuous number — a machine’s remaining useful life — usually calls for mean squared error, which punishes large mistakes disproportionately by squaring them. Classifying something into categories — will it fail or won’t it — calls for cross-entropy, which measures the gap between two probability distributions and comes down hardest on predictions that are confident and wrong.

Nobody told the neural network machine vibration readings above a certain threshold, combined with temperature, predict a bearing failure. It found that out for itself, by being wrong thousands of times and adjusting. It’s brute force with a compass.

Why depth changes what’s possible

There’s a result called the Universal Approximation Theorem worth knowing: a feedforward network with even a single hidden layer, given enough neurons, can approximate any continuous function to any desired accuracy. Not some functions. Any of them. That’s not an incremental step up from linear regression — it’s a different category of tool, which is the argument this whole series is built on.

There’s a catch worth naming honestly, though. A fully connected feedforward network treats every input as independent and unordered. Feed it a photograph as a flat list of pixel values, and it has no innate sense that neighbouring pixels are related. Feed it a sentence word by word, and it has no sense that word order carries meaning. It can technically learn these relationships given enough data, but it’s fighting its own architecture to do so — like teaching someone geography from an alphabetically sorted list of place names. That gap is exactly the seam the next two parts of this series stitch into: convolutional networks give a network something like an eye, recurrent networks and LSTMs give it something like a memory.

Back to the factory floor

Which brings us back to where we started. Before any of this works, the temperature and vibration readings need normalising onto comparable footing — feed the network raw values and the numerically larger temperature readings will dominate the weighted sums regardless of which feature actually matters more. Then comes the real design decision: are you predicting whether the machine fails in the next 24 hours — a classification problem, one output neuron, sigmoid activation, cross-entropy loss — or how many days of useful life remain — a regression problem, one output neuron with no squashing activation, mean squared error as the loss?

And one trap worth knowing before you build anything like this: in a working manufacturing plant, failures are rare, maybe one in a hundred. A classifier trained carelessly on that data can hit 99% accuracy by predicting “no failure” every time — right 99 times out of 100, and wrong about the one thing you built it to catch. It’s why the industry leans on precision and recall instead of accuracy here, and why, for maintenance specifically, missing a real failure usually costs more than a false alarm.

Miss Smith’s two-line solution to her dot puzzle wasn’t a trick. It was the whole idea in miniature: some problems need more than one boundary before the answer comes into view. Neural networks scaled that idea up from two lines to millions of weighted connections — enough to learn, from nothing but historical sensor readings, when a machine is about to fail. The next two posts are about what happens when you shape those connections to match the problem in front of you, starting with images.

---