Artificial Neural Network (ANN) by example

Schematic of a neural network with input features including age, state, income, and politics leading to two hidden layers and an output layer predicting class based on dropout connections.

As kids growing up in the 70s with no internet and no video games, we explored the world around us. Our curiosity fueled adventure and discovery. For me, I needed to understand how things worked. I remember one day in my bedroom having the thought, “I wonder what electricity looks like?” So I took the back off a plug and plugged it into the wall socket. I didn’t get to see electricity, but I certainly got to feel it. My right arm jolted as 240 volts held me pinned to the wall plug. Finally, with some effort, I grabbed my right arm with my left and pulled myself away. Of course, I never told my parents.

Growing up in an age of childhood freedom gave me curiosity. And that same curiosity is why this blog exists. I’m not going to walk through a neural network diagram with dots and lines. In this blog. Instead I’ll walk through the code of a simple ANN. Don’t worry, it’s not shocking.

In the last two posts we built up the ideas. A single artificial neuron: weighted inputs, a bias, an activation function, a decision. Then a network of them: layers stacked so that each one hands its output to the next, learning boundaries no single neuron could draw alone. We talked about backpropagation, loss functions, and why depth changes what’s possible.

Now let’s get under the hood.

The code below is a real, working neural network — not a toy illustration, not pseudocode. It’s written in Python using TensorFlow and Keras, which are the tools most practitioners actually reach for. The problem it solves is deliberately simple: given a person’s age, state, income, and political leaning, predict whether they work in Engineering or Education. Two possible answers. One network to find the boundary between them.

That’s a binary classification problem — the same category as “will this machine fail in the next 24 hours?” from the factory floor. Same structure, different data.

Step 1: The Data

Python code snippet for loading datasets using Pandas, including reading training and testing data from CSV files and defining column names.

Two hundred people for training, forty held back for testing. Each row is one person. The Sector column is the answer — 1 for Engineering, 0 for Education. Everything else is a feature: the inputs the network will learn from.

Look at the numbers in the data and you’ll notice something. Age isn’t stored as 32 or 45 — it’s stored as 0.32 or 0.45. Income isn’t 82,000 — it’s 0.82. Everything has been scaled down onto a comparable footing, roughly between 0 and 1. This is data normalisation, and it matters more than it sounds. Without it, the raw income figure — a number in the tens of thousands — would dominate every weighted sum in the network simply by being numerically large, regardless of whether income is actually the most informative feature. Data normalisation levels the playing field so the network can decide for itself what matters.

The state and political leaning columns work differently. A state isn’t a number on a scale — Oregon isn’t “more” than Maine in any meaningful sense. So instead of encoding state as a single column with values 1, 2, 3, each state gets its own binary column: State_OR, State_ME, State_OK. A person from Maine gets a 1 in State_ME and a 0 everywhere else. This is called one-hot encoding, and it’s the standard way of feeding categorical data into a neural network without accidentally implying an order that doesn’t exist.

Step 2: The Architecture

Code snippet showing the architecture definition of an Artificial Neural Network (ANN) using TensorFlow and Keras, including layers, activation functions, and dropout.

Stop for a second. This is it. This is the neural network. Not a diagram. Not a concept. Five lines of Python code that define the entire structure of an ANN — the layers, the neurons, the activation functions, everything. When people talk about “building a neural network,” this is what they mean. Let’s walk through each line.

Input(shape=(8,)) — the doorway. Eight features come in: age, three state columns, income, three political leaning columns. This layer doesn’t compute anything; it just tells the network what shape to expect.

Dense(16, activation=’relu’) — the first hidden layer. Sixteen neurons, each one connected to all eight inputs. Each neuron takes a weighted sum of those eight values, adds a bias, and passes the result through ReLU. If the result is positive, it passes through unchanged. If it’s negative, it becomes zero. Sixteen different weighted combinations of the same eight inputs, each one learning to notice something slightly different about the data.

Dropout(0.2) — this one doesn’t appear in most diagrams, but it’s doing something important. During training, it randomly switches off 20% of the neurons in the previous layer on each pass. Not the same 20% every time — a different random selection each time. This sounds counterproductive. Why would you deliberately break your own network? Because it forces the network to learn redundant, overlapping representations rather than leaning too heavily on any single neuron. A network that can only function when every neuron is present is a fragile one. Dropout is the training equivalent of practising a skill with one hand tied behind your back so that when both hands are free, you’re stronger for it. At prediction time, Dropout switches off entirely — all neurons active, full strength.

Dense(16, activation=’relu’) — a second hidden layer, identical in structure to the first. Its inputs are the outputs of the first hidden layer, not the raw data. This is where depth earns its keep: the second layer isn’t looking at age and income directly, it’s looking at whatever the first layer learned to notice about age and income. Abstractions built on abstractions.

Dense(1, activation=’sigmoid’) — the output layer. One neuron, because there are two possible answers and one probability is enough to express both. Sigmoid squashes the result to a number between 0 and 1. Above 0.5, the network says Engineering. Below 0.5, it says Education. The closer to the extremes, the more confident the prediction.

The whole network has 433 trainable parameters — 433 individual weights and biases that start as random numbers and will be adjusted, thousands of times, until they mean something. Think about that for a moment. Two hidden layers of 16 neurons each, connecting 8 inputs to a single output, and the entire intelligence of this network — everything it knows about predicting whether someone works in Engineering or Education — lives in 433 numbers. The model summary tells you the memory footprint too: 1.69 KB. Not megabytes. Not gigabytes. One point six nine kilobytes. A text message takes up more space. And yet those 433 numbers, tuned correctly, can make predictions a human couldn’t easily articulate as a rule.

Step 3: Compile — Choosing the Learning Rules

Code snippet showing a summary of a model and its compilation using 'adam' optimizer and 'binary_crossentropy' loss function in Python.

Before training can begin, the network needs two things: a way to measure how wrong it is, and a strategy for getting less wrong.

binary_crossentropy is the loss function. For a binary classification problem — two possible outcomes — this is the standard choice. It measures the gap between what the network predicted and what the correct answer actually was, and it comes down hardest on predictions that were confident and wrong. A network that says “90% sure it’s Engineering” when it’s actually Education gets penalised far more than one that hedged at 55%.

adam is the optimiser — the strategy for adjusting the weights after each error is measured. Adam is an adaptive optimiser, meaning it doesn’t nudge every weight by the same amount. It keeps track of how each weight has been moving and adjusts its step size accordingly. Weights that have been oscillating get smaller nudges; weights that have been consistently moving in one direction get larger ones. In practice, Adam converges faster and more reliably than plain gradient descent for most problems, which is why it’s become the default starting point for almost everyone.

Step 4: Training — Being Wrong, Repeatedly, on Purpose

Code snippet demonstrating the training of a machine learning model using the fit method with parameters for training data, validation split, number of epochs, batch size, and verbosity.

This is where the learning happens. model.fit runs the training loop.

epochs=50 means the network will see the entire training dataset fifty times. Each pass through the data is one epoch. On each pass, it makes predictions, measures its error with the loss function, and uses backpropagation to work out how every one of those 433 weights contributed to that error. Then Adam nudges each weight in the direction that reduces it.

batch_size=16 means the network doesn’t wait to see all 200 training examples before updating its weights. It processes 16 at a time, updates, processes the next 16, updates again. This is called mini-batch gradient descent, and it’s faster and often more stable than waiting for the full dataset.

validation_split=0.2 holds back 20% of the training data — 40 people — as a validation set. The network never trains on these; they’re used purely to check, after each epoch, whether the network is actually learning something general or just memorising the training data. Watch the training output and you’ll see two accuracy figures per epoch: training accuracy and validation accuracy. If training accuracy climbs while validation accuracy stalls or falls, the network is overfitting — learning the training data by heart rather than learning the underlying pattern.

Here’s what the first few epochs look like:

Text output displaying training progress of a machine learning model across epochs with metrics including accuracy, loss, and validation loss.

Epoch 1: the network is barely better than a coin flip. The weights are random, the predictions are noise. By epoch 40, training accuracy has climbed to 71%. The network is learning. The validation accuracy tells a more cautious story — it peaks around 50%, suggesting the network is finding real patterns but the problem is genuinely hard with only 200 training examples. More data would help.

Step 5: Evaluate — Honest Accounting

Python code snippet demonstrating model evaluation, including loss and accuracy calculations, prediction generation for a confusion matrix, and printing of classification report for binary classification.

Test accuracy comes in at 65%. On 40 unseen people, the network gets 26 right. That’s better than chance, but the classification report tells the more interesting story:     

A classification report displaying precision, recall, f1-score, and support for Engineering and Education categories.

When the network predicts Engineering, it’s right 93% of the time — high precision. But it only catches half the actual engineers — low recall. It’s cautious about calling Engineering, so it misses a lot of them. Education is the mirror image: it catches almost everyone who works in Education (93% recall), but half its Education predictions are wrong (50% precision).

This is exactly the precision-recall trade-off we touched on with the factory floor. Accuracy alone — 65% — doesn’t tell you this. The confusion matrix does:

A confusion matrix displaying values in a 2x2 format with counts: [[13, 13], [1, 13]].

Thirteen engineers correctly identified. Thirteen engineers incorrectly called Education. One educator incorrectly called Engineering. Thirteen educators correctly identified. The network has a bias toward predicting Education. Understanding why — whether it’s the class balance in the training data, the features available, or the architecture — is where the real diagnostic work begins.

Step 6: Prediction — The Network in Use

A screenshot of Python code that demonstrates a function to collect user information such as age, state, income, and political leaning, and then predict a sector based on the input data.

The network is trained. Now it earns its keep.

A new person walks in: age 32, from Maine, income $82,000, politically moderate. The code normalises those inputs exactly as the training data was normalised — age becomes 0.32, income becomes 0.82 — and feeds them through the network. The 433 weights, now tuned through 50 epochs of training, do their work. The output neuron returns a single number: 0.7066.

Above 0.5. The network says: Education.

That number — 0.7066 — isn’t a certainty. It’s a probability. The network is saying “I think there’s about a 71% chance this person works in Education.” The threshold of 0.5 is a choice, not a law. In a real application, you might raise it to 0.7 or 0.8 if the cost of a wrong prediction is high. Lower it if missing a true positive is the bigger risk. The threshold is yours to set; the probability is the network’s to produce.

What Just Happened

Six steps. One hundred and something lines of Python. A network that started knowing nothing — random weights, random predictions — and ended up with a view of the world encoded in 433 numbers that no human chose or wrote down.

The architecture was ours: eight inputs, two hidden layers of sixteen neurons each, one output. The activation functions were ours: ReLU in the hidden layers, sigmoid at the output. The loss function and optimiser were ours. But the weights — the actual knowledge, the thing that makes a prediction possible — those were discovered, not designed. The network found them by being wrong thousands of times and adjusting.

That’s the idea this whole series is built on. Not a model we wrote. A model that wrote itself, in a language of numbers, from nothing but examples and error.

In the next post, we’ll dive into why more parameters isn’t always best. And like with everything, bigger takes up more memory and more GPUs. So choose wisely.

Leave a Reply