The best teachers and writers have always understood that the goal isn’t to demonstrate how much they know. It’s to change what their audience knows.
Last night my adult son walked past my office door just after midnight and stopped. He’d noticed the light still on.
“What you doing, Dad? It’s late.”
“Writing my next blog,” I said.
He paused, looked at me for a second longer than the question needed, and just asked, “Why?”
I didn’t have a good answer at midnight. I do now. This post is my reply.
Not many people are wired that way.
Years ago, in a law class I only sat through because my degree required it, a lecturer taught me legal principles I can still recall today, more than two decades on. I couldn’t tell you what he looked like. I couldn’t tell you his voice, his style, anything about him as a person. He wasn’t trying to be remembered. He was trying to change what was in my head by the time I left the room, and he succeeded so completely that he disappeared entirely — but the knowledge remained.
That’s the whole job, as far as I can tell. Not to be remembered. But to leave your audience knowing more than they did an hour ago.
And somewhere along the way, I’ve noticed most technical content — most AI content especially — does the opposite.
Sit through the average AI webinar or keynote and watch what’s actually happening in the room. Someone is demonstrating how much they know. The architecture diagrams, the fluent jargon, the confident sweep through six use cases in four minutes — it’s a performance of expertise, and it’s often genuinely impressive to watch. But impressive and useful aren’t the same thing. The audience applauds and leaves the room knowing exactly as much as they did walking in an hour ago. Nothing.
That’s the answer I didn’t have at midnight.
I write because I’ve spent thirty years watching smart people fail to change what the room knows, while accidentally proving how much they know instead. And I got tired of watching it happen in my own industry, around a topic — AI — that’s drowning in exactly that failure mode right now. Everyone’s demonstrating. Hardly anyone’s changing understanding. AI should not be mysterious.
So when I write about a neuron, or a factory floor, or a winery that doesn’t exist, or a fictional CTO called Max who is quietly just me twenty years ago — none of that is decoration. It’s the mechanism. A story is how transfer actually happens. A diagram of dots and lines mostly isn’t.
If you’ve read this far and wondered why there’s always a person and a story before there’s ever a neural network in these posts — this is why. I’m not trying to show you what I know. I’m trying to change what you know. Everything else is just the vehicle.
That’s what I was doing at midnight. That’s the answer, Jack, if you’re reading this.
Knowledge and understanding is a gift. My gift, to whoever reads my blogs and books.
I’ve got a confession — I’m as bad as the next man for mentioning reasoning in AI conversations, but without fully understanding what reasoning is. But as a guy that likes to get under the hood and understand real stuff — and then share what I find — I’ve recently been covering Reasoning in my MSc studies, so here’s a series on Reasoning 101.
That’s a problem, because “reasoning” isn’t just one thing. It’s several genuinely different mechanisms, each with its own guarantees, its own blind spots, and its own honest limits. Mix them up and you end up nodding along to claims that don’t actually hold up once you look underneath. Separate them properly, and most of the confident “our AI reasons” pitches start to look a lot thinner — while some of the quieter, more specific claims start to look a lot more impressive.
So this series is me doing the thing I always do: getting under the bonnet, taking the thing apart, and being straight with you about what I find. Some of it comes from my MSc studies. Some of it comes from a genuinely intermittent starting fault on the MGB last summer that turned out to be a masterclass in three different kinds of reasoning, back to back, without me realising it at the time.
Here’s where we’re headed over the coming posts:
Deduction — the only one that actually guarantees anything, and why that guarantee is far narrower than people assume
Induction — the pattern-spotting most people probably mean when they say “reasoning,” and why it’s never certain, no matter how much data sits behind it
Abduction — the detective’s move, inference to the best explanation, and the one most easily faked by something that just sounds plausible
Spatial and temporal reasoning — the two dimensions almost nobody brings up, and exactly where AI still trips over its own feet
How AI actually learns to reason — which turns out to be a very different story from the one being sold in most keynotes
Deduction: The Only Certainty You’ll Ever Get
Last summer my ’79 MGB wouldn’t start some mornings. Cranked fine, just wouldn’t fire. First thing I did wasn’t guess — it was rule things out with total confidence. The manual is unambiguous: no spark means the fault sits in the points, the condenser, or the coil. I hooked up tool between the HT lead and spark plug No. 1, checked for spark, and got it, strong and consistent spark every single time .
That let me conclude, with zero doubt, that the ignition spark path wasn’t the problem. Full stop. Not “probably.” Not “unlikely.” Certain. That’s deduction, and it’s the only reasoning type in this series that actually earns that word.
How the certainty works Deduction doesn’t guess. It starts with a general rule — no spark means a fault in points, condenser, or coil — and a specific fact — spark is present — and the conclusion falls out automatically. It’s not new information. It was already sitting inside the two starting facts; the “reasoning” just makes it visible. Logicians call this a closed system for exactly that reason: nothing gets added, nothing gets discovered, everything just gets unpacked. The classic shape looks like this:
All mammals have backbones. A dolphin is a mammal. Therefore, a dolphin has a backbone. Swap in the MGB and it’s identical: no spark means a wiring fault; spark is present; therefore, no wiring fault. Same logic, same certainty, different garage.
Here’s the part almost nobody mentions: certain doesn’t mean true….
This is where most confident claims about “reasoning” quietly fall apart, and it’s worth sitting with, because it’s counterintuitive. A deductive argument can be airtight in structure and still be complete nonsense in content. Try this one:
All birds can breathe fire. A pigeon is a bird. Therefore, a pigeon can breathe fire. Logically flawless. Every bit as valid as the dolphin example. And obviously wrong, because the first premise is rubbish. That’s the difference between an argument being valid — the logic holds together — and being sound — the logic holds together and every premise is actually true. Validity you can check from an armchair. Soundness means going and checking the premise against reality, which is exactly what the spark testing tool was for.
I think about the fire-breathing pigeon a lot, honestly, because I hear its cousin in plenty of meetings. Someone builds a perfectly logical-sounding case, step by step, and everyone nods along because the structure is doing all the persuading — nobody’s checking whether step one was actually true to begin with. Valid reasoning is cheap. You can build an infinite supply of it. Sound reasoning is expensive, because it means somebody actually went and put a spark testing tool on the HT lead.
Where you’ll actually find it in AI
Here’s what matters for the meetings I mentioned at the start of this series. Almost none of what gets called “AI reasoning” today is deduction in this strict sense — the large language models doing the “reasoning” in most demos are pattern-matching machines, not closed logical systems, and I’ll get into exactly why that matters a few posts from now.
But deduction hasn’t disappeared from AI — it’s just been pushed into the parts nobody puts in the keynote. Within fraud detection and credit decisioning tools often sits a genuine rule engine underneath the flashier model: if transaction amount exceeds £X and originates from a new device and the account was opened under 30 days ago, flag for review. That’s the exact same closed-system logic as the spark test — a fixed rule, a specific fact, a guaranteed conclusion. No probability, no “the model thinks.” It either fires or it doesn’t.
It shows up more rigorously in formal verification — proving, with mathematical certainty, that a system will never do a specific thing. Aviation autopilot software and medical device firmware get checked this way: not “we tested it a lot and it seemed fine,” but a proof that for every possible input in a defined range, a bad output is impossible. That’s the one corner of modern AI safety where “certain” is still allowed to mean certain.
The tell, once you know to look for it: genuine deduction in AI is boring. It’s a rule engine, a compliance check, a hard safety boundary — unglamorous, unmarketed, and usually the part nobody’s pitching you in the keynote. Which tells you something about which kind of “reasoning” actually earns its certainty. And those who know me hear me say “Boring AI wins every time.”
Where this leaves deduction
It’s the only reasoning type that hands you a genuine guarantee — but only within a closed system, and only if you’ve actually verified the premises rather than just trusting how confident they sound. That’s a much narrower promise than “the model reasons” implies, and it’s worth holding onto, because the next post is about the reasoning type most people actually mean when they say that — and it comes with no such guarantee at all.
Next up: Induction — the pattern-spotting that runs most of real-world judgement, and why no amount of evidence ever makes it certain.
Written by me, copy edited and proof read in partnership with AI.
AI doesn’t make us lesser humans. AI helps shape our capabilities — if you’re honest about which C is doing the work.
A week ago I published my first book, AI Reality. Written as business fiction, it’s the result of three decades of experience, industry insights, and knowledge I’ve been accumulating since long before “AI” was a word anyone said at a keynote. A colleague, who hasn’t read it, suggested AI may have written it for me.
That got my goat.
But it also got me thinking, in a more useful way than I expected. Because AI did help me write it — as a thought partner working through arguments, a copy editor catching my worse habits, a proofreader tidying the parts I’d read past twenty times and stopped seeing. That’s real, and I’m not precious about it. What that colleague missed is the wit in that book is mine. The story is mine. The voice is mine. The characters are built from people I’ve actually worked alongside, and the scenes are built from places I’ve actually worked at — decades of them. The judgement calls, the technical argument I fought with myself over before it made it onto the pages — that’s human intelligence, and it isn’t something a model can borrow or replicate. That’s how a good story is crafted.
That distinction, between what AI can genuinely do and what stays irreducibly human, is what this blog is about. It happened to land in the middle of some MSc studies I’ve been doing on what intelligence actually is — human intelligence, and the emulated kind we’ve started calling artificial intelligence.
It’s worth pausing on that word, artificial. The term was coined in 1956 at a summer research conference at Dartmouth College, proposed by John McCarthy, with an ambition striking in its confidence: that every aspect of learning, and every feature of intelligence, could in principle be so precisely described that a machine could be made to simulate it. Seventy years on, we’re still working out what that means, and whether “artificial” was ever quite the right word for what we’ve built.
The clearest lens I’ve found for cutting through the noise didn’t come from an AI researcher at all. It came from reading psychologist Scott Barry Kaufman’s work on human giftedness and potential, which argues that human intelligence isn’t one thing you either have or don’t. It’s several distinct capacities that develop in sequence, each dependent on the one before it. I’ve taken that broader idea and adapted it into a four-part lens of my own that maps more directly onto the AI question: capacity, competence, commitment, and creativity. This isn’t Kaufman’s own framework restated — it’s my own synthesis, built on his underlying argument that intelligence is multidimensional rather than a single score.
That reframing exposes something worth saying plainly: IQ was never a complete measure of intelligence. The test was designed to capture a narrow band of cognitive ability, largely logical reasoning and pattern recognition, and it captures that slice well. But it tells you almost nothing about creative range, very little about capacity for growth under pressure, and nothing whatsoever about commitment — the willingness to keep showing up when the problem is hard and the outcome is uncertain. Two people with identical IQ scores can lead radically different lives for exactly this reason. And an AI model that aces every cognitive benchmark will still be missing the things that matter most.
Competence is AI’s home ground. It’s demonstrated skill: code that runs, a pattern-match that would take a person hours to find by hand. This is the layer most benchmarks measure, which is exactly why benchmark scores feel so persuasive. When people say AI is impressive, they’re almost always pointing here.
Creativity is more nuanced than either side of the AI debate tends to admit. GPT-4 has scored in the top percentile for originality and elaboration on standardised creativity tests, and that’s real. But the same research shows humans still hold the edge on sustained, finished creative work — novels, speeches, whole pieces judged as wholes rather than in fragments. AI is strong at divergent thinking, generating many varied ideas fast. Humans are still stronger at the harder, slower thing: carrying one idea all the way through to something finished and true.
Capacity is where AI has nothing to offer. Capacity is unexercised potential, ability sitting dormant until conditions call it out. A model’s weights are fixed the moment training ends. There’s no dormant skill waiting for a hard year or the right problem to draw it out, no growth through adversity, no quiet deepening over time. This isn’t a limitation better hardware or more data will fix. It’s structural.
Commitment is the quietest ingredient, and arguably the one that matters most. It’s what turns capacity into competence through sustained effort over years — showing up on the bad days as well as the good ones, pushing through when the first ten attempts fail, staying with a problem because it matters. AI has no version of this. No stakes, no continuity, no reason to keep going.
Deficiency isn’t the end of the story here, though. It’s the beginning of the real argument. Every gap on AI’s side of the ledger is a place human intelligence is naturally strong, and every gap on the human side is somewhere AI quietly compensates. But before we get to the bigger creative and strategic partnership, there’s something more immediate that often gets overlooked: AI’s single greatest gift to human intelligence is the return of time.
Think about what knowledge workers actually spend their days doing. Drafting routine emails. Reformatting reports. Summarising meeting notes. Searching for information buried in documents. Generating first drafts of things that will be heavily edited anyway. None of this requires capacity, commitment, or deep creativity. It’s the cognitive equivalent of administrative overhead — necessary, but not where human intelligence is at its best. AI handles all of it, and handles it well, because these tasks live squarely in the competence layer. The result isn’t that humans are replaced. It’s that humans are freed — to think harder about the problem that actually matters, to bring commitment to the work that genuinely requires it, to exercise the capacity and creativity no machine can replicate. That’s augmentation in its truest sense: not AI doing your job, but AI clearing the path so you can do the parts of your job that only you can do. The organisations that pull ahead won’t be the ones that use AI to cut headcount — they’ll be the ones that use it to multiply the depth of human thinking, on problems that are genuinely hard, genuinely ambiguous, and genuinely worth solving.
Put together, the four C’s don’t compete. They compound. A person with genuine capacity and commitment, paired with an AI generating competent, creatively varied raw material on demand, can move through more genuine iterations of an idea in a week than either could alone in a month. The AI doesn’t need capacity or commitment, because the human supplies it. The human doesn’t need to generate every variation by hand, because the AI supplies that. A researcher who once spent three days synthesising literature can now spend three hours, and use the rest of the time to ask better questions. A strategist who spent a morning building a slide deck can spend that morning stress-testing the argument inside it instead. In every case, the human isn’t doing less. They’re doing more of what matters.
So what does this mean in practice? Whenever you’re sizing up an AI use case, the question worth asking is which C is actually doing the work. If it’s competence, or the divergent, generate-many-options side of creativity, AI can carry a significant share of the load, and probably should. If it’s capacity or commitment, that’s yours to bring, and no amount of model capability changes that.
There’s a risk buried inside all of this that deserves naming directly, though. If AI consistently handles the competence layer, people will do less of it — and competence isn’t just an output, it’s also how capacity develops. The researcher who never struggles through a literature review never builds the instinct for what a good source looks like. The engineer who never writes boilerplate never internalises why the architecture matters. Capacity is built through difficulty; remove the difficulty entirely, and you quietly erode the very thing that makes human intelligence irreplaceable. The same erosion threatens commitment. When a passable answer is one prompt away, the temptation is to stop there — but the best work has never come from stopping at good enough. It comes from staying with a problem longer than was comfortable, caring about the outcome enough to keep going. AI cannot manufacture that. Only you can bring it.
This isn’t an argument against using AI. It’s an argument for using it deliberately. The question isn’t just “can AI do this?” It’s “should I let it?” The professionals who thrive in an AI-augmented world won’t be the ones who delegate everything they can. They’ll be the ones honest about which tasks build them and which merely occupy them, and who use AI to eliminate the latter while fiercely protecting the former.
So here’s what I’d say back to that colleague.
“Don’t tell me AI wrote my book. Go write your own. Go be creative. Go build something that outlasts you. Go use AI the way it deserves to be used — not as a replacement for the hard, human parts of the work, but as a way of unleashing more of your own creativity than you thought you had time for. I did exactly that.”
My book exists because I brought the capacity, the competence, the commitment, and the creativity — thirty years of myself, characters and rooms that AI could never have invented — and because AI cleared enough of the noise out of the way that I actually had the hours to do it. That’s the real argument here, against both the hype and the dismissal. AI isn’t a lesser version of us, and it isn’t a toy. It’s a different shape of capability that fits neatly into the missing space in ours, if you’re honest about which C is doing the work at any given moment. Because judgement like that has to be built in, not bolted on.
This piece adapts an idea from psychologist Scott Barry Kaufman’s work on human giftedness and potential. The four-part framing applied to AI here — capacity, competence, commitment, creativity — is my own synthesis, not Kaufman’s original model.
EDI, Event-Driven Architecture and the Ancestors of Agentic AI
Recent conversations at customer AI Roadshow in Dublin got me to thinking about the days when I was IT Manger for a start-up retail business. I was introduced to EDI. Electronic Data Interchange — the ability to send a structured purchase order from one company’s system to another without a human typing it in. It was, genuinely, transformative. Before EDI, someone had to pick up the phone, read out an order number, and hope the person on the other end wrote it down correctly.
EDI removed the human from the data transfer. That was the revolution.
But here’s what EDI couldn’t do: think.
The First Generation: EDI — Moving Structured Data Between Systems
EDI was brilliant at one thing — moving a known, structured message from point A to point B, reliably, at scale.
A purchase order goes out. An invoice comes back. A shipping confirmation follows. Every field in its place. Every transaction logged. The pipes worked.
What EDI required, though, was that both sides agreed on the format in advance. EDIFACT. X12. Fixed-width files. The message had to be perfectly structured, perfectly mapped, and perfectly expected. If something fell outside the agreed schema, the transaction failed. There was no interpretation. No judgement. No handling of the unexpected.
EDI moved data. It didn’t move meaning.
And it was entirely reactive. Something happened upstream — a human placed an order, a warehouse confirmed a shipment — and EDI carried the news downstream. The system itself initiated nothing. It waited.
The Second Generation: Event-Driven Architecture — Systems That React
Event-Driven Architecture was the next evolution. Instead of scheduled batch transfers between agreed parties, EDA introduced the idea of a continuous stream of events — things that happened — flowing through a system in real time.
A sensor reading changes. An event fires. A customer clicks “buy”. An event fires. A stock level drops below threshold. An event fires.
Systems subscribed to the events they cared about and responded accordingly. Kafka, RabbitMQ, AWS EventBridge — the plumbing of the modern enterprise. EDA made systems faster, more loosely coupled, and far more responsive than the batch-and-transfer world of EDI.
But EDA still had a fundamental constraint: the responses were pre-programmed.
When the stock level drops below threshold, the system does exactly what it was told to do — raise a replenishment order, send an alert, update a dashboard. The logic was written by a developer, tested, deployed, and fixed. The system reacted, but it didn’t reason.
If the stock drop happened because of an unusual combination of factors — a supplier delay, a seasonal spike, and a logistics disruption all at once — the pre-programmed response might be exactly wrong. The system would fire its rule. The rule would execute. And nobody would know it was the wrong call until the consequences arrived.
EDA moved meaning. It still couldn’t interpret it.
Now the Grandchild – The Third Generation: Agentic AI — Systems That Reason
This is where we are now. And this is why it’s different.
Agentic AI in business promises to not just just move data. And not just react to events. But reason across context, construct execution plans appropriate to situations, and take action — using tools, calling systems, adapting when the outcome isn’t what was expected.
The same scenario that would have broken an EDI transaction or misfired an EDA rule — the supplier delay, the seasonal spike, the logistics disruption, all at once — is exactly the kind of multi-signal, ambiguous situation an agentic system can handle. Because it holds all of that context simultaneously. It weighs it. It decides.
What makes this possible is the LLM at the centre of the agent. The LLM is the part that EDI and EDA never had.
Natural language understanding means that events described in unstructured form — a technician’s note, a customer complaint, a supplier email — can now trigger workflows. Not just structured data events. Any signal. Reasoning across context means the agent doesn’t just react to one event; it holds multiple events simultaneously and reasons across them. The order, the stock level, the maintenance alert — connected. Dynamic planning means that instead of following a pre-defined decision tree, the agent constructs a plan appropriate to the specific situation, novel exceptions included. And natural language output means the agent doesn’t just act — it explains. The shift supervisor gets a recommendation in plain English with the reasoning visible. Trust is built.
Tool selection is the final piece. The LLM decides which tools to call and in what order. The workflow isn’t hardcoded. It’s reasoned.
EDI moved structured data between systems. EDA moved events between systems in real time. LLMs move meaning.
The Family Tree
It helps to see this as a lineage, not a replacement.
Generation
Technology
What it moved
What it couldn’t do
1st
EDI
Structured data between agreed parties
Handle the unexpected
2nd
EDA
Events between loosely coupled systems
Reason about what they meant
3rd
Agentic AI B2B Agentic Workflows
Goals, plans and actions across systems and across organisations
Still being discovered
Each generation inherited the infrastructure of the one before. Agentic AI runs on top of the event streams that EDA built. EDA ran on top of the integration patterns that EDI established. You don’t throw away the pipes. You add a brain.
The Democratisation Wave
There’s a second story running alongside the technical one, and it matters just as much.
EDI required specialists. You needed someone who understood EDIFACT schemas, VAN networks, and trading partner agreements. It was powerful, but it was not accessible. The people who could build EDI integrations were a small, expensive, specialist group.
EDA democratised some of that — cloud-native event brokers, managed Kafka, serverless functions. A competent developer could wire up event-driven workflows without a decade of integration experience. But it was still firmly in the hands of engineers.
Agentic AI is doing something different. It’s democratising the building of intelligent workflows to people who have never written a line of code.
n8n, Make, Zapier AI — visual workflow builders with LLM nodes — mean a marketing manager can build an agentic workflow in an afternoon. Microsoft Copilot Studio puts a drag-and-drop agent builder on top of the M365 data layer without IT involvement for every use case. Salesforce Agentforce puts CRM-native agents in the hands of sales operations teams, not engineers. ServiceNow puts ITSM agents in the hands of process owners who understand the work. And DataRobot — an AI and ML platform that has evolved rapidly into agentic territory — allows data teams to build, deploy, and govern agents on top of existing model infrastructure. For organisations that already have a mature ML practice and want to extend it into agentic workflows without starting from scratch, it is particularly powerful.
These are just the established names. The startup ecosystem is moving fast — new agentic workflow builders are appearing monthly, each with their own approach to orchestration, tool calling, memory, and governance. The space is genuinely exciting. It is also, right now, genuinely fragmented.
Which brings us to the standards problem.
The Standards Problem — And Why It Matters
EDI had this problem too. In the early days, every organisation implemented EDI slightly differently. EDIFACT versus X12. Different field mappings. Different interpretations of the same schema. Trading partners had to negotiate bespoke integration agreements with every counterparty. The technology worked. The interoperability didn’t — not without significant effort.
EDA had the same growing pains. Kafka versus RabbitMQ versus Pulsar. Different event schemas. Different serialisation formats. Different approaches to ordering, delivery guarantees, and error handling. The ecosystem fragmented before it consolidated.
Agentic AI is at exactly that point right now.
Every platform has its own way of defining tools. Its own agent memory model. Its own approach to multi-agent orchestration. Its own security and governance primitives. An agent built in LangChain doesn’t natively interoperate with one built in AutoGen. A tool wrapped for Copilot Studio doesn’t automatically work in n8n. The capability is real. The standards are not yet.
This is not a reason to wait. It is a reason to build with portability in mind — and to watch one emerging standard very closely.
MCP — The Standard That Could Change Everything
Model Context Protocol, introduced by Anthropic and rapidly gaining adoption across the industry, is the most important standards development in agentic AI right now. And it maps almost perfectly onto what EDI did for data exchange — and what event schemas did for EDA.
MCP is a standard protocol that defines how an AI agent connects to external tools, data sources, and systems. Instead of every agent builder implementing bespoke integrations with every tool, MCP provides a universal connector layer. Build an MCP server for your system once, and any MCP-compatible agent can use it.
Think about what that means in practice. For the agent builder, instead of writing custom integration code for every tool the agent needs to call, you connect to MCP servers. The agent discovers what tools are available, what they do, and how to call them — dynamically, at runtime. For the tool owner, instead of building integrations for every agent framework that might want to use your system, you build one MCP server. Any compliant agent can connect. For the enterprise, instead of a fragmented estate of bespoke agent-to-tool integrations that nobody fully understands, you have a governed, discoverable, auditable layer of MCP servers that defines exactly what your agents can and cannot do.
This is the EDI moment for agentic AI. EDI gave us agreed schemas for business documents. MCP gives us an agreed protocol for agent-to-tool communication.
The analogy goes further. EDI worked best when the underlying data was clean and the trading partners had agreed on what the fields actually meant — not just their format. MCP works best when the tools behind the servers are well-behaved, well-documented, and governed. The protocol solves the connection problem. It doesn’t solve the data quality problem or the governance problem. Those are still yours to own.
MCP also does something that matters enormously for any enterprise running multiple agents across multiple systems. It makes multi-agent architectures composable. Consider a manufacturer running a maintenance agent, a production scheduling agent, and a procurement agent in parallel. Without MCP, connecting them into a coherent workflow requires custom orchestration code that understands the internals of each agent — change one and you potentially break the others. With MCP, each agent exposes its capabilities as MCP tools. Other agents can discover and call those capabilities through the standard protocol, with a standard contract, logged and auditable. Three systems. One control plane. Connected through a standard that didn’t exist two years ago.
The pattern is the same as every democratisation wave in technology history.
Era
What got democratised
The tool that did it
1990s
Desktop publishing
PageMaker, Word
2000s
Web presence
WordPress, Dreamweaver
2010s
Data visualisation
Tableau, Power BI
2020s
Agentic workflows
n8n, Copilot Studio, Agentforce
Each wave put powerful capability into the hands of people who previously had to ask a specialist. Each wave also created a new class of problems — because power without understanding is just faster chaos.
B2B Agentic Workflows — The Next Frontier
Most of the conversation about agentic AI is pointing inward. Automated reports. Automated configurations. Automated competitive analysis. Internal workflows made faster, smarter, less dependent on human intervention. That’s real, and it matters.
But it’s only half the story. Possibly the smaller half.
Because EDI was never an internal tool. It was born as a between-business protocol — two organisations’ systems talking directly to each other, without human intermediaries on either side. If the lineage holds, agentic AI follows the same arc. And the scenario that makes that concrete isn’t hard to imagine. It’s already forming.
Consider a retailer. Its demand forecasting agent is reading sales velocity, seasonal patterns, and external signals continuously — and generating a forward-looking inventory position without being asked. That signal reaches the procurement agent, which reasons across current stock levels, lead times, and supplier terms, and decides to place an order. That order doesn’t go to a human at the supplier. It goes to the supplier’s availability agent, which checks production schedules and fulfilment capacity and responds — or negotiates. Behind that, the supplier’s manufacturing scheduling agent has already slotted the order into the production run. And further upstream, the supplier’s own procurement agent is talking to their suppliers.
Agent talking to agent, all the way up the chain. Each one reasoning locally. The chain reasoning collectively.
This is B2B Agentic — and it is categorically different from EDI. EDI moved a deterministic message between agreed parties. B2B Agentic moves a reasoned decision across organisational boundaries, without a human initiating either side of the transaction. The demand signal propagates. The factory schedules. And at no point did a person decide to place an order.
That’s not science fiction. The components exist today. What doesn’t exist yet is the inter-enterprise trust and governance framework that makes it safe to run at scale. Who owns the decision when two agents from different organisations get it wrong together? How do you audit a transaction that involved reasoning, not just data transfer? What does liability look like when the purchasing agent acted autonomously on a signal the forecasting agent generated?
These are the questions that will define whether B2B Agentic scales cleanly or creates a new class of enterprise risk. MCP is a candidate for the protocol layer. The governance model is still being written.
The ancestors built the pipes. The current generation added the brain. The next frontier is teaching that brain to do business across organisational boundaries — reliably, safely, and at scale.
The Critical Caveat
Here’s the thing that gets lost in the excitement about low-code agent builders.
The tool is not the answer. The tool is the last ten percent.
A marketing manager who wires up an agentic workflow in Copilot Studio in an afternoon has built something that runs on top of data. If that data is ungoverned, siloed, or untrustworthy — the agent will be fast, autonomous, and wrong. In a B2B Agentic world, that problem doesn’t stay inside your firewall. It propagates upstream, downstream and crossstream.
The democratisation of the interface doesn’t democratise the need for foundations.
EDI required clean, agreed data schemas. EDA required reliable, well-defined events. Agentic AI requires trustworthy data, well-behaved tools, clear governance, and someone who owns the outcome. The new hire arrives fluent in n8n. They can build an agent before lunch. But if nobody has done the work to make the underlying data and systems agent-ready, they’ve built a very sophisticated way to automate a bad process.
The low-code builder is the interface. The foundations are still the foundations.
What This Means in Practice
If you’re thinking about agentic AI in your organisation, the lineage matters.
The organisations that got the most out of EDI were the ones that had clean master data and agreed standards before they plugged in the integration. The organisations that got the most out of EDA were the ones that had well-defined events and reliable systems of record before they started subscribing to streams.
The organisations that will get the most out of agentic AI — and that will be ready when B2B Agentic arrives — are the ones that have data their agents can actually use: authoritative, accessible, reasonably clean. Systems that expose safe, well-defined actions: APIs, automations, tools with clear contracts. Governance that decides where autonomy is allowed and where a human stays in the loop. And someone who owns the agent as a product, not a side project.
The technology has changed dramatically across three generations. The discipline required to use it well has not.
Agentic AI is the most capable generation yet. It can reason, plan, adapt, and explain. It can handle the unexpected in ways that EDI and EDA never could. And if B2B Agentic develops as the lineage suggests it will, the boundaries of what agents can do — and who they can do it with — are about to expand significantly.
The ancestors built the pipes. The next generation will build the trust layer.
Just like EDI and EDA before, B2B Agentic Workflows will become cross-organizational pipelines – now with intelligence built in.
I’ve seen many customer presentations. You know the ones: the pitch is well rehearsed, the content is polished and the delivery is flawless. At that point I watch the room. I’m checking the attention span. I see disengagement. Phones being scrolled, emails that could wait being sent, faces just saying “This isn’t relevant.”
However, the presenter goes home thinking he’s done a good job. In fact, so good he thinks — no one asked any questions.
When my boys were troublesome teenagers they’d say in a rebellious way, with more than a hint of sarcasm, “Cool story dad.” This was teenager speak for — what you’re saying is just not important to me.
I believe customers need more than that Cool Story. This is why business acumen and domain knowledge matters.
The Comfort Zone Problem
Here’s what I observe in the technology channel every day. Vendors and partners are deeply comfortable talking about servers, storage, networking, and cloud. They know the architecture. They know the benchmarks. They know the competitive positioning. Ask them about IOPS, latency, or core counts and they’ll talk for an hour.
But walk into a room with a CFO, an Operations Director, or a Head of Supply Chain — and ask them to have a conversation about how AI can genuinely help their business — and the comfort zone disappears fast.
Because that’s not a technology conversation. That’s a business conversation. And most of the technology channel hasn’t been trained to have it.
The result? Another Cool Story. Well rehearsed. Polished. Flawless.
And completely irrelevant to the person sitting across the table.
What Business Acumen Actually Looks Like
I’ll be honest — I take this for granted. I probably shouldn’t.
My first degree was in Business and Finance. Before I spent thirty years in enterprise technology I worked in the motor industry, retail, and pharma. I’ve sat on both sides of the table — as someone trying to run a business and as someone trying to help businesses use technology better.
That background gives me a lens that I’ve come to realise isn’t as common as it should be.
When I walk into a customer conversation I’m not thinking about products, features, feeds and speeds. I’m thinking about how the business actually makes its money, and where value is quietly being left on the table. I’m thinking about which decisions are being made badly, or not being made at all, and which processes are broken, slow, or dependent on people doing work they shouldn’t need to do by hand. Above all, I’m thinking about what good would actually look like, in terms a board would recognise — not in terms a vendor slide deck would recognise.
These aren’t AI questions. They’re business questions. And they’re the questions that determine whether an AI conversation goes somewhere real or becomes another Cool Story.
The Discovery Conversation
The most important AI conversation isn’t about models or infrastructure. It’s the discovery conversation — the one where you sit with a business leader and genuinely explore where AI and data analytics can make a difference.
Not future promises. Not theoretical possibilities. Real, practical impact. Here and now.
That conversation requires two things that most technology partners simply don’t have enough of. Business acumen — understanding how organisations work, how value is created, how decisions get made, and how to quantify the cost of a problem worth solving. And domain knowledge — understanding the specific industry, its language, its constraints, its regulation, and what good looks like in that context.
You cannot have a credible AI conversation in pharma if you don’t understand what a clinical trial looks like. You cannot have a credible AI conversation in retail if you don’t understand margin pressure, shrinkage, and the brutal reality of demand forecasting. You cannot have a credible AI conversation in manufacturing if you don’t understand what unplanned downtime actually costs.
Generic AI knowledge is becoming table stakes. Contextual judgment is the differentiator.
AI Is Not a Catalogue Item
Some can be forgiven for thinking AI is a catalogue of use cases. Scroll through any vendor presentation and you’ll find them — neatly packaged, impressively labelled, complete with ROI estimates and customer logos.
If only it was that simple.
Use cases can be interesting. They can be positive pointers to possible business solutions. They can spark a conversation, open a door, or help a business leader see what’s possible in their industry. I’m not dismissing them.
But here’s the problem. A catalogue of use cases is someone else’s answer to someone else’s question. And the moment you lead with it, you’ve already made a fundamental mistake — you’ve started with the solution before you’ve understood the problem.
The real work starts somewhere else entirely.
It starts by understanding the business. Its processes. Its people. And understanding the data — not just the data that exists, but the data that’s missing and hasn’t even been acquired yet. It means following the processes and the way decisions actually get made, not how the org chart says they should be made, but how they really get made. The friction that everyone knows about but nobody has fixed. The data that exists but nobody trusts. The process that runs on spreadsheets and institutional memory because nobody ever got around to doing it properly.
That understanding cannot come from a use case catalogue. It can only come from a conversation — a genuine discovery conversation, grounded in business acumen and domain knowledge, where the right questions get asked and the answers actually get heard.
A use case tells you what AI has done for someone else.
Business acumen tells you what AI can do for this business, right now, with the data and processes and people that actually exist.
That’s the difference. And it’s everything.
The Golden Process
In my AI Reality framework I talk about finding the Golden Process — the business process that is ready for AI right now. The one where the data exists, the problem is real, the ROI is quantifiable, and the organisation has the appetite to act.
You cannot find the Golden Process from a slide deck.
You find it by asking the right business questions, listening carefully to the answers, and having enough domain knowledge to recognise the opportunity when it appears.
A partner who only knows technology will walk past the Golden Process every time. Not because they’re not intelligent. But because they’re looking at the wrong map.
The right map isn’t a product catalogue. It’s an understanding of how the business works and where it hurts.
What the Room Looks Like When You Get It Right
I presented at a Partner event recently. I talked about AI Reality — practical AI, data readiness, real use cases, honest outcomes. No hype. No future promises. Just what’s working now and why.
Not one person was on their phone.
Afterwards they asked for my slides. I pointed them to my blog site. “It’s all here,” I said. “And I wrote it.”
The LinkedIn connection requests came in before I’d left the building.
That’s what happens when a room full of people hear a business conversation instead of a technology pitch. They lean in. They engage. They want more.
Because it’s relevant. Because it’s real. Because it respects their intelligence and their time.
No Cool Story. Just signal.
The Evolution of the Technology Partner
Let me take you back thirty years.
A technology partner’s job was straightforward. Understand the product. Know the speeds and feeds. Build the relationship. Win the deal. Rack the kit. Repeat.
And for thirty years — that was enough. Customers needed servers. Partners sold servers. Customers needed storage. Partners sold storage. Networking. Cloud. Virtualisation. Each wave brought new products to learn, new certifications to collect, and broadly the same conversation to have.
“Here’s what it does. Here’s what it costs. Here’s why ours is better than theirs.”
It worked. For a long time it worked very well.
But something has shifted. And the Partners who haven’t noticed yet are about to find out the hard way.
The World Changed. The Conversation Didn’t.
AI didn’t just bring a new product category. It brought a fundamentally different kind of customer conversation.
Because the customer isn’t asking “what does it do?” anymore.
They’re asking “what problem does it solve — for my business, in my industry, with my data, right now?”
That’s a different question. And it requires a different kind of partner to answer it.
The technology partner who built their career on product knowledge, competitive positioning, and relationship selling isn’t suddenly irrelevant. But they are incomplete. Because the customer has moved on — and the conversation they want to have requires something that thirty years of selling servers simply doesn’t build.
Business acumen. Business domain knowledge. The ability to walk into a room and talk about outcomes before products.
What Good Looks Like Today
The most relevant technology partner in 2026 isn’t the one with the most certifications. It isn’t the one with the biggest product portfolio or the most competitive pricing.
It’s the one who walks into a customer conversation and demonstrates — before a single slide is shown — that they understand their business, and that they’re genuinely interested in making it run better.
That partner leads with the problem, not the product. The first question isn’t “let me tell you about our AI portfolio.” It’s “tell me about the process that’s costing you the most time, money, or risk right now.”
That partner speaks the language of the business, not IOPS and latency. The CFO cares about margin. The Operations Director cares about throughput and efficiency. The Risk Director cares about compliance. Speak their language, not yours — because the conversation that lands in a board meeting sounds nothing like the conversation that lands in a data centre.
That partner understands the industry, not generically but specifically — the regulatory environment, the competitive pressures, the operational realities, the difference between what the industry says it does and what it actually does.
And that partner connects AI to outcomes. Not “here’s a large language model,” but “here’s how three organisations in your sector reduced their claims processing time by 40 percent — and here’s what had to be true about their data before that was possible.”
That last point is the real test, because earning the follow-up meeting is what separates the partner who gets invited back from the one who gets a polite thank you and a door that doesn’t open again. Not by being impressive. By being relevant.
How to Be the Most Relevant Partner on the RFP
Here’s the uncomfortable truth about most RFP responses: they look identical.
Same structure. Same claims. Same case studies. Same promises about partnership and innovation and transformation. The customer reads three responses and struggles to remember which was which.
The partner who stands out on an RFP isn’t the one with the longest list of capabilities. It’s the one who demonstrates — clearly, specifically, and credibly — that they understand the customer’s business well enough to solve their actual problem.
That starts with showing you’ve done the homework. Reference their industry. Reference their specific challenges. Show that you understand the context, not just the requirement — because anyone can respond to what the RFP says, but the best partners respond to what it means.
It means leading with outcomes, not outputs. Don’t describe what you’ll deliver. Describe what will be different when you’re done, in terms the business cares about — time saved, risk reduced, revenue protected, decisions improved.
It means being honest about the prerequisites. The partners who win long-term trust are the ones who tell the truth about what has to be true before AI delivers value — data readiness, process clarity, governance, stakeholder alignment. Customers have been burned by partners who promised transformation and delivered complexity. Honesty is a differentiator.
And it means demonstrating domain credibility. Reference real outcomes in their sector. Use their language. Show that you’ve been in rooms like theirs before, and that you know what good looks like from the inside.
The best RFP responses don’t just answer the question. They reframe it. They show the customer something about their own problem that they hadn’t fully articulated. That’s the moment a partner stops being a vendor and starts being an advisor.
The most relevant partner on the RFP isn’t the one who answered the question best.
It’s the one who made the customer think: “These people actually understand us.”
The Gap — And What To Do About It
The gap between what AI can deliver and what most organisations are actually getting from it isn’t a technology gap. The technology is good enough. In many cases it’s extraordinary.
The gap is a translation gap. Between AI capability and business outcome. Between what the technology can do and what the business actually needs.
Closing that gap requires people who can have the business conversation first and the technology conversation second. Who understand the industry they’re working in deeply enough to know where the problems are. Who can sit with a business leader and ask the questions that matter — and understand the answers.
That’s not a certification. That’s not a product training. That’s thirty years of paying attention.
But it can be developed. It starts with curiosity — genuine curiosity about how businesses work, how industries operate, and what keeps leaders awake at night. It continues with discipline — doing the reading, asking the questions, building the domain knowledge that makes the conversation credible.
And it shows up in the room. Every time.
The Bottom Line
It’s no longer a conversation about products, features, feeds and speeds.
Business wants to know how Data and AI can help their business. Reality, not future promises.
The technology partners who will thrive in the AI era aren’t necessarily the ones with the deepest technical knowledge. They’re the ones who combine enough technical literacy to be credible with enough business acumen and domain knowledge to be useful.
The ones who walk into a room and have the conversation that matters.
The ones who earn the follow up meeting.
The ones whose audience isn’t on their phones.
Because when you get it right — when the conversation is real, relevant, and grounded in genuine understanding of the business — nobody needs to check their emails.
They’re too busy thinking about what you just said.
There is a moment in every data science project where the model produces a result that doesn’t feel right. Not wrong enough to be obviously broken. Just wrong enough to make you pause. You look at the output, you look at the data, and something doesn’t add up. That moment is not a failure of the model. It is an invitation to think. The practitioners who take that invitation seriously are the ones who build systems worth trusting. The ones who don’t are the ones who find out the hard way that a model left unquestioned will eventually answer a question nobody wanted to ask.
The Bottle That Started the Argument
Clare’s random forest classifier had been running for three weeks when the winemaker walked in with a printout.
A 2019 reserve vintage — hand-harvested, barrel-aged, the kind of wine the winery staked its reputation on — had been flagged as Standard. Not Premium. Standard. The model had looked at its chemical profile, compared it against everything it had learned, and placed it in the middle tier.
The winemaker was not pleased.
Clare looked at the numbers. The acidity was slightly elevated for that varietal. The residual sugar sat at the lower end of the Premium range. On paper, the model had not made an obviously unreasonable call. But the winemaker knew that vintage. He had watched it develop. The chemical snapshot the model had seen was a moment in time, and that moment had not told the whole story.
If Clare had not questioned the result, the wine would have been priced accordingly. A Premium vintage sold at Standard margin. Revenue lost. Reputation quietly eroded. Nobody would have known — until the wine won an award the following year, and someone went looking for more of it at the price they remembered.
The model’s own numbers supported the winemaker’s instinct. Premium recall across the test set was 0.58 — meaning four in ten genuine Premium wines were being misclassified. That is not a rounding error. That is a systematic commercial leak. And it was visible in the confusion matrix before a single bottle hit the shelf, for anyone willing to look.
Always question the model. Not because models are wrong, but because they are not always right in the ways you expect — and the consequences of uncritical trust compound quietly until they don’t.
When the Stakes Rise
The winery’s problem was commercial. A mislabelled bottle costs margin. That is recoverable.
Now consider the same logic applied somewhere less forgiving.
A clinical prediction model assesses patient risk from incoming diagnostic data. It has been trained on historical cases and performs well on aggregate accuracy metrics. But accuracy is an average, and averages hide the distribution of errors. The model misclassifies a subset of high-risk patients as low-risk. They are triaged accordingly. The cost of that error is not a margin shortfall. It is a missed diagnosis.
In healthcare, algorithm selection must account for disease prevalence and risk tolerance in ways that pure accuracy metrics do not capture. Missing a high-risk patient carries a categorically different cost from issuing an unnecessary diagnostic test. The confusion matrix Clare used in the winery — precision, recall, the asymmetry of error types — becomes a clinical instrument. The domain priority is not balanced accuracy. It is minimising the consequences of the most dangerous failure mode.
Public sector applications carry a different kind of weight. Algorithmic decisions in government — benefits eligibility, sentencing guidance, resource allocation — must withstand scrutiny regarding fairness, bias, and transparency. An opaque model that produces accurate aggregate outcomes may still embed and amplify historical inequities present in its training data. High predictive accuracy does not confer fairness. And in public decision-making, a model that cannot explain itself is a model that cannot be held accountable.
Financial institutions sit at the intersection of both pressures. Regulatory frameworks demand explainability. Credit decisions, fraud detection, and risk assessment carry legal accountability. A model that cannot be audited cannot be deployed — regardless of how well it performs on a test set.
The common thread is this: algorithm choice is shaped by accountability, transparency, regulatory compliance, and stakeholder trust, not solely by predictive accuracy. Clare’s logic map holds in every environment. But in high-stakes contexts, the interpretability question is not a preference. It is a constraint.
Trusted AI systems tend to mirror the traits we associate with reliable human decision-makers. When stakeholders perceive a model’s recommendations as misaligned with domain logic — when the output feels wrong to the people who know the problem best, the way the winemaker felt when he saw that printout — acceptance declines regardless of technical performance. The winemaker did not need to understand random forests. He needed to trust the result. And on that day, he didn’t.
Business Constraints Are Real Constraints
A model that cannot be deployed is not a model. It is an academic exercise.
Clare’s winery operates on practical terms. There is no dedicated ML infrastructure, no model monitoring pipeline, no team of engineers to retrain the classifier when the harvest changes the distribution of incoming data. The model has to run on available compute, be maintained by people with other responsibilities, and produce outputs that a non-technical winemaker can act on without a statistics degree.
These are not edge cases. They are the normal conditions under which most real-world models operate.
Computational cost is a genuine selection criterion. A model requiring significant infrastructure to train and serve introduces cost and operational complexity the organisation may not sustain. In enterprise environments this extends further — cloud compute costs, latency requirements, batch versus real-time inference, and the overhead of version control and retraining schedules all sit within the scope of algorithm selection.
Team capability shapes what is maintainable. A model built by a specialist that nobody else understands becomes a liability the moment that specialist leaves. Interpretable models are easier to hand over, easier to audit, and easier to debug when something unexpected happens.
Maintenance burden is the constraint most often underestimated. Models are trained on historical data. The world changes. Distributions shift. A wine classifier trained on three years of data from a specific terroir will degrade as growing conditions evolve. The simpler the model, the lower the cost of keeping it current.
Clare’s random forest was the right choice for her context. In a different context — regulated environment, higher interpretability requirements, leaner team — logistic regression might have been the more responsible choice precisely because of its constraints, not despite them.
Too Tight, Too Loose, or Just Right
Clare’s model had performed well on the test set. But the bias-variance results told a more complicated story — and this is where the pipeline revealed something the accuracy number alone would never have shown.
The base random forest achieved a training accuracy of 1.0000. Every single training example classified correctly. That sounds impressive. It isn’t. It is a warning sign. The model had memorised the training data — its patterns, its noise, its sampling accidents — rather than learning the underlying relationship between chemistry and quality. When cross-validated against unseen folds, accuracy dropped to 0.6857, a generalisation gap of 0.3143. Well beyond the threshold that signals a problem.
This is overfitting. High variance. The model is too loose — it has fitted itself so precisely to what it has seen that it struggles with what it hasn’t.
Logistic regression told the opposite story. Training accuracy 0.6450, cross-validation mean 0.6356, a gap of just 0.0094. What it learned from training data held up cleanly across every fold. It is not memorising. It is generalising. The trade-off is that its linear assumptions leave predictive performance on the table — particularly for Premium recall, where the true decision boundary in the data is not a straight line.
This is the bias-variance trade-off made real. Not a textbook diagram. Actual numbers from Clare’s actual data.
Clare attempted to close the gap by constraining the random forest — limiting tree depth, requiring minimum samples at leaf nodes, reducing the features considered at each split. The gap narrowed from 0.3143 to 0.2221, and training accuracy came down from 1.0000 to a more credible 0.8835. Better. But not solved. The overfitting in the base model ran deeper than these constraints alone could fix.
The final comparison told the full story clearly:
Logistic regression: test accuracy 0.61, CV mean 0.64, gap 0.009. Honest, stable, consistent. Random forest base: test accuracy 0.75, CV mean 0.69, gap 0.314. Impressive on paper, unreliable under scrutiny. Random forest regularised: test accuracy 0.70, CV mean 0.66, gap 0.222. Better behaved, but the overfitting problem persists.
The model with the highest test accuracy had the worst generalisation profile. The model with the lowest test accuracy generalised almost perfectly. Neither is definitively the right answer — the choice depends on what the winery can tolerate, what the winemaker needs to trust, and what Clare is prepared to stand behind.
That is not a statistical question. It is a judgement call. And it is exactly the kind of call that separates a data scientist from a model runner.
Always Question the Model
Clare did not discard her classifier after the winemaker’s challenge. She documented the edge case, flagged it for review at the next retraining cycle, and added a human review step for wines scoring within a defined confidence margin of a class boundary. The model remained useful. It also remained accountable.
The bias-variance results reinforced what the winemaker’s challenge had already suggested: the model’s confidence in its own outputs was not fully earned. A training accuracy of 1.0 is not mastery. It is a model that has stopped learning and started remembering. The discipline is in knowing the difference.
That is the lesson Clare carried out of the winery and into every subsequent project. Not blind trust. Not reflexive scepticism. A structured, ongoing interrogation of what the model knows, what it doesn’t, and where the consequences of its errors are highest.
Supervised learning is a rigorous analytical discipline, not a collection of tools. Performance metrics quantify error according to defined priorities. Algorithm selection balances statistical performance with operational feasibility. And deployment is not the end of the process — it is the beginning of a new one.
The data told Clare what the wine was. The model learned to say it again. But Clare never stopped asking whether it was right.
That habit — more than any algorithm, any metric, any architecture — is what makes the difference between a model that serves a business and one that quietly misleads it.
Always question the model. The ones worth trusting can take it.
As I’ve studied AI and Machine Learning, I’ve come to realise it’s called data science for a reason. And that reason is simple: it’s not easy. Finding the right signal in the noise takes a genuinely analytical approach, logical thinking, and a willingness to sit with uncertainty before reaching for an answer. There’s no shortcut through the discipline. I’ve learned that the hard way more than once working on assignments late in to the night. In this post, our data scientist hero Clare faces exactly that challenge. She has her clusters, she has her labels, and now she has to decide how supervised learning — and which algorithm — will predict wine quality with the fewest errors.
The Work Doesn’t End With the Clusters
When we last left Clare, she had done something quietly impressive. Working through the winery’s historical tasting data with an unsupervised learning approach, she had let the data organise itself. No labels, no guidance — just structure emerging from pattern. Three clusters had formed, and the winemaker had given them names that meant something: Premium, Standard, and Reject.
That was discovery. What Clare needs now is prediction.
The distinction matters more than it first appears. Unsupervised learning asks the data what it contains. Supervised learning asks a model to learn from what the data already knows, so it can make decisions about data it has never seen before. Clare’s clusters gave her the labels. Now those labels become the target. The question sitting on her desk is straightforward to state and genuinely hard to answer: given the measurable chemical properties of a new vintage, can a model predict its quality grade before it ever reaches the tasting panel?
That shift — from discovering structure to predicting outcomes — is the conceptual foundation of supervised learning. The model is given inputs and known outputs. It learns the mapping between them. Then it applies that mapping to inputs where the output is unknown. The discipline lies in how rigorously that estimation is made.
Clare opens her notebook. The data is ready. The labels exist. Now she has to make some decisions.
Reading the Target
The first decision in supervised learning is not which algorithm to use. It is what kind of thing you are trying to predict.
Clare’s target variable is wine quality grade: Premium, Standard, or Reject. That is not a number on a continuous scale. It is a category. And that single observation rules out an entire family of approaches before she has written a line of code.
Linear regression predicts continuous numerical outputs — price, temperature, yield, revenue. It is the right tool when the answer sits anywhere along a spectrum. Classification algorithms predict discrete categorical outputs. They assign an observation to one of a defined set of classes. Clare’s problem is a multi-class classification problem. Linear regression is off the table.
This is not a trivial distinction. Applying linear regression to a categorical target produces nonsense. Treating category labels as if they were ordered numbers — as if Reject equals one, Standard equals two, and Premium equals three — imposes a mathematical relationship the data does not actually contain. The model learns the wrong thing. Weak conceptual understanding at this stage leads to flawed modelling decisions later, and Clare knows it.
She writes classification at the top of her notebook and moves on.
The Logic Map
Professional guidance frameworks for algorithm selection begin with a logic map. The questions are simple. The discipline is in answering them honestly.
Is the problem supervised or unsupervised? Clare’s problem is supervised. She has labelled data. The clusters gave her that.
Is the target continuous or categorical? Categorical. Three classes. Classification confirmed.
How many samples are available? Clare’s dataset contains 1,599 wines — modest by enterprise standards. That steers her toward algorithms that generalise well on smaller datasets rather than those demanding millions of observations.
Are the features interpretable? Clare will need to explain her model’s recommendations to the winemaker. If the model flags a batch as Reject, there will be a conversation. Interpretability is not a nice-to-have. It is a functional requirement.
Is computational cost a constraint? At 1,599 wines and eleven features, this is a modest workload — Clare’s Dell Precision 7 laptop handles the full pipeline without breaking a sweat. Compute is not the bottleneck here. That matters, because it means her selection criteria can focus entirely on what the model learns and how well it explains itself, rather than what the hardware can afford to run.
Each answer narrows the field. By the time Clare reaches the end of the logic map, she is not choosing between dozens of algorithms. She is choosing between a handful of serious candidates.
Shortlisting the Candidates
Clare’s shortlist takes shape around four candidates, each considered on its own terms.
Logistic regression is the natural starting point. It is interpretable, computationally inexpensive, and well understood. For a three-class problem it extends cleanly through a one-versus-rest approach. Its coefficients can be read directly — Clare can tell the winemaker that a unit increase in volatile acidity pushes a wine toward Reject with a quantifiable effect. The limitation is that logistic regression assumes a roughly linear relationship between features and the log-odds of class membership. If the true decision boundary is more complex, the model will underfit.
Decision trees offer a different kind of interpretability. The model produces a flowchart of decisions the winemaker can follow without any statistical training. They handle non-linear boundaries well and make no assumptions about feature distributions. The problem is instability — small changes in the training data can produce substantially different trees, and a single decision tree overfits easily.
Random forest addresses that instability by building many trees and aggregating their votes. It is more robust, typically more accurate, and handles real-world data noise gracefully. The trade-off is interpretability. The winemaker cannot follow the logic of a hundred trees simultaneously.
K-nearest neighbours classifies a new observation by finding the most similar training examples and taking a majority vote. It is intuitive but sensitive to irrelevant features and offers no explanatory power. Clare sets it aside.
Then there is the question that sits at the edge of her shortlist. Deep neural networks are capable of extraordinary things. But Clare’s dataset is modest, her compute is limited, and the winemaker is waiting for an explanation. Neural networks are opaque by nature — the internal representations they learn do not translate into human-readable reasoning. Clare notes them for future reference and removes them from consideration.
She circles random forest as her primary candidate, with logistic regression as a baseline to measure against. The reasoning is sound. The decision is defensible. But before she commits, she needs to know whether her model actually works.
Did It Actually Work?
A model that produces predictions is not necessarily a model that produces good predictions. Clare runs both classifiers on a held-out test set — 320 wines the models have never seen during training — and looks at what comes back.
The first instinct is to reach for overall accuracy. Logistic regression scores 61%. Random forest scores 75%. Random forest wins — but Clare has learned not to stop there.
Her dataset is imbalanced. Of the 1,599 wines, 46.5% are Reject, 39.9% are Standard, and just 13.6% are Premium. A model that simply predicted Reject or Standard for everything would achieve a deceptively high accuracy figure while being commercially useless. The warning is in the numbers before the model even runs.
The confusion matrix gives her a more honest picture. It lays out, for each actual class, how many examples the model correctly identified and how many it misclassified — and crucially, what it misclassified them as. From it, two metrics earn Clare’s attention.
Precision asks: of all the wines the model predicted as Premium, how many actually were Premium? Recall asks: of all the wines that actually were Premium, how many did the model correctly identify?
The results are illuminating. Logistic regression finds only 26% of actual Premium wines — a recall of 0.26. Three quarters of the winery’s best bottles are being miscalled as Standard. Random forest improves this substantially, reaching a Premium recall of 0.58. Still not perfect, but it is now finding more than half the genuine Premium wines rather than missing most of them.
Reject performance is strong across both models — F1 of 0.73 for logistic regression, 0.81 for random forest. The majority class, with the most training examples, is the easiest to learn.
Standard sits in the middle, chemically and statistically. It is the hardest class to call cleanly, and both models reflect that. The soft boundaries the unsupervised clustering revealed — a silhouette score of 0.1892, indicating genuine overlap rather than clean separation — are still present in the supervised results. The models did not invent that difficulty. It was always in the data.
The winemaker asks Clare a pointed question: which error costs more — calling a Premium wine Standard and pricing it down, or calling a Standard wine Premium and disappointing a customer? That question does not have a statistical answer. It has a business answer. And it is exactly where algorithm evaluation and operational reality meet.
Clare documents both models. The random forest performs better across all three classes. But before she declares a winner, she runs one more check.
What Clare Learned
Clare closes her Jupyter Notebook with something that took longer to arrive than she expected: not confidence in the model, but confidence in the process.
Algorithm selection is not a guess dressed up in technical language. It begins with understanding what the data contains and what the prediction task actually requires. It follows a logic that can be stated, examined, and defended. It weighs interpretability, computational cost, and domain requirements alongside statistical performance. And it ends not with a declaration that the model is correct, but with evidence — real numbers, per class, honestly read.
The feature importance analysis added one final thread. The random forest’s three most influential predictors were alcohol, sulphates, and volatile acidity — in that order. These were precisely the features that defined the unsupervised clusters in the previous analysis. The supervised model, trained independently and without knowledge of the clustering work, arrived at the same conclusion the clusters had already reached. The data gave the same answer twice, through two entirely different methods. That is not coincidence. That is signal.
Supervised learning is not a collection of tools. It is a way of thinking about prediction — rigorously, honestly, and always in service of a decision that someone, somewhere, actually needs to make.
The data told Clare what the wine was. The model learned to say it again, about wine it had never tasted.
In Part 2, Clare’s model leaves the winery and enters environments where the stakes are higher, the constraints are harder, and the bias-variance results raise a question she wasn’t expecting.
AI Winter is nothing new. We’ve been there before. The last AI winter was 1987 – 1993. Businesses over-invested in “expert systems” — expensive, specialised computers designed to mimic corporate decision-making. When these systems proved too difficult to maintain and update, the commercial market for AI collapsed. This time winter may come because Frontier AI is hitting a scaling ceiling it can’t engineer its way past. This post suggests the answer isn’t bigger. It’s smaller. And the proof of concept has been running quietly for twenty years.
If you want an overview of how AI actually developed — from 1950s foundations through to today — I’ve covered that in AI Didn’t Come From Nowhere. The short version: AI has been building for seventy years. ChatGPT didn’t start it. It just made it visible.
The Clock Got Reset
When ChatGPT launched in late 2022 it did something remarkable and something damaging simultaneously.
Remarkable — it made AI legible to everyone. For the first time, a non-technical person could sit down, ask a question, and feel the capability directly. That democratisation matters. It changed the conversation permanently.
Damaging — it reset the public clock to zero. Suddenly AI had a birthday. And a generation of business leaders started building AI strategies as if nothing had existed before that date.
I meet smart, senior people regularly who genuinely believe AI started with ChatGPT. That 2023 was year one.
It wasn’t. Not even close.
This is what I’ve called the Magpie Effect — organisations distracted by the latest glittering capability, dropping whatever was already working to chase the new thing. The frontier AI conversation feeds it perfectly. New model. New benchmark. New promise. The magpie pivots. The roadmap restarts. The quiet AI that was already delivering sits ignored in the corner.
But before we get to what came before, we need to talk about where frontier AI is heading. Because the ceiling that’s approaching makes the Magpie Effect not just wasteful — but genuinely risky.
The Scaling Wall
The Frontier AI story has been built on a single premise: scale it out and it gets better. Bigger models. More parameters. Larger data centres. More powerful GPUs. The scaling hypothesis held, the results were real, and the industry committed.
But scaling isn’t a strategy. Scaling by laws of physics and economics always has limits. It’s a phase. And this phase is approaching a physical wall that no amount of engineering enthusiasm will move.
A single large-scale AI training run consumes energy comparable to a small town. Inference at scale runs continuously — millions of queries, every day, globally, without pause. The major cloud providers are signing nuclear power agreements not because nuclear is cheap or fast, but because conventional grid capacity cannot keep pace with demand. Microsoft has already reactivated Three Mile Island. Amazon is investing in small modular reactors. They have run out of other options.
Data centre construction can’t keep up. From planning decision to operational facility takes four to seven years in most developed jurisdictions. Grid connection queues in the UK, the US, and across the EU are already measured in years. Water rights for cooling are becoming a site selection constraint. Liquid cooling manages the thermal symptom. It does not touch the energy consumption problem.
The ceiling is lower than the industry’s public projections suggest. The people closest to the infrastructure already know it. The investment cycle is still running hot enough that saying so publicly is commercially inconvenient.
Efficiency gains — model distillation, inference optimisation, new chip architectures — will buy time. They won’t move the ceiling. The physics and the economics don’t negotiate.
As I write this, I use Claude daily. Claude for many of us has become a genuine thinking partner — including in writing this post. I’m not dismissing Frontier AI. I’m being honest about its infrastructure cost and scalability limits. Even Claude will hit this ceiling. That’s not a criticism. It’s physics, it’s economics. It’s a practical viewpoint.
The Transistor Parallel
We’ve been here before.
In the 1940s the valve computer proved the concept. ENIAC, Colossus — genuinely capable machines. But enormous, power hungry, heat generating, and impossible to scale to meet the demand that was already visible. The resource constraint was real. The engineering community knew it.
The answer wasn’t a bigger valve. It was the transistor.
The transistor didn’t just miniaturise the valve. It changed the trajectory entirely. Compute escaped the specialist room. It went into consumer products, industrial equipment, vehicles, eventually into every pocket on the planet. Nobody called it a compromise. The smartphone was always a better destination than a room full of valves — we just couldn’t see it from inside the valve paradigm.
Frontier AI is the valve computer. Capable. Proven. Impressive. And running into the same kind of physical ceiling that made the transistor not just desirable but inevitable.
The transistor moment for AI is smaller, distributed, efficient models — running where the work actually happens, without depending on infrastructure the planet cannot sustainably provide at the rate being demanded.
And here is the thing that gets lost in the Frontier AI conversation.
That transistor moment isn’t coming. It’s already here. It’s been here for twenty years. We just haven’t been paying attention to it.
The AI That Never Made the Headlines
While the world was discovering chatbots, a different AI had already been running quietly for years. Doing real work. Solving defined problems. Delivering measurable outcomes with ROI.
Netflix’s recommendation engine has been shaping what you watch since 2006. Gmail’s spam filter has used machine learning since 2004. Amazon’s demand forecasting and warehouse optimisation runs on ML models that have been refining themselves for over a decade. Google Maps learns from billions of journeys in real time.
Credit card fraud detection catches billions of fraudulent transactions annually using models that make decisions in milliseconds, invisibly, at the edge.
Aircraft engines predict their own failures. Factory production lines inspect their own output. Rail networks scan their own infrastructure at track speed.
None of this started in 2023. None of it needed a press release. None of it requires a nuclear power agreement or a seven year planning queue.
In my post on Machine Learning: The Backbone of Enterprise AI I looked at what this looks like in practice. DHL’s parcel inspection system uses computer vision models running at the edge — cameras over conveyor belts, ML models trained on millions of images, making pass/fail decisions on every parcel at line speed. Duos Technologies scans every railcar at track speed using AI-enabled imaging, catching defects before they become safety incidents. Factory quality control systems watch production lines continuously, catching defects too subtle for the human eye.
None of these run on hyperscale infrastructure. None send data to a cloud API and wait for a response. They run locally, efficiently, on hardware sized to the task — because the task is defined, bounded, and understood.
This is the transistor analogy for AI. Already in production. Already sustainable. Already delivering ROI.
The Right Tool for the Right Problem
The AI decisions that make genuine business sense share a common structure.
A defined problem — not “use AI” but “reduce defect escape rate on line 4” or “inspect every railcar at track speed without taking it out of service” or “predict supply chain disruption before it hits.”
A defined process — the AI is embedded in a workflow with a measurable before and after. It replaces or augments a specific activity with a specific outcome.
A defined return — because the problem is specific and the process is clear, the ROI calculation is straightforward. Not “we have AI” but “our cost per inspection dropped, our throughput increased, our safety incidents reduced.”
When AI is aligned to business process optimisation this way, the technology choices become obvious. Computer vision for inspection. Predictive models for maintenance. Anomaly detection for risk. Small, focused, efficient. Running on hardware sized to the job, not to the ambition.
The organisations getting genuine return from AI are overwhelmingly doing this. They’re not in the headlines. They’re on the production floor. They’ve been there for years.
I’ve written about what that structured approach looks like in practice in Getting AI Right First Time. The organisations that get AI right treat it as process optimisation first, technology second. Start small. Prove value. Escape the pilot graveyard. Make AI boring. That five step discipline is what separates durable capability from expensive experimentation.
The starting point is finding your critical process — the one business process that, if it stopped tomorrow, the business would stop with it. Name it. Find the friction within it. That friction is your AI use case shortlist. Not a vendor briefing. Not a capability demo. The process the business already depends on, and the specific points where better prediction, classification, or inspection removes a real constraint.
The Obvious Outcome
The Frontier AI scaling ceiling isn’t a maybe. It’s a when. Efficiency gains will buy time. The laws of physics and economics won’t be moved.
When it arrives, the organisations that built deep dependencies on hyperscale infrastructure for processes that didn’t require it will be most exposed. The expertise put to waste won’t be quickly rebuilt. The processes restructured around availability assumptions that stop being true won’t easily revert. That’s what winter looks like.
But for the organisations that chose the right tool for the right business problem — summer is already here.
Distributed. Local. Efficient. Sustainable. Aligned to real business processes. Delivering real ROI.
Not because they anticipated the ceiling. Because they were asking the right question from the start — not “how do we use AI” but “where does better prediction, classification, or inspection create measurable value, and what is the smallest, most reliable tool that delivers it.”
Electronics couldn’t scale with valves. It scaled with the transistor. And everything that followed — the PC, the phone, the computer in your car — emerged from that single shift in direction.
AI will follow the same arc. Not because anyone planned it. Because the ceiling makes it inevitable.
The quiet AI already running on the production floor isn’t the consolation prize.
How AI is following the same arc as every technology before it — and why that’s the most important thing happening in enterprise AI right now.
The 22 Binders Problem
In the 1990s I worked for Volvo Trucks, responsible for technical documentation. Every authorised dealership workshop had 22 steel binders crammed with service manuals, torque specifications, fault code references, and replacement procedures. Covering every model, every variant, every engine configuration.
Getting those binders to the right workshop, in the right language, at the right time was a logistical nightmare. Print runs. Translation cycles. Physical distribution across multiple countries. And the moment a binder left the building, you started losing control of it.
Because trucks don’t stand still. Specifications change. Procedures get revised. Safety-critical updates happen. So we’d issue interim service bulletins — printed updates mailed out to dealerships, hoping they’d find their way into the right binder, in the right place, before a technician needed them.
The knowledge was good. The people who wrote those manuals understood those trucks deeply. The content was authoritative, structured, procedural. But the distribution model was broken by design. The moment you printed something, the clock started on it becoming out of date.
I spent years managing that problem. I never solved it. The technology didn’t exist to solve it.
It does now.
The Sledgehammer Era
When generative AI arrived at scale it came in like a sledgehammer. Vast data centres. Enormous compute. Hundreds of billions of parameters. Everything centralised, everything cloud-dependent, everything expensive.
That was necessary. You needed that scale to prove the capability existed. GPT-4, Gemini, Claude — these required hyperscale infrastructure to demonstrate what large language models could actually do. The sledgehammer era wasn’t wrong. It was the only way to get here.
But it’s not the end state.
There’s a pattern in technology that repeats so reliably it’s almost boring once you’ve seen it enough times. A new capability emerges at scale, centralised, specialist, expensive. Then the algorithms get smarter. The hardware gets smaller. The economics shift. And compute escapes the specialist environment and goes where the work actually happens.
We’ve seen this cycle before. Several times.
The Transistor Moment
In the 1940s the valve computer proved the concept. ENIAC, Colossus — genuinely capable machines doing real computation. Enormous, power hungry, heat generating, fragile, requiring specialist environments and specialist people just to keep running.
Then the transistor arrived. It didn’t just miniaturise the valve. It changed what was possible. Compute escaped the air-conditioned room. It went into consumer products, industrial equipment, vehicles. Eventually into the Volvo workshop — diagnostic computers, engine management systems, electronic service tools.
Nobody staring at a room full of valves in 1955 predicted the smartphone. But it was inevitable once the transistor existed. The use cases emerged from the distribution.
We are at that moment with AI.
The evidence is already here. Models like Mistral 7B, Microsoft’s Phi-3, and Meta’s Llama 3.1 8B perform remarkably well on focused tasks. Quantisation techniques mean a model that needed 80GB of GPU memory two years ago runs comfortably in 8GB today with minimal quality loss. NVIDIA has put one petaflop of AI compute into a desktop machine — the GB10 Grace Blackwell — that sits on a workbench. Apple put a capable language model in a phone.
The sledgehammer is giving way to the scalpel. Smarter algorithms. Smaller models. Distributed inference. The transistor moment.
The Architecture That Changes Everything
So let me describe what I’d build for Volvo Trucks today.
A GB10 desktop in every dealership workshop. On it: a small language model, a local MCP server, and a local cache of the workshop knowledge base. No internet connection required. No cloud dependency. No data leaving the building.
The knowledge base lives centrally — one authoritative source, maintained by a team of technical authors with a proper authoring and approval workflow. Single version of the truth. When a torque specification changes, it changes once, in one place, approved by the right person. Overnight, a delta sync pushes only the changed content to every workshop box in every country. By morning every technician in Europe has the current procedure.
The technician doesn’t type freeform queries. They select from predefined prompt templates — fault code lookup, torque specification, replacement procedure, service interval. Each template fires a carefully engineered retrieval query behind the scenes, pulling the right content from the local knowledge base, passing it to the local model, generating a precise, grounded answer.
The model never wanders off domain. There’s no internet to reach out to. No risk of the model hallucinating — confabulating a procedure it half-remembers from training. The answer comes from the authoritative knowledge base, retrieved precisely, generated locally, watermarked with the sync timestamp so you know exactly which version of the truth informed it.
This isn’t a chatbot. It’s a precision workshop tool that happens to use an AI model internally. The AI is an implementation detail. The value proposition is the right answer, instantly, for a truck that needs to move.
And crucially — this couldn’t be done with a naive RAG implementation bolted onto an ungoverned file store. The intelligence isn’t in the retrieval mechanism. It’s in the governance that happened before retrieval was ever involved. The single version of truth. The approval workflow. The deprecation process that ensures superseded procedures stop being retrieved. The content discipline that technical authors like my 1990s self spent years trying to maintain manually.
The AI amplifies good knowledge management. It doesn’t replace it.
The Pendulum
I’ve watched enough technology cycles to see the pattern clearly.
The PC era distributed compute out of the data centre and into the hands of individuals. The internet centralised it again — everything running on servers you didn’t own. Mobile distributed it once more, putting capable compute in every pocket. Cloud AI is centralising again — everything phoning home to a hyperscale data centre to answer a query.
Each time, the dominant narrative says this is how it will always be now. Each time, the pendulum swings back. Not because the technology fails, but because the same forces reassert themselves: latency becomes unacceptable, data sovereignty pressure builds, cost economics shift, and capability catches up to the point where distribution becomes viable again.
We are at peak centralisation with AI right now. The distribution forces are already building. Sovereignty regulation is tightening globally. Edge hardware is catching up fast. Algorithmic efficiency is compressing capable models into deployable sizes. Enterprises are growing uncomfortable with sensitive data leaving their premises to answer a query.
The pendulum will swing. It always does.
The Control Plane
But there’s something different this time that could break the historical pattern — and it’s the idea I find most interesting in enterprise AI right now.
Every previous distribution wave eventually lost coherence. The PC era fragmented into version chaos, security nightmares, and unmanageable sprawl. Mobile created a device estate that IT departments are still trying to govern. Distribution without discipline becomes a different kind of problem — one that often ends up being worse than the centralisation it replaced.
The question is whether AI distribution can be done differently. Whether you can have local inference without local chaos.
I think you can. The architecture looks like this: local inference running in your server room, your data never leaving your premises, your domain knowledge locked in a governed local knowledge base. But the model’s values, safety alignment, and capability updates maintained centrally by the people who built it. The enterprise controls the execution environment. The model provider maintains the model itself.
Distribution without fragmentation. Sovereignty without chaos.
It’s the Volvo KB architecture applied to the model itself. Central truth. Distributed execution. The same principle that would have solved my 22-binder problem in 1993 — one authoritative source, pushed to the edge, with discipline about what changes and who approves it.
This isn’t a theoretical position. The infrastructure to do it exists today. What’s missing, in most organisations, is the governance thinking that makes it safe. And governance thinking, it turns out, is not a technology problem. It’s a knowledge management problem. Which is a very old problem indeed.
What This Means
The use cases for distributed, domain-locked AI are not going to come from hyperscale thinking. They’re going to emerge from people who understand specific domains deeply — who know where the knowledge lives, what governance it needs, and what questions actually matter in that environment.
The Volvo workshop is one example. But the same architecture applies anywhere that has a bounded domain, authoritative knowledge, and real decisions being made by people who need the right answer quickly.
Consider a hospital ward. A clinician needs the current drug interaction protocol for a specific combination — not a general answer from a model trained on the internet, but the approved formulary for this trust, this version, signed off by the chief pharmacist last Tuesday. The architecture is identical to the workshop: local inference, governed knowledge base, delta sync, no data leaving the building. The AI is an implementation detail. The value is the right answer, for this patient, right now.
Or a field engineer on an offshore platform, no reliable connectivity, needing the current maintenance procedure for a specific valve configuration. Or a legal team needing to retrieve the approved contract clause library — not a hallucinated approximation, but the version that compliance signed off.
In each case the AI isn’t the interesting part. The interesting part is the governed knowledge layer underneath it — built by people who understand the domain, maintained with discipline, versioned and approved and auditable.
The sledgehammer era gave us the capability. The transistor moment gives us the distribution. What makes it useful is the same thing that always made the difference — knowing your domain, respecting your data, and being honest about what the technology actually does.
I learned that managing 22 steel binders for Volvo Trucks in the 1990s.
If you look back over the last 30 years, our technology history is riddled with failed projects, projects that never got off the ground, and projects that went massively over budget. And then there are the scandals.
The UK Post Office Horizon scandal stands as one of the most serious technology failures in modern British history. Between 1999 and 2015, around 1,000 sub-postmasters were wrongfully prosecuted after the Fujitsu-supplied Horizon accounting software recorded losses that did not exist. The total cost of redress now stands at around £2 billion. More than 13 people took their own lives. The technology did not just fail — it was trusted when it should have been questioned, and the consequences were devastating.
AI is just another technology. That is worth saying plainly. Although many of the possibilities being talked about are genuinely achievable, reality always kicks in. Every project — AI or otherwise — is a complicated combination of people, software, hardware, and process. That has been true for 30 years. It remains true now.
But we can always learn from the past. The question is whether we choose to.
AI is no different. And right now, the data on AI project failure should give everyone pause. The numbers are in. And they are not improving.
RAND Corporation, in one of the most rigorous independent analyses of AI project failure to date, interviewed 65 experienced data scientists and engineers across industries and company sizes. Their finding: more than 80% of AI projects fail to reach meaningful production deployment. That is twice the failure rate of IT projects without an AI component.
S&P Global’s 2025 survey of over 1,000 organisations across North America and Europe found that 42% of companies abandoned most of their AI initiatives that year. In 2024, that figure was 17%. The abandonment rate more than doubled in a single year. MIT’s Project NANDA, published in July 2025, found that 95% of organisations deploying generative AI saw zero measurable return. Not low return. Zero.
With global AI spending projected to reach $630 billion by 2028, these failure rates are not a statistic. They represent hundreds of billions of dollars in wasted investment, stalled initiatives, and businesses no closer to the outcomes they were promised.
What makes this harder to ignore is that the failure rate is moving in the wrong direction. The technology is more capable than it has ever been. The investment is larger than it has ever been. And yet more organisations are abandoning AI initiatives today than they were twelve months ago.
So what is actually going wrong?
The Research Points to Three Things
Across RAND, McKinsey, Gartner, MIT, and Informatica’s CDO Insights survey, the diagnosis is remarkably consistent. AI projects fail for three reasons, and they repeat themselves across industries, geographies, and organisation sizes.
The wrong use case. The process targeted for AI is not the right one, the problem definition is vague, or the initiative is chasing a technology rather than solving a business problem.
Data that is not ready. The data that would be needed to make AI work in production does not exist in the form required — it is fragmented, inconsistent, ungoverned, or simply not there.
A skills gap. The people needed to build, deploy, and sustain AI in a business context are not in place — and the organisation has not yet found a way to close that gap.
None of these are surprises. But the uncomfortable truth is that most organisations are still walking into all three of them, often simultaneously.
The Wrong Use Case: The Magpie Effect at Work
The first failure mode — choosing the wrong process to target — is something I have written about in depth. The Magpie Effect describes what happens when AI strategy is driven by possibility rather than process: the endless pivot towards the latest model, the newest capability, the most impressive vendor demo. Every pivot consumes time, burns budget, and erodes the momentum that comes from doing one thing properly and scaling it.
The RAND report is direct on this point. Stakeholders often misunderstand or miscommunicate what problem needs to be solved. Models get built and deployed optimised for the wrong metrics, or ones that simply do not fit into the real business workflow.
The antidote is starting with the Golden Process — the one business process that everything else hinges on — and asking what AI can do to remove the constraints within it. That conversation has to happen before any vendor is in the room, before any use case is evaluated, and certainly before any infrastructure is selected.
If you have not read the Magpie Effect post, the practical AI test at the end of it is a useful filter for any use case that lands on your desk.
Data That Is Not Ready: The Silent Failure
The second failure mode is the one that catches organisations by surprise — because it tends not to surface until a project is already in trouble.
The pattern is familiar. A proof of concept is built on a carefully selected, cleaned-up sample dataset. The demo runs well. Leadership approves production. And then everything stalls.
Production data is fragmented across systems that were never designed to talk to each other. Basic business terms — “customer”, “order”, “revenue” — are defined differently across departments. Historical records have gaps. Formatting is inconsistent. The clean sample that powered the demo bears almost no resemblance to the messy reality of how the business actually runs.
Informatica’s 2025 CDO Insights survey found that data quality and readiness was the top obstacle to AI success, cited by 43% of organisations. McKinsey found that organisations reporting significant AI returns are twice as likely to have invested in data infrastructure before selecting modelling techniques. Gartner predicts that 60% of AI projects lacking AI-ready data will be abandoned entirely.
This is not a new problem. It is the same problem that has existed since organisations first tried to build anything useful on top of their data. The difference is that AI amplifies it. A flawed report can be corrected. A model trained on broken foundations will confidently produce broken outputs at scale — and the damage compounds before anyone notices.
The third failure mode is the one that receives the least attention — and may be the hardest to solve quickly.
The last 30 years of enterprise IT have built deep, hard-won expertise in infrastructure. Server architecture. Storage design. Networking and virtualisation. Security and compliance. That expertise is not obsolete — it remains essential. The physical and virtual foundations that AI runs on still need people who understand them properly.
But AI demands a different and additional skill set that most organisations are still building.
Traditional IT Skills (Still Relevant)
New Skills Now Required
Server architecture and management
Data engineering
Storage design and optimisation
AI/ML engineering
Networking and virtualisation
Data science
Security and compliance
Business domain knowledge
McKinsey’s 2024 State of AI survey found that 58% of businesses are hampered by internal AI skill shortages. Informatica’s CDO Insights survey placed skills and data literacy third in the list of top AI obstacles. PwC found that almost 65% of executives acknowledge their AI initiatives are not succeeding because of a lack of executive sponsorship — a leadership gap that is itself a symptom of not having people in the room who can translate between business outcomes and AI capability.
The skills gap is not a technology problem. It is a people and partnership problem. And it is the reason most AI pilots never leave the room they were born in.
Picture a typical scenario. An organisation identifies a strong AI use case, aligned to a real business process. The data is in reasonable shape. Leadership is supportive. A vendor is engaged. And then the questions start: who is going to build the data pipeline? Who owns the model in production? Who bridges the gap between what the model does and what the business actually needs it to do? The room goes quiet. Not because the will isn’t there — but because the people aren’t.
The honest message is that organisations do not need to replace their existing IT teams — they need to extend them. The people who understand the infrastructure are still essential. But they need colleagues who understand data pipelines, model behaviour, and the business processes those models need to serve. That combination is rare. It takes time to build. And in its absence, even the right use case with clean data will stall.
This is why Partner selection matters as much as use case selection. The right Partner does not just bring technical capability — they bring the scars of what does not work. An AI and data practice built over years has already made the mistakes most organisations have not made yet. That is not a credential. That is insurance.
AI is not a product, it’s an eco-system built on partnership
Three Causes. One Pattern.
What the research is describing — even when it does not use these words — is organisations that jumped to the model before they had solved the three problems that sit upstream of it.
They chased the possibility rather than the process. They skipped the data foundations. And they underestimated how different the skills requirement would be.
McKinsey’s summary of what separates the 6% of genuine AI high performers from everyone else is as sharp as it gets: AI is 20% algorithms, 80% organisational rewiring.
The organisations building durable AI capability are not necessarily the ones with the most sophisticated models. They are the ones that got the process right, made the data ready, and put the right people in place — before they wrote a single line of model code.
The failure statistics look alarming until you understand the causes. Then they look entirely predictable.
The good news: all three are solvable. None of them require waiting for the next frontier model. They just require doing the less glamorous work first.
I have written about what that looks like in practice. Getting AI Right First Time sets out a five-step path from honest awareness through to durable, enterprise-wide capability — the journey from AI as a science project to AI as part of how the business actually runs. And In Praise of Boring, Everyday AI makes the case for what success actually looks like in production: not the demo, not the headline, but the quiet system that runs on a Tuesday afternoon without anyone noticing — because it has simply become part of how the business works.
That is the goal. Not AI that impresses. AI that endures.
AI success has to be built on the right foundations — not retrofitted onto broken ones.
