The "Average" Trap: Why Decentralised AI Is One Bad Actor Away From Collapse

By Bob John  | Technology Correspondent

Co Authored by AI

Thursday, 5 February 2026

In the sterile, humming server rooms of London’s major teaching hospitals, a quiet revolution is taking place. It is called Federated Learning.

The promise is seductive: hospitals can collaborate on an AI to detect rare cancers without ever sharing a single patient file. The data stays local; only the "intelligence" travels. It is the ultimate digital privacy shield.

But there is a ghost in the machine.

While we are busy celebrating the privacy benefits, we are ignoring a fundamental flaw in the way these systems listen. Most current DIY setups rely on a mathematical instrument as blunt as a sledgehammer: the simple average.

By treating every participant as an equal, we aren't creating a digital democracy. We are creating a system vulnerable to manipulation, incompetence, and catastrophic failure. Here is why the "average" is failing us, and why the next generation of AI must learn to discriminate.

1. The Rotten Borough Problem

The Flaw: Data Imbalance

Imagine a general election where a farmhouse in the Hebrides has the exact same voting power as the entirety of Greater Manchester. You wouldn't call that democracy; you’d call it a rigged system. Yet, this is exactly how simple averaging works in early-stage or naive Federated Learning implementations.

Consider two hospitals training a diagnostic AI. Hospital A is a massive urban trauma centre seeing 10,000 patients a month. Hospital B is a tiny, rural specialist clinic with 10 patients.

If the central brain treats them equally (a simple mean), the quirky, statistical noise from the tiny clinic has just as much influence as the massive, verified trends from the urban centre. The model doesn't learn the truth; it learns a compromise between a mountain of evidence and a handful of anecdotes.

The Upgrade: Proportional Representation The fix is Weighted Federated Averaging (FedAvg). It is the AI equivalent of proportional representation. We stop counting "heads" and start weighing "volume." The central brain must assign influence based on the amount of data contributing to the update. The noise of the few is acknowledged, but it is no longer allowed to drown out the reality of the many.

2. The Heckler’s Veto

The Flaw: Poisoning & Outliers

In any town hall meeting, there is an unwritten rule: the person shouting the loudest usually clears the room.

In the mathematical world, this is known as "Model Poisoning" or a scaling attack. Because the simple average is obedient, it listens to magnitude. If a thousand honest phones whisper "this is a cat," and one malicious, hacked device screams "THIS IS A TOASTER" at a volume of billion decibels, the simple average panics. It drags the entire global model towards "Toaster."

We have seen this vulnerability in early predictive text models destroyed by trolls, and we risk it again with critical infrastructure. If a saboteur wants to break a power grid AI, they don't need to hack the server; they just need one compromised smart meter sending back impossibly large numbers.

The Upgrade: The Digital Bouncer The industry term is Norm Clipping, but think of it as a strict bouncer at the door of the club. The system sets a "speed limit" on influence. It looks at the incoming update and says, “I’ll take your suggestion, but I’m capping your volume.” No matter how loud the malicious actor shouts, their input is scaled down to a whisper before it enters the room.

3. The Bill Gates Paradox

The Flaw: Adversarial Robustness

Even without the screaming hecklers, the "average" is famously fragile. It has a glass jaw.

The classic example is simple: Imagine you are in your local pub. You calculate the average wealth of the drinkers—perhaps £35,000. Then, Bill Gates walks in. Suddenly, the average wealth in the pub is £100 million.

Has everyone in the pub suddenly become a millionaire? No. The average has lied to you. It has been pulled entirely off course by a single outlier. In AI, this allows a coordinated group of "bad actors" to slowly, subtly drift a model off course without ever triggering alarms.

The Upgrade: The Wisdom of the Median Professional-grade systems are abandoning the mean in favour of the Coordinate-wise Median.

Unlike the average, the median is stubborn. If Bill Gates walks into the pub, the median income stays exactly where it was: with the regular people in the middle of the room. By taking the median value of all the updates, the AI effectively ignores the fringes. It listens to the consensus, not the exceptions. It is a mathematical shield that says: "I don't care how rich or crazy the guy at the end of the bar is; I'm sticking with the crowd."

The Verdict

The Verdict

We are moving past the "move fast and break things" era of AI. When these models run our traffic lights, diagnose our hearts, and filter our news, "breaking things" is no longer an option. The simple average was a convenient placeholder, a relic of the research lab. But for the real world? It is too naive, too vulnerable, and too easily gamed.

To rely on it today is to voluntarily trap your algorithm in a scene of tragicomic horror: imagine the existential angst of Edvard Munch’s The Scream, but wearing Onslow’s string vest. It is the realization that your pristine, Hyacinth Bucket-esque aspirations for "perfect" AI are currently being held hostage by the slobiest, most chaotic data point in the network.

It is time to stop averaging, and start governing.