Five hospitals want to train one model to spot a rare tumour on MRI scans. Between them they have enough cases to make the model genuinely good. Individually, none of them does. And none of them can send a single scan across the border to the others, not because of red tape somebody could cut through with a signature, but because the data is patient records, and patient records do not travel between institutions or countries just because a research project would benefit.
Federated learning is the answer that gets reached for in exactly that situation: train a shared model without moving the data that trains it. Instead of pooling records in one place, each site trains locally on its own data, and only the resulting model updates, not the raw records, get sent back and combined into a shared model. The data never leaves the building it started in. What travels is a description of what the model learned there.
We build data and AI systems at Shipshape Data, and federated learning is one of the ideas clients arrive having half-understood from a vendor slide, usually pitched as the solution to a data privacy problem that a much simpler fix would have solved. This guide covers how the mechanics actually work, the real cases that made the approach credible, what it does and does not do for privacy and regulation, the engineering and communication costs that rarely make the pitch deck, and where it earns its keep rather than adding a distributed systems problem to a problem you already had.
How training without centralising the data actually works
Strip away the vendor language and the loop is not complicated. A central server holds a global model and sends a copy of it out to a set of participants, which might be hospitals, phones, banks or factories. Each participant trains that copy for a while on its own local data. Nobody else sees the data; it stays exactly where it was. Each participant then sends back not its data but its model update, the change in weights that its local training produced. The server averages those updates across all participants, folds the result into the global model, and sends the improved model back out for another round. Repeat until the model stops improving.
The part that actually crosses the network
What crosses the wire is gradients or weight deltas, numbers describing how a model changed, not the rows of data that changed it. That distinction is the entire pitch of federated learning, and it is worth sitting with, because it is also the source of most of the confusion people have about what the approach guarantees. A weight update is not the data. But it was computed from the data, and information can leak through it in ways we will come back to.
Non-IID is the polite name for "everyone's data looks different"
The averaging step assumes, implicitly, that the participants are training on data that behaves roughly the same way. In practice it rarely does. One hospital sees more advanced cases because it is a referral centre. One phone user types mostly in one language and another mixes three. A rural bank's transaction patterns do not resemble a city bank's. Statisticians call this non-independent, non-identically distributed data, non-IID for short, and it is the reason a naive average of updates can pull the global model towards whichever participants happen to have the most data or the loudest gradients, rather than towards a model that genuinely serves everyone.
The documented cases that made the idea credible
This is not a hypothetical exercise. Google popularised the approach through Gboard, its Android keyboard, using on-device training to improve next-word prediction from what people actually type, without uploading the typed text itself to a central server. Apple has run something similar for on-device personalisation in iOS. Neither case is medical or dramatic. Both are exactly the situation federated learning suits: huge numbers of small, similar devices, each holding a sliver of behavioural data too sensitive and too voluminous to sensibly centralise, all contributing to one shared model. Notice what makes both cases tractable: the devices are running near-identical software, the task is narrow, and nobody is trying to reconcile five different definitions of what a "word" or a "keystroke" means. That uniformity is doing a lot of quiet work.
The FeTS project, and what it actually took
On the clinical side, the Federated Tumour Segmentation project, FeTS, trained a shared model to segment brain tumours on MRI scans across dozens of institutions worldwide, none of which shared patient images with the others. It is a genuine, published, multi-year academic and clinical collaboration, not a vendor case study, and it is worth reading precisely because the papers are honest about the cost: years of coordination, careful protocol design, and infrastructure most of the participating hospitals did not have lying around. The model got built. It also took a research consortium with dedicated funding to build it.
Federated learning does not remove the cost of collaboration. It moves the cost from moving the data to coordinating the training, and coordinating the training is its own kind of expensive.
The privacy and regulatory case for it
The pull towards federated learning almost always starts with a rule, not a preference. Patient data cannot leave a hospital's jurisdiction. Bank transaction records cannot cross certain borders. Under GDPR, moving personal data between organisations, or out of the EU, is not a paperwork step you complete once, it is an ongoing legal exposure with real teeth. When five organisations each hold a piece of the picture and none of them is allowed to hand its piece to the others, training one model on all the data by copying it into a warehouse is not a hard engineering problem. It is not allowed at all.
Where the law actually points at architecture
This is the genuine case for federated learning, and it is narrower than most pitches suggest. It applies when the constraint is legal or contractual rather than technical, when the data genuinely cannot be pooled under any reasonable governance arrangement, and when the value of a shared model still clears the bar despite that constraint. Competing banks that want a fraud model trained on transaction patterns across all of them, but are barred by competition law and client confidentiality from simply exchanging customer data, are in that position. So are hospitals bound by different national health data rules who still want one model rather than five weaker ones.
What this is not a fix for
It is not a fix for data that is merely inconvenient to move. If the barrier is that nobody built the pipeline, or that two departments never agreed on a schema, federated learning adds a distributed training system on top of a problem that a proper integration project would have solved directly, and more cheaply. We see this distinction blurred constantly. A team facing ordinary data silos, systems that could legally and technically share data but never got connected, reaches for federated learning because the word "federated" sounds like it addresses fragmentation. It does not. It is built for data that cannot be centralised, not data that merely has not been.
What it doesn't automatically give you
Here is the part that gets glossed over in the simpler explanations. Not sending raw data across the network is a real privacy improvement over the alternative. It is not the same thing as a formal privacy guarantee, and treating it as one is where projects get into trouble later.
Gradients leak more than people expect
A model update was computed from real data, and under the right conditions that data can be partially reconstructed from the update itself. Research on gradient inversion has shown that, for some model architectures and some batch sizes, an attacker who can see the gradients can recover recognisable fragments of the training images or text that produced them. This is not a reason to abandon the approach. It is a reason to treat "the data never left the device" as the start of a privacy design, not the end of one.
The techniques that actually close the gap
Two additions do most of the real work here. Differential privacy adds calibrated statistical noise to the updates before they leave the participant, so the contribution of any single record is mathematically bounded and cannot be isolated afterwards, at some cost to model accuracy that has to be tuned rather than assumed away. Secure aggregation uses cryptographic protocols so the central server only ever sees the sum of many participants' updates, never any individual one, which closes off a different attack entirely. Serious federated learning deployments use one or both. Deployments that skip them are trading a real vulnerability for a comforting name.
The engineering cost that rarely makes the pitch deck
Assume the legal case is genuine and the privacy design is sound. There is still a system to build, and it behaves nothing like training a model on a table sitting in one warehouse.
Communication is the bottleneck, not compute
In centralised training, the expensive resource is usually compute: GPUs, training time, the size of the dataset. In federated learning, the expensive resource is often the network. Every round of training means sending a full model out to every participant and getting an update back from each of them, over connections that in practice range from a good hospital network to a phone on patchy mobile data. Do that for hundreds of rounds, across dozens or thousands of participants, and the traffic adds up to something that has to be engineered, not assumed. Techniques exist to compress updates and reduce how often full synchronisation happens, and none of them are free; they trade communication cost for either accuracy or engineering complexity. A team that budgeted for GPU time and forgot to budget for bandwidth and round-trip latency across a dozen institutional firewalls tends to find that out partway through the first real deployment, which is a slow and expensive place to learn it.
Not every participant shows up, and the round has to survive that
Centralised training runs on a machine you control, on a schedule you set. Federated training runs on hospitals with their own IT priorities, phones that go offline, factories whose edge devices lose power mid-round. The system has to tolerate participants dropping out partway through a round without corrupting the aggregate, handle stragglers without blocking everyone else indefinitely, and keep some sensible record of which version of the model each participant is actually running. None of this is exotic. All of it needs building, testing and monitoring, on top of the machine learning itself.
You cannot debug the way you are used to
When a centralised model behaves oddly, you pull the offending rows and look at them. In a federated setup you often cannot, because that is precisely the data you were never allowed to see. Diagnosing why the model performs badly for one class of participant, and not others, becomes an exercise in reading aggregate statistics and update patterns rather than looking at the records themselves. Teams used to debugging by staring at data find this genuinely disorienting the first time, and it is worth budgeting the extra time that adjustment costs. We have watched capable engineers lose the better part of a week hunting a training bug through log statistics alone, purely because their instinct was to open the offending file and look, and the whole point of the system was that they could not.
Where it quietly turns into a data warehouse problem in disguise
There is a version of this that catches organisations out. They adopt federated learning because centralising the data feels impossible, build the coordination layer, run the rounds, and eventually discover that half their actual difficulty was never about where the data physically sat. It was that nobody had agreed what a "customer" or a "transaction" or a "patient record" meant across the participating systems in the first place.
Federated learning does not resolve that. It assumes reasonably comparable data and reasonably comparable definitions walking into the training loop, and if participant A's "positive case" means something different from participant B's, no amount of gradient averaging fixes it. Where the constraint really is legal, we still tell clients to do the unglamorous groundwork first, the shared definitions, the comparable feature engineering, the same work that ordinary data integration would require, because federated learning inherits every one of those problems and adds distributed systems engineering on top rather than instead.
When it's genuinely the right answer
Put the case plainly. Federated learning earns its cost when the data legally or contractually cannot be pooled, when the participants are numerous enough or the data sensitive enough that the privacy design is worth building properly, and when a shared model is meaningfully better than several separate ones trained in isolation. Hospital consortia across jurisdictions. Competing banks barred from exchanging customer records but wanting a shared fraud signal. Millions of consumer devices, each holding a scrap of typed or spoken data too personal and too voluminous to ever centralise sensibly.
It is the wrong answer, expensive theatre rather than a solution, when the real blocker is an unbuilt pipeline, an unsigned data-sharing agreement that could plausibly get signed, or a handful of participants small enough that a properly governed central dataset, with access controls and audit logging, would do the same job for a fraction of the engineering effort. A federated system for three willing internal business units that could legally share a database is a distributed systems project you did not need.
Where to start
Work out first whether the constraint is legal or merely organisational. That single question decides almost everything else, and it is worth getting a straight answer from whoever owns the compliance position rather than assuming. If the data genuinely cannot be pooled, and it is regulation or contract rather than inertia keeping it apart, only then does the rest of this become worth doing.
From there, treat privacy as a design decision rather than a side effect of the architecture. Decide early whether differential privacy or secure aggregation, or both, are going into the build, because retrofitting them after the coordination layer exists is far harder than including them from the start. Budget for the communication cost and the debugging difficulty honestly, because both are real line items, not footnotes. And before any of that, check whether the definitions each participant is training against actually match. A shared model trained on mismatched data still ends up wrong, and quietly wrong in a way that is harder to spot than the version built on data sitting in one warehouse. If synthetic data would clear the same legal bar without the coordination overhead, our guide to synthetic data is worth reading before committing to the federated route.
If you are weighing this up for a real project, with genuine cross-institutional constraints rather than an internal turf war dressed up as one, talk to us. We would rather tell you honestly that a governed shared database solves your problem for a tenth of the cost than watch you build the distributed version because it sounded more sophisticated.