Federated learning: how it works and when it's the right answer

Five hospitals aim to train one model that can detect a rare tumor on MRI scans. Together, these hospitals possess sufficient cases to build an effective model. Individually, no hospital has enough cases to accomplish this.

These institutions are unable to exchange scan data with each other. Simple agreements cannot solve this problem. The data comprises patient records, and such records are not allowed to be transferred between different organisations or across national borders for research.

Federated learning builds a shared model without centralising the data. Each participant trains the model locally and transmits only the update to be merged into a single shared version. This approach is useful when data cannot be pooled for legal reasons, for example between hospitals or competing banks. It is meant for these restrictions rather than for situations where linking data is merely inconvenient.

Federated learning is employed to train a shared model without relocating the underlying data. Each site trains the model locally with its own records rather than pooling them centrally. Only the resulting model updates are returned to be merged into the shared model. The raw data stays at its original site. The transmitted information is limited to a description of what the model learned at that location.

Shipshape Data constructs data and AI systems. Clients frequently enquire about federated learning after observing it in vendor presentations. It is often promoted as a privacy solution even when simpler methods would be more effective. This guide describes the mechanics of the technology and the specific use cases that have established its credibility. It details the real impact on privacy and regulation. It also outlines the engineering and communication costs that are often omitted from sales pitches. Finally, it pinpoints where federated learning is useful without creating unnecessary technical complications.

How training operates without centralising data

If you remove the vendor language, the loop is not complicated. A central server maintains a global model and distributes a copy to a set of participants, which may include hospitals, phones, banks, or factories. Each participant trains that copy for a period using its own local data. No one else sees the data; it remains exactly where it was. Each participant then returns not its data but its model update, the change in weights produced by its local training. The server averages those updates across all participants, incorporates the result into the global model, and distributes the improved model for another round. Repeat this process until the model stops improving.

The portion that travels across the network

The transmitted information consists of gradients or weight deltas. These numbers describe how a model changed rather than the specific rows of data used for training. This distinction forms the primary basis for federated learning. It also explains why many people are confused about the specific guarantees this approach provides.

A weight update is not the data. But it was computed from the data, and information can leak through it in ways we'll come back to.

Non-IID is the polite name for "everyone's data looks different"

The averaging step assumes participants train on data with similar characteristics. In practice, this is rarely true. A referral hospital encounters more advanced cases than a general clinic. One phone user may type in a single language while another uses three. Transaction patterns at a rural bank differ from those at a city bank. Statisticians call this non-independent and non-identically distributed data, or non-IID. Consequently, a simple average of updates can bias the global model towards participants with the most data or the strongest gradients. This hinders the model from serving all users effectively.

The documented cases that made the idea credible

This is not a hypothetical exercise. Google popularised the approach via Gboard, its Android keyboard, using on-device training to improve next-word prediction from what people actually type, without uploading the typed text to a central server. Apple has implemented something similar for on-device personalisation in iOS. Neither case is medical or dramatic. Both precisely match the situation federated learning suits: huge numbers of small, similar devices, each holding a sliver of behavioural data too sensitive and too voluminous to sensibly centralise, all contributing to one shared model. Notice what makes both cases tractable: the devices run near-identical software, the task is narrow, and no one is trying to reconcile five different definitions of what a "word" or a "keystroke" means. That uniformity does a lot of quiet work.

The FeTS project, and what it took

In the clinical field, the Federated Tumour Segmentation project (FeTS) trained a shared model to detect brain tumors on MRI scans. The project involved dozens of institutions worldwide that did not exchange patient images with each other. This is a published academic and clinical collaboration rather than a vendor case study. The project papers detail the actual costs involved, including years of coordination, detailed protocol design, and infrastructure that most participating hospitals had to acquire specifically for this work.

We built the model. A research consortium with dedicated funding was needed to complete the work.

Federated learning does not remove the costs associated with collaboration. It shifts the expense from data transfer to coordinating the training process. This coordination demands its own significant resources.

The privacy and regulatory case for it

The shift to federated learning usually starts with a requirement. Patient data cannot exit a hospital's control. Bank transaction records cannot be transferred across specific boundaries. According to GDPR, moving personal data between organisations, or outside the EU, poses a continuous legal risk with significant penalties, and is not a one-time administrative task. When five organisations each hold part of a complete dataset and none can share their portion with the others, training a single model using all the data by copying it to a central location presents no technical challenge. It is prohibited.

Where the law points at architecture

The real argument for federated learning is more specific than typical marketing claims. This approach is useful when constraints are legal or contractual rather than technical. It applies when data cannot be centralised under any governance structure, yet the benefits of a shared model remain high enough to justify the effort. Competing banks fit this description when they want a fraud model trained on collective transaction patterns but are prohibited from sharing customer data by competition law and confidentiality rules. Hospitals are in a similar situation when they are bound by different national health data regulations but require one effective shared model instead of several weaker versions.

What this does not fix

Federated learning is not a solution for data that is difficult to move. If the problem is that a pipeline does not exist or that departments have not agreed on a schema, this technology adds a distributed training system to a situation that a standard integration project would solve more effectively and at a lower cost; this distinction is often misunderstood. Teams dealing with typical data silos, which are systems that could legally and technically share data but remain disconnected, often choose federated learning because the term suggests it addresses fragmentation; it does not. This method is designed for data that cannot be centralised, rather than data that simply has not been centralised yet.

What the system does not provide automatically

Simple explanations often omit this detail. Transmitting processed data instead of raw data provides more privacy than the alternative method. However, this approach does not constitute a formal privacy guarantee. Projects often face difficulties when they treat this technical improvement as a definitive safeguard.

Gradients leak more than people expect

Model updates are calculated using real data. In specific circumstances, an attacker can use these updates to reconstruct parts of the original data. Research into gradient inversion indicates that for certain model architectures and batch sizes, an observer can recover recognisable pieces of training images or text from the gradients. These findings do not mean the approach should be discarded. Instead, they indicate that engineers should treat the fact that "the data never left the device" as the beginning of a privacy design rather than the conclusion.

The techniques that reduce the difference

Two specific methods provide the security in this process. Differential privacy adds statistical noise to updates before they are sent by the participant. This ensures that a single record's contribution is mathematically limited and cannot be identified later. This method reduces model accuracy, so the noise levels must be carefully adjusted. Secure aggregation uses cryptographic protocols to ensure the central server only views the total sum of updates from many participants rather than individual ones. This prevents a different type of security breach. Professional federated learning systems use one or both of these techniques. Systems that do not include them remain vulnerable despite the name of the technology.

The engineering cost that rarely makes the pitch deck

Suppose the legal case is authentic and the privacy design is functional.

We still need to build a system. This process is different from training a model on a single table located in one data warehouse.

Communication is the bottleneck, not compute

In centralised training, the primary expense usually comes from compute resources such as GPUs, training time, and dataset size. In federated learning, the dominant cost is typically the network. Each training round involves transmitting the full model to every participant and receiving their update, using connections that may range from a stable hospital network to a mobile phone with a weak signal. Repeating this for hundreds of rounds and dozens or thousands of participants creates large network traffic that must be carefully planned, not assumed. There are methods to compress updates and lower the frequency of full synchronisation, but they carry costs; they trade communication expense against accuracy or engineering complexity. A team that budgets for GPU time but neglects bandwidth and round-trip latency across several institutional firewalls typically discovers the issue during initial deployment, resulting in a costly and time-consuming learning experience.

The round must keep running even if some participants are absent

Centralised training runs on hardware you control and follows a schedule you set. Federated training happens on external devices, for example hospital servers with particular IT requirements, mobile phones that may disconnect, or factory equipment that can lose power. The system must operate when participants drop out during a training round to avoid data corruption. It must also handle slow devices so the whole process is not delayed. The infrastructure must track which model version each participant is using. These requirements are ordinary engineering tasks, yet they need development, testing, and monitoring alongside the machine learning components.

You cannot apply your usual debugging methods

When a centralised model behaves unexpectedly, you can inspect the particular rows of data that cause it. In a federated system, you often cannot do this because the system is built to keep that data private. Finding out why a model performs poorly for one group of participants but not others requires analysing aggregate statistics and update patterns rather than individual records. Teams used to debugging by inspecting raw data find this shift challenging initially. You should set aside extra time for this adjustment. We have observed experienced engineers spend almost a week searching for a training bug using only log statistics. This occurred because their natural instinct was to open the data files, which the system is specifically designed to prevent.

Where it quietly turns into a data warehouse problem in disguise

Some organisations encounter a particular problem with this approach. They select federated learning because centralising their data is not possible. They construct the coordination layer and execute the training rounds, yet they later discover that the data's physical location was only one aspect of the difficulty. The main problem was that the participating systems had never established standard definitions for terms such as "customer," "transaction," or "patient record."

Federated learning does not address this issue. It demands that data and definitions be comparable prior to the start of the training process. If participants use different definitions of a positive case, averaging the gradients will not fix the discrepancy. When legal constraints are present, we recommend that clients finish the foundational work first. This involves establishing shared definitions and comparable feature engineering. These are the same tasks needed for standard data integration. Federated learning does not replace these requirements; it preserves all of these problems and adds the complexities of distributed systems engineering.

When it is the appropriate answer

Put the case plainly. Federated learning earns its cost when the data legally or contractually cannot be pooled, when the participants are numerous enough or the data sensitive enough that the privacy design is worth building properly, and when a shared model is meaningfully better than several separate ones trained in isolation. Hospital consortia across jurisdictions. Competing banks barred from exchanging customer records but wanting a shared fraud signal. Millions of consumer devices, each holding a scrap of typed or spoken data too personal and too voluminous to ever centralise sensibly.

This approach is an incorrect and costly substitute for a solution. It is unnecessary when the real problem is an unbuilt pipeline, a pending data-sharing agreement, or a small group of participants. A central dataset with access controls and audit logging would deliver the same results with far less engineering effort. Building a federated system for three internal business units that are legally permitted to share a database is a distributed systems project you do not need.

Where to start

First, determine whether the constraint is legal or organisational. This question determines the subsequent steps. Obtain a clear answer from the compliance officer rather than assuming. Proceed with these steps only if regulations or contracts, not internal habits, prevent data pooling.

Consider privacy a deliberate design decision instead of an unintended consequence of the system architecture. At the project start, decide whether to employ differential privacy, secure aggregation, or both. Adding these features after constructing the coordination layer is difficult. You need to consider the high communication costs and the debugging challenges of these systems. These represent significant technical requirements. Also verify that each participant uses the same data definitions. Training a shared model on inconsistent data will produce incorrect results. Such errors are more difficult to detect in federated systems than in centralised databases. If synthetic data satisfies your legal requirements without requiring complex coordination, read our guide to synthetic data before beginning a federated project.

If you are evaluating this for an actual project with genuine cross-institutional constraints, not an internal turf war in disguise, talk to us. We prefer to honestly tell you that a governed shared database solves your problem at one-tenth the cost rather than watching you build a distributed version just because it seemed more sophisticated.

When moving from research to planning, our AI consulting services support everything from strategy to delivering a production-ready build.

Frequently asked questions