The encyclopedia · Engineering & Operations · Technical decision · 2016
Google's federated learning trained a model by moving code, not data.
Google's FedAvg trains a shared model by sending the model to phones and averaging updates, so raw data never leaves the device.
the move
Standard machine learning moves training data to a data center, but on-device data for keyboards and photo ranking is both privacy-sensitive and too large to upload wholesale.
Federated learning inverts that: the phone downloads the model, trains on its own local data, and uploads only a compact update, which the server averages with everyone else's to improve the shared model. No individual update or raw data is ever stored in the cloud.
The Federated Averaging algorithm made this practical by having each device compute higher-quality updates, cutting the communication rounds needed by 10-100x versus a straightforward federated approach, and Google deployed it to Gboard's query-suggestion model.
why it works
- Private data never leaves the device, so privacy is preserved
- Only a small average update leaves the phone, saving bandwidth
- Averaging updates lets the shared model improve without seeing the data
- The device also keeps the improved model for immediate local use
what transfers
When the raw material is private and enormous, stop bringing it to the factory; bring the factory to it, and ship the small thing the factory learned.
what came after
Federated learning became a foundation of on-device privacy-preserving AI, deployed in Gboard and across Google products and adopted widely in mobile and healthcare applications. Secure aggregation followed to ensure even individual updates could not be inspected.
references
- Communication-Efficient Learning of Deep Networks from Decentralized Data
- Federated Learning: Collaborative Machine Learning without Centralized Training Data
spotted an error? The archive wants to know.