Private analytics via zero-trust aggregation

By processing data locally, on-device AI can provide enhanced protection and timely alerts while keeping user information private. For example, Android uses a system called SafetyCore to provide privacy-preserving on-device features and common infrastructure to protect users from unwanted content. When developing on-device technologies, teams need to understand how well their systems work across millions of individual smartphones, each with unique data distributions, varying hardware constraints, and different user behaviors. To achieve this in a way that reveals only collective trends without revealing individual user data, teams can leverage cryptographic secure aggregation as a key building block. Like all cryptographic protocols, secure aggregation uses advanced mathematical tools to provide its security assurance.

Today, we set a higher bar for efficient cryptographic aggregation in a private analytics service. We follow a zero-trust principle, which aims to reduce trust necessary in any single entity. We achieve this through a new security design that combines cryptographic and hardware protection mechanisms. Our solution leverages a new cryptographic aggregation method that provably guarantees only anonymized, aggregated insights about a population can be obtained by Google. Additionally, trusted execution environments (TEEs) are used to provide a strict layer of attestation and transparency.

The challenge

When models are deployed locally on-device, simply knowing that a model is 'running' isn't enough to understand its behavior, effectiveness, or failure modes. This limits the ability to answer critical questions like:

• Is the model drifting? (e.g., Does a translation model struggle with new slang emerging in a specific region?)

• Are there hidden biases? (e.g., Is an image classifier less accurate under specific lighting conditions common in certain geographic areas?)