Intel, in collaboration with Avast and Borsetta, launched the Private AI Collaborative Research Institute to advance and develop technologies in privacy and trust for decentralized artificial intelligence (AI). The companies issued a call for research proposals earlier this year and selected the first nine research projects to be supported by the institute at eight universities worldwide.
The Private AI Collaborative Research Institute was originally established by Intel’s University Research & Collaboration Office, which then invited Avast, a global leader in digital security and privacy products, and Borsetta, an AI software-defined secure compute hardware company, to collaborate on the institute.
“The Private AI Collaborative Research Institute will be committed to advancing technologies using ethical principles that put people first, and keep individuals safe and secure,” said Richard Uhlig, Intel Senior Fellow, vice president and director of Intel Labs.
Need for Decentralized Analytics at the Edge
Industry is trending toward intelligent edge systems. Algorithms such as neural networks and distributed ledgers are gaining traction at the edge on the device level without reliance on cloud infrastructure. To be effective, this requires huge amounts of data that is often sensed at the edge, such as vehicle routing, industrial monitoring, security threat monitoring, or search term predictions.
However, training for AI models is centralized with large amounts of data pooled in the data center of a trusted provider. To perform classification, the resulting model is then distributed to the edge. In many cases, today’s centralized approach is limiting performance. For example, health data is siloed and cannot be used for centralized training due to privacy and regulatory constraints. Autonomous cars generate terabytes of traffic data where bandwidth prevents centralized training. Personal computers and phones in billions of homes generate vast amounts of data daily, which cannot be uploaded due to privacy concerns.
The Private AI Collaborative Research Institute will focus its efforts on overcoming five main obstacles of the current centralized approach:
· Training data is decentralized in isolated silos and often inaccessible. Most data stored at the edge is privacy-sensitive and cannot be moved to the cloud.
· Today’s solutions are insecure and require a single trusted data center. Centralized training can be easily attacked by modifying data anywhere between collection and cloud. There is no framework for decentralized secure training among potentially untrusting participants.
· Centralized models become obsolete quickly. Infrequent batch cycles of collection, training, and deployment lead to outdated models, making continuous and differential retraining not possible.
· Centralized compute resources are costly and throttled by communication and latency. It requires vast data storage and compute as well as dedicated accelerators to make training viable.
· Federated machine learning (FL) is limited. While FL can access data at the edge, it cannot reliably guarantee privacy and security.
Research at the Private AI Collaborative Research Institute will address secure, trusted, and decentralized analytics, and compute at the edge. By decentralizing AI, the institute plans to liberate data from silos, protect privacy and security, and maintain efficiency. The 2021 inaugural university awardees will focus on the following research areas:
Carnegie Mellon University (CMU)
Unified Framework for the Competing Constraints in Federated Learning
Principal investigator: Virginia Smith
CMU will focus on federated learning systems with competing constraints, such as accuracy, fairness, robustness, and privacy. CMU will investigate statistical heterogeneity as a root cause for tension between these constraints. In particular, CMU will tackle heterogeneity via a unified framework for robust, privacy-preserving multi-task learning — unlocking a new generation of FL systems that can holistically address the constraints of realistic federated networks.
National University of Singapore (NUS)
Robust and Privacy-Preserving Knowledge Transfer for Heterogeneous Decentralized Learning
Principal investigator: Reza Shokri
When decentralized learning algorithms exchange model parameters, it limits the scalability of such algorithms, prevents the support of heterogeneous networks, and introduces many robustness and privacy issues. NUS will design knowledge transfer algorithms which are robust, privacy-preserving, and support heterogeneous networks. NUS also will focus on designing a theoretical framework and algorithms for certifiably robust and differentially private knowledge transfer algorithms for decentralized learning. The team will evaluate and test its scalability and efficiency for secure multi-party computation algorithms.
Université Catholique de Louvain (UCLouvain)
Federated Private Learning on Heterogeneous Devices
Principal investigators: Axel Legay and Thomas Given-Wilson
UCLouvain is focused on the detection, classification, and analysis of malware. With malware, one must gather and analyze diverse and incomplete information to build a unified understanding of a sample program. Many heterogeneous devices can be exposed to a new malware sample and data from these devices can be combined to learn a highly accurate model of the sample. UCLouvain will focus on how to learn from diverse information from heterogeneous devices while ensuring privacy.
University of California, San Diego (UCSD)
Private Decentralized Analytics on the Edge (PriDEdge)
Principal investigator: Farinaz Koushanfar
The training and data management in FL, especially when executed through secure protocols, entails a large amount of computation that makes its practical deployment a challenge. To ease the burden on these computations, the UCSD team focuses on evaluation of the cryptographic primitives and devising new hardware-based primitives that complement the existing resources on Intel processors. The new primitives include accelerators for homomorphic encryption, Yao's garbled circuit, and Shamir's secret sharing. Placing several cryptographic primitives on the same chip will ensure optimal usage by enabling resource sharing among these primitives. Furthermore, the UCSD team plans to design efficient systems through the co-optimization of the FL algorithms, defense mechanisms, cryptographic primitives, and the hardware primitives.
University of Southern California (USC)
Secure and Privacy Preserving Machine Learning: Foundations and Scalable System Design (PPML)
Principal investigators: Salman Avestimehr and Murali Annavaram
With PPML, USC will address critical requirements of decentralization, security, and scalability in distributed machine learning (ML). USC will also expand the PPML framework by leveraging trusted execution environments to enhance the security and improve the performance of the approach. USC will demonstrate how multiple data-owners can jointly train a machine learning model while keeping individual datasets private and secure.
University of Toronto (U of T)
Cryptography in Privacy-Preserving Machine Learning
Principal investigator: Nicolas Papernot
The predictions of ML systems often reveal private information contained in training data, necessitating learning algorithms that provide confidentiality and privacy guarantees. U of T will focus on the collaborative training of ML models across few participants with sensitive datasets and will construct a protocol for collaborative ML providing both confidentiality and privacy. U of T will rely on cryptography for participants to query others without revealing the input queried. In conjunction, differential privacy will prevent the querying participant from learning about other participants’ data.
University of Waterloo (UW)
Confidence in Distributed AI Systems
Principal investigator: N. Asokan
UW will focus on protecting the confidentiality of two types of sensitive data involved in machine learning: model parameters and training data. Real-world deployment of ML-based systems requires convincing confidentiality protection. UW will devise leakage-resistant aggregation mechanisms and effective model watermarking techniques for federated learning systems. The team will also explore design options for side-channel-resistant accelerator architectures for deep learning.
Technical University of Darmstadt (TU Darmstadt)
Engineering Private AI Systems (EPAI), Cryptography and Privacy Engineering Group (ENCRYPTO)
Principal investigator: Thomas Schneider
For EPAI, the ENCRYPTO group at TU Darmstadt will develop basic technologies to build private AI systems, investigate their orchestration strategies to optimize efficiency and costs on a given network and compute infrastructure, and systematically validate these to allow for automatically selecting the most efficient solution for a specific usage scenario. As underlying technologies, the ENCRYPTO group will mix different building blocks from cryptography and hardware, including secure multi-party computation, hardware acceleration, and trusted execution environments.
Technical University of Darmstadt (TU Darmstadt) and University of Würzburg
Decentralized Trustworthy Federated Learning (TRUFFLE)
Principal investigators: Ahmad-Reza Sadeghi and Alexandra Dmitrienko
For TRUFFLE, TU Darmstadt and University of Würzburg will design a framework for FL that provides comprehensive security and privacy. The design will be resilient against various attacks, such as poisoning, and will incorporate privacy-enhancing technologies based on decentralized aggregators and advanced crypto-based primitives to address privacy requirements of FL. TRUFFLE will incorporate hardware-assisted security and trusted execution environments of varying capabilities.