Videos

FedCBO: Reaching Group Consensus in Clustered Federated Learning and Robustness to Backdoor Adversarial Attacks.

May 10, 2024
Abstract
Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user with their own local data set, in a way that is sensitive to the users’ data privacy and to communication cost constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train models that are useful for each group, rather than training a single global model for all users. In the first part of this talk, I will present a novel solution to the problem of clustered federated learning that is inspired by ideas in consensus-based optimization (CBO). Our new CBO-type method is based on a system of interacting particles that is oblivious to group memberships. Our algorithm is accompanied by theoretical justification that is illustrated by real data experiments. I will then discuss an additional point of concern in federated learning: the vulnerability of federated learning protocols to “backdoor” adversarial attacks. This discussion will motivate the introduction of a modified, improved particle system with enhanced robustness properties that, at an abstract level, can be interpreted as a bi-level optimization algorithm based on interacting particle dynamics. The talk is based on joint works with Jose A. Carrillo, Sixu Li, and Yuhua Zhu; as well as with Sixu Li, Konstantin Riedl, and Yuhua Zhu.
Supplementary Materials