Aligning before Aggregating: Enabling Cross-domain Federated Learning via Consistent Feature Extraction

Federated learning (FL) is an emerging machine learning paradigm where multiple distributed clients collaboratively train a model without centrally collecting their raw data. In FL setting, it is a common case that the data on local clients come from different domains, e.g., photos taken by different mobile phones can vary in intensity and contrast due to the difference of imaging parameters. In such a cross-domain case, features extracted from data of different clients deviate from each other in the feature space, leading to the so-called feature shift. The feature shift can reduce the discrimination of features and degrade the performance of the learned model. However, most existing FL methods are not particularly designed for cross-domain setting. In this paper, we propose a novel cross-domain FL method, named AlignFed. In AlignFed, the model on each client is separated to a personalized feature extractor and a shared classifier. The former extracts consistent features among clients by aligning features of different clients to some specific points in the feature space. The latter aggregates the knowledge across clients over the consistent feature space, which can mitigate the performance degradation caused by the feature shift in cross-domain FL. We conduct experiments on common-used multi-domain datasets, including Digits-Five, Office-Caltech10, and DomainNet. The experimental results demonstrate that AlignFed can outperform the state-of-art FL methods.