FedHST Tackles Data Diversity Challenges in Federated Learning

Federated learning has become a popular way to train AI models across many devices. However, when data comes from different sources, it often varies a lot. This difference, called non-IID data, makes standard optimization methods struggle. Traditional server-side optimizers like FedAdam try to adjust updates over time, but they don't account for how much clients disagree with each other at any single moment. Under heavy data heterogeneity, these methods can overshoot or oscillate too much.

FedHST was created to fix this specific problem. It adds a new layer to the existing temporal adaptation by including an instantaneous spatial variance term. Think of it as an extra brake that kicks in when clients have conflicting views. This spatial damping term only activates when there is significant disagreement between clients. When everyone agrees, the term disappears and the system behaves like a standard temporal-only optimizer. The approach achieves a convergence rate of O(1/T) and shows how the damping coefficient controls the impact of data heterogeneity.

Testing on several image datasets revealed strong performance. On FashionMNIST, CIFAR-10, and CIFAR-100, FedHST matched or beat FedAdam by up to half a percentage point. It also outperformed FedDyn by five to twenty-eight percentage points across all datasets. Importantly, the computational cost stays close to FedAdam, only increasing by one to four percent. The ablation studies confirm that the method's edge grows stronger with more local training rounds and higher data heterogeneity. This makes FedHST a promising solution for real-world federated learning scenarios.

actions