Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization

This paper focuses on the challenges related to gradient aggregation in online distributed stochastic optimization within directed graphs, where tracking time-varying optima is a critical issue. By making use of the Projection Onto Convex Sets (POCS) algorithm, the gradient information is extrapo-lated, thus increasing the efficient use of gradient information. Tested on datasets with time-varying distributions, the optimizers employing our method outperform the traditional ones with improvements of 9% and 15.5% on the CLEAR-10 and CLEAR-100 datasets, respectively. Additionally, the robustness of the method has been demonstrated across different graph network scales, highlighting its adaptability to a broad range of distributed environments.