Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization

Yifu Lin; Wenling Li

doi:10.1109/CCDC65474.2025.11091074

Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization

Yifu Lin
, Wenling Li

School of Automation Science and Electrical Engineering

Beihang University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper focuses on the challenges related to gradient aggregation in online distributed stochastic optimization within directed graphs, where tracking time-varying optima is a critical issue. By making use of the Projection Onto Convex Sets (POCS) algorithm, the gradient information is extrapo-lated, thus increasing the efficient use of gradient information. Tested on datasets with time-varying distributions, the optimizers employing our method outperform the traditional ones with improvements of 9% and 15.5% on the CLEAR-10 and CLEAR-100 datasets, respectively. Additionally, the robustness of the method has been demonstrated across different graph network scales, highlighting its adaptability to a broad range of distributed environments.

Original language	English
Title of host publication	Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1812-1817
Number of pages	6
ISBN (Electronic)	9798331510565
DOIs	https://doi.org/10.1109/CCDC65474.2025.11091074
State	Published - 2025
Event	37th Chinese Control and Decision Conference, CCDC 2025 - Xiamen, China Duration: 16 May 2025 → 19 May 2025

Publication series

Name	Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025

Conference

Conference	37th Chinese Control and Decision Conference, CCDC 2025
Country/Territory	China
City	Xiamen
Period	16/05/25 → 19/05/25

Keywords

Directed Graph
Gradient Extrapolation
Online Distributed Stochastic Optimization
Projection Onto Con-vex Sets

Access to Document

10.1109/CCDC65474.2025.11091074

Cite this

Lin, Y., & Li, W. (2025). Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization. In Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025 (pp. 1812-1817). (Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CCDC65474.2025.11091074

@inproceedings{75e8f3d92b684bcf8e0281f9a53019f1,

title = "Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization",

abstract = "This paper focuses on the challenges related to gradient aggregation in online distributed stochastic optimization within directed graphs, where tracking time-varying optima is a critical issue. By making use of the Projection Onto Convex Sets (POCS) algorithm, the gradient information is extrapo-lated, thus increasing the efficient use of gradient information. Tested on datasets with time-varying distributions, the optimizers employing our method outperform the traditional ones with improvements of 9\% and 15.5\% on the CLEAR-10 and CLEAR-100 datasets, respectively. Additionally, the robustness of the method has been demonstrated across different graph network scales, highlighting its adaptability to a broad range of distributed environments.",

keywords = "Directed Graph, Gradient Extrapolation, Online Distributed Stochastic Optimization, Projection Onto Con-vex Sets",

author = "Yifu Lin and Wenling Li",

note = "Publisher Copyright: {\textcopyright} 2025 IEEE.; 37th Chinese Control and Decision Conference, CCDC 2025 ; Conference date: 16-05-2025 Through 19-05-2025",

year = "2025",

doi = "10.1109/CCDC65474.2025.11091074",

language = "英语",

series = "Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1812--1817",

booktitle = "Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025",

address = "美国",

}

Lin, Y & Li, W 2025, Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization. in Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025. Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025, Institute of Electrical and Electronics Engineers Inc., pp. 1812-1817, 37th Chinese Control and Decision Conference, CCDC 2025, Xiamen, China, 16/05/25. https://doi.org/10.1109/CCDC65474.2025.11091074

Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization. / Lin, Yifu; Li, Wenling.
Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025. Institute of Electrical and Electronics Engineers Inc., 2025. p. 1812-1817 (Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization

AU - Lin, Yifu

AU - Li, Wenling

PY - 2025

Y1 - 2025

N2 - This paper focuses on the challenges related to gradient aggregation in online distributed stochastic optimization within directed graphs, where tracking time-varying optima is a critical issue. By making use of the Projection Onto Convex Sets (POCS) algorithm, the gradient information is extrapo-lated, thus increasing the efficient use of gradient information. Tested on datasets with time-varying distributions, the optimizers employing our method outperform the traditional ones with improvements of 9% and 15.5% on the CLEAR-10 and CLEAR-100 datasets, respectively. Additionally, the robustness of the method has been demonstrated across different graph network scales, highlighting its adaptability to a broad range of distributed environments.

AB - This paper focuses on the challenges related to gradient aggregation in online distributed stochastic optimization within directed graphs, where tracking time-varying optima is a critical issue. By making use of the Projection Onto Convex Sets (POCS) algorithm, the gradient information is extrapo-lated, thus increasing the efficient use of gradient information. Tested on datasets with time-varying distributions, the optimizers employing our method outperform the traditional ones with improvements of 9% and 15.5% on the CLEAR-10 and CLEAR-100 datasets, respectively. Additionally, the robustness of the method has been demonstrated across different graph network scales, highlighting its adaptability to a broad range of distributed environments.

KW - Directed Graph

KW - Gradient Extrapolation

KW - Online Distributed Stochastic Optimization

KW - Projection Onto Con-vex Sets

UR - https://www.scopus.com/pages/publications/105013966469

U2 - 10.1109/CCDC65474.2025.11091074

DO - 10.1109/CCDC65474.2025.11091074

M3 - 会议稿件

AN - SCOPUS:105013966469

T3 - Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025

SP - 1812

EP - 1817

BT - Proceedings of the 37th Chinese Control and Decision Conference, CCDC 2025

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 37th Chinese Control and Decision Conference, CCDC 2025

Y2 - 16 May 2025 through 19 May 2025

ER -

Extrapolated Gradient for Accelerating Online Distributed Stochastic Optimization

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this