The Variance Reduced Douglas-Rachford Splitting Method for Large-Scale Optimization Problems

This paper addresses large-scale composite optimization problems consisting of a finite-sum smooth (possibly nonconvex) loss function and a nonsmooth regularization term. By integrating the variance-reduction mechanism of Point-SAGA with the Douglas-Rachford (DR) splitting paradigm, we propose an efficient framework termed the Variance-Reduced Douglas-Rachford (VR-DR) method. Unlike existing stochastic schemes, VR-DR sequentially evaluates the proximal operators of a sampled loss component and the regularization term, consistent with the classical DR splitting framework. To enhance computational tractability, we further develop an inexact variant (iVR-DR), which allows subproblems to be solved approximately under a controlled inexactness criterion. Rigorous convergence analysis in expectation is provided for both VR-DR and iVR-DR. Specifically, we establish sublinear convergence rates with respect to the proximal gradient mapping when the regularization term is convex. In the fully nonconvex and nonsmooth regime, we prove the global convergence and establish convergence rates for the generated iterative sequences by leveraging the Kurdyka-Łojasiewicz framework. Numerical experiments on sparse logistic regression, sparse least squares, and sparse support vector machines demonstrate the superior efficiency of our methods compared to state-of-the-art solvers. Notably, while our theoretical analysis assumes smoothness, empirical results reveal that VR-DR remains highly effective for nonsmooth loss functions, significantly broadening its practical utility beyond its theoretical constraints.