A class of optimal estimators for the covariance operator in reproducing kernel Hilbert spaces

The covariance operator plays an important role in modern statistical methods and is critical for inference. It is most often estimated by the empirical covariance operator. In spite of its simple and appealing properties, however, this estimator can be improved by a class of shrinkage operators. In this paper, we study shrinkage estimation of the covariance operator in reproducing kernel Hilbert spaces. A data-driven shrinkage estimator enjoying desirable theoretical and computational properties is proposed. The procedure is easily implemented and its numerical performance is investigated through simulations. In finite samples, the estimator outperforms the empirical covariance operator, especially when the data dimension is much larger than the sample size. We also show that the rate of convergence in Hilbert–Schmidt norm is of the order n^−1∕2. Furthermore, we establish the minimax optimal rate of convergence over suitable classes of probability measures and demonstrate that these shrinkage operators are all minimax rate-optimal.