Large Covariance Matrix Estimation by Composite Minimization

The present thesis concerns large covariance matrix estimation via composite minimization under the assumption of low rank plus sparse structure. Existing methods like POET (Principal Orthogonal complEment Thresholding) perform estimation by extracting principal components and then applying a soft thresholding algorithm. In contrast, our method recovers the low rank plus sparse decomposition of the covariance matrix by least squares minimization under nuclear norm plus $l_1$ norm penalization. This non-smooth convex minimization procedure is based on semidefinite programming and subdifferential methods, resulting in two separable problems solved by a singular value thresholding plus soft thresholding algorithm. The most recent estimator in literature is called LOREC (Low Rank and sparsE Covariance estimator) and provides non-asymptotic error rates as well as identifiability conditions in the context of algebraic geometry. Our work shows that the unshrinkage of the estimated eigenvalues of the low rank component improves the performance of LOREC considerably. The same method also recovers covariance structures with very spiked latent eigenvalues like in the POET setting, thus overcoming the necessary condition $p\leq n$. In addition, it is proved that our method recovers structures with intermediate degrees of spikiness, obtaining a loss which is bounded accordingly. Then, an ad hoc model selection criterion which detects the optimal point in terms of composite penalty is proposed. Empirical results coming from a wide original simulation study where various low rank plus sparse settings are simulated according to different parameter values are described outlining in detail the improvements upon existing methods. Two real data-sets are finally explored highlighting the usefulness of our method in practical applications.