Joint Estimation of Multiple Conditional Gaussian Graphical Models

Feihu Huang, Songcan Chen, Sheng-Jun Huang

Abstract: In this paper, we propose a joint conditional graphical Lasso to learn multiple conditional Gaussian graphical models, also known as Gaussian conditional random fields, with some similar structures. Our model builds on the maximum likelihood method with the convex sparse group Lasso penalty. Moreover, our model is able to model multiple multivariate linear regressions with unknown noise covariances via a convex formulation. In addition, we develop an efficient approximated Newton’s method for optimizing our model. Theoretically, we establish the asymptotic properties of our model on consistency and sparsistency under the high-dimensional settings. Finally, extensive numerical results on simulations and real data sets demonstrate that our method outperforms the compared methods on structure recovery and structured output prediction. To the best of our knowledge, the joint learning of multiple multivariate regressions with unknown covariance is first studied.