Latent variable models play an increasingly crucial role in modern statistics and machine learning for analyzing large-scale and complex-structured data, with wide-ranging applications across various scientific fields. For instance, in educational assessments, latent variable models capture unobservable traits, such as intelligence, personality, and attitude. In biology and genomics, latent variable models uncover underlying genetic factors, gene expression patterns, or hidden biological mechanisms. By inferring the latent variables, researchers gain a deeper understanding of the mechanisms governing the observed data. Despite wide applications of latent variable models, the large scale and complex structures of data, and the involvement of covariates pose numerous challenges for its identifiability, estimation, and inference. This dissertation focuses on developing identifiability theory, estimation approaches, and inference methodologies for interpretable latent variable models, addressing three important problems:
(I): The first part addresses the identifiability issues of latent class models with covariates. Despite that these models are widely used in various applications, the fundamental identifiability issue of the latent class models with covariates has not been fully addressed. To address this open identifiability issue, we establish conditions to ensure the global identifiability of the model parameters in both strict and generic sense. Moreover, our results extend to polytomous-response Cognitive Diagnosis Models (CDMs) with covariates, which generalizes the existing identifiability results for CDMs.
(II) The second part develops estimation and inference methods for generalized linear framework with latent confounders. Statistical inferences for high-dimensional regression models have been extensively studied for their wide applications ranging from genomics, neuroscience, to economics. However, in practice, there are often potential unmeasured confounders associated with both the response and covariates, which can lead to invalidity of standard debiasing methods. In this part, we focus on a generalized linear regression framework with hidden confounding and propose a debiasing approach to address this high-dimensional problem, by adjusting for the effects induced by the unmeasured confounders. We establish consistency and asymptotic normality for the proposed debiased estimator.
(III) The third part focuses on statistical inference for covariate-adjusted generalized factor models. In addition to understanding the latent factors, the covariate effects on responses controlling for latent factors is also of great scientific interest and has wide applications, such as evaluating the fairness of educational testing, where the covariate effect reflects whether a test question is biased toward certain individual characteristics (e.g. gender and race) taking into account their latent abilities. However, the large sample size, substantial covariate dimension, and great test length pose great challenges to developing efficient methods and drawing valid inferences. Moreover, to accommodate the commonly encountered discrete type of responses, nonlinear factor models are often assumed, bringing in further complexity to the problem. To address these challenges, we consider a covariate-adjusted generalized factor model and develop novel and interpretable conditions to address the identifiability issue. Based on the identifiability conditions, we propose a joint maximum likelihood estimation method and establish estimation consistency and asymptotic normality results for the covariate effects under a practical yet challenging asymptotic regime. Furthermore, we derive estimation and inference results for latent factors and the factor loadings.