Abstract:



Decomposition models for understanding mean and covariance structures from high-dimensional data have attracted a lot of attention in recent years. This thesis visits selected machine learning problems with applications in topic modeling, neuroimaging, and experimental designs and tackles challenges in these applications by incorporating decomposable structures.

The first part of the thesis looks into the statistical learning problems for the applications with decomposable mean structures, namely topic modeling and multi-spectral imaging. The goal of topic modeling and multi-spectral unmixing is to decompose the spectrum for each document (or pixel) in the corpus (or the image of a scene) to find latent topics (or spectrums of materials present in multi-spectral images). In topic modeling applications, the number of latent variables is a lot less than the ambient dimension. This allows us to estimate the topic simplex with the geometric approach by minimizing the volume of the topic polytope. In our second application, we aim to trace neurons present in multi-spectral images, called Brainbow images, which capture individual neurons in the brain and allow researchers to distinguish different neurons based on unique com binations of fluorescent colors. Brainbow images, however, have an over-defined problem as the number of unique neuron color combinations is greater than the number of spectral channels. Thus, we reformulate the neuron tracing problem as a hidden Markov model with underlying neuronal processes as latent variables to decompose the observed Brainbow images into individual neurons.

The second part of the thesis studies the decomposition of covariance models for tensor-variate data to introduce a scalable and interpretable structure. In the tensor-variate analysis, the observed data often exhibit spatio-temporal structure, and it is desirable to simultaneously learn partial cor relation for each mode of the tensor data. However, estimating the unstructured covariance model for tensor-variate data scales quadratically in terms of the product of all dimensions of the tensor. Instead, we introduce a Kronecker sum model for the square root factor of the precision matrix. This model assumption results in a decomposable covariance matrix motivated by a well-known Sylvester equation.

For the last part of the thesis, we visit the linear contextual bandit problem with missing values to understand the effect of missing probabilities on the cumulative regret, showing that the regret degrades due to missingness by at most the square of minimum sampling probability. By separating the missing values from the context vectors in the covariance model, we can estimate the linear parameter over time without explicitly imputing the missing values. Our method is ap plied to the experimental design for collecting gene expression data by sequentially selecting class discriminating DNA probes.