We study the spectral theory of a family of random matrices that are variants on empirical covariance matrices, with entries that can have correlations through projections of the data onto some O(1)-many directions. These matrices arise naturally when looking at the empirical Hessians of many high-dimensional statistical tasks at different points in parameter space, to probe the local geometry of their loss landscapes. We prove limits for the bulk distribution and any outlier eigenvalues, in a way that only depends on the point in parameter space through finitely many “summary statistics” of the parameter. This allows us to probe the evolution of the Hessian as one moves through parameter space via some training dynamics like stochastic gradient descent, establishing interesting phenomena like splitting and emergence of outliers over the course of training in basic classification tasks. Based on joint work with G. Ben Arous, J. Huang, and A. Jagannath.