Abstract: Deep generative models, such as Variational Autoencoders (VAE) and diffusion probabilistic models, have transformed high-dimensional data modeling. However, these approaches often rely on variational approximations or computationally intensive ordinary differential equation (ODE) solvers, trading exact Bayesian inference for scalability. In this talk, I present the Bayesian Deep Noise Neural Network (B-DeepNoise), a framework originally developed for density regression that possesses inherent yet under-explored generative capabilities. Unlike standard Bayesian neural networks that place priors only on network weights, the B-DeepNoise framework injects stochastic noise into every hidden layer of a deep architecture. We show that this construction is mathematically equivalent to a deep hierarchical latent variable model, yielding rich conditional distributions through layer-wise noise propagation. By exploiting piecewise-linear activation functions, specifically ReLU function, we derive a closed-form Gibbs sampling algorithm that enables asymptotically exact posterior inference, avoiding the approximation errors commonly associated with variational methods. I will demonstrate how this framework unifies three closely related tasks: (1) uncertainty quantification in regression, (2) density regression for complex conditional distributions, and (3) extensions to generative modeling, where layer-wise noise injection enables flexible sample generation and data imputation. These results bridge flexible deep learning architectures with rigorous Bayesian inference and computational statistics, providing a principled approach to density learning and generative modeling.