Abstract: Deep learning has had a significant impact on both science and society due to its successful application of data-driven artificial intelligence. One of the key characteristics of deep learning is that its accuracy improves with an increase in the model size and the amount of training data. This feature has notably enhanced state-of-the-art learning architectures across various fields over the past decade. However, the lack of a solid mathematical and statistical foundation has restricted the development of deep learning to specific applications and has hindered its broader, high-confidence implementation. This foundational gap becomes particularly evident when deep learning is applied to statistical estimation and inference, especially with limited training sample sizes. To address this issue, we aim to develop a statistically principled framework and theory that can validate the application of deep learning and support the creation of interpretable models. Our approach is grounded in Bayesian statistical theory and methodology, as well as scalable computation. We will demonstrate our methods across a wide range of applications.