AI may change how science answers its hardest open questions, and even which questions it asks. Today's vast datasets hold physical information we do not yet know how to look for, and AI offers a way to search for the unexpected. To trust AI discoveries, we need to judge models by whether they obey the laws of physics. How convincing their outputs look is not enough. In this talk I will discuss the promise and challenges of cutting-
edge AI for science, and how to define and build physical realism into our models. I will present recent progress
on physical, probabilistic, and generative models for astronomy. I will also show where classical physics
models, machine learning, and agentic AI each reach their limits.