Contrary to most modern mathematics, the field of AI safety sits on less well-developed theoretical foundations, and as such can feel more sporadic and hard to orient around. I will try to describe how I currently think about AI safety, what kind of work is being done, and why I believe mathematicians can (and should!) help.