This semester, our reading group will focus on Interpretability Theory.

For this week, we are reading Toy Models of Superposition, Sections 1, 2, 3(optional), 4, and 8.
https://transformer-circuits.pub/2022/toy_model/index.html
The paper studies how neural networks can represent more features than they have dimensions by storing multiple features in overlapping directions — a phenomenon called superposition. This has a number of implications. In particular, there is a trade off between the benefit of representing more concepts and interference between features. This seminar will start with a 10 minute summary lecture followed by group discussions.