I will describe a construction of algebras of observables associated with local subregions in quantum gravity in the small G_N limit. This algebra consists of operators dressed to a semiclassical observer degree of freedom which serves as an anchor defining the subregion. I will argue that properly implementing the gravitational constraints on this algebra results in a type II von Neumann algebra, which possesses a well-defined notion of entropy. Up to a state-independent constant, this entropy agrees with the UV-finite generalized entropy of the subregion, consisting of a Bekenstein-Hawking area term and a bulk entropy term. This gives an algebraic explanation for the finiteness of the generalized entropy, and provides a number of tools for investigating aspects of semiclassical gravitational entropy, including the generalized second law, the quantum focusing conjecture, and the quantum extremal surface prescription in holography.