The strong interactions in Rydberg atoms make them an ideal system for the study of correlated many-body physics, both in the presence and absence of dissipation. Using such highly excited atomic states requires addressing challenges posed by the dense spectrum of Rydberg levels, the detrimental effects of spontaneous emission, and strong interactions. A full understanding of the scope and limitations of many Rydberg-based proposals requires including these effects, which typically cannot be described by a mean-field treatment due to correlations in the quantum coherent and dissipative processes. We study a driven, dissipative system of Rydberg atoms in a 3D optical lattice, and observe substantial deviation from single-particle excitation rates. We attribute these effects to unavoidable blackbody-induced transitions to nearby Rydberg states of opposite parity, which have large, resonant dipole-dipole interactions with the state of interest. Even at low densities of Rydberg atoms, uncontrolled production of atoms in other states significantly modifies the transition energies of the remaining atoms. These off-diagonal exchange interactions result in complex many-body states of the system and have implications for off-resonant Rydberg dressing proposals.