In this talk, I will discuss some recent progress on understanding the dynamics of binary astrophysical objects (Kerr black holes or neutron stars) from the EFT perspective. We model the spinning objects by a higher spin field theory with non-minimal couplings to gravity. The order G^2 classical effective Hamiltonian of the binary system is then obtained by matching the gauge invariant scattering amplitudes at one-loop. Up to the fifth order in spin, we find that a unique result can be obtained by imposing a spin shift symmetry and improved high energy behavior in the scattering amplitudes of the binaries. We conjecture that this result corresponds to Kerr black holes. We also find that we need more Wilson coefficients than conventional world line approach to describe a neutron star. I will end the talk by discussing some preliminary attempts to resolve this descrepancy.