We develop a novel estimation and inference procedure for the common parameters in linear panel data regression models with nonparametric two-way specification of unobserved heterogeneity. Our procedure builds on three main components: First, we develop moment conditions for the common parameters that are Neyman orthogonal with respect to nonparametric component to reduce the effect of the first-step estimation of this. Second, we develop a novel two-step estimator of the nonparametric component, where the second step assumes that the nonparametric component is well-proxied by eigenfunctions estimated in the first step. Third, we develop a novel adjustment of the nonparametric estimator so the estimated eigenfunctions do not generate incidental parameter biases. Together, these ensure that the resulting estimator of the common parameters is root-NT-asymptotically normally distributed thereby allowing for valid inference on the linear parameters in the model using standard methods. A numerical study shows that the proposed estimators perform well in finite samples.