Comparing Preferences Across Actors

Abstract:
Achen’s (1978) famous critique of Miller and Stokes (1963) shows that correlations between the policy stances taken by legislators and the policy stances taken by constituents do not establish whether or not these policy stances are proximate to one another. In general, proximity cannot be established when the two measures are not on the same scale. The recent literature on joint scaling proposes a solution to this problem. Indeed, the proposed solution is general enough that it has been applied to a variety of contexts, comparing the political positions of everyone from interest groups (Bonica, 2013) to Twitter users (Barbera, 2015). We show that joint scaling works well in contexts where the underlying assumptions are correct. However, it fails in some of its most well-known applications. In particular, we show that methods for jointly scaling between media outlets and legislators, and between legislators and constituents, are problematic. Researchers who accept these methods for these datasets must also accept that the uncertainty associated with them precludes useful proximity comparisons across groups.