In analogy to the curve complex and its role in the geometry of mapping class groups of finite-type surfaces, a number of authors have defined graphs whose vertices are arcs or curves on a given infinite-type surface S, and on which the mapping class group Mod(S) acts by isometries. We show that for a broad class of such graphs, including the grand arc graph, the omnipresent arc graph, and all others defined comparably to Masur-Minsky, the asymptotic dimension is infinite. In particular, if one could construct a graph in this class admitting a Švarc-Milnor-type action of Mod(S), then Mod(S) would have infinite asymptotic dimension.