Learning is usually associated with neural networks. But non-neural systems can also accumulate incremental changes over time and thus respond better to future environments. We show how seemingly 'dumb' physical systems like DNA crystals and elastic materials can learn to recognize complex patterns in chemical or mechanical stimuli, much like a neural network. We outline the potential and limits of such 'mechanical intelligence' due to physically realizable learning dynamics.