Virtual screening of commercial make-on-demand chemical libraries is a promising strategy for rapid, low-cost drug discovery. However, due to the uncertain predictive accuracy, it is not clear how to best integrate docking into discovery campaigns, an instance of a general problem for applying complex prediction methods. To address this challenge, I will describe how we designed a Bayesian optimal experiment to estimate the hit-rate as a function of predicted free energy of binding by carefully selecting ~500 compounds test in an in vitro binding assay. Using this an example, I will then describe a novel statistical and computational framework for efficiently computing Bayesian optimal designs. The core idea is to use stochastic gradient descent to simultaneously optimize the parameters of variational bounds of the expected information gain and the experimental degrees of freedom. Through implementing this in Pyro a probabilistic programming language built on PyTorch, this method can scale to designing highly informative experiments to calibrate a wide range of predictive models.