Accurately benchmarking variant calling accuracy is critical for the continued improvement of human whole genome sequencing. In this work, we show that current variant calling evaluations are biased towards certain variant representations and may misrepresent the relative performance of different variant calling pipelines. We propose solutions, first exploring the affine gap parameter design space for complex variant representation and suggesting a standard. Next, we present our tool “vcfdist” and demonstrate the importance of enforcing local phasing for evaluation accuracy. We then introduce the notion of partial credit for mostly-correct calls and present an algorithm for clustering dependent variants. Lastly, we motivate using alignment distance metrics to supplement precision-recall curves for understanding variant calling performance. We evaluate the performance of 64 phased “Truth Challenge V2” submissions and show that vcfdist improves measured insertion and deletion performance consistency across variant representations from R^2= 0.97243 for baseline vcfeval to 0.99996 for vcfdist.

Tool link: https://github.com/timd1/vcfdist

This presentation will be held in 2036 Palmer Commons. There will also be a remote viewing option via Zoom.