Rocky 2021

OP-36
Robustness and reproducibility of computational genomics tools

Presenting Author: Serghei Mangul, USC

Co-Author(s):
Pelin Icer Baykal, ETH Zurich

Abstract: Reproducibility and robustness of genomic tools are two important factors to assess the accuracy of bioinformatics analysis. Such assessment based on these criteria requires repetition of experiments across lab facilities which is usually costly and time-consuming and sometimes even impossible. In this study, we report the development of CompRep, a novel scalable method able to generate computational replicates by altering the properties of sequencing data. Computational replicates are created by randomly shuffling the order of reads and by taking the reverse complement of the reads. Despite that our method is not able to capture full variability across real technical replicates, our method is able to provide a robust low bound estimate of the reproducibility of genomic tools.

We analyzed two different groups of genomic tools: genomic read alignment tools and structural variant (SV) detection. We observed that for some genomic tools handling reverse complement data is much more challenging than handling randomly shuffled data. This analysis reflects substantial variability across different genomic tools.

The model that we propose will enable broad biomedical communities to easily access the robustness and reproducibility of genomic tools allowing choose genomics tools able to preserve consistent results across technical replicates. Furthermore, our method will enable routine robustness and reproducibility evaluation of newly published tools at scale at no cost.