1. Critical Assessment of Metagenome Interpretation: the second round of challenges
- Author
-
Fernando Meyer, Adrian Fritz, Zhi-Luo Deng, David Koslicki, Till Robin Lesker, Alexey Gurevich, Gary Robertson, Mohammed Alser, Dmitry Antipov, Francesco Beghini, Denis Bertrand, Jaqueline J. Brito, C. Titus Brown, Jan Buchmann, Aydin Buluç, Bo Chen, Rayan Chikhi, Philip T. L. C. Clausen, Alexandru Cristian, Piotr Wojciech Dabrowski, Aaron E. Darling, Rob Egan, Eleazar Eskin, Evangelos Georganas, Eugene Goltsman, Melissa A. Gray, Lars Hestbjerg Hansen, Steven Hofmeyr, Pingqin Huang, Luiz Irber, Huijue Jia, Tue Sparholt Jørgensen, Silas D. Kieser, Terje Klemetsen, Axel Kola, Mikhail Kolmogorov, Anton Korobeynikov, Jason Kwan, Nathan LaPierre, Claire Lemaitre, Chenhao Li, Antoine Limasset, Fabio Malcher-Miranda, Serghei Mangul, Vanessa R. Marcelino, Camille Marchet, Pierre Marijon, Dmitry Meleshko, Daniel R. Mende, Alessio Milanese, Niranjan Nagarajan, Jakob Nissen, Sergey Nurk, Leonid Oliker, Lucas Paoli, Pierre Peterlongo, Vitor C. Piro, Jacob S. Porter, Simon Rasmussen, Evan R. Rees, Knut Reinert, Bernhard Renard, Espen Mikal Robertsen, Gail L. Rosen, Hans-Joachim Ruscheweyh, Varuni Sarwal, Nicola Segata, Enrico Seiler, Lizhen Shi, Fengzhu Sun, Shinichi Sunagawa, Søren Johannes Sørensen, Ashleigh Thomas, Chengxuan Tong, Mirko Trajkovski, Julien Tremblay, Gherman Uritskiy, Riccardo Vicedomini, Zhengyang Wang, Ziye Wang, Zhong Wang, Andrew Warren, Nils Peder Willassen, Katherine Yelick, Ronghui You, Georg Zeller, Zhengqiao Zhao, Shanfeng Zhu, Jie Zhu, Ruben Garrido-Oter, Petra Gastmeier, Stephane Hacquard, Susanne Häußler, Ariane Khaledi, Friederike Maechler, Fantin Mesny, Simona Radutoiu, Paul Schulze-Lefert, Nathiana Smit, Till Strowig, Andreas Bremges, Alexander Sczyrba, Alice Carolyn McHardy, Braunschweig Integrated Centre of Systems Biology [Braunschweig] (BRICS), Technische Universität Braunschweig = Technical University of Braunschweig [Braunschweig]-Helmholtz Centre for Infection Research (HZI), Pennsylvania State University (Penn State), Penn State System, German Center for Infection Research - partner site Hannover-Braunschweig (DZIF), Saint Petersburg State University (SPBU), Department of Information Technology and Electrical Engineering [Zürich] (D-ITET), Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich] (ETH Zürich), Center for Algorithmic Biotechnology [Saint Petersburg], Institute of Translational Biomedicine [Saint-Petersburg], Saint Petersburg University (SPBU)-Saint Petersburg University (SPBU), Centre for Integrative Biology (CIBIO), University of Trento (CIBIO), University of Trento [Trento], Genome Institute of Singapore (GIS), University of Southern California (USC), University of California [Davis] (UC Davis), University of California (UC), Heinrich Heine Universität Düsseldorf = Heinrich Heine University [Düsseldorf], Lawrence Berkeley National Laboratory [Berkeley] (LBNL), Institut Pasteur [Paris] (IP), National Food Institute [Lyngby] (Forside), Drexel University, Robert Koch Institute [Berlin] (RKI), University of Technology Sydney (UTS), DOE Joint Genome Institute [Walnut Creek], University of California [Los Angeles] (UCLA), Intel Corporation [Santa Clara], Intel Corporation [USA], Department of Plant and Environmental Sciences [Frederiksberg], University of Copenhagen = Københavns Universitet (UCPH), Fudan University [Shanghai], Beijing Genomics Institute [Shenzhen] (BGI), Novo Nordisk Foundation Center for Biosustainability, Danmarks Tekniske Universitet = Technical University of Denmark (DTU), Université de Genève = University of Geneva (UNIGE), The Arctic University of Norway [Tromsø, Norway] (UiT), Charité - UniversitätsMedizin = Charité - University Hospital [Berlin], University of California [San Diego] (UC San Diego), University of Wisconsin-Madison, Scalable, Optimized and Parallel Algorithms for Genomics (GenScale), Inria Rennes – Bretagne Atlantique, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-GESTION DES DONNÉES ET DE LA CONNAISSANCE (IRISA-D7), Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Université de Rennes (UR)-Institut National des Sciences Appliquées - Rennes (INSA Rennes), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT)-Institut de Recherche en Informatique et Systèmes Aléatoires (IRISA), Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université de Bretagne Sud (UBS)-École normale supérieure - Rennes (ENS Rennes)-CentraleSupélec-Centre National de la Recherche Scientifique (CNRS)-IMT Atlantique (IMT Atlantique), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Centre National de la Recherche Scientifique (CNRS), Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189 (CRIStAL), Centrale Lille-Université de Lille-Centre National de la Recherche Scientifique (CNRS), Hasso Plattner Institute [Potsdam, Germany], The University of Sydney, Inria Lille - Nord Europe, Institut National de Recherche en Informatique et en Automatique (Inria), Amsterdam UMC - Amsterdam University Medical Center, Structural and Computational Biology, European Molecular Biology Laboratory [Heidelberg] (EMBL), DTU Electrical Engineering [Lyngby], National Institutes of Health [Bethesda] (NIH), Department of Biology [ETH Zürich] (D-BIOL), University of Virginia, IT University of Copenhagen (ITU), Freie Universität Berlin, University of Potsdam = Universität Potsdam, Florida International University [Miami] (FIU), National Research Council of Canada (NRC), Phase Genomics [Seattle], Max Planck Institute for Plant Breeding Research (MPIPZ), Helmholtz Centre for Infection Research (HZI), Aarhus University [Aarhus], Center for Biotechnology (CeBiTec), Universität Bielefeld = Bielefeld University, Open access funding provided by Helmholtz-Zentrum für Infektionsforschung GmbH (HZI), ANR-16-CONV-0005,INCEPTION,Institut Convergences pour l'étude de l'Emergence des Pathologies au Travers des Individus et des populatiONs(2016), ANR-19-P3IA-0001,PRAIRIE,PaRis Artificial Intelligence Research InstitutE(2019), and Medical Microbiology and Infection Prevention
- Subjects
06 Biological Sciences, 10 Technology, 11 Medical and Health Sciences ,Reproducibility of Results ,Cell Biology ,DNA ,Sequence Analysis, DNA ,Biochemistry ,Archaea ,Software ,Metagenome ,Metagenomics ,[INFO.INFO-BI]Computer Science [cs]/Bioinformatics [q-bio.QM] ,Molecular Biology ,Sequence Analysis ,Biotechnology ,Developmental Biology - Abstract
Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the Initiative for the Critical Assessment of Metagenome Interpretation (CAMI). The CAMI II challenge engaged the community to assess methods on realistic and complex datasets with long- and short-read sequences, created computationally from around 1,700 new and known genomes, as well as 600 new plasmids and viruses. Here we analyze 5,002 results by 76 program versions. Substantial improvements were seen in assembly, some due to long-read data. Related strains still were challenging for assembly and genome recovery through binning, as was assembly quality for the latter. Profilers markedly matured, with taxon profilers and binners excelling at higher bacterial ranks, but underperforming for viruses and Archaea. Clinical pathogen detection results revealed a need to improve reproducibility. Runtime and memory usage analyses identified efficient programs, including top performers with other metrics. The results identify challenges and guide researchers in selecting methods for analyses., Nature Methods, 19 (8), ISSN:1548-7105, ISSN:1548-7091
- Published
- 2022
- Full Text
- View/download PDF