1. Critical Assessment of Metagenome Interpretation - the second round of challenges
- Author
-
Zho. Wang, Ariane Khaledi, Alice C. McHardy, Anton Korobeynikov, A. Cristian, Gherman Uritskiy, Huijue Jia, Philip Thomas Lanken Conradsen Clausen, Till Strowig, Denis Bertrand, N. Smit, Niranjan Nagarajan, Enrico Seiler, Adam G. Thomas, David Koslicki, Piotr Wojtek Dabrowski, Vitor C. Piro, Andreas Bremges, L. Oliker, Petra Gastmeier, Steven Hofmeyr, Zhe Wang, Jason C. Kwan, Alessio Milanese, Tue Sparholt Jørgensen, Mohammed Alser, J. S. Porter, Alexander Sczyrba, Georg Zeller, Bernhard Y. Renard, Chenhao Li, Riccardo Vicedomini, Chengxuan Tong, Andrew S. Warren, Jaqueline J. Brito, Alexey Gurevich, Axel Kola, C.T. Brown, Julien Tremblay, Shinichi Sunagawa, F. Maechler, G. Robertson, Jakob Nybo Nissen, Ruben Garrido-Oter, Rob Egan, Simon Rasmussen, Katherine Yelick, Fernando Meyer, Zhengqiao Zhao, Daniel R Mende, Shanfeng Zhu, Lizhen Shi, F. Malcher-Miranda, Fengzhu Sun, Zi. Wang, Lars Hestbjerg Hansen, J. Buchmann, S. D. Kieser, Jie Zhu, E. M. Robertsen, Fantin Mesny, Sergey Nurk, Pierre Marijon, Dmitry Meleshko, Gail L. Rosen, Nicola Segata, Nathan LaPierre, Eugene Goltsman, Varuni Sarwal, Mirko Trajkovski, Dmitry Antipov, P. Huang, Vanesa R. Marcelino, Francesco Beghini, Antoine Limasset, Rayan Chikhi, Eleazar Eskin, M. A. Gray, Camille Marchet, Lucas Paoli, Adrian Fritz, Evangelos Georganas, Zhi-Luo Deng, T. Klemetsen, Hans-Joachim Ruscheweyh, Evan R. Rees, S. Häußler, Simona Radutoiu, Stéphane Hacquard, Paul Schulze-Lefert, Mikhail Kolmogorov, N. P. Willassen, Pierre Peterlongo, Knut Reinert, Claire Lemaitre, Ronghui You, Søren J. Sørensen, Aydin Buluc, Luiz Irber, Serghei Mangul, B. Chen, and Aaron E. Darling
- Subjects
Taxon ,Pathogen detection ,Microbial Genomes ,Metagenomics ,Computer science ,Critical assessment ,Computational biology ,Taxonomic rank ,Genome - Abstract
Evaluating metagenomic software is key for optimizing metagenome interpretation and focus of the community-driven initiative for the Critical Assessment of Metagenome Interpretation (CAMI). In its second challenge, CAMI engaged the community to assess their methods on realistic and complex metagenomic datasets with long and short reads, created from ∼1,700 novel and known microbial genomes, as well as ∼600 novel plasmids and viruses. Altogether 5,002 results by 76 program versions were analyzed, representing a 22x increase in results.Substantial improvements were seen in metagenome assembly, some due to using long-read data. The presence of related strains still was challenging for assembly and genome binning, as was assembly quality for the latter. Taxon profilers demonstrated a marked maturation, with taxon profilers and binners excelling at higher bacterial taxonomic ranks, but underperforming for viruses and archaea. Assessment of clinical pathogen detection techniques revealed a need to improve reproducibility. Analysis of program runtimes and memory usage identified highly efficient programs, including some top performers with other metrics. The CAMI II results identify current challenges, but also guide researchers in selecting methods for specific analyses.
- Published
- 2021