Search

Publisher Copyright: Copyright © 2021 Maarala et al. Computational pan-genomics utilizes information from multiple individual genomes in largescale comparative analysis. Genetic variation between case-controls, ethnic groups, or species can be discovered thoroughly using pan-genomes of such subpopulations. Wholegenome sequencing (WGS) data volumes are growing rapidly, making genomic data compression and indexing methods very important. Despite current space-efficient repetitive sequence compression and indexing methods, the deployed compression methods are often sequential, computationally time-consuming, and do not provide efficient sequence alignment performance on vast collections of genomes such as pan-genomes. For performing rapid analytics with the ever-growing genomics data, data compression and indexing methods have to exploit distributed and parallel computing more efficiently. Instead of strict genome data compression methods, we will focus on the efficient construction of a compressed index for pan-genomes. Compressed hybrid-index enables fast sequence alignments to several genomes at once while shrinking the index size significantly compared to traditional indexes. We propose a scalable distributed compressed hybrid-indexing method for large genomic data sets enabling pan-genome-based sequence search and read alignment capabilities. We show the scalability of our tool, DHPGIndex, by executing experiments in a distributed Apache Spark-based computing cluster comprising 448 cores distributed over 26 nodes. The experiments have been performed both with human and bacterial genomes. DHPGIndex built a BLAST index for n = 250 human pan-genome with an 870:1 compression ratio (CR) in 342 minutes and a Bowtie2 index with 157:1 CR in 397 minutes. For n = 1,000 human pan-genome, the BLAST index was built in 1520 minutes with 532:1 CR and the Bowtie2 index in 1938 minutes with 76:1 CR. Bowtie2 aligned 14.6 GB of paired-end reads to the compressed (n = 1,000) index in 31.7 minutes on a single node. Compressing n = 13,375,031 (488 GB) GenBank database to BLAST index resulted in CR of 62:1 in 575 minutes. BLASTing 189,864 Crispr-Cas9 gRNA target sequences (23 MB in total) to the compressed index of human pan-genome (n = 1,000) finished in 45 minutes on a single node. 30 MB mixed bacterial sequences were (n = 599) were blasted to the compressed index of 488 GB GenBank database (n = 13,375,031) in 26 minutes on 25 nodes. 78 MB mixed sequences (n = 4,167) were blasted to the compressed index of 18 GB E. coli sequence database (n = 745,409) in 5.4 minutes on a single node

Next-generation sequencing (NGS), in particular Illumina sequencing, is the current stateof- the-art DNA sequencing technology. However, when it comes to time-critical analysis, Illumina sequencing lacks sufficiently short turnaround times due to the sequential paradigm of data acquisition and analysis. For clinical application and infectious disease outbreaks, a significant reduction of time needed from sample arrival to analysis outcome is crucial to optimally treat patients and to prevent further spread of disease. At the same time, nucleotidelevel analysis is required to enable (sub-)species level classification and determination of organism-specific properties such as, for example, antimicrobial resistances. To accelerate the generation of NGS analysis results, the real-time read aligner HiLive was developed that performs read alignment while sequencing. Still, HiLive delivers results only at the end of the sequencing process and lacks sufficient resolution and scalability. In this thesis, a novel real-time alignment algorithm is introduced that was implemented in HiLive2. Unlike its predecessor, HiLive2 provides results at any desired stage of sequencing at full nucleotide-level resolution. The novel approach is based on an FM-index and is more scalable with respect to reference database size and sample size. HiLive2 enables high-quality downstream analysis as shown by performing variant calling based on realtime alignments of human sequencing data. Further, PathoLive is presented, a pipeline for real-time pathogen identification from metagenomic datasets. Based on the output of HiLive2, PathoLive performs a weighted ranking of identified species. Thereby, sequences that typically do not occur in samples from non-infected human individuals are assumed to be of high clinical significance and therefore highlighted in the results. PathoLive also provides an intuitive and interactive visualization that significantly facilitates the interpretation of results. In a case study of a real-world sample from Sudan, PathoLive enables the correct identification of Crimean–Congo hemorrhagic fever virus based on only a few dozen related reads. Besides analytical challenges, samples from human individuals are problematic with respect to data protection as reads from a human host can be used for the identification of the patient. To address this issue, PriLive was developed that enables the irrevocable removal of human sequences from Illumina sequencing data during the ongoing sequencing process. This enables a much higher level of data protection than conventional post hoc host removal approaches as the human sequences are at no time available in full length., ‘Next-Generation Sequencing’, im Speziellen die Illumina Sequenzierung, ist die derzeit meistgenutzte DNA-Sequenziertechnologie. Jedoch sind für zeitkritische Analysen aufgrund des sequentiellen Paradigmas der Datenerzeugung und -analyse die Durchlaufzeiten zu hoch. In der klinischen Anwendung und bei Ausbrüchen von Infektionskrankheiten ist es entscheidend, die Zeit vom Probeneingang zum Analyseergebnis zu verkürzen um Patienten optimal zu behandeln und einer weitere Krankheitsausbreitung zu verhindern. Gleichzeitig ist eine Analyse auf Nukleotidebene erforderlich um eine Spezies-Level-Klassifizierung und die Bestimmung spezifischer Eigenschaften, wie z.B. antimikrobiellen Resistenzen, zu ermöglichen. Um eine frühere Verfügbarkeit von Analyse-Ergebnissen zu erreichen wurde die Echtzeit-Alignierungssoftware HiLive entwickelt, welche DNA-Sequenzen während der Sequenzierung aligniert. Jedoch lieferte HiLive die Ergebnisse bislang nur am Ende eines Sequenzierlaufs und hatte keine ausreichende Auflösung und Skalierbarkeit. In dieser Arbeit präsentiere ich einen neuen Echtzeit-Alignierungsalgorithmus, der in HiLive2 implementiert wurde. HiLive2 basiert auf dem FM-index, kann zu jedem Zeitpunkt der Sequenzierung Ergebnisse liefern und erreicht eine höhere Skalierbarkeit der Größe von Referenzdatenbank und Datensatz. Durch die Detektion von Varianten basierend auf den Echtzeit-Alignierungen von humanen Sequenzierdaten zeige ich, dass HiLive2 qualitativ hochwertige Folgeanalysen ermöglicht. Außerdem stelle ich PathoLive vor, eine Pipeline zur Echtzeit-Identifizierung von Krankheitserregern aus metagenomischen Datensätzen. Basierend auf den Ergebnissen von HiLive2 führt PathoLive eine gewichtete Einstufung der identifizierten Organismen durch. Dabei werden Sequenzen, die auch in Proben von gesunden Menschen vorkommen, in den Ergebnissen weniger stark berücksichtigt. PathoLive bietet eine intuitive und interaktive Visualisierung, welche die Interpretation der Ergebnisse erleichtert. Ich zeige, dass PathoLive basierend auf nur wenigen Dutzend Sequenzen die Identifizierung des Krim-Kongo-Hämorrhagisches-Fieber-Virus in einer Probe aus dem Sudan ermöglicht. Neben den analytischen Herausforderungen sind Patientenproben im Hinblick auf den Datenschutz problematisch, da die Daten des humanen Wirts zur Identifizierung des Patienten verwendet werden könnten. Für diese Problematik präsentiere ich PriLive, welches noch während des Sequenzierlaufs das Entfernen humaner Sequenzen aus den Rohdaten ermöglicht. Hierdurch kann ein deutlich höheres Datenschutzniveau erreicht werden als mit herkömmlichen post hoc Ansätzen, da die humanen Sequenzen auch während des Sequenzierungsprozesses zu keinem Zeitpunkt in voller Länge vorliegen.

Background Cold temperatures and their alleviation affect many plant traits including the abundance of protein coding gene transcripts. Transcript level changes that occur in response to cold temperatures and their alleviation are shared or vary across genotypes. In this study we identify individual transcripts and groups of functionally related transcripts that consistently respond to cold and its alleviation. Genes that respond differently to temperature changes across genotypes may have limited functional importance. We investigate if these genes share functions, and if their genotype-specific gene expression levels change in magnitude or rank across temperatures. Results We estimate transcript abundances from over 22,000 genes in two unrelated Zea mays inbred lines during and after cold temperature exposure. Genotype and temperature contribute to many genes’ abundances. Past cold exposure affects many fewer genes. Genes up-regulated in cold encode many cytokinin glucoside biosynthesis enzymes, transcription factors, signalling molecules, and proteins involved in diverse environmental responses. After cold exposure, protease inhibitors and cuticular wax genes are newly up-regulated, and environmentally responsive genes continue to be up-regulated. Genes down-regulated in response to cold include many photosynthesis, translation, and DNA replication associated genes. After cold exposure, DNA replication and translation genes are still preferentially downregulated. Lignin and suberin biosynthesis are newly down-regulated. DNA replication, reactive oxygen species response, and anthocyanin biosynthesis genes have strong, genotype-specific temperature responses. The ranks of genotypes’ transcript abundances often change across temperatures. Conclusions We report a large, core transcriptome response to cold and the alleviation of cold. In cold, many of the core suite of genes are up or downregulated to control plant growth and photosynthesis and limit cellular damage. In recovery, core responses are in part to prepare for future stress. Functionally related genes are consistently and greatly up-regulated in a single genotype in response to cold or its alleviation, suggesting positive selection has driven genotype-specific temperature responses in maize. Electronic supplementary material The online version of this article (10.1186/s12864-018-5134-7) contains supplementary material, which is available to authorized users.

Most current approaches for analyzing metagenomic data rely on comparisons to reference genomes, but the microbial diversity of many environments extends far beyond what is covered by reference databases. De novo segregation of complex metagenomic data into specific biological entities, such as particular bacterial strains or viruses, remains a largely unsolved problem. Here we present a method, based on binning co-abundant genes across a series of metagenomic samples, that enables comprehensive discovery of new microbial organisms, viruses and co-inherited genetic entities and aids assembly of microbial genomes without the need for reference sequences. We demonstrate the method on data from 396 human gut microbiome samples and identify 7,381 co-abundance gene groups (CAGs), including 741 metagenomic species (MGS). We use these to assemble 238 high-quality microbial genomes and identify affiliations between MGS and hundreds of viruses or genetic entities. Our method provides the means for comprehensive profiling of the diversity within complex metagenomic samples.

[Abstract]: The growth of next-generation sequencing (NGS) datasets poses a challenge to the alignment of reads to reference genomes in terms of alignment quality and execution speed. Some available aligners have been shown to obtain high quality mappings at the expense of long execution times. Finding fast yet accurate software solutions is of high importance to research, since availability and size of NGS datasets continue to increase. In this work we present an efficient parallelization approach for NGS short-read alignment on multi-core clusters. Our approach takes advantage of a distributed shared memory programming model based on the new UPC++ language. Experimental results using the CUSHAW3 aligner show that our implementation based on dynamic scheduling obtains good scalability on multi-core clusters. Through our evaluation, we are able to complete the single-end and paired-end alignments of 246 million reads of length 150 base-pairs in 11.54 and 16.64 minutes, respectively, using 32 nodes with four AMD Opteron 6272 16-core CPUs per node. In contrast, the multi-threaded original tool needs 2.77 and 5.54 hours to perform the same alignments on the 64 cores of one node. The source code of our parallel implementation is publicly available at the CUSHAW3 homepage (http://cushaw3.sourceforge.net). [Resumen]: El crecimiento de los conjuntos de datos de "secuenciamiento de próxima generación" (NGS por sus siglas en inglés) es un reto respecto a la calidad y a la velocidad de alineamientos de secuencias a genomas de referencia. Algunos alineadores disponibles obtienen mapeados de alta calidad a expensas de largos tiempos de ejecución. Desarrollar software rápido y preciso es muy importante para la investigación, ya que la disponibilidad y tamaño de los conjuntos NGS continua creciendo. En este trabajo presentamos una paralelización eficiente para el alineamiento de secuencias cortas de NGS en sistemas con nodos de múltiples núcleos de computación. Nuestra aproximación se aprovecha de un modelo de programación distribuida-compartida basado en el nuevo lenguaje UPC++. Los resultados experimentales usando el alineador CUSHAW3 muestran que nuestra implementación basada en reparto dinámico de trabajo obtiene buena escalabilidad. En nuestra evaluación somos capaces de completar alineamientos sencillos y en parejas de 246 millones de secuencias de longitud 150 en 11.54 y 16.64 minutos, respectivamente, usando 32 nodos con cuatro AMD Opteron 6272 y 16 núcleos de CPU cada uno. Sin embargo, la herramienta multi-hilo original necesita 2.77 y 5.54 horas para completar los mismos alineamientos en los 64 núcleos de un nodo. El código fuente de nuestra implementación paralela está disponible públicamente en la web de CUSHAW3 (http://cushaw3.sourceforge.net). [Resumo]: O medre dos conxuntos de datos de "secuenzamento de próxima xeración" (NGS polas súas siglas en inglés) é un reto respecto á calidade e á velocidade dos aliñamentos de secuencias a xenomas de referencia. Algúns aliñadores disponibles obteñen mapeados de alta calidade a expensas de largos tempos de execución. Desenvolver software rápido e preciso é moi importante para a investigación, xa que a disponibilidade e tamaño dos conxuntos NGS continua a medrar. Neste traballo presentamos unha paralelización eficiente para o aliñamiento de secuencias cortas de NGS en sistemas con nodos de múltiples núcleos de computación. A nosa aproximación aproveitase dun modelo de programación distribuida-compartida basado na nova linguaxe UPC++. Os resultados experimentais que fan uso do aliñador CUSHAW3 mostran que a nosa implementación baseada en reparto dinámico de traballo obtén boa escalabilidade. Na nosa avaliación somos capaces de completar aliñamentos sinxelos e en parellas de 246 millóns de secuencias de lonxitude 150 en 11.54 e 16.64 minutos, respectivamente, empregando 32 nodos con catro AMD Opteron 6272 e 16 núcleos de CPU cada un. Sen embargo, a ferramenta multi-fío oxiginal necesita 2.77 e 5.54 horas para completar os mesmos aliñamientos nos 64 núcleos dun nodo. O código fonte da nosa implementación paralela está disponible públicamente na web de CUSHAW3 (http://cushaw3.sourceforge.net).

Background The alignment of short reads generated by next-generation sequencers to genomes is an important problem in many biomedical and bioinformatics applications. Although many proposed methods work very well on narrow ranges of read lengths, they tend to suffer in performance and alignment quality for reads outside of these ranges. Results We introduce RandAL, a novel method that aligns DNA sequences to reference genomes. Our approach utilizes two FM indices to facilitate efficient bidirectional searching, a pruning heuristic to speed up the computing of edit distances, and most importantly, a randomized strategy that enables effective estimation of key parameters. Extensive comparisons showed that RandAL outperformed popular aligners in most instances and was unique in its consistent and accurate performance over a wide range of read lengths and error rates. The software package is publicly available at https://github.com/namsyvo/RandAL. Conclusions RandAL promises to align effectively and accurately short reads that come from a variety of technologies with different read lengths and rates of sequencing error.

Motivation: The growth of next-generation sequencing (NGS) has not only dramatically accelerated the pace of research in the field of genomics, but it has also opened the door to personalized medicine and diagnostics. The resulting flood of data has led to the rapid development of large numbers of bioinformatic tools for data analysis, creating a challenging situation for researchers when choosing and configuring a variety of software for their analysis, and for other researchers trying to replicate their analysis. As NGS technology continues to expand from the research environment into clinical laboratories, the challenges associated with data analysis have the potential to slow the adoption of this technology. Results: Here we discuss the potential of virtual machines (VMs) to be used as a method for sharing entire installations of NGS software (bioinformatic ‘pipelines’). VMs are created by programs designed to allow multiple operating systems to co-exist on a single physical machine, and they can be made following the object-oriented paradigm of encapsulating data and methods together. This allows NGS data to be distributed within a VM, along with the pre-configured software for its analysis. Although VMs have historically suffered from poor performance relative to native operating systems, we present benchmarking results demonstrating that this reduced performance can now be minimized. We further discuss the many potential benefits of VMs as a solution for NGS analysis and describe several published examples. Lastly, we consider the benefits of VMs in facilitating the introduction of NGS technology into the clinical environment. Contact: brian.wilhelm@umontreal.ca

National audience; Mapping short reads against a reference genome is classically the first step of many next-generation sequencing data analyses, and it should be as accurate as possible. Because of the large number of reads to handle, numerous sophisticated algorithms have been developped in the last 3 years to tackle this problem. In this article, we first review the underlying algorithms used in most of the existing mapping tools, and then we compare the performance of nine of these tools on a well controled benchmark built for this purpose. We built a set of reads that exist in single or multiple copies in a reference genome and for which there is no mismatch, and a set of reads with three mismatches. We considered as reference genome both the human genome and a concatenation of all complete bacterial genomes. On each dataset, we quantified the capacity of the different tools to retrieve all the occurrences of the reads in the reference genome. Special attention was paid to reads uniquely reported and to reads with multiple hits.

This work was partially supported by FCT through the PIDDAC Program funds (INESC-ID multiannual funding) and under grant PEst-OE/EEI/LA0008/2011 (IT multiannual funding). In addition, it was also partially funded by projects HIVCONTROL (PTDC/EEA-CRO/100128/2008, S. Vinga, PI), TAGS (PTDC/EIA-EIA/112283/2009) and NEUROCLINOMICS (PTDC/EIA-EIA/111239/2009) from FCT (Portugal). Background: Chaos Game Representation (CGR) is an iterated function that bijectively maps discrete sequences into a continuous domain. As a result, discrete sequences can be object of statistical and topological analyses otherwise reserved to numerical systems. Characteristically, CGR coordinates of substrings sharing an L-long suffix will be located within 2(-L) distance of each other. In the two decades since its original proposal, CGR has been generalized beyond its original focus on genomic sequences and has been successfully applied to a wide range of problems in bioinformatics. This report explores the possibility that it can be further extended to approach algorithms that rely on discrete, graph-based representations. Results: The exploratory analysis described here consisted of selecting foundational string problems and refactoring them using CGR-based algorithms. We found that CGR can take the role of suffix trees and emulate sophisticated string algorithms, efficiently solving exact and approximate string matching problems such as finding all palindromes and tandem repeats, and matching with mismatches. The common feature of these problems is that they use longest common extension (LCE) queries as subtasks of their procedures, which we show to have a constant time solution with CGR. Additionally, we show that CGR can be used as a rolling hash function within the Rabin-Karp algorithm. Conclusions: The analysis of biological sequences relies on algorithmic foundations facing mounting challenges, both logistic (performance) and analytical (lack of unifying mathematical framework). CGR is found to provide the latter and to promise the former: graph-based data structures for sequence analysis operations are entailed by numerical-based data structures produced by CGR maps, providing a unifying analytical framework for a diversity of pattern matching problems. publishersversion published

International audience; Mapping short reads against a reference genome is classically the first step of many next-generation sequencing data analyses, and it should be as accurate as possible. Because of the large number of reads to handle, numerous sophisticated algorithms have been developped in the last 3 years to tackle this problem. In this article, we first review the underlying algorithms used in most of the existing mapping tools, and then we compare the performance of nine of these tools on a well controled benchmark built for this purpose. We built a set of reads that exist in single or multiple copies in a reference genome and for which there is no mismatch, and a set of reads with three mismatches. We considered as reference genome both the human genome and a concatenation of all complete bacterial genomes. On each dataset, we quantified the capacity of the different tools to retrieve all the occurrences of the reads in the reference genome. Special attention was paid to reads uniquely reported and to reads with multiple hits.

The combination of incessant advances in sequencing technology producing large amounts of data and innovative bioinformatics approaches, designed to cope with this data flood, has led to new interesting results in the life sciences. Given the magnitude of sequence data to be processed, many bioinformatics tools rely on efficient solutions to a variety of complex string problems. These solutions include fast heuristic algorithms and advanced data structures, generally referred to as index structures. Although the importance of index structures is generally known to the bioinformatics community, the design and potency of these data structures, as well as their properties and limitations, are less understood. Moreover, the last decade has seen a boom in the number of variant index structures featuring complex and diverse memory-time trade-offs. This article brings a comprehensive state-of-the-art overview of the most popular index structures and their recently developed variants. Their features, interrelationships, the trade-offs they impose, but also their practical limitations, are explained and compared.

Polyploidization, both ancient and recent, is frequent among plants. A ‘‘two-step theory’’ was proposed to explain the meso-triplication of the Brassica ‘‘A’’ genome: Brassica rapa. By accurately partitioning of this genome, we observed that genes in the less fractioned subgenome (LF) were dominantly expressed over the genes in more fractioned subgenomes (MFs: MF1 and MF2), while the genes in MF1 were slightly dominantly expressed over the genes in MF2. The results indicated that the dominantly expressed genes tended to be resistant against gene fractionation. By re-sequencing two B. rapa accessions: a vegetable turnip (VT117) and a Rapid Cycling line (L144), we found that genes in LF had less nonsynonymous or frameshift mutations than genes in MFs; however mutation rates were not significantly different between MF1 and MF2. The differences in gene expression patterns and on-going gene death among the three subgenomes suggest that ‘‘two-step’’ genome triplication and differential subgenome methylation played important roles in the genome evolution of B. rapa.

Background In vivo detection of protein-bound genomic regions can be achieved by combining chromatin-immunoprecipitation with next-generation sequencing technology (ChIP-seq). The large amount of sequence data produced by this method needs to be analyzed in a statistically proper and computationally efficient manner. The generation of high copy numbers of DNA fragments as an artifact of the PCR step in ChIP-seq is an important source of bias of this methodology. Results We present here an R package for the statistical analysis of ChIP-seq experiments. Taking the average size of DNA fragments subjected to sequencing into account, the software calculates single-nucleotide read-enrichment values. After normalization, sample and control are compared using a test based on the ratio test or the Poisson distribution. Test statistic thresholds to control the false discovery rate are obtained through random permutations. Computational efficiency is achieved by implementing the most time-consuming functions in C++ and integrating these in the R package. An analysis of simulated and experimental ChIP-seq data is presented to demonstrate the robustness of our method against PCR-artefacts and its adequate control of the error rate. Conclusions The software ChIP-seq Analysis in R (CSAR) enables fast and accurate detection of protein-bound genomic regions through the analysis of ChIP-seq experiments. Compared to existing methods, we found that our package shows greater robustness against PCR-artefacts and better control of the error rate.

Efforts such as International HapMap Project and 1000 Genomes Project resulted in a catalog of millions of single nucleotides and insertion/deletion (INDEL) variants of the human population. Viewed as a reference of existing variants, this resource commonly serves as a gold standard for studying and developing methods to detect genetic variants. Our analysis revealed that this reference contained thousands of INDELs that were constructed in a biased manner. This bias occurred at the level of aligning short reads to reference genomes to detect variants. The bias is caused by the existence of many theoretically optimal alignments between the reference genome and reads containing alternative alleles at those INDEL locations. We examined several popular aligners and showed that these aligners could be divided into groups whose alignments yielded INDELs that agreed strongly or disagreed strongly with reported INDELs. This finding suggests that the agreement or disagreement between the aligners’ called INDEL and the reported INDEL is merely a result of the arbitrary selection of one of the optimal alignments. The existence of bias in INDEL calling might have a serious influence in downstream analyses. As such, our finding suggests that this phenomenon should be further addressed.