Search

The authors acknowledge the economical support from MICINN to the Instruct Image Processing Center (I2PC) as part of the Spanish participation in Instruct-ERIC, the European Strategic Infrastructure Project (ESFRI) in the area of Structural Biology and Grant PID2019- 104757RB-I00 funded by MCIN/AEI/ 10.13039/501100011033/ and “ERDF A way of making Europe”, by the “European Union”, the support of the French National Research Agency - ANR (ANR-20-CE11-0020-03 and ANR-19-CE11-0008-01 to S.J.). The project that gave rise to these results received the support of a fellowship from the “la Caixa’’ Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660021. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. European Union (EU) and Horizon 2020 through grant: HighResCells (ERC - 2018 - SyG, Proposal: 810057) EOSC Life (INFRAEOSC-04-2018, Proposal: 824087) (...), Jiménez de la Morena J, Conesa P, Fonseca YC, de Isidro-Gómez FP, Herreros D, Fernández-Giménez E, Strelak D, Moebel E, Buchholz TO, Jug F, Martinez-Sanchez A, Harastani M, Jonic S, Conesa JJ, Cuervo A, Losana P, Sánchez I, Iceta M, Del Cano L, Gragera M, Melero R, Sharov G, Castaño-Díez D, Koster A, Piccirillo JG, Vilas JL, Otón J, Marabini R, Sorzano COS, Carazo JM

Cryo-electron tomography (cryo-ET) is an imaging technique that allows three-dimensional visualization of macro-molecular assemblies under near-native conditions. Cryo-ET comes with a number of challenges, mainly low signal-to-noise and inability to obtain images from all angles. Computational methods are key to analyze cryo-electron tomograms. To promote innovation in computational methods, we generate a novel simulated dataset to benchmark different methods of localization and classification of biological macromolecules in tomograms. Our publicly available dataset contains ten tomographic reconstructions of simulated cell-like volumes. Each volume contains twelve different types of complexes, varying in size, function and structure. In this paper, we have evaluated seven different methods of finding and classifying proteins. Seven research groups present results obtained with learning-based methods and trained on the simulated dataset, as well as a baseline template matching (TM), a traditional method widely used in cryo-ET research. We show that learning-based approaches can achieve notably better localization and classification performance than TM. We also experimentally confirm that there is a negative relationship between particle size and performance for all methods., Eurographics Workshop on 3D Object Retrieval, Short Papers, 5, 17, Ilja Gubins, Marten L. Chaillet, Gijs van der Schot, M. Cristina Trueba, Remco C. Veltkamp, Friedrich Förster, Xiao Wang, Daisuke Kihara, Emmanuel Moebel, Nguyen P. Nguyen, Tommi White, Filiz Bunyak, Giorgos Papoulias, Stavros Gerolymatos, Evangelia I. Zacharaki, Konstantinos Moustakas, Xiangrui Zeng, Sinuo Liu, Min Xu, Yaoyu Wang, Cheng Chen, Xuefeng Cui, and Fa Zhang, CCS Concepts: Information systems --> Evaluation of retrieval results; Specialized information retrieval; Multimedia and multimodal retrieval; Retrieval models and ranking

Cryo-electron tomography (cryo-ET) is an imaging technique that allows us to three-dimensionally visualize both the structural details of macro-molecular assemblies under near-native conditions and its cellular context. Electrons strongly interact with biological samples, limiting electron dose. The latter limits the signal-to-noise ratio and hence resolution of an individual tomogram to about 50 (5 nm). Biological molecules can be obtained by averaging volumes, each depicting copies of the molecule, allowing for resolutions beyond 4 (0.4 nm). To this end, the ability to localize and classify components is crucial, but challenging due to the low signal-to-noise ratio. Computational innovation is key to mine biological information from cryo-electron tomography. To promote such innovation, we provide a novel simulated dataset to benchmark different methods of localization and classification of biological macromolecules in cryo-electron tomograms. Our publicly available dataset contains ten tomographic reconstructions of simulated cell-like volumes. Each volume contains twelve different types of complexes, varying in size, function and structure. In this paper, we have evaluated seven different methods of finding and classifying proteins. Six research groups present results obtained with learning-based methods and trained on the simulated dataset, as well as a baseline template matching, a traditional method widely used in cryo-ET research. We find that method performance correlates with particle size, especially noticeable for template matching which performance degrades rapidly as the size decreases. We learn that neural networks can achieve significantly better localization and classification performance, in particular convolutional networks with focus on high-resolution details such as those based on U-Net architecture.

Cryo-electron tomography (cryo-ET) allows one to visualize and study the 3D spatial distribution of macromolecules, in their native states and at nanometer resolution in single cells. While this label-free cryogenic imaging technology produces data containing rich structural information, automatic localization and identification of macromolecules are prone to noise and reconstruction artifacts, and to the presence of many molecular species in small areas. Hence, we present a computational procedure that uses artificial neural networks to accurately localize several macromolecular species in cellular cryo-electron tomograms. The DeepFinder algorithm leverages deep learning and outperforms the commonly-used template matching method on synthetic datasets. Meanwhile, DeepFinder is very fast when compared to template matching, and is better capable of localizing and identifying small macro-molecules than other competitive deep learning methods. On experimental cryo-ET data depicting ribosomes, the localization and structure resolution (determined through subtomogram averaging) results obtained with DeepFinder are consistent with those obtained by experts. The DeepFinder algorithm is able to imitate the analysis performed by experts, and is therefore a very promising algorithm to investigate efficiently the contents of cellular tomograms. Furthermore, we show that DeepFinder can be combined with a template matching procedure to localize the missing macromolecules not found by one or the other method. Application of this collaborative strategy allowed us to find additional 20.5% membrane-bound ribosomes that had been missed or discarded during manual template matching-assisted annotation.

Different imaging techniques allow us to study the organization of life at different scales. Cryo-electron tomography (cryo-ET) has the ability to three-dimensionally visualize the cellular architecture as well as the structural details of macro-molecular assemblies under near-native conditions. Due to beam sensitivity of biological samples, an inidividual tomogram has a maximal resolution of 5 nanometers. By averaging volumes, each depicting copies of the same type of a molecule, resolutions beyond 4 Å have been achieved. Key in this process is the ability to localize and classify the components of interest, which is challenging due to the low signal-to-noise ratio. Innovation in computational methods remains key to mine biological information from the tomograms. To promote such innovation, we organize this SHREC track and provide a simulated dataset with the goal of establishing a benchmark in localization and classification of biological particles in cryo-electron tomograms. The publicly available dataset contains ten reconstructed tomograms obtained from a simulated cell-like volume. Each volume contains twelve different types of proteins, varying in size and structure. Participants had access to 9 out of 10 of the cell-like ground-truth volumes for learning-based methods, and had to predict protein class and location in the test tomogram. Five groups submitted eight sets of results, using seven different methods. While our sample size gives only an anecdotal overview of current approaches in cryo-ET classification, we believe it shows trends and highlights interesting future work areas. The results show that learning-based approaches is the current trend in cryo-ET classification research and specifically end-to-end 3D learning-based approaches achieve the best performance., Eurographics Workshop on 3D Object Retrieval, SHREC Session 1, 49, 54, Ilja Gubins, Gijs van der Schot, Remco C. Veltkamp, Friedrich Förster, Xuefeng Du, Xiangrui Zeng, Zhenxi Zhu, Lufan Chang, Min Xu, Emmanuel Moebel, Antonio Martinez-Sanchez, Charles Kervrann, Tuan M. Lai, Xusi Han, Genki Terashi, Daisuke Kihara, Benjamin A. Himes, Xiaohua Wan, Jingrong Zhang, Shan Gao, Yu Hao, Zhilong Lv, Xiaohua Wan, Zhidong Yang, Zijun Ding, Xuefeng Cui, and Fa Zhang, CCS Concepts: Information systems --> Evaluation of retrieval results; Specialized information retrieval; Multimedia and multimodal retrieval; Retrieval models and ranking

Cryo electron tomography (cryo-ET) is an imaging technique capable of producing 3D views of biological specimens. This technology enables to capture large field of views of vitrified cells at nanometer resolution. These features allow to combine several scales of understanding of the cellular machinery, from the interactions between groups of proteins to their atomic structure. Cryo-ET therefore has the potential to act as a link between in vivo cell imaging and atomic resolution techniques.However, cryo-ET images suffer from a high amount of noise and imaging artifacts, and the interpretability of these images heavily depends on computational image analysis methods. Existing methods allow to identify large macromolecules such as ribosomes, but there is evidence that the detections are incomplete. In addition, these methods are limited when searched objects are smaller and have more structural variability.The purpose of this thesis is to propose new image analysis methods, in order to enable a more robust identification of macromolecules of interest. We propose two computational methods to achieve this goal. The first aims at reducing the noise and imaging artifacts, and operates by iteratively adding and removing artificial noise to the image. We provide both mathematical and experimental evidence that this concept allows to enhance signal in cryo-ET images. The second method builds on recent advances in machine learning to improve macromolecule localization. The method is based on a convolutional neural network, and we show how it can be adapted to achieve better detection rates than the current state-of- the-art.; La cryo-tomographie électronique (cryo-ET) est une technique d'imagerie capable de produire des vues 3D de spécimens biologiques. Cette technologie permet d’imager de larges portions de cellules vitrifiées à une résolution nanométrique. Elle permet de combiner plusieurs échelles de compréhension de la machinerie cellulaire, allant des interactions entre les groupes de protéines à leur structure atomique. La cryo-ET a donc le potentiel d'agir comme un lien entre l'imagerie cellulaire in vivo et les techniques atteignant la résolution atomique.Cependant, ces images sont corrompues par un niveau de bruit élevé et d'artefacts d'imagerie. Leur interprétabilité dépend fortement des méthodes de traitement d'image. Les méthodes computationelles existantes permettent actuellement d'identifier de larges macromolécules telles que les ribosomes, mais il est avéré que ces détections sont incomplètes. De plus, ces méthodes sont limitées lorsque les objets recherchés sont de très petite taille ou présentent une plus grande variabilité structurelle.L'objectif de cette thèse est de proposer de nouvelles méthodes d'analyse d'images, afin de permettre une identification plus robuste des macromolécules d'intérêt. Nous proposons deux méthodes computationelles pour atteindre cet objectif. La première vise à réduire le bruit et les artefacts d'imagerie, et fonctionne en ajoutant et en supprimant de façon itérative un bruit artificiel à l'image. Nous fournissons des preuves mathématiques et expérimentales de ce concept qui permet d'améliorer le signal dans les images de cryo-ET. La deuxième méthode s'appuie sur les progrès récents de l'apprentissage automatique et les méthodes convolutionelles pour améliorer la localisation des macromolécules. La méthode est basée sur un réseau neuronal convolutif et nous montrons comment l'adapter pour obtenir des taux de détection supérieur à l'état de l'art.

International audience; Cryo-electron tomography (cryo-ET) allows one to capture 3D images of cells in a close to native state, at sub-nanometer resolution. However, noise and artifact levels are such that heavy computational processing is needed to access the image content. We propose a deep learning framework to accurately and jointly localize multiple types and states of macromolecules in cellular cryo-electron tomograms. We compare this framework to the commonly-used template matching method on both synthetic and experimental data. On synthetic image data, we show that our framework is very fast and produces superior detection results. On experimental data, the detection results obtained by our method correspond to an overlap rate of 86% with the expert annotations. In addition, we show that our method can be combined to template matching procedures to reliably increase the number of expected detections. In our experiments, this strategy was able to find additional 24.3% membrane-bound ribosomes that were missed or discarded during manual annotation.

Different imaging techniques allow us to study the organization of life at different scales. Cryo-electron tomography (cryo-ET) has the ability to three-dimensionally visualize the cellular architecture as well as the structural details of macro-molecular assemblies under near-native conditions. Due to beam sensitivity of biological samples, an inidividual tomogram has a maximal resolution of 5 nanometers. By averaging volumes, each depicting copies of the same type of a molecule, resolutions beyond 4 Å have been achieved. Key in this process is the ability to localize and classify the components of interest, which is challenging due to the low signal-to-noise ratio. Innovation in computational methods remains key to mine biological information from the tomograms. To promote such innovation, we organize this SHREC track and provide a simulated dataset with the goal of establishing a benchmark in localization and classification of biological particles in cryo-electron tomograms. The publicly available dataset contains ten reconstructed tomograms obtained from a simulated cell-like volume. Each volume contains twelve different types of proteins, varying in size and structure. Participants had access to 9 out of 10 of the cell-like ground-truth volumes for learning-based methods, and had to predict protein class and location in the test tomogram. Five groups submitted eight sets of results, using seven different methods. While our sample size gives only an anecdotal overview of current approaches in cryo-ET classification, we believe it shows trends and highlights interesting future work areas. The results show that learning-based approaches is the current trend in cryo-ET classification research and specifically end-to-end 3D learning-based approaches achieve the best performance.

In this paper, we describe a statistical method to address an important issue in cryo-electron tomography image analysis: reduction of a high amount of noise and artifacts due to the presence of a missing wedge (MW) in the spectral domain. The method takes as an input a 3D tomo-gram derived from limited-angle tomography, and gives as an output a 3D denoised and artifact compensated volume. The artifact compensation is achieved by filling up the MW with meaningful information. To address this inverse problem, we compute a Minimum Mean Square Error (MMSE) estimator of the uncorrupted image. The underlying high-dimensional integral is computed by applying a dedicated Markov Chain Monte-Carlo (MCMC) sampling procedure based on the Metropolis-Hasting (MH) algorithm. The proposed computational method can be used to enhance visualization or as a pre-processing step for image analysis, including segmentation and classification of macromolecules. Results are presented for both synthetic data and real 3D cryo-electron images.

Cryo-electron tomography (cryo-ET) allows one to capture 3D images of cells in a close to native state, at sub-nanometer resolution. However, noise and artifact levels are such that heavy computational processing is needed to access the image content. In this paper, we propose a deep learning framework to accurately and jointly localize multiple types and states of macromolecules in cellular cryo-electron tomograms. We compare this framework to the commonly-used template matching method on both synthetic and experimental data. On synthetic image data, we show that our framework is very fast and produces superior detection results. On experimental data, the detection results obtained by our method correspond to an overlap rate of 86% with the expert annotations, and comparable resolution is achieved when applying subtomogram averaging. In addition, we show that our method can be combined to template matching procedures to reliably increase the number of expected detections. In our experiments, this strategy was able to find additional 20.5% membrane-bound ribosomes that were missed or discarded during manual annotation.

International audience; In this study, we have addressed two important issues in cryo electron tomography images: the lowsignal-to-noise ratio and the presence of a missing wedge (MW) of information in the spectral domain. Indeed,according to the Fourier slice theorem, limited angle tomography results into an incomplete sampling of theFourier domain. Therefore, the Fourier domain is separated into two regions: the known spectrum (KS) and theunknown spectrum, the latter having the shape of a missing wedge (see Figure). The proposed method tacklesboth issues jointly, by iteratively applying a denoising algorithm in order to fill up the MW, and proceeds asfollows:[1] Excitation step: Add noise into the MW[2] Denoising step: Apply a patch-based denoising algorithm[3] Repeat steps 1 and 2, by keeping KS constant through the iterationsThe excitation step is used to randomly initialize the coefficients of the MW, whereas the denoising step acts as aspatial regularization. The employed denoising algorithm, which exploits the self-similarity of the image, filtersout coefficient values which are dissimilar to KS, thereby keeping similar ones. By iterating these steps, we areable to diffuse the information contained in KS into the MW.An application example on experimental data can be seen on the Figure, which shows the data in both spectraland spatial domain. The data contains a spherical gold particle, deformed by MW induced artifacts: elongation ofthe object, side- and ray-artifacts. From the residue image it can be seen that noise and MW artifacts have beenreduced, while preserving the details of the image. Experiments are being performed to verify if particledetection and alignment are enhanced by using the method as a pre-processing step.

Actes électroniques; International audience

Background and ObjectivesCryo electron tomography visualizes native cells at nanometer resolution, but analysis is challenged by noise and artifacts. Recently, supervised deep learning methods have been applied to decipher the 3D spatial distribution of macromolecules. However, in order to discover unknown objects, unsupervised classification techniques are necessary. In this paper, we provide an overview of unsupervised deep learning techniques, discuss the challenges to analyze cryo-ET data, and provide a proof-of-concept on real data.MethodsWe propose an unsupervised sub-tomogram classification method based on transfer learning. We use a deep neural network to learn a clustering friendly representation able to capture 3D shapes in the presence of noise and artifacts. This representation is learned here from a synthetic data set.ResultsWe show that when applying k-means clustering given a learning-based representation, it becomes possible to satisfyingly classify real sub-tomograms according to structural similarity. It is worth noting that no manual annotation is used for performing classification.ConclusionsWe describe the advantages and limitations of our proof-of-concept and raise several perspectives for improving classification performance.