Search

Image denoising is probably the oldest and still one of the most active research topic in image processing. Many methodological concepts have been introduced in the past decades and have improved performances significantly in recent years, especially with the emergence of convolutional neural networks and supervised deep learning. In this paper, we propose a survey of guided tour of supervised and unsupervised learning methods for image denoising, classifying the main principles elaborated during this evolution, with a particular concern given to recent developments in supervised learning. It is conceived as a tutorial organizing in a comprehensive framework current approaches. We give insights on the rationales and limitations of the most performant methods in the literature, and we highlight the common features between many of them. Finally, we focus on on the normalization equivariance properties that is surprisingly not guaranteed with most of supervised methods. It is of paramount importance that intensity shifting or scaling applied to the input image results in a corresponding change in the denoiser output.

We propose a unified view of non-local methods for single-image denoising, for which BM3D is the most popular representative, that operate by gathering noisy patches together according to their similarities in order to process them collaboratively. Our general estimation framework is based on the minimization of the quadratic risk, which is approximated in two steps, and adapts to photon and electronic noises. Relying on unbiased risk estimation (URE) for the first step and on ``internal adaptation'', a concept borrowed from deep learning theory, for the second, we show that our approach enables to reinterpret and reconcile previous state-of-the-art non-local methods. Within this framework, we propose a novel denoiser called NL-Ridge that exploits linear combinations of patches. While conceptually simpler, we show that NL-Ridge can outperform well-established state-of-the-art single-image denoisers., Comment: arXiv admin note: text overlap with arXiv:2203.00570

In many information processing systems, it may be desirable to ensure that any change of the input, whether by shifting or scaling, results in a corresponding change in the system response. While deep neural networks are gradually replacing all traditional automatic processing methods, they surprisingly do not guarantee such normalization-equivariance (scale + shift) property, which can be detrimental in many applications. To address this issue, we propose a methodology for adapting existing neural networks so that normalization-equivariance holds by design. Our main claim is that not only ordinary convolutional layers, but also all activation functions, including the ReLU (rectified linear unit), which are applied element-wise to the pre-activated neurons, should be completely removed from neural networks and replaced by better conditioned alternatives. To this end, we introduce affine-constrained convolutions and channel-wise sort pooling layers as surrogates and show that these two architectural modifications do preserve normalization-equivariance without loss of performance. Experimental results in image denoising show that normalization-equivariant neural networks, in addition to their better conditioning, also provide much better generalization across noise levels.

Generators of space-time dynamics in bioimaging have become essential to build ground truth datasets for image processing algorithm evaluation such as biomolecule detectors and trackers, as well as to generate training datasets for deep learning algorithms. In this contribution, we leverage a stochastic model, called birth-death-move (BDM) point process, in order to generate joint dynamics of biomolecules in cells. This approach is very flexible and allows us to model a system of particles in motion, possibly in interaction, that can each possibly switch from a motion regime (e.g. Brownian) to another (e.g. a directed motion), along with the appearance over time of new trajectories and their death after some lifetime, all of these features possibly depending on the current spatial configuration of all existing particles. We explain how to specify all characteristics of a BDM model, with many practical examples that are relevant for bioimaging applications. Based on real fluorescence microscopy datasets, we finally calibrate our model to mimic the joint dynamics of Langerin and Rab11 proteins near the plasma membrane. We show that the resulting synthetic sequences exhibit comparable features as those observed in real microscopy image sequences.

In the past decade, deep neural networks have revolutionized image denoising in achieving significant accuracy improvements by learning on datasets composed of noisy/clean image pairs. However, this strategy is extremely dependent on training data quality, which is a well-established weakness. To alleviate the requirement to learn image priors externally, single-image (a.k.a., self-supervised or zero-shot) methods perform denoising solely based on the analysis of the input noisy image without external dictionary or training dataset. This work investigates the effectiveness of linear combinations of patches for denoising under this constraint. Although conceptually very simple, we show that linear combinations of patches are enough to achieve state-of-the-art performance. The proposed parametric approach relies on quadratic risk approximation via multiple pilot images to guide the estimation of the combination weights. Experiments on images corrupted artificially with Gaussian noise as well as on real-world noisy images demonstrate that our method is on par with the very best single-image denoisers, outperforming the recent neural network based techniques, while being much faster and fully interpretable.

We propose an extension of the well-known Space-Time Cube (STC) visualization technique in order to visualize time-varying 3D spatial data, taking advantage of the interaction capabilities of Virtual Reality (VR). The analysis of multidimensional time-varying datasets, which size grows as recording and simulating techniques advance, faces challenges on the representation and visualization of dense data, as well as on the study of temporal variations. First, we propose the Space-Time Hypercube (STH) as an abstraction for 3D temporal data, extended from the STC concept. Second, through the example of embryo development imaging dataset, we detail the construction and visualization of a STC based on a user-driven projection of the spatial and temporal information. This projection yields a 3D STC visualization, which can also encode additional numerical and categorical data. Additionally, we propose a set of tools allowing the user to filter and manipulate the 3D STC which benefits from the visualization, exploration and interaction possibilities offered by VR. Finally, we evaluated the proposed visualization method in the context of the visualization of spatio-temporal biological data. Several biology experts accompanied the application design to provide insight on how the STC visualization could be used to explore such data. We report a user study (n=12) using non-expert users performing a set of exploration and query tasks to evaluate the system., Comment: 21 1-column pages, 7 figures

Timelines are common visualizations to represent and manipulate temporal data, from historical events storytelling to animation authoring. However, timeline visualizations rarely consider spatio-temporal 3D data (e.g. mesh or volumetric models) directly, which are typically explored using 3D visualizers only displaying one time-step at a time. In this paper, leveraging the increased workspace and 3D interaction capabilities of virtual reality, we propose to use timelines for the visualization of 3D temporal data to support exploration and analysis. First, we propose a timeline design space for 3D temporal data extending the timeline design space proposed by Brehmer et al. The proposed design space adapts the scale, layout and representation dimensions to account for the depth dimension and how 3D temporal data can be partitioned and structured. In our approach, an additional dimension is introduced, the support, which further characterizes the 3D dimension of the visualization. To complement the design space and the interaction capabilities of VR systems, we discuss the interaction methods required for the efficient visualization of 3D timelines. Then, to evaluate the benefits of 3D timelines, we conducted a formal evaluation with two main objectives: comparing the proposed visualization with a traditional visualization method; exploring how users interact with different 3D timeline designs. Our results showed that time-related tasks can be achieved more comfortably using timelines, and more efficiently for specific tasks requiring the analysis of the surrounding temporal context. Though the comparison between the different timeline designs were inconclusive, participants reported a clear preference towards the timeline design that did not occupy the vertical space. Finally, we illustrate the use of the 3D timelines to a real use-case on the analysis of biological 3D temporal datasets., Comment: 11 pages, 6 figures

We propose a unified view of unsupervised non-local methods for image denoising that linearily combine noisy image patches. The best methods, established in different modeling and estimation frameworks, are two-step algorithms. Leveraging Stein's unbiased risk estimate (SURE) for the first step and the "internal adaptation", a concept borrowed from deep learning theory, for the second one, we show that our NL-Ridge approach enables to reconcile several patch aggregation methods for image denoising. In the second step, our closed-form aggregation weights are computed through multivariate Ridge regressions. Experiments on artificially noisy images demonstrate that NL-Ridge may outperform well established state-of-the-art unsupervised denoisers such as BM3D and NL-Bayes, as well as recent unsupervised deep learning methods, while being simpler conceptually.

This work tackles the issue of noise removal from images, focusing on the well-known DCT image denoising algorithm. The latter, stemming from signal processing, has been well studied over the years. Though very simple, it is still used in crucial parts of state-of-the-art "traditional" denoising algorithms such as BM3D. Since a few years however, deep convolutional neural networks (CNN) have outperformed their traditional counterparts, making signal processing methods less attractive. In this paper, we demonstrate that a DCT denoiser can be seen as a shallow CNN and thereby its original linear transform can be tuned through gradient descent in a supervised manner, improving considerably its performance. This gives birth to a fully interpretable CNN called DCT2net. To deal with remaining artifacts induced by DCT2net, an original hybrid solution between DCT and DCT2net is proposed combining the best that these two methods can offer; DCT2net is selected to process non-stationary image patches while DCT is optimal for piecewise smooth patches. Experiments on artificially noisy images demonstrate that two-layer DCT2net provides comparable results to BM3D and is as fast as DnCNN algorithm composed of more than a dozen of layers.

Investigation of dynamic processes in cell biology very often relies on the observation in two dimensions of 3D biological processes. Consequently, the data are partial and statistical methods and models are required to recover the parameters describing the dynamical processes. In the case of molecules moving over the 3D surface, such as proteins on walls of bacteria cell, a large portion of the 3D surface is not observed in 2D-time microscopy. It follows that biomolecules may disappear for a period of time in a region of interest, and then reappear later. Assuming Brownian motion with drift, we address the mathematical problem of the reconstruction of biomolecules trajectories on a cylindrical surface. A subregion of the cylinder is typically recorded during the observation period, and biomolecules may appear or disappear in any place of the 3D surface. The performance of the method is demonstrated on simulated particle trajectories that mimic MreB protein dynamics observed in 2D time-lapse fluorescence microscopy in rod-shaped bacteria.

Colocalization aims at characterizing spatial associations between two fluorescently-tagged biomolecules by quantifying the co-occurrence and correlation between the two channels acquired in fluorescence microscopy. Colocalization is presented either as the degree of overlap between the two channels or the overlays of the red and green images, with areas of yellow indicating colocalization of the molecules. This problem remains an open issue in diffraction-limited microscopy and raises new challenges with the emergence of super-resolution imaging, a microscopic technique awarded by the 2014 Nobel prize in chemistry. We propose GcoPS, for Geo-coPositioning System, an original method that exploits the random sets structure of the tagged molecules to provide an explicit testing procedure. Our simulation study shows that GcoPS unequivocally outperforms the best competitive methods in adverse situations (noise, irregularly shaped fluorescent patterns, different optical resolutions). GcoPS is also much faster, a decisive advantage to face the huge amount of data in super-resolution imaging. We demonstrate the performances of GcoPS on two biological real datasets, obtained by conventional diffraction-limited microscopy technique and by super-resolution technique, respectively.

Single-particle tracking allows to infer the motion of single molecules in living cells. When we observe a long trajectory (more than 100 points), it is possible that the particle switches mode of motion over time. Then, fitting a single model to the trajectory can be misleading. In this paper, we propose a method to detect the temporal change points : the times at which a change of dynamics occurs. More specifically, we consider that the particle switches between three main modes of motion : Brownian motion, subdiffusion and superdiffusion. We use an algorithm based on a statistic (Briane et al. 2016) computed on local windows along the trajectory. The method is non parametric as the statistic is not related to any particular model. This algorithm controls the number of false change point detections in the case where the trajectory is fully Brownian. A Monte Carlo study is proposed to demonstrate the performances of the method and also to compare the procedure to two competitive algorithms. At the end, we illustrate the utility of the method on real data depicting the motion of mRNA complexes (called mRNP) in neuronal dendrites.

Recent advances in molecular biology and fluorescence microscopy imaging have made possible the inference of the dynamics of single molecules in living cells. Such inference allows to determine the organization and function of the cell. The trajectories of particles in the cells, computed with tracking algorithms, can be modelled with diffusion processes. Three types of diffusion are considered : (i) free diffusion; (ii) subdiffusion or (iii) superdiffusion. The Mean Square Displacement (MSD) is generally used to determine the different types of dynamics of the particles in living cells (Qian, Sheetz and Elson 1991). We propose here a non-parametric three-decision test as an alternative to the MSD method. The rejection of the null hypothesis -- free diffusion -- is accompanied by claims of the direction of the alternative (subdiffusion or a superdiffusion). We study the asymptotic behaviour of the test statistic under the null hypothesis, and under parametric alternatives which are currently considered in the biophysics literature, (Monnier et al,2012) for example. In addition, we adapt the procedure of Benjamini and Hochberg (2000) to fit with the three-decision test setting, in order to apply the test procedure to a collection of independent trajectories. The performance of our procedure is much better than the MSD method as confirmed by Monte Carlo experiments. The method is demonstrated on real data sets corresponding to protein dynamics observed in fluorescence microscopy., Comment: Revised introduction. A clearer and shorter description of the model (section 2)

Handling all together large displacements, motion details and occlusions remains an open issue for reliable computation of optical flow in a video sequence. We propose a two-step aggregation paradigm to address this problem. The idea is to supply local motion candidates at every pixel in a first step, and then to combine them to determine the global optical flow field in a second step. We exploit local parametric estimations combined with patch correspondences and we experimentally demonstrate that they are sufficient to produce highly accurate motion candidates. The aggregation step is designed as the discrete optimization of a global regularized energy. The occlusion map is estimated jointly with the flow field throughout the two steps. We propose a generic exemplar-based approach for occlusion filling with motion vectors. We achieve state-of-the-art results in computer vision benchmarks, with particularly significant improvements in the case of large displacements and occlusions., Comment: Submission,IEEE Transactions on Image Processing (2014)