Search

Orthodontic treatment usually requires regular face-to-face examinations to monitor dental conditions of the patients. When in-person diagnosis is not feasible, an alternative is to utilize five intra-oral photographs for remote dental monitoring. However, it lacks of 3D information, and how to reconstruct 3D dental models from such sparse view photographs is a challenging problem. In this study, we propose a 3D teeth reconstruction framework, named TeethDreamer, aiming to restore the shape and position of the upper and lower teeth. Given five intra-oral photographs, our approach first leverages a large diffusion model's prior knowledge to generate novel multi-view images with known poses to address sparse inputs and then reconstructs high-quality 3D teeth models by neural surface reconstruction. To ensure the 3D consistency across generated views, we integrate a 3D-aware feature attention mechanism in the reverse diffusion process. Moreover, a geometry-aware normal loss is incorporated into the teeth reconstruction process to enhance geometry accuracy. Extensive experiments demonstrate the superiority of our method over current state-of-the-arts, giving the potential to monitor orthodontic treatment remotely. Our code is available at https://github.com/ShanghaiTech-IMPACT/TeethDreamer, Comment: MICCAI2024

Digital Subtraction Angiography (DSA) is one of the gold standards in vascular disease diagnosing. With the help of contrast agent, time-resolved 2D DSA images deliver comprehensive insights into blood flow information and can be utilized to reconstruct 3D vessel structures. Current commercial DSA systems typically demand hundreds of scanning views to perform reconstruction, resulting in substantial radiation exposure. However, sparse-view DSA reconstruction, aimed at reducing radiation dosage, is still underexplored in the research community. The dynamic blood flow and insufficient input of sparse-view DSA images present significant challenges to the 3D vessel reconstruction task. In this study, we propose to use a time-agnostic vessel probability field to solve this problem effectively. Our approach, termed as vessel probability guided attenuation learning, represents the DSA imaging as a complementary weighted combination of static and dynamic attenuation fields, with the weights derived from the vessel probability field. Functioning as a dynamic mask, vessel probability provides proper gradients for both static and dynamic fields adaptive to different scene types. This mechanism facilitates a self-supervised decomposition between static backgrounds and dynamic contrast agent flow, and significantly improves the reconstruction quality. Our model is trained by minimizing the disparity between synthesized projections and real captured DSA images. We further employ two training strategies to improve our reconstruction quality: (1) coarse-to-fine progressive training to achieve better geometry and (2) temporal perturbed rendering loss to enforce temporal consistency. Experimental results have demonstrated superior quality on both 3D vessel reconstruction and 2D view synthesis., Comment: 12 pages, 13 figures, 5 tables

Accurately localizing and identifying vertebrae from CT images is crucial for various clinical applications. However, most existing efforts are performed on 3D with cropping patch operation, suffering from the large computation costs and limited global information. In this paper, we propose a multi-view vertebra localization and identification from CT images, converting the 3D problem into a 2D localization and identification task on different views. Without the limitation of the 3D cropped patch, our method can learn the multi-view global information naturally. Moreover, to better capture the anatomical structure information from different view perspectives, a multi-view contrastive learning strategy is developed to pre-train the backbone. Additionally, we further propose a Sequence Loss to maintain the sequential structure embedded along the vertebrae. Evaluation results demonstrate that, with only two 2D networks, our method can localize and identify vertebrae in CT images accurately, and outperforms the state-of-the-art methods consistently. Our code is available at https://github.com/ShanghaiTech-IMPACT/Multi-View-Vertebra-Localization-and-Identification-from-CT-Images., Comment: MICCAI 2023

Two-dimensional arrays of magnetically coupled nanomagnets provide a mesoscopic platform for exploring collective phenomena as well as realizing a broad range of spintronic devices. In particular, the magnetic coupling plays a critical role in determining the nature of the cooperative behaviour and providing new functionalities in nanomagnet-based devices. Here, we create coupled Ising-like nanomagnets in which the coupling between adjacent nanomagnetic regions can be reversibly converted between parallel and antiparallel through solid-state ionic gating. This is achieved with the voltage-control of magnetic anisotropies in a nanosized region where the symmetric exchange interaction favours parallel alignment and the antisymmetric exchange interaction, namely the Dzyaloshinskii-Moriya interaction, favours antiparallel alignment. Applying this concept to a two-dimensional lattice, we demonstrate a voltage-controlled phase transition in artificial spin ices. Furthermore, we achieve an addressable control of the individual couplings and realize an electrically programmable Ising network, which opens up new avenues to design nanomagnet-based logic devices and neuromorphic computers

Strong, adjustable magnetic couplings are of great importance to all devices based on magnetic materials. Controlling the coupling between adjacent regions of a single magnetic layer, however, is challenging. In this work, we demonstrate strong exchange-based coupling between arbitrarily shaped regions of a single ferrimagnetic layer. This is achieved by spatially patterning the compensation temperature of the ferrimagnet by either oxidation or He+ irradiation. The coupling originates at the lateral interface between regions with different compensation temperature and scales inversely with their width. We show that this coupling generates large lateral exchange coupling fields and we demonstrate its application to control the switching of magnetically compensated dots with an electric current.

Cone Beam Computed Tomography (CBCT) is the most widely used imaging method in dentistry. As hundreds of X-ray projections are needed to reconstruct a high-quality CBCT image (i.e., the attenuation field) in traditional algorithms, sparse-view CBCT reconstruction has become a main focus to reduce radiation dose. Several attempts have been made to solve it while still suffering from insufficient data or poor generalization ability for novel patients. This paper proposes a novel attenuation field encoder-decoder framework by first encoding the volumetric feature from multi-view X-ray projections, then decoding it into the desired attenuation field. The key insight is when building the volumetric feature, we comply with the multi-view CBCT reconstruction nature and emphasize the view consistency property by geometry-aware spatial feature querying and adaptive feature fusing. Moreover, the prior knowledge information learned from data population guarantees our generalization ability when dealing with sparse view input. Comprehensive evaluations have demonstrated the superiority in terms of reconstruction quality, and the downstream application further validates the feasibility of our method in real-world clinics.

High resolution imaging is achieved using increasingly larger apertures and successively shorter wavelengths. Optical aperture synthesis is an important high-resolution imaging technology used in astronomy. Conventional long baseline amplitude interferometry is susceptible to uncontrollable phase fluctuations, and the technical difficulty increases rapidly as the wavelength decreases. The intensity interferometry inspired by HBT experiment is essentially insensitive to phase fluctuations, but suffers from a narrow spectral bandwidth which results in a lack of detection sensitivity. In this study, we propose optical synthetic aperture imaging based on spatial intensity interferometry. This not only realizes diffraction-limited optical aperture synthesis in a single shot, but also enables imaging with a wide spectral bandwidth. And this method is insensitive to the optical path difference between the sub-apertures. Simulations and experiments present optical aperture synthesis diffraction-limited imaging through spatial intensity interferometry in a 100 $nm$ spectral width of visible light, whose maximum optical path difference between the sub-apertures reach $69.36\lambda$. This technique is expected to provide a solution for optical aperture synthesis over kilometer-long baselines at optical wavelengths.

Spectral camera based on ghost imaging via sparsity constraints (GISC spectral camera) obtains three-dimensional (3D) hyperspectral information with two-dimensional (2D) compressive measurements in a single shot, which has attracted much attention in recent years. However, its imaging quality and real-time performance of reconstruction still need to be further improved. Recently, deep learning has shown great potential in improving the reconstruction quality and reconstruction speed for computational imaging. When applying deep learning into GISC spectral camera, there are several challenges need to be solved: 1) how to deal with the large amount of 3D hyperspectral data, 2) how to reduce the influence caused by the uncertainty of the random reference measurements, 3) how to improve the reconstructed image quality as far as possible. In this paper, we present an end-to-end V-DUnet for the reconstruction of 3D hyperspectral data in GISC spectral camera. To reduce the influence caused by the uncertainty of the measurement matrix and enhance the reconstructed image quality, both differential ghost imaging results and the detected measurements are sent into the network's inputs. Compared with compressive sensing algorithm, such as PICHCS and TwIST, it not only significantly improves the imaging quality with high noise immunity, but also speeds up the reconstruction time by more than two orders of magnitude.

The establishment of chiral coupling in thin magnetic films with inhomogeneous anisotropy has led to the development of artificial systems of fundamental and technological interest. The chiral coupling itself is enabled by the Dzyaloshinskii-Moriya interaction (DMI) enforced by the patterned noncollinear magnetization. Here, we create a domain wall track with out-of-plane magnetization coupled on each side to a narrow parallel strip with in-plane magnetization. With this we show that the chiral torques emerging from the DMI at the boundary between the regions of noncollinear magnetization in a single magnetic layer can be used to bias the domain wall velocity. To tune the chiral torques, the design of the magnetic racetracks can be modified by varying the width of the tracks or the width of the transition region between noncollinear magnetizations, reaching effective chiral magnetic fields of up to 7.8 mT. Furthermore, we show how the magnitude of the chiral torques can be estimated by measuring asymmetric domain wall velocities, and demonstrate spontaneous domain wall motion propelled by intrinsic torques even in the absence of any external driving force.

Since its selection as the method of the year in 2013, single-cell technologies have become mature enough to provide answers to complex research questions. With the growth of single-cell profiling technologies, there has also been a significant increase in data collected from single-cell profilings, resulting in computational challenges to process these massive and complicated datasets. To address these challenges, deep learning (DL) is positioning as a competitive alternative for single-cell analyses besides the traditional machine learning approaches. Here we present a processing pipeline of single-cell RNA-seq data, survey a total of 25 DL algorithms and their applicability for a specific step in the processing pipeline. Specifically, we establish a unified mathematical representation of all variational autoencoder, autoencoder, and generative adversarial network models, compare the training strategies and loss functions for these models, and relate the loss functions of these models to specific objectives of the data processing step. Such presentation will allow readers to choose suitable algorithms for their particular objective at each step in the pipeline. We envision that this survey will serve as an important information portal for learning the application of DL for scRNA-seq analysis and inspire innovative use of DL to address a broader range of new challenges in emerging multi-omics and spatial single-cell sequencing., Comment: 74 pages

Just noticeable difference (JND) of natural images refers to the maximum pixel intensity change magnitude that typical human visual system (HVS) cannot perceive. Existing efforts on JND estimation mainly dedicate to modeling the diverse masking effects in either/both spatial or/and frequency domains, and then fusing them into an overall JND estimate. In this work, we turn to a dramatically different way to address this problem with a top-down design philosophy. Instead of explicitly formulating and fusing different masking effects in a bottom-up way, the proposed JND estimation model dedicates to first predicting a critical perceptual lossless (CPL) counterpart of the original image and then calculating the difference map between the original image and the predicted CPL image as the JND map. We conduct subjective experiments to determine the critical points of 500 images and find that the distribution of cumulative normalized KLT coefficient energy values over all 500 images at these critical points can be well characterized by a Weibull distribution. Given a testing image, its corresponding critical point is determined by a simple weighted average scheme where the weights are determined by a fitted Weibull distribution function. The performance of the proposed JND model is evaluated explicitly with direct JND prediction and implicitly with two applications including JND-guided noise injection and JND-guided image compression. Experimental results have demonstrated that our proposed JND model can achieve better performance than several latest JND models. In addition, we also compare the proposed JND model with existing visual difference predicator (VDP) metrics in terms of the capability in distortion detection and discrimination. The results indicate that our JND model also has a good performance in this task., Comment: 16 pages, 16 figures

When light travels through scattering media, speckles (spatially random distribution of fluctuated intensities) are formed due to the interference of light travelling along different optical paths, preventing the perception of structure, absolute location and dimension of a target within or on the other side of the medium. Currently, the prevailing techniques such as wavefront shaping, optical phase conjugation, scattering matrix measurement, and speckle autocorrelation imaging can only picture the target structure in the absence of prior information. Here we show that a scattering medium can be conceptualized as an assembly of randomly packed pinhole cameras, and the corresponding speckle pattern is a superposition of randomly shifted pinhole images. This provides a new perspective to bridge target, scattering medium, and speckle pattern, allowing one to localize and profile a target quantitatively from speckle patterns perceived from the other side of the scattering medium, which is impossible with all existing methods. The method also allows us to interpret some phenomena of diffusive light that are otherwise challenging to understand. For example, why the morphological appearance of speckle patterns changes with the target, why information is difficult to be extracted from thick scattering media, and what determines the capability of seeing through scattering media. In summary, the concept, whilst in its infancy, opens a new door to unveiling scattering media and information extraction from scattering media in real time.

By encoding the high-dimensional light-field imaging information into a detectable two-dimensional speckle plane, ghost imaging camera via sparsity constraints (GISC camera) can directly catch the high-dimensional light-field imaging information with only one snapshot. This makes it worth to revisit the spatial resolution limit of this optical imaging system. In this paper we show both theoretically and experimentally that, while the resolution in high-dimensional light-field space is still limited by diffraction, the statistical spatial resolution of GISC camera can be greatly improved comparing to classical Rayleigh's criterion by utilizing the discernibility in high-dimensional light-field space. The interaction between imaging resolution, degrees of freedom of light field, and degrees of freedom of objects in high-dimensional light-field space is also demonstrated., Comment: 7pages, 6 figures

A spectral camera based on ghost imaging via sparsity constraints (GISC) acquires a spectral data-cube (x; y; {\lambda}) through a single exposure. The noise immunity of the system is one of the important factors affecting the quality of the reconstructed images, especially at low sampling rates. Tailoring the intensity to generate super-Rayleigh speckle patterns which have superior noise immunity may offer an effective route to promote the imaging quality of GISC spectral camera. According to the structure of GISC spectral camera, we proposed a universal method for generating super-Rayleigh speckle patterns with customized intensity statistics based on the principle of reversibility of light. Simulation and experimental results demonstrate that, within a wide imaging spectral bandwidth, GISC spectral camera with super-Rayleigh modulator not only has superior noise immunity, but also has higher imaging quality at low sampling rates. This work will promote the application of GISC spectral camera by improving the quality of imaging results, especially in weak-light illumination., Comment: 11 pages, 12 figures

The applications of present nanoscopy techniques for live cell imaging are limited by the long sampling time and low emitter density. Here we developed a new single frame wide-field nanoscopy based on ghost imaging via sparsity constraints (GISC Nanoscopy), in which a spatial random phase modulator is applied in a wide-field microscopy to achieve random measurement for fluorescence signals. This new method can effectively utilize the sparsity of fluorescence emitters to dramatically enhance the imaging resolution to 80 nm by compressive sensing (CS) reconstruction for one raw image. The ultra-high emitter density of 143 {\mu}m-2 has been achieved while the precision of single-molecule localization below 25 nm has been maintained. Thereby working with high-density of photo-switchable fluorophores GISC nanoscopy can reduce orders of magnitude sampling frames compared with previous single-molecule localization based super-resolution imaging methods.

Ghost imaging (GI) is a novel imaging technique based on the second-order correlation of light fields. Due to limited number of samplings in practice, traditional GI methods often reconstruct objects with unsatisfactory quality. To improve the imaging results, many reconstruction methods have been developed, yet the reconstruction quality is still fundamentally restricted by the modulated light fields. In this paper, we propose to improve the imaging quality of GI by optimizing the light fields, which is realized via matrix optimization for a learned dictionary incorporating the sparsity prior of objects. A closed-form solution of the sampling matrix, which enables successive sampling, is derived. Through simulation and experimental results, it is shown that the proposed scheme leads to better imaging quality compared to the state-of-the-art optimization methods for light fields, especially at a low sampling rate., Comment: 14 pages, 6 figures

Recent behavioral and electroencephalograph (EEG) studies have defined ways that auditory spatial attention can be allocated over large regions of space. As with most experimental studies, behavior EEG was averaged over 10s of minutes because identifying abstract feature spatial codes from raw EEG data is extremely challenging. The goal of this study is to design a deep learning model that can learn from raw EEG data and predict auditory spatial information on a trial-by-trial basis. We designed a convolutional neural networks (CNN) model to predict the attended location or other stimulus locations relative to the attended location. A multi-task model was also used to predict the attended and stimulus locations at the same time. Based on the visualization of our models, we investigated features of individual classification tasks and joint feature of the multi-task model. Our model achieved an average 72.4% in relative location prediction and 90.0% in attended location prediction individually. The multi-task model improved the performance of attended location prediction by 3%. Our results suggest a strong correlation between attended location and relative location., Comment: 6 pages, 5 tables, 9 figures

Ghost imaging LiDAR via sparsity constraints using push-broom scanning is proposed. It can image the stationary target scene continuously along the scanning direction by taking advantage of the relative movement between the platform and the target scene. Compared to conventional ghost imaging LiDAR that requires multiple speckle patterns staring the target, ghost imaging LiDAR via sparsity constraints using push-broom scanning not only simplifies the imaging system, but also reduces the sampling number. Numerical simulations and experiments have demonstrated its efficiency.