Search

Accurate depth estimation is crucial for many fields, including robotics, navigation, and medical imaging. However, conventional depth sensors often produce low-resolution (LR) depth maps, making detailed scene perception challenging. To address this, enhancing LR depth maps to high-resolution (HR) ones has become essential, guided by HR-structured inputs like RGB or grayscale images. We propose a novel sensor fusion methodology for guided depth super-resolution (GDSR), a technique that combines LR depth maps with HR images to estimate detailed HR depth maps. Our key contribution is the Incremental guided attention fusion (IGAF) module, which effectively learns to fuse features from RGB images and LR depth maps, producing accurate HR depth maps. Using IGAF, we build a robust super-resolution model and evaluate it on multiple benchmark datasets. Our model achieves state-of-the-art results compared to all baseline models on the NYU v2 dataset for $\times 4$, $\times 8$, and $\times 16$ upsampling. It also outperforms all baselines in a zero-shot setting on the Middlebury, Lu, and RGB-D-D datasets. Code, environments, and models are available on GitHub.

Optical brain imaging technologies are promising due to their relatively high temporal resolution, portability and cost-effectiveness. However, the highly scattering nature of near-infrared light in human tissue makes it challenging to collect photons emerging from more than 4 cm below the scalp, or with source-detector separation larger than several centimeters. We explore the physical limits of photon transport in the head and show that despite an extreme attenuation of ~10^(18), we can experimentally detect light that is transmitted diametrically through the entire adult human head. Analysis of various photon migration pathways through the head also indicates how the source-detector configuration can be used to isolate photons interacting with deep regions of the brain that are inaccessible with current optical techniques.

In the field of detection and ranging, multiple complementary sensing modalities may be used to enrich the information obtained from a dynamic scene. One application of this sensor fusion is in public security and surveillance, whose efficacy and privacy protection measures must be continually evaluated. We present a novel deployment of sensor fusion for the discrete detection of concealed metal objects on persons whilst preserving their privacy. This is achieved by coupling off-the-shelf mmWave radar and depth camera technology with a novel neural network architecture that processes the radar signals using convolutional Long Short-term Memory (LSTM) blocks and the depth signal, using convolutional operations. The combined latent features are then magnified using a deep feature magnification to learn cross-modality dependencies in the data. We further propose a decoder, based on the feature extraction and embedding block, to learn an efficient upsampling of the latent space to learn the location of the concealed object in the spatial domain through radar feature guidance. We demonstrate the detection of presence and inference of 3D location of concealed metal objects with an accuracy of up to 95%, using a technique that is robust to multiple persons. This work provides a demonstration of the potential for cost effective and portable sensor fusion, with strong opportunities for further development.

How does the genome encode the form of the organism? What is the nature of this genomic code? Inspired by recent work in machine learning and neuroscience, we propose that the genome encodes a generative model of the organism. In this scheme, by analogy with variational autoencoders, the genome comprises a connectionist network, embodying a compressed space of latent variables, with weights that get encoded by the learning algorithm of evolution and decoded through the processes of development. The generative model analogy accounts for the complex, distributed genetic architecture of most traits and the emergent robustness and evolvability of developmental processes, while also offering a conception that lends itself to formalisation., Comment: 33 pages, 4 figures