Search

We introduce Stereo Anywhere, a novel stereo-matching framework that combines geometric constraints with robust priors from monocular depth Vision Foundation Models (VFMs). By elegantly coupling these complementary worlds through a dual-branch architecture, we seamlessly integrate stereo matching with learned contextual cues. Following this design, our framework introduces novel cost volume fusion mechanisms that effectively handle critical challenges such as textureless regions, occlusions, and non-Lambertian surfaces. Through our novel optical illusion dataset, MonoTrap, and extensive evaluation across multiple benchmarks, we demonstrate that our synthetic-only trained model achieves state-of-the-art results in zero-shot generalization, significantly outperforming existing solutions while showing remarkable robustness to challenging cases such as mirrors and transparencies., Comment: Code: https://github.com/bartn8/stereoanywhere - Project page: https://stereoanywhere.github.io/

Characterizing quantum states of the electromagnetic field at microwave frequencies requires fast and sensitive detectors that can simultaneously probe the field time-dependent amplitude and its quantum fluctuations. In this work, we demonstrate a quantum sensor that exploits the phase of a single electron wavefunction, measured in an electronic Fabry-Perot interferometer, to detect a classical time-dependent electric field. The time resolution, limited by the temporal width of the electronic wavepacket, is a few tens of picoseconds. The interferometry technique provides a voltage resolution of a few tens of microvolts, corresponding to a few microwave photons. Importantly, our detector simultaneously probes the amplitude of the field from the phase of the measured interference pattern and its fluctuations from the interference contrast. This capability paves the way for on-chip detection of quantum radiation, such as squeezed or Fock states., Comment: Main (8 pages and 3 figures) and supplementary files (17 pages and 12 figures)

Event stereo matching is an emerging technique to estimate depth from neuromorphic cameras; however, events are unlikely to trigger in the absence of motion or the presence of large, untextured regions, making the correspondence problem extremely challenging. Purposely, we propose integrating a stereo event camera with a fixed-frequency active sensor -- e.g., a LiDAR -- collecting sparse depth measurements, overcoming the aforementioned limitations. Such depth hints are used by hallucinating -- i.e., inserting fictitious events -- the stacks or raw input streams, compensating for the lack of information in the absence of brightness changes. Our techniques are general, can be adapted to any structured representation to stack events and outperform state-of-the-art fusion methods applied to event-based stereo., Comment: ECCV 2024. Code: https://github.com/bartn8/eventvppstereo/ - Project Page: https://eventvppstereo.github.io/

Stereo matching is close to hitting a half-century of history, yet witnessed a rapid evolution in the last decade thanks to deep learning. While previous surveys in the late 2010s covered the first stage of this revolution, the last five years of research brought further ground-breaking advancements to the field. This paper aims to fill this gap in a two-fold manner: first, we offer an in-depth examination of the latest developments in deep stereo matching, focusing on the pioneering architectural designs and groundbreaking paradigms that have redefined the field in the 2020s; second, we present a thorough analysis of the critical challenges that have emerged alongside these advances, providing a comprehensive taxonomy of these issues and exploring the state-of-the-art techniques proposed to address them. By reviewing both the architectural innovations and the key challenges, we offer a holistic view of deep stereo matching and highlight the specific areas that require further investigation. To accompany this survey, we maintain a regularly updated project page that catalogs papers on deep stereo matching in our Awesome-Deep-Stereo-Matching (https://github.com/fabiotosi92/Awesome-Deep-Stereo-Matching) repository., Comment: Extended version of CVPR 2024 Tutorial "Deep Stereo Matching in the Twenties" (https://sites.google.com/view/stereo-twenties)

This paper presents a novel general-purpose stereo and depth data fusion paradigm that mimics the active stereo principle by replacing the unreliable physical pattern projector with a depth sensor. It works by projecting virtual patterns consistent with the scene geometry onto the left and right images acquired by a conventional stereo camera, using the sparse hints obtained from a depth sensor, to facilitate the visual correspondence. Purposely, any depth sensing device can be seamlessly plugged into our framework, enabling the deployment of a virtual active stereo setup in any possible environment and overcoming the severe limitations of physical pattern projection, such as the limited working range and environmental conditions. Exhaustive experiments on indoor and outdoor datasets featuring both long and close range, including those providing raw, unfiltered depth hints from off-the-shelf depth sensors, highlight the effectiveness of our approach in notably boosting the robustness and accuracy of algorithms and deep stereo without any code modification and even without re-training. Additionally, we assess the performance of our strategy on active stereo evaluation datasets with conventional pattern projection. Indeed, in all these scenarios, our virtual pattern projection paradigm achieves state-of-the-art performance. The source code is available at: https://github.com/bartn8/vppstereo., Comment: extended version of ICCV 2023: "Active Stereo Without Pattern Projector"

This paper proposes a new framework for depth completion robust against domain-shifting issues. It exploits the generalization capability of modern stereo networks to face depth completion, by processing fictitious stereo pairs obtained through a virtual pattern projection paradigm. Any stereo network or traditional stereo matcher can be seamlessly plugged into our framework, allowing for the deployment of a virtual stereo setup that is future-proof against advancement in the stereo field. Exhaustive experiments on cross-domain generalization support our claims. Hence, we argue that our framework can help depth completion to reach new deployment scenarios., Comment: 3DV 2024. Code: https://github.com/bartn8/vppdc - Project page: https://vppdc.github.io/

This paper proposes a novel framework integrating the principles of active stereo in standard passive camera systems without a physical pattern projector. We virtually project a pattern over the left and right images according to the sparse measurements obtained from a depth sensor. Any such devices can be seamlessly plugged into our framework, allowing for the deployment of a virtual active stereo setup in any possible environment, overcoming the limitation of pattern projectors, such as limited working range or environmental conditions. Experiments on indoor/outdoor datasets, featuring both long and close-range, support the seamless effectiveness of our approach, boosting the accuracy of both stereo algorithms and deep networks., Comment: ICCV 2023. Code: https://github.com/bartn8/vppstereo - Project page: https://vppstereo.github.io

Efficient exploration strategies are vital in tasks such as search-and-rescue missions and disaster surveying. Unmanned Aerial Vehicles (UAVs) have become particularly popular in such applications, promising to cover large areas at high speeds. Moreover, with the increasing maturity of onboard UAV perception, research focus has been shifting toward higher-level reasoning for single- and multi-robot missions. However, autonomous navigation and exploration of previously unknown large spaces still constitutes an open challenge, especially when the environment is cluttered and exhibits large and frequent occlusions due to high obstacle density, as is the case of forests. Moreover, the problem of long-distance wireless communication in such scenes can become a limiting factor, especially when automating the navigation of a UAV swarm. In this spirit, this work proposes an exploration strategy that enables UAVs, both individually and in small swarms, to quickly explore complex scenes in a decentralized fashion. By providing the decision-making capabilities to each UAV to switch between different execution modes, the proposed strategy strikes a great balance between cautious exploration of yet completely unknown regions and more aggressive exploration of smaller areas of unknown space. This results in full coverage of forest areas of variable density, consistently faster than the state of the art. Demonstrating successful deployment with a single UAV as well as a swarm of up to three UAVs, this work sets out the basic principles for multi-root exploration of cluttered scenes, with up to 65% speed up in the single UAV case and 40% increase in explored area for the same mission time in multi-UAV setups.

Squeezing of the quadratures of the electromagnetic field has been extensively studied in optics and microwaves. However, previous works focused on the generation of squeezed states in a low impedance ($Z_0 \approx 50 \Omega$) environment. We report here on the demonstration of the squeezing of bosonic edge magnetoplasmon modes in a quantum Hall conductor whose characteristic impedance is set by the quantum of resistance ($R_K \approx 25 k \Omega$), offering the possibility of an enhanced coupling to low-dimensional quantum conductors. By applying a combination of dc and ac drives to a quantum point contact, we demonstrate squeezing and observe a noise reduction 18\% below the vacuum fluctuations. This level of squeezing can be improved by using more complex conductors, such as ac driven quantum dots or mesoscopic capacitors., Comment: 6+2 pages, 3+1 figures

Molecular dynamics calculations of inelastic collisions of atomic oxygen with molecular nitrogen are known to show orders of magnitude discrepancies with experimental results in the range from room temperature to many thousands of degrees Kelvin. In this work, we have achieved an unprecedented quantitative agreement with experiments even at low temperature, by including a non-adiabatic treatment involving vibronic states on newly developed potential energy surfaces. This result paves the way to the calculation of accurate and detailed databases of vibrational energy exchange rates for this collisional system. This is bound to have an impact on air plasma simulations in a wide range of conditions and on the development of Very Low Earth Orbit (VLEO) satellites, operating in the low thermosphere, objects of great technological interest due to their potential at a competitive cost., Comment: 17 pages, 8 figures

We report accurate quantum calculations of the sieving of Helium atoms by two-dimensional (2D) graphtriyne layers with a new interaction potential. Thermal rate constants and permeances in an ample temperature range are computed and compared for both Helium isotopes. With a pore larger than graphdiyne, the most common member of the gamma - graphyne family, it could be expected that the appearance of quantum effects were more limited. We find, however, a strong quantum behavior that can be attributed to the presence of selective adsorption resonances, with a pronounced effect in the low temperature regime. This effect leads to the appearance of some selectivity at very low temperatures and the possibility for the heavier isotope to cross the membrane more efficiently than the lighter, contrarily to what happened with graphdiyne membranes, where the sieving at low energy is predominantly ruled by quantum tunneling. The use of more approximate methods could be not advisable in these situations and prototypical transition state theory (TST) treatments might lead to large errors.

Two-dimensional systems can host exotic particles called anyons whose quantum statistics are neither bosonic nor fermionic. For example, the elementary excitations of the fractional quantum Hall effect at filling factor $\nu=1/m$ (where m is an odd integer) have been predicted to obey abelian fractional statistics, with a phase $\varphi$ associated with the exchange of two particles equal to $\pi/m$. However, despite numerous experimental attempts, clear signatures of fractional statistics remain elusive. Here we experimentally demonstrate abelian fractional statistics at filling factor $\nu=1/3$ by measuring the current correlations resulting from the collision between anyons at a beam-splitter. By analyzing their dependence on the anyon current impinging on the splitter and comparing with recent theoretical models, we extract $\varphi=\pi/3$, in agreement with predictions., Comment: 5 pages, supplementary material can be downloaded at https://science.sciencemag.org/content/368/6487/173/suppl/DC1

Epilepsy is a dynamic and complex neurological disease affecting about 1% of the worldwide population, among which 30% of the patients are drug-resistant. Epilepsy is characterized by recurrent episodes of paroxysmal neural discharges (the so-called seizures), which manifest themselves through a large-amplitude rhythmic activity observed in depth-EEG recordings, in particular in local field potentials (LFPs). The signature characterizing the transition to seizures involves complex oscillatory patterns, which could serve as a marker to prevent seizure initiation by triggering appropriate therapeutic neurostimulation methods. To investigate such protocols, neurophysiological lumped-parameter models at the mesoscopic scale, namely neural mass models, are powerful tools that not only mimic the LFP signals but also give insights on the neural mechanisms related to different stages of seizures. Here, we analyze the multiple time-scale dynamics of a neural mass model and explain the underlying structure of the complex oscillations observed before seizure initiation. We investigate population-specific effects of the stimulation and the dependence of stimulation parameters on synaptic timescales. In particular, we show that intermediate stimulation frequencies (>20 Hz) can abort seizures if the timescale difference is pronounced. Those results have the potential in the design of therapeutic brain stimulation protocols based on the neurophysiological properties of tissue.

Depth perception is paramount to tackle real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image represents the most versatile solution, since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit its practical deployment: i) the low reliability when deployed in-the-wild and ii) the demanding resource requirements to achieve real-time performance, often not compatible with such devices. Therefore, in this paper, we deeply investigate these issues showing how they are both addressable adopting appropriate network design and training strategies -- also outlining how to map the resulting networks on handheld devices to achieve real-time performance. Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications. Indeed, to further support this evidence, we report experimental results concerning real-time depth-aware augmented reality and image blurring with smartphones in-the-wild., Comment: 11 pages, 9 figures

We present a fully-integrated sensing and control system which enables mobile manipulator robots to execute building tasks with millimeter-scale accuracy on building construction sites. The approach leverages multi-modal sensing capabilities for state estimation, tight integration with digital building models, and integrated trajectory planning and whole-body motion control. A novel method for high-accuracy localization updates relative to the known building structure is proposed. The approach is implemented on a real platform and tested under realistic construction conditions. We show that the system can achieve sub-cm end-effector positioning accuracy during fully autonomous operation using solely on-board sensing.

We report on a combined experimental and theoretical study of Li$^+$ ions solvated by up to 50 He atoms. The experiments show clear enhanced abundances associated with He$_n$Li$^+$ clusters where $n=2$, 6, 8, and 14. We find that classical methods, e.g.\ Basin-Hopping (BH), give results that qualitatively agree with quantum mechanical methods such as path integral Monte Carlo, diffusion Monte Carlo and quantum free energy, regarding both energies and the solvation structures that are formed. The theory identifies particularly stable structures for $n=4$, 6 and 8 which line up with some of the most abundant features in the experiments., Comment: 8 pages, 7 figures, published in Physical Chemistry Chemical Physics

Unmanned aerial vehicles combined with computer vision systems, such as convolutional neural networks, offer a flexible and affordable solution for terrain monitoring, mapping, and detection tasks. However, a key challenge remains the collection and annotation of training data for the given sensors, application, and mission. We introduce an informative path planning system that incorporates novelty estimation into its objective function, based on research for uncertainty estimation in deep learning. The system is designed for data collection to reduce both the number of flights and of annotated images. We evaluate the approach on real world terrain mapping data and show significantly smaller collected training dataset compared to standard lawnmower data collection techniques., Comment: presented at RSS'19 "2nd Workshop on Informative Path Planning and Adaptive Sampling"

Strongly correlated low-dimensional systems can host exotic elementary excitations carrying a fractional charge $q$ and potentially obeying anyonic statistics. In the fractional quantum Hall effect, their fractional charge has been successfully determined owing to low frequency shot noise measurements. However, a universal method for sensing them unambiguously and unraveling their intricate dynamics was still lacking. Here, we demonstrate that this can be achieved by measuring the microwave photons emitted by such excitations when they are transferred through a potential barrier biased with a dc voltage $V_{\text{dc}}$. We observe that only photons at frequencies $f$ below $qV_{\text{dc}}/h$ are emitted. This threshold provides a direct and unambiguous determination of the charge $q$, and a signature of exclusion statistics. Derived initially within the Luttinger model, this feature is also predicted by universal non-equilibrium fluctuation relations which agree fully with our measurements. Our work paves the way for further exploration of anyonic statistics using microwave measurements., Comment: 16 pages, 4 figures

Topological effects in edge states are clearly visible on short lengths only, thus largely impeding their studies. On larger distances, one may be able to dynamically enhance topological signatures by exploiting the high mobility of edge states with respect to bulk carriers. Our work on microwave spectroscopy highlights the responses of the edges which host very mobile carriers, while bulk carriers are drastically slowed down in the gap. Though the edges are denser than expected, we establish that charge relaxation occurs on short timescales, and suggests that edge states can be addressed selectively on timescales over which bulk carriers are frozen., Comment: Main text (5 pages, 4 figures) + Supplementary Material