Search

The large abundance of perspective camera datasets facilitated the emergence of novel learning-based strategies for various tasks, such as camera localization, single image depth estimation, or view synthesis. However, panoramic or omnidirectional image datasets, including essential information, such as pose and depth, are mostly made with synthetic scenes. In this work, we introduce a large scale 360$^{\circ}$ videos dataset in the wild. This dataset has been carefully scraped from the Internet and has been captured from various locations worldwide. Hence, this dataset exhibits very diversified environments (e.g., indoor and outdoor) and contexts (e.g., with and without moving objects). Each of the 25K images constituting our dataset is provided with its respective camera's pose and depth map. We illustrate the relevance of our dataset for two main tasks, namely, single image depth estimation and view synthesis.

Although the study of X-ray binaries has led to major breakthroughs in high-energy astrophysics, their circumbinary environment at scales of $\sim$100--10,000 astronomical units has not been thoroughly investigated. In this paper, we undertake a novel and exploratory study by employing direct and high-contrast imaging techniques on a sample of X-ray binaries, using adaptive optics and the vortex coronagraph on Keck/NIRC2. High-contrast imaging opens up the possibility to search for exoplanets, brown dwarfs, circumbinary companion stars, and protoplanetary disks in these extreme systems. Here, we present the first near-infrared high-contrast images of 13 high-mass X-ray binaries located within $\sim$2--3 kpc. The key results of this campaign involve the discovery of several candidate circumbinary companions ranging from sub-stellar (brown dwarf) to stellar masses. By conducting an analysis based on galactic population models, we discriminate sources that are likely background/foreground stars and isolate those that have a high probability ($\gtrsim 60 - 99\%$) of being gravitationally bound to the X-ray binary. This publication seeks to establish a preliminary catalog for future analyses of proper motion and subsequent observations. With our preliminary results, we calculate the first estimate of the companion frequency and the multiplicity frequency for X-ray binaries: $\approx$0.6 and 1.8 $\pm$ 0.9 respectively, considering only the sources that are most likely bound to the X-ray binary. In addition to extending our comprehension of how brown dwarfs and stars can form and survive in such extreme systems, our study opens a new window to our understanding of the formation of X-ray binaries., Comment: 26 pages, 6 figures, accepted for publication in ApJ

With the widespread use of NeRF-based implicit 3D representation, the need for camera localization in the same representation becomes manifestly apparent. Doing so not only simplifies the localization process -- by avoiding an outside-the-NeRF-based localization -- but also has the potential to offer the benefit of enhanced localization. This paper studies the problem of localizing cameras in NeRF using a diffusion model for camera pose adjustment. More specifically, given a pre-trained NeRF model, we train a diffusion model that iteratively updates randomly initialized camera poses, conditioned upon the image to be localized. At test time, a new camera is localized in two steps: first, coarse localization using the proposed pose diffusion process, followed by local refinement steps of a pose inversion process in NeRF. In fact, the proposed camera localization by pose diffusion (CaLDiff) method also integrates the pose inversion steps within the diffusion process. Such integration offers significantly better localization, thanks to our downstream refinement-aware diffusion process. Our exhaustive experiments on challenging real-world data validate our method by providing significantly better results than the compared methods and the established baselines. Our source code will be made publicly available.

Giant exoplanets have been directly imaged over orders of magnitude of orbital separations, prompting theoretical and observational investigations of their formation pathways. In this paper, we present new VLTI/GRAVITY astrometric data of HIP 65426 b, a cold, giant exoplanet which is a particular challenge for most formation theories at a projected separation of 92 au from its primary. Leveraging GRAVITY's astrometric precision, we present an updated eccentricity posterior that disfavors large eccentricities. The eccentricity posterior is still prior-dependent, and we extensively interpret and discuss the limits of the posterior constraints presented here. We also perform updated spectral comparisons with self-consistent forward-modeled spectra, finding a best fit ExoREM model with solar metallicity and C/O=0.6. An important caveat is that it is difficult to estimate robust errors on these values, which are subject to interpolation errors as well as potentially missing model physics. Taken together, the orbital and atmospheric constraints paint a preliminary picture of formation inconsistent with scattering after disk dispersal. Further work is needed to validate this interpretation. Analysis code used to perform this work is available at https://github.com/sblunt/hip65426., Comment: 28 pages, 14 figures. Revised and resubmitted to AJ

Tension remains between the observed and modeled properties of substellar objects, but objects in binary orbits, with known dynamical masses can provide a way forward. HD 72946 B is a recently imaged brown dwarf companion to the nearby, solar type star. We achieve $\sim100~\mu\mathrm{as}$ relative astrometry of HD 72946 B in the K-band using VLTI/GRAVITY, unprecedented for a benchmark brown dwarf. We fit an ensemble of measurements of the orbit using orbitize! and derive a strong dynamical mass constraint $\mathrm{M_B}=69.5\pm0.5~\mathrm{M_{Jup}}$ assuming a strong prior on the host star mass $\mathrm{M_A}=0.97\pm0.01~\mathrm{M_\odot}$ from an updated stellar analysis. We fit the spectrum of the companion to a grid of self-consistent BT-Settl-CIFIST model atmospheres, and perform atmospheric retrievals using petitRADTRANS. A dynamical mass prior only marginally influences the sampled distribution on effective temperature, but has a large influence on the surface gravity and radius, as expected. The dynamical mass alone does not strongly influence retrieved pressure-temperature or cloud parameters within our current retrieval setup. Independent of cloud prescription and prior assumptions, we find agreement within $\pm2\,\sigma$ between the C/O ratio of the host ($0.52\pm0.05)$ and brown dwarf ($0.43$ to $0.63$), as expected from a molecular cloud collapse formation scenario, but our retrieved metallicities are implausibly high ($0.6-0.8$) in light of an excellent agreement of the data with the solar abundance model grid. Future work on our retrieval framework will seek to resolve this tension. Additional study of low surface-gravity objects is necessary to assess the influence of a dynamical mass prior on atmospheric analysis., Comment: Accepted for publication in the Astrophysical Journal. 32 pages, 14 figures, 7 tables. v2&3 correct errors in co-author's affiliations, figure rendering, and some grant acknowledgements

The Near-Infrared Imager and Slitless Spectrograph (NIRISS) is the science module of the Canadian-built Fine Guidance Sensor (FGS) onboard the James Webb Space Telescope (JWST). NIRISS has four observing modes: 1) broadband imaging featuring seven of the eight NIRCam broadband filters, 2) wide-field slitless spectroscopy (WFSS) at a resolving power of $\sim$150 between 0.8 and 2.2 $\mu$m, 3) single-object cross-dispersed slitless spectroscopy (SOSS) enabling simultaneous wavelength coverage between 0.6 and 2.8 $\mu$m at R$\sim$700, a mode optimized for exoplanet spectroscopy of relatively bright ($J<6.3$) stars and 4) aperture masking interferometry (AMI) between 2.8 and 4.8 $\mu$m enabling high-contrast ($\sim10^{-3}-10^{-4}$) imaging at angular separations between 70 and 400 milliarcsec for relatively bright ($M<8$) sources. This paper presents an overview of the NIRISS instrument, its design, its scientific capabilities, and a summary of in-flight performance. NIRISS shows significantly better response shortward of $\sim2.5\,\mu$m resulting in 10-40% sensitivity improvement for broadband and low-resolution spectroscopy compared to pre-flight predictions. Two time-series observations performed during instrument commissioning in the SOSS mode yield very stable spectro-photometry performance within $\sim$10% of the expected noise. The first space-based companion detection of the tight binary star AB Dor AC through AMI was demonstrated.

Reducing the number of bits needed to encode the weights and activations of neural networks is highly desirable as it speeds up their training and inference time while reducing memory consumption. It is unsurprising that considerable attention has been drawn to developing neural networks that employ lower-precision computation. This includes IEEE 16-bit, Google bfloat16, 8-bit, 4-bit floating-point or fixed-point, 2-bit, and various mixed-precision algorithms. Out of these low-precision formats, IEEE 16-bit stands out due to its universal compatibility with contemporary GPUs. This accessibility contrasts with bfloat16, which needs high-end GPUs, or other non-standard fewer-bit designs, which typically require software simulation. This study focuses on the widely accessible IEEE 16-bit format for comparative analysis. This analysis involves an in-depth theoretical investigation of the factors that lead to discrepancies between 16-bit and 32-bit models, including a formalization of the concepts of floating-point error and tolerance to understand the conditions under which a 16-bit model can approximate 32-bit results. Contrary to literature that credits the success of noise-tolerated neural networks to regularization effects, our study-supported by a series of rigorous experiments-provides a quantitative explanation of why standalone IEEE 16-bit floating-point neural networks can perform on par with 32-bit and mixed-precision networks in various image classification tasks. Because no prior research has studied IEEE 16-bit as a standalone floating-point precision in neural networks, we believe our findings will have significant impacts, encouraging the adoption of standalone IEEE 16-bit networks in future neural network applications.

Segment anything model (SAM) developed by Meta AI Research has recently attracted significant attention. Trained on a large segmentation dataset of over 1 billion masks, SAM is capable of segmenting any object on a certain image. In the original SAM work, the authors turned to zero-short transfer tasks (like edge detection) for evaluating the performance of SAM. Recently, numerous works have attempted to investigate the performance of SAM in various scenarios to recognize and segment objects. Moreover, numerous projects have emerged to show the versatility of SAM as a foundation model by combining it with other models, like Grounding DINO, Stable Diffusion, ChatGPT, etc. With the relevant papers and projects increasing exponentially, it is challenging for the readers to catch up with the development of SAM. To this end, this work conducts the first yet comprehensive survey on SAM. This is an ongoing project and we intend to update the manuscript on a regular basis. Therefore, readers are welcome to contact us if they complete new works related to SAM so that we can include them in our next version., Comment: First survey on Segment Anything Model (SAM), work under progress

Generative AI (AIGC, a.k.a. AI generated content) has made significant progress in recent years, with text-guided content generation being the most practical as it facilitates interaction between human instructions and AIGC. Due to advancements in text-to-image and 3D modeling technologies (like NeRF), text-to-3D has emerged as a nascent yet highly active research field. Our work conducts the first comprehensive survey and follows up on subsequent research progress in the overall field, aiming to help readers interested in this direction quickly catch up with its rapid development. First, we introduce 3D data representations, including both Euclidean and non-Euclidean data. Building on this foundation, we introduce various foundational technologies and summarize how recent work combines these foundational technologies to achieve satisfactory text-to-3D results. Additionally, we present mainstream baselines and research directions in recent text-to-3D technology, including fidelity, efficiency, consistency, controllability, diversity, and applicability. Furthermore, we summarize the usage of text-to-3D technology in various applications, including avatar generation, texture generation, shape editing, and scene generation.

Twenty-six years ago a small committee report, building on earlier studies, expounded a compelling and poetic vision for the future of astronomy, calling for an infrared-optimized space telescope with an aperture of at least $4m$. With the support of their governments in the US, Europe, and Canada, 20,000 people realized that vision as the $6.5m$ James Webb Space Telescope. A generation of astronomers will celebrate their accomplishments for the life of the mission, potentially as long as 20 years, and beyond. This report and the scientific discoveries that follow are extended thank-you notes to the 20,000 team members. The telescope is working perfectly, with much better image quality than expected. In this and accompanying papers, we give a brief history, describe the observatory, outline its objectives and current observing program, and discuss the inventions and people who made it possible. We cite detailed reports on the design and the measured performance on orbit., Comment: Accepted by PASP for the special issue on The James Webb Space Telescope Overview, 29 pages, 4 figures

The HR 2562 system is a rare case where a brown dwarf companion resides in a cleared inner hole of a debris disk, offering invaluable opportunities to study the dynamical interaction between a substellar companion and a dusty disk. We present the first ALMA observation of the system as well as the continued GPI monitoring of the companion's orbit with 6 new epochs from 2016 to 2018. We update the orbital fit and, in combination with absolute astrometry from GAIA, place a 3$\sigma$ upper limit of 18.5 $M_J$ on the companion's mass. To interpret the ALMA observations, we used radiative transfer modeling to determine the disk properties. We find that the disk is well resolved and nearly edge on. While the misalignment angle between the disk and the orbit is weakly constrained due to the short orbital arc available, the data strongly support a (near) coplanar geometry for the system. Furthermore, we find that the models that describe the ALMA data best have an inner radius that is close to the companion's semi-major axis. Including a posteriori knowledge of the system's SED further narrows the constraints on the disk's inner radius and place it at a location that is in reasonable agreement with, possibly interior to, predictions from existing dynamical models of disk truncation by an interior substellar companion. HR\,2562 has the potential over the next few years to become a new testbed for dynamical interaction between a debris disk and a substellar companion., Comment: 22 pages, 17 figures

Inferring traffic object such as lane information is of foremost importance for deployment of autonomous driving. Previous approaches focus on offline construction of HD map inferred with GPS localization, which is insufficient for globally scalable autonomous driving. To alleviate these issues, we propose online HD map learning framework that detects HD map elements from onboard sensor observations. We represent the map elements as a graph; we propose InstaGraM, instance-level graph modeling of HD map that brings accurate and fast end-to-end vectorized HD map learning. Along with the graph modeling strategy, we propose end-to-end neural network composed of three stages: a unified BEV feature extraction, map graph component detection, and association via graph neural networks. Comprehensive experiments on public open dataset show that our proposed network outperforms previous models by up to 13.7 mAP with up to 33.8X faster computation time., Comment: Workshop on Vision-Centric Autonomous Driving (VCAD) at Conference on Computer Vision and Pattern Recognition (CVPR) 2023

Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences. To produce beautiful composites, the camera needs to be geometrically calibrated, which can be tedious and requires a physical calibration target. In place of the traditional multi-image calibration process, we propose to infer the camera calibration parameters such as pitch, roll, field of view, and lens distortion directly from a single image using a deep convolutional neural network. We train this network using automatically generated samples from a large-scale panorama dataset, yielding competitive accuracy in terms of standard `2 error. However, we argue that minimizing such standard error metrics might not be optimal for many applications. In this work, we investigate human sensitivity to inaccuracies in geometric camera calibration. To this end, we conduct a large-scale human perception study where we ask participants to judge the realism of 3D objects composited with correct and biased camera calibration parameters. Based on this study, we develop a new perceptual measure for camera calibration and demonstrate that our deep calibration network outperforms previous single-image based calibration methods both on standard metrics as well as on this novel perceptual measure. Finally, we demonstrate the use of our calibration network for several applications, including virtual object insertion, image retrieval, and compositing. A demonstration of our approach is available at https://lvsn.github.io/deepcalib ., Comment: 12 pages, 12 figures, project page (including live demo) available at https://lvsn.github.io/deepcalib. arXiv admin note: text overlap with arXiv:1712.01259

Long term precise radial velocity (RV) monitoring of the nearby star HD206893, as well as anomalies in the system proper motion, have suggested the presence of an additional, inner companion in the system. Here we describe the results of a multi-epoch search for the companion responsible for this RV drift and proper motion anomaly using the VLTI/GRAVITY instrument. Utilizing information from ongoing precision RV measurements with the HARPS spectrograph, as well as Gaia host star astrometry, we report a high significance detection of the companion HD206893c over three epochs, with clear evidence for Keplerian orbital motion. Our astrometry with $\sim$50-100 $\mu$arcsec precision afforded by GRAVITY allows us to derive a dynamical mass of 12.7$^{+1.2}_{-1.0}$ M$_{\rm Jup}$ and an orbital separation of 3.53$^{+0.08}_{-0.06}$ au for HD206893c. Our fits to the orbits of both companions in the system utilize both Gaia astrometry and RVs to also provide a precise dynamical estimate of the previously uncertain mass of the B component, and therefore derive an age of $155\pm15$ Myr. We find that theoretical atmospheric/evolutionary models incorporating deuterium burning for HD206893c, parameterized by cloudy atmospheres provide a good simultaneous fit to the luminosity of both HD206893B and c. In addition to utilizing long-term RV information, this effort is an early example of a direct imaging discovery of a bona fide exoplanet that was guided in part with Gaia astrometry. Utilizing Gaia astrometry is expected to be one of the primary techniques going forward to identify and characterize additional directly imaged planets. Lastly, this discovery is another example of the power of optical interferometry to directly detect and characterize extrasolar planets where they form at ice-line orbital separations of 2-4\,au., Comment: Accepted to A&A

Training models dedicated to semantic segmentation requires a large amount of pixel-wise annotated data. Due to their costly nature, these annotations might not be available for the task at hand. To alleviate this problem, unsupervised domain adaptation approaches aim at aligning the feature distributions between the labeled source and the unlabeled target data. While these strategies lead to noticeable improvements, their effectiveness remains limited. To guide the domain adaptation task more efficiently, previous works attempted to include human interactions in this process under the form of sparse single-pixel annotations in the target data. In this work, we propose a new domain adaptation framework for semantic segmentation with annotated points via active selection. First, we conduct an unsupervised domain adaptation of the model; from this adaptation, we use an entropy-based uncertainty measurement for target points selection. Finally, to minimize the domain gap, we propose a domain adaptation framework utilizing these target points annotated by human annotators. Experimental results on benchmark datasets show the effectiveness of our methods against existing unsupervised domain adaptation approaches. The propose pipeline is generic and can be included as an extra module to existing domain adaptation strategies.

The direct characterization of exoplanetary systems with high contrast imaging is among the highest priorities for the broader exoplanet community. As large space missions will be necessary for detecting and characterizing exo-Earth twins, developing the techniques and technology for direct imaging of exoplanets is a driving focus for the community. For the first time, JWST will directly observe extrasolar planets at mid-infrared wavelengths beyond 5$\mu$m, deliver detailed spectroscopy revealing much more precise chemical abundances and atmospheric conditions, and provide sensitivity to analogs of our solar system ice-giant planets at wide orbital separations, an entirely new class of exoplanet. However, in order to maximise the scientific output over the lifetime of the mission, an exquisite understanding of the instrumental performance of JWST is needed as early in the mission as possible. In this paper, we describe our 55-hour Early Release Science Program that will utilize all four JWST instruments to extend the characterisation of planetary mass companions to $\sim$15$\mu$m as well as image a circumstellar disk in the mid-infrared with unprecedented sensitivity. Our program will also assess the performance of the observatory in the key modes expected to be commonly used for exoplanet direct imaging and spectroscopy, optimize data calibration and processing, and generate representative datasets that will enable a broad user base to effectively plan for general observing programs in future cycles., Comment: 21 pages, 7 figures. Accepted for Publication in PASP

X-ray binaries provide exceptional laboratories for understanding the physics of matter under the most extreme conditions. Until recently, there were few, if any, observational constraints on the circumbinary environments of X-ray binaries at $\sim$ 100-5000 AU scales; it remains unclear how the accretion onto the compact objects or the explosions giving rise to the compact objects interact with their immediate surroundings. Here, we present the first high-contrast adaptive optics images of X-ray binaries. These observations target all X-ray binaries within $\sim$ 3 kpc accessible with the Keck/NIRC2 vortex coronagraph. This paper focuses on one of the first key results from this campaign: our images reveal the presence of 21 sources potentially associated with the $\gamma$ Cassiopeiae analog high-mass X-ray binary RX J1744.7$-$2713. By conducting different analyses - a preliminary proper motion analysis, a color-magnitude diagram and a probability of chance alignment calculation - we found that three of these 21 sources have a high probability of being bound to the system. If confirmed, they would be in wide orbits ($\sim$ 450 AU to 2500 AU). While follow-up astrometric observations will be needed in $\sim$ 5-10 years to confirm further the bound nature of these detections, these discoveries emphasize that such observations may provide a major breakthrough in the field. In fact, they would be useful not only for our understanding of stellar multiplicity but also for our understanding of how planets, brown dwarfs and stars can form even in the most extreme environments., Comment: Accepted for publication in AJ

In this thesis, we propose a pioneering work on sparse keypoints tracking across images using transformer networks. While deep learning-based keypoints matching have been widely investigated using graph neural networks - and more recently transformer networks, they remain relatively too slow to operate in real-time and are particularly sensitive to the poor repeatability of the keypoints detectors. In order to address these shortcomings, we propose to study the particular case of real-time and robust keypoints tracking. Specifically, we propose a novel architecture which ensures a fast and robust estimation of the keypoints tracking between successive images of a video sequence. Our method takes advantage of a recent breakthrough in computer vision, namely, visual transformer networks. Our method consists of two successive stages, a coarse matching followed by a fine localization of the keypoints' correspondences prediction. Through various experiments, we demonstrate that our approach achieves competitive results and demonstrates high robustness against adverse conditions, such as illumination change, occlusion and viewpoint differences.

Point cloud obtained from 3D scanning is often sparse, noisy, and irregular. To cope with these issues, recent studies have been separately conducted to densify, denoise, and complete inaccurate point cloud. In this paper, we advocate that jointly solving these tasks leads to significant improvement for point cloud reconstruction. To this end, we propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse stacked-hourglass network as for the initial densification and denoising, 2) a refinement via transformers converting the discrete voxels into 3D points. In particular, we further improve the performance of transformer by a newly proposed module called amplified positional encoding. This module has been designed to differently amplify the magnitude of positional encoding vectors based on the points' distances for adaptive refinements. Extensive experiments demonstrate that our network achieves state-of-the-art performance among the recent studies in the ScanNet, ICL-NUIM, and ShapeNetPart datasets. Moreover, we underline the ability of our network to generalize toward real-world and unmet scenes., Comment: ICLR 2022 accepted

MLP-Mixer has newly appeared as a new challenger against the realm of CNNs and transformer. Despite its simplicity compared to transformer, the concept of channel-mixing MLPs and token-mixing MLPs achieves noticeable performance in visual recognition tasks. Unlike images, point clouds are inherently sparse, unordered and irregular, which limits the direct use of MLP-Mixer for point cloud understanding. In this paper, we propose PointMixer, a universal point set operator that facilitates information sharing among unstructured 3D points. By simply replacing token-mixing MLPs with a softmax function, PointMixer can "mix" features within/between point sets. By doing so, PointMixer can be broadly used in the network as inter-set mixing, intra-set mixing, and pyramid mixing. Extensive experiments show the competitive or superior performance of PointMixer in semantic segmentation, classification, and point reconstruction against transformer-based methods., Comment: Accepted to ECCV 2022

Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task that often relies on the so-called scene rigidity assumption. When observing a dynamic environment, this assumption is violated which leads to an ambiguity between the ego-motion of the camera and the motion of the objects. To solve this problem, we present a self-supervised learning framework for 3D object motion field estimation from monocular videos. Our contributions are two-fold. First, we propose a two-stage projection pipeline to explicitly disentangle the camera ego-motion and the object motions with dynamics attention module, called DAM. Specifically, we design an integrated motion model that estimates the motion of the camera and object in the first and second warping stages, respectively, controlled by the attention module through a shared motion encoder. Second, we propose an object motion field estimation through contrastive sample consensus, called CSAC, taking advantage of weak semantic prior (bounding box from an object detector) and geometric constraints (each object respects the rigid body motion model). Experiments on KITTI, Cityscapes, and Waymo Open Dataset demonstrate the relevance of our approach and show that our method outperforms state-of-the-art algorithms for the tasks of self-supervised monocular depth estimation, object motion segmentation, monocular scene flow estimation, and visual odometry., Comment: ICCV 2021

We aim to demonstrate that the presence and mass of an exoplanet can now be effectively derived from the astrometry of another exoplanet. We combined previous astrometry of $\beta$ Pictoris b with a new set of observations from the GRAVITY interferometer. The orbital motion of $\beta$ Pictoris b is fit using Markov chain Monte Carlo simulations in Jacobi coordinates. The inner planet, $\beta$ Pictoris c, was also reobserved at a separation of 96\,mas, confirming the previous orbital estimations. From the astrometry of planet b only, we can (i) detect the presence of $\beta$ Pictoris c and (ii) constrain its mass to $10.04^{+4.53}_{-3.10}\,M_{\rm Jup}$. If one adds the astrometry of $\beta$ Pictoris c, the mass is narrowed down to $9.15^{+1.08}_{-1.06}\,M_{\rm Jup}$. The inclusion of radial velocity measurements does not affect the orbital parameters significantly, but it does slightly decrease the mass estimate to $8.89^{+0.75}_{-0.75}\,M_{\rm Jup}$. With a semimajor axis of $2.68\pm0.02$\,au, a period of $1221\pm15$ days, and an eccentricity of $0.32\pm0.02$, the orbital parameters of $\beta$ Pictoris c are now constrained as precisely as those of $\beta$ Pictoris b. The orbital configuration is compatible with a high-order mean-motion resonance (7:1). The impact of the resonance on the planets' dynamics would then be negligible with respect to the secular perturbations, which might have played an important role in the eccentricity excitation of the outer planet., Comment: Accepted in A&A Letter

Since the rise of the SARS-CoV-2 pandemic in the winter of 2019, the need for an affordable and efficient drug has not yet been met. Leveraging its unique, fast and precise binding free energy prediction technology, Aqemia screened and ranked FDA-approved molecules against the 3ClPro protein. This protease is key to the post-translational modification of two polyproteins produced by the viral genome. We propose in our top 10 predicted molecules some drugs or prodrugs that could be repurposed and used in the treatment of COVID cases.

To reconstruct a 3D scene from a set of calibrated views, traditional multi-view stereo techniques rely on two distinct stages: local depth maps computation and global depth maps fusion. Recent studies concentrate on deep neural architectures for depth estimation by using conventional depth fusion method or direct 3D reconstruction network by regressing Truncated Signed Distance Function (TSDF). In this paper, we advocate that replicating the traditional two stages framework with deep neural networks improves both the interpretability and the accuracy of the results. As mentioned, our network operates in two steps: 1) the local computation of the local depth maps with a deep MVS technique, and, 2) the depth maps and images' features fusion to build a single TSDF volume. In order to improve the matching performance between images acquired from very different viewpoints (e.g., large-baseline and rotations), we introduce a rotation-invariant 3D convolution kernel called PosedConv. The effectiveness of the proposed architecture is underlined via a large series of experiments conducted on the ScanNet dataset where our approach compares favorably against both traditional and deep learning techniques., Comment: ICCV 2021 Accepted

We aim to reveal the nature of the reddest known substellar companion HD 206893 B by studying its near-infrared colors and spectral morphology and by investigating its orbital motion. We fit atmospheric models for giant planets and brown dwarfs and perform spectral retrievals with petitRADTRANS and ATMO on the observed GRAVITY, SPHERE, and GPI spectra of HD 206893 B. To recover its unusual spectral features, we include additional extinction by high-altitude dust clouds made of enstatite grains in the atmospheric model fits. We also infer the orbital parameters of HD 206893 B by combining the $\sim 100~\mu\text{as}$ precision astrometry from GRAVITY with data from the literature and constrain the mass and position of HD 206893 C based on the Gaia proper motion anomaly of the system. The extremely red color and the very shallow $1.4~\mu\text{m}$ water absorption feature of HD 206893 B can be fit well with the adapted atmospheric models and spectral retrievals. Altogether, our analysis suggests an age of $\sim 3$-$300~\text{Myr}$ and a mass of $\sim 5$-$30~\text{M}_\text{Jup}$ for HD 206893 B, which is consistent with previous estimates but extends the parameter space to younger and lower-mass objects. The GRAVITY astrometry points to an eccentric orbit ($e = 0.29^{+0.06}_{-0.11}$) with a mutual inclination of $< 34.4~\text{deg}$ with respect to the debris disk of the system. While HD 206893 B could in principle be a planetary-mass companion, this possibility hinges on the unknown influence of the inner companion on the mass estimate of $10^{+5}_{-4}~\text{M}_\text{Jup}$ from radial velocity and Gaia as well as a relatively small but significant Argus moving group membership probability of $\sim 61\%$. However, we find that if the mass of HD 206893 B is $< 30~\text{M}_\text{Jup}$, then the inner companion HD 206893 C should have a mass between $\sim 8$-$15~\text{M}_\text{Jup}$., Comment: 28 pages, 19 figures, accepted for publication in A&A

The direct detection of exoplanets with high-contrast instruments can be boosted with high spectral resolution. For integral field spectrographs yielding hyperspectral data, this means that the field of view consists of diffracted starlight spectra and a spatially localized planet. Analysis usually relies on cross-correlation with theoretical spectra. In a purely blind-search context, this supervised strategy can be biased with model mismatch and/or be computationally inefficient. Using an approach that is inspired by the remote-sensing community, we aim to propose an alternative to cross-correlation that is fully data-driven, which decomposes the data into a set of individual spectra and their corresponding spatial distributions. This strategy is called spectral unmixing. We used an orthogonal subspace projection to identify the most distinct spectra in the field of view. Their spatial distribution maps were then obtained by inverting the data. These spectra were then used to break the original hyperspectral images into their corresponding spatial distribution maps via non-negative least squares. The performance of our method was evaluated and compared with a cross-correlation using simulated hyperspectral data with medium resolution from the ELT/HARMONI integral field spectrograph. We show that spectral unmixing effectively leads to a planet detection solely based on spectral dissimilarities at significantly reduced computational cost. The extracted spectrum holds significant signatures of the planet while being not perfectly separated from residual starlight. The sensitivity of the supervised cross-correlation is three to four times higher than with unsupervised spectral unmixing, the gap is biased toward the former because the injected and correlated spectrum match perfectly. The algorithm was furthermore vetted on real data obtained with VLT/SINFONI of the beta Pictoris system., Comment: 13 pages, 8 figures

We propose a novel framework to generate clean video frames from a single motion-blurred image. While a broad range of literature focuses on recovering a single image from a blurred image, in this work, we tackle a more challenging task i.e. video restoration from a blurred image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors, and the blurred image as an observation. Our framework is based on an encoder-decoder structure with spatial transformer network modules to restore a video sequence and its underlying motion in an end-to-end manner. We design a loss function and regularizers with complementary properties to stabilize the training and analyze variant models of the proposed network. The effectiveness and transferability of our network are highlighted through a large set of experiments on two different types of datasets: camera rotation blurs generated from panorama scenes and dynamic motion blurs in high speed videos., Comment: Accepted to CVPRW, NTIRE 2021

This paper presents a stereo object matching method that exploits both 2D contextual information from images as well as 3D object-level information. Unlike existing stereo matching methods that exclusively focus on the pixel-level correspondence between stereo images within a volumetric space (i.e., cost volume), we exploit this volumetric structure in a different manner. The cost volume explicitly encompasses 3D information along its disparity axis, therefore it is a privileged structure that can encapsulate the 3D contextual information from objects. However, it is not straightforward since the disparity values map the 3D metric space in a non-linear fashion. Thus, we present two novel strategies to handle 3D objectness in the cost volume space: selective sampling (RoISelect) and 2D-3D fusion (fusion-by-occupancy), which allow us to seamlessly incorporate 3D object-level information and achieve accurate depth performance near the object boundary regions. Our depth estimation achieves competitive performance in the KITTI dataset and the Virtual-KITTI 2.0 dataset., Comment: Accepted at ICRA 2021

In most of computer vision applications, motion blur is regarded as an undesirable artifact. However, it has been shown that motion blur in an image may have practical interests in fundamental computer vision problems. In this work, we propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner. We design our network with transformer networks to learn globally and locally varying motions from encoded features of a motion-blurred input, and decode left and right frame features without explicit frame supervision. A flow estimator network is then used to estimate optical flow from the decoded features in a coarse-to-fine manner. We qualitatively and quantitatively evaluate our model through a large set of experiments on synthetic and real motion-blur datasets. We also provide in-depth analysis of our model in connection with related approaches to highlight the effectiveness and favorability of our approach. Furthermore, we showcase the applicability of the flow estimated by our method on deblurring and moving object segmentation tasks., Comment: Accepted to AAAI 2021

Abrupt motion of camera or objects in a scene result in a blurry video, and therefore recovering high quality video requires two types of enhancements: visual enhancement and temporal upsampling. A broad range of research attempted to recover clean frames from blurred image sequences or temporally upsample frames by interpolation, yet there are very limited studies handling both problems jointly. In this work, we present a novel framework for deblurring, interpolating and extrapolating sharp frames from a motion-blurred video in an end-to-end manner. We design our framework by first learning the pixel-level motion that caused the blur from the given inputs via optical flow estimation and then predict multiple clean frames by warping the decoded features with the estimated flows. To ensure temporal coherence across predicted frames and address potential temporal ambiguity, we propose a simple, yet effective flow-based rule. The effectiveness and favorability of our approach are highlighted through extensive qualitative and quantitative evaluations on motion-blurred datasets from high speed videos., Comment: Accepted to AAAI 2021