Search

Although existing monocular depth estimation methods have made great progress, predicting an accurate absolute depth map from a single image is still challenging due to the limited modeling capacity of networks and the scale ambiguity issue. In this paper, we introduce a fully Visual Attention-based Depth (VADepth) network, where spatial attention and channel attention are applied to all stages. By continuously extracting the dependencies of features along the spatial and channel dimensions over a long distance, VADepth network can effectively preserve important details and suppress interfering features to better perceive the scene structure for more accurate depth estimates. In addition, we utilize geometric priors to form scale constraints for scale-aware model training. Specifically, we construct a novel scale-aware loss using the distance between the camera and a plane fitted by the ground points corresponding to the pixels of the rectangular area in the bottom middle of the image. Experimental results on the KITTI dataset show that this architecture achieves the state-of-the-art performance and our method can directly output absolute depth without post-processing. Moreover, our experiments on the SeasonDepth dataset also demonstrate the robustness of our model to multiple unseen environments., Comment: Published on IEEE Robotics and Automation Letters (RA-L)

Aims: There is a spatially resolved star-forming main sequence (rSFMS) and mass-metallicity relation (rMZR) of galaxies in local universe. We know that the global mass-metallicity relation (MZR) results from the integral of rMZR, and it will evolve with the redshift. However, the evolution of rMZR with redshift is still unclear due to the low spatial resolution and signal-to-noise ratio. There are currently too few observations beyond local universe, and only simulations can reproduce the evolution of rMZR with redshift. Methods: In this work, we select ten emission-line galaxies with an average redshift of $z\sim 0.26$ from MUSE-Wide DR1. We obtain the spatially resolved star formation rate (SFR) and metallicity from the integral field spectroscopy (IFS), as well as the stellar mass surface density from the 3D-HST photometry. We derive the rSFMS and rMZR at $z\sim 0.26$ and compare them with local galaxies. Results: We find the rSFMS of galaxies at $z\sim 0.26$ has a slope of $\sim$0.771. The rMZR exists at $z\sim 0.26$, showing a similar shape to the local universe but a lower average metallicity about $\sim$0.11 dex than the local one. In addition, we also study their spatially resolved fundamental metallicity relation (rFMR). However, there is no obvious evidence that rFMR exists at $z\sim$0.26 and it is not an extension of rMZR at a high SFR. Conclusions: Similar to their global versions, the rSFMS and rMZR of galaxies also evolve with redshift. Given the fixed stellar mass, galaxies at higher redshift show higher SFR and lower metallicity. These suggest that the evolution of the global galaxy properties with redshift may result from integrating the evolution of spatially resolved properties of galaxies., Comment: 9 pages, 8 figures, accepted for publication in A&A

Achieving realistic, vivid, and human-like synthesized conversational gestures conditioned on multi-modal data is still an unsolved problem due to the lack of available datasets, models and standard evaluation metrics. To address this, we build Body-Expression-Audio-Text dataset, BEAT, which has i) 76 hours, high-quality, multi-modal data captured from 30 speakers talking with eight different emotions and in four different languages, ii) 32 millions frame-level emotion and semantic relevance annotations. Our statistical analysis on BEAT demonstrates the correlation of conversational gestures with facial expressions, emotions, and semantics, in addition to the known correlation with audio, text, and speaker identity. Based on this observation, we propose a baseline model, Cascaded Motion Network (CaMN), which consists of above six modalities modeled in a cascaded architecture for gesture synthesis. To evaluate the semantic relevancy, we introduce a metric, Semantic Relevance Gesture Recall (SRGR). Qualitative and quantitative experiments demonstrate metrics' validness, ground truth data quality, and baseline's state-of-the-art performance. To the best of our knowledge, BEAT is the largest motion capture dataset for investigating human gestures, which may contribute to a number of different research fields, including controllable gesture synthesis, cross-modality analysis, and emotional gesture recognition. The data, code and model are available on https://pantomatrix.github.io/BEAT/., Comment: 28 pages, 15 figures, Accepted by ECCV2022

The scaling relationship is a fundamental probe of the evolution of galaxies. Using the integral field spectroscopic data from the Mapping Nearby Galaxies at Apache Point Observatory survey, we select 1698 spaxels with significant detection of the auroral emission line \oiii$\lambda$4363 from 52 galaxies to investigate the scaling relationships at the low-metallicity end. We find that our sample's star formation rate is higher and its metallicity is lower in the scaling relationship than the star-forming sequence after removing the contribution of the Fundamental Metallicity Relation.We also find that the stellar ages of our sample are younger ($<$ 1 Gyr) and the stellar metallicities are also lower. Morphological parameters from Deep Learning catalog indicate that our galaxies are more likely to be merger. These results suggest that their low metallicity regions may be related to interaction, the inflow of metal-poor gas may dilute the interstellar medium and form new metal-poor stars in these galaxies during interaction., Comment: 12 pages, 10 figures, accepted in ApJ

Bottom-up based multi-person pose estimation approaches use heatmaps with auxiliary predictions to estimate joint positions and belonging at one time. Recently, various combinations between auxiliary predictions and heatmaps have been proposed for higher performance, these predictions are supervised by the corresponding L2 loss function directly. However, the lack of more explicit supervision results in low features utilization and contradictions between predictions in one model. To solve these problems, this paper proposes (i) a new loss organization method which uses self-supervised heatmaps to reduce prediction contradictions and spatial-sequential attention to enhance networks' features extraction; (ii) a new combination of predictions composed by heatmaps, Part Affinity Fields (PAFs) and our block-inside offsets to fix pixel-level joints positions and further demonstrates the effectiveness of proposed loss function. Experiments are conducted on the MS COCO keypoint dataset and adopting OpenPose as the baseline model. Our method outperforms the baseline overall. On the COCO verification dataset, the mAP of OpenPose trained with our proposals outperforms the OpenPose baseline by over 5.5%.