Search

The task of 3D human-body reconstruction (3DHR), which mostly utilizes parametric pose and shape representations, has witnessed significant advances in recent years. However, the application of 3DHR techniques in handling real-world in-the-wild data, still faces limitations. Training the 3DHR model in such scenario with 3D human pose’s ground truth (GT) is non-trivial. Curating the accurate 3D human pose GT for in-the-wild scenes remains difficult due to various factors. Recent test-time refinement approaches on 3DHR task leverage 2D off-the-shelf human keypoints information to support the lack of 3D supervision on in-the-wild data. However, we observed that additional 2D supervision alone could cause overfitting issue on common 3DHR backbones, making the 3DHR test-time refinement task seem intractable. We answer this challenge by proposing a strategy that complements 3DHR test-time refinement work under a collaborative approach. Specifically, we initially apply a pre-adaptation approach that works by collaborating various 3DHR models in a single framework to directly improve their initial outputs. This approach is then further combined with the test-time adaptation work under specific settings that minimize the overfitting issue to further boost the 3DHR performance. The whole framework is termed as 3DHR-Co, and on the experiment side, the proposed work can significantly enhance the scores of common classic 3DHR backbones up to -34 mm pose error suppression, putting them among the top list on the in-the-wild benchmark data. Such achievement shows that our approach helps unveil the true potential of the common classic 3DHR backbones. Based on these findings, we further investigate various settings on the proposed framework to better elaborate the capability of our collaborative strategy in the 3DHR task.

In this paper, we propose a novel disease detection framework based on translating a given chest X-ray (CXR) to a corresponding normal CXR image. To train a model for normal CXR translation, we synthesize a paired image dataset from existing public CXR datasets. More specifically, we synthesize new images that appear to be from the same patient but without abnormalities, i.e., matching-pairs, that are not present in the dataset. Here, we search for normal local regions in images contained within the dataset that have a similar appearance, and we apply a blending process to effectively erase the abnormal regions. A conditional GAN can then be trained on the synthetic matching-pair images for a normal CXR translator model that transforms the given CXR images into their corresponding normal CXRs. We incorporate this pretrained translator model into a deep learning based object detection model. Here we define the difference between the translated normal CXR and the given input as the residual-saliency map. As the residual-saliency map will be most active at diseased regions, we use this as the attention within the detection model. We also utilize the matching-pair synthesis pipeline to synthesize abnormal images for data augmentation when we train the detector. We evaluate the proposed approach on two public chest X-ray datasets: RSNA and VinBigData.

We formulate the visual tracking problem as a semi-supervised continual learning problem, where only an initial frame is labeled. In contrast to conventional meta-learning based approaches that regard visual tracking as an instance detection problem with a focus on finding good weights for model initialization, we consider both initialization and online update processes simultaneously under our adaptive continual meta-learning framework. The proposed adaptive meta-learning strategy dynamically generates the hyperparameters needed for fast initialization and online update to achieve more robustness via adaptively regulating the learning process. In addition, our continual meta-learning approach based on knowledge distillation scheme helps the tracker adapt to new examples while retaining its knowledge on previously seen examples. We apply our proposed framework to deep learning-based tracking algorithm to obtain noticeable performance gains and competitive results against recent state-of-the-art tracking algorithms while performing at real-time speeds.