Search

Image Inpainting, the task of recovering missing information in an image, is of great practical interest to users who wish to recover damaged photographs or remove unwanted parts of images. Advances in Convolutional Neural Networks and Adversarial Networks have produced large gains in this problem. Nevertheless, Inpainting algorithms may fail to recover a decent image, and even produce jarring artifacts. In this work, we seek to uncover what causes such issues. We investigate the relationship between the artifacts and the type of degradation mask, the size of the mask, and the algorithms used. To that end, we employ rectangular, free-form and segmentation-based degradation masks, of varying sizes. We find that large, continuous regions of missing information are often to blame for the most egregious failures, and not really how complex the shape of the region is. Moreover, we observe that no algorithm is completely immune to these failure modes.

3D reconstruction is one of the main topics in computer vision and is heavily applied for creating virtual environments. Photogrammetry is a technique for obtaining 3D information by mapping objects and scenarios using only images. However, this process can take a long time when using large datasets. In this paper, a permissive open-source pipeline is proposed focusing on robustness, efficiency, and low execution time in batch processing. The permissive license allows commercial use without the need of keeping the code open. We mixed an enhanced structure from motion algorithm and a recurrent multi-view reconstruction. We also use Point Cloud Library for normal estimation, surface reconstruction, and texture mapping. We compared our results with COLMAP and MVE techniques using the DTU MVS dataset and real-world scenarios with our own datasets. The results showed a decrease of 69.4% on average time (compared to the best result of other techniques), but also demonstrated the need for more images to generate a complete reconstructed model.

Image-based body tracking algorithms are useful in several scenarios, such as avatar animations and gesture interaction for VR applications. In the last few years, the best-ranked solutions presented on the state of the art of body tracking (according to the most popular datasets in the field) are intensively based on Convolutional Neural Networks (CNNs) algorithms and use large datasets for training and validation. Although these solutions achieve high precision scores while evaluated with some of these datasets, there are particular tracking challenges (for example, upside-down cases) that are not well-modeled and, therefore, not correctly tracked. Instead of lurking an all-in-one solution for all cases, we propose HuTrain, a framework for creating datasets quickly and easily. HuTrain comprises a series of steps, including automatic camera calibration, refined human pose estimation, and known dataset formats conversion. We show that, with our system, the user can generate human pose datasets, targeting specific tracking challenges for the desired application context, with no need to annotate human pose instances manually.

In the past years, captures made by dual-fisheye lens cameras have been used for virtual reality, 360 broadcasting and many other applications. For these scenarios, to provide a good- quality experience, the alignment of the boundaries between the two images to be stitched must be done properly. However, due to the peculiar design of dual-fisheye cameras and the high variance between different captured scenes, the stitching process can be very challenging. In this work, we present a 360 stitching solution based on feature cluster matching. It is an adaptive stitching technique based on the extraction of feature cluster templates from the stitching region. It is proposed an alignment based on the template matching of these clusters, successfully reducing the discontinuities in the full-view panorama. We evaluate our method on a dataset built from captures made with an existing camera of this kind, the Samsung's Gear 360. It is also described how we can extend these concepts from image stitching to video stitching using the temporal information of the media. Finally, we show that our matching method outperforms a state-of-the-art matching technique for image and video stitching.

In this work we address the problem of landmark recognition. We extend PlaNet, a model based on deep neural networks that approaches the problem of landmark recognition as a classification problem and performs the recognition of places around the world. We propose an extension of the PlaNet technique in which we use a voting scheme to perform the classification, dividing the image into previously defined regions and inferring the landmark based on these regions. The prediction of the model depends not only on the information of the features learned by the deep convolutional neural network architecture during training, but also uses local information from each region in the image for which the classification is made. To validate our proposal, we performed the training of the original PlaNet model and our variation using a database built with images from Flickr, and evaluated the models in the Paris and Oxford Buildings datasets. It was possible to notice that the addition of image division and voting structure improves the accuracy result of the model by 5-11 percentage points on average, reducing the level of ambiguity found during the inference of the model.