Search

We present the design of an entire on-device system for large-scale urban localization using images. The proposed design integrates compact image retrieval and 2D-3D correspondence search to estimate the location in extensive city regions. Our design is GPS agnostic and does not require network connection. In order to overcome the resource constraints of mobile devices, we propose a system design that leverages the scalability advantage of image retrieval and accuracy of 3D model-based localization. Furthermore, we propose a new hashing-based cascade search for fast computation of 2D-3D correspondences. In addition, we propose a new one-many RANSAC for accurate pose estimation. The new one-many RANSAC addresses the challenge of repetitive building structures (e.g. windows and balconies) in urban localization. Extensive experiments demonstrate that our 2D-3D correspondence search achieves the state-of-the-art localization accuracy on multiple benchmark datasets. Furthermore, our experiments on a large Google street view image dataset show the potential of large-scale localization entirely on a typical mobile device.

Recent successes in Generative Adversarial Networks (GAN) have affirmed the importance of using more data in GAN training. Yet it is expensive to collect data in many domains such as medical applications. Data Augmentation (DA) has been applied in these applications. In this work, we first argue that the classical DA approach could mislead the generator to learn the distribution of the augmented data, which could be different from that of the original data. We then propose a principled framework, termed Data Augmentation Optimized for GAN (DAG), to enable the use of augmented data in GAN training to improve the learning of the original distribution. We provide theoretical analysis to show that using our proposed DAG aligns with the original GAN in minimizing the Jensen-Shannon (JS) divergence between the original distribution and model distribution. Importantly, the proposed DAG effectively leverages the augmented data to improve the learning of discriminator and generator. We conduct experiments to apply DAG to different GAN models: unconditional GAN, conditional GAN, self-supervised GAN and CycleGAN using datasets of natural images and medical images. The results show that DAG achieves consistent and considerable improvements across these models. Furthermore, when DAG is used in some GAN models, the system establishes state-of-the-art Frechet Inception Distance (FID) scores. Our code is available., Comment: Accepted in IEEE Transactions on Image Processing

While Generative Adversarial Networks (GANs) are fundamental to many generative modelling applications, they suffer from numerous issues. In this work, we propose a principled framework to simultaneously mitigate two fundamental issues in GANs: catastrophic forgetting of the discriminator and mode collapse of the generator. We achieve this by employing for GANs a contrastive learning and mutual information maximization approach, and perform extensive analyses to understand sources of improvements. Our approach significantly stabilizes GAN training and improves GAN performance for image synthesis across five datasets under the same training and evaluation conditions against state-of-the-art works. In particular, compared to the state-of-the-art SSGAN, our approach does not suffer from poorer performance on image domains such as faces, and instead improves performance significantly. Our approach is simple to implement and practical: it involves only one auxiliary objective, has a low computational cost, and performs robustly across a wide range of training settings and datasets without any hyperparameter tuning. For reproducibility, our code is available in Mimicry: https://github.com/kwotsin/mimicry., Comment: Accepted to WACV 2021. An initial version was accepted to NeurIPS 2019 Workshop on Information Theory and Machine Learning

We introduce effective training algorithms for Generative Adversarial Networks (GAN) to alleviate mode collapse and gradient vanishing. In our system, we constrain the generator by an Autoencoder (AE). We propose a formulation to consider the reconstructed samples from AE as “real” samples for the discriminator. This couples the convergence of the AE with that of the discriminator, effectively slowing down the convergence of discriminator and reducing gradient vanishing. Importantly, we propose two novel distance constraints to improve the generator. First, we propose a latent-data distance constraint to enforce compatibility between the latent sample distances and the corresponding data sample distances. We use this constraint to explicitly prevent the generator from mode collapse. Second, we propose a discriminator-score distance constraint to align the distribution of the generated samples with that of the real samples through the discriminator score. We use this constraint to guide the generator to synthesize samples that resemble the real ones. Our proposed GAN using these distance constraints, namely Dist-GAN, can achieve better results than state-of-the-art methods across benchmark datasets: synthetic, MNIST, MNIST-1K, CelebA, CIFAR-10 and STL-10 datasets. Our code is published here (https://github.com/tntrung/gan) for research.

We propose two new techniques for training Generative Adversarial Networks (GANs). Our objectives are to alleviate mode collapse in GAN and improve the quality of the generated samples. First, we propose neighbor embedding, a manifold learning-based regularization to explicitly retain local structures of latent samples in the generated samples. This prevents generator from producing nearly identical data samples from different latent samples, and reduces mode collapse. We propose an inverse t-SNE regularizer to achieve this. Second, we propose a new technique, gradient matching, to align the distributions of the generated samples and the real samples. As it is challenging to work with high-dimensional sample distributions, we propose to align these distributions through the scalar discriminator scores. We constrain the difference between the discriminator scores of the real samples and generated ones. We further constrain the difference between the gradients of these discriminator scores. We derive these constraints from Taylor approximations of the discriminator function. We perform experiments to demonstrate that our proposed techniques are computationally simple and easy to be incorporated in existing systems. When Gradient matching and Neighbour embedding are applied together, our GN-GAN achieves outstanding results on 1D/2D synthetic, CIFAR-10 and STL-10 datasets, e.g. FID score of $30.80$ for the STL-10 dataset. Our code is available at: https://github.com/tntrung/gan, Comment: Published as a conference paper at AAAI 2019

International audience; 3D face tracking using one monocular camera is an important topic, since it is useful in many domains such as: video surveillance system, human machine interaction, biometrics, etc. In this paper, we propose a new 3D face tracking which is robust to large head rotations. Underlying cascaded regression approach for 2D landmark detection, we build an extension in context of 3D pose tracking. To better work with out-of-plane issues, we extend the training dataset by including a new set of synthetic images. For evaluation, we propose to use a new recording system to capture automatically face pose ground-truth, and create a new test dataset, named U3PT (Unconstrained 3D Pose Tracking). Theperformance of our method along with the state-of-the-art methods are carried out to analyze advantage as well as limitations need to be improved in the future.

La réforme agraire qui fait suite à l’indépendance du Viêt-Nam met fin à la propriété individuelle des terres. La terre est déclarée propriété du peuple vietnamien et un système de coopératives est mis en place. Entre la fin des années 70 et le début des années 80, ce système rentre dans une phase de crise caractérisée par une baisse de la production rizicole et une démotivation des coopérateurs (Jésus et Dao Thê Anh, 1998). Deux réformes successives en 1982 (« résolution 100 ») et 1986 (« co...

Our method can track simultaneously rigid and non-rigid face using synthetic data.It is robust with fast movements, illumination, partly occlusion.It can recover in term of lost tracking without waiting for the frontal face reset.It can keep tracking the face up to profile (even 90?) and complex motions.It is comparable to state-of-the-art methods in terms of pose and landmark tracking. This paper presents a robust framework for simultaneously tracking rigid pose and non-rigid animation of a single face with a monocular camera. Our proposed method consists of two phases: training and tracking. In the training phase, using automatically detected landmarks and the three-dimensional face model Candide-3, we built a cohort of synthetic face examples with a large range of the three axial rotations. The representation of a face's appearance is a set of local patches of landmarks that are characterized by Scale Invariant Feature Transform (SIFT) descriptors. In the tracking phase, we propose an original approach combining geometric and appearance models. The purpose of the geometric model is to provide a SIFT baseline matching between the current frame and an adaptive set of keyframes for rigid parameter estimation. The appearance model uses nearest synthetic examples of the training set to re-estimate rigid and non-rigid parameters. We found a tracking capability up to 90? of vertical axial rotation, and our method is robust even in the presence of fast movements, illumination changes and tracking losses. Numerical results on the rigid and non-rigid parameter sets are reported using several annotated public databases. Compared to other published algorithms, our method provides an excellent compromise between rigid and non-rigid parameter accuracies. The approach has some potential, providing good pose estimation (average error less than 4? on the Boston University Face Tracking dataset) and landmark tracking precision (6.3?pixel error compared to 6.8 of one of state-of-the-art methods on Talking Face video).

International audience; Face alignment is a fundamental problem in computer vision to localize the landmarks of eyes, nose or mouth in 2D images. In this paper, our method for face alignment integrates three aspects not seen in previous approaches: First, learning local descriptors using Restricted Boltzmann Machine (RBM) to model the local appearance of each facial points independently. Second, proposing the coarse-to-fine regression to localize the landmarks after the estimation of the shape configuration via global regression. Third, and using synthetic data as training data to enable our approach to work better with the profile view, and to forego the need of increasing the number of annotations for training. Our results on challenging datasets compare favorably with state of the art results. The combination with the synthetic data allows our method yielding good results in profile alignment. That highlights the potential of using synthetic data for in-the-wild face alignment.

International audience; 3D pose tracking using monocular cameras is an important topic, which has been receiving a great attention since last decades. It is useful in many domains such as: Video Surveillance, Human-Computer Interface, Biometrics, etc. The problem gets much challenging if occurring, for example, fast motion, out-of-plane rotation, the illumination changes, expression, or occlusions. In the literature, there are some datasets reported for 3D pose tracking evaluation, however, all of them retains simple background, no-expression, slow motion, frontal rotation, or no-occlusion. It is not enough to test advances of in-the-wild tracking. Indeed, collecting accurate ground-truth of 3D pose is difficult because some special devices or sensors are required. In addition, the magnetic sensors usually used for 3D pose ground-truth, is uncomfortable to wear and move because of their wires. In this paper, we propose a new recording system that allows people move more comfortable. We create a new challenging dataset, named U3PT (Unconstrained 3D Pose Tracking). It could be considered as a benchmark to evaluate and compare the robustness and precision of state-of-the-art methods that aims to work in-the-wild. This paper will also present the performances of two well-known state-of-the-art methods compared to our method on face tracking when applied to this database. We have carried out several experiments and have reported advantages and some limitations to be improved in the future.

International audience; The non-rigid face tracking has been achieved many advances in recent years, but most of empirical experiments are restricted at near-frontal face. This report introduces a robust framework for pose-free tracking of non-rigid face. Our method consists of two phases: training and tracking. In the training phase, a large offline synthesized database is built to train landmark appearance models using linear Support Vector Machine (SVM). In the tracking phase, a two-step approach is proposed: the first step, namely initialization, benefits 2D SIFT matching between the current frame and a set of adaptive keyframes to estimate the rigid parameters. The second step obtains the whole set of parameters (rigid and non-rigid) using a heuristic method via pose-wise SVMs. The combination of these aspects makes our method work robustly up to 90 degree of vertical axial rotation. Moreover, our method appears to be robust even in the presence of fast movements and tracking losses. Comparing to other published algorithms, our method offers a very good compromise of rigid and non-rigid parameter accuracies. This study gives a promising perspective because of the good results in terms of pose estimation (average error is less than 4 o on BUFT dataset) and landmark tracking precision (5.8 pixel error compared to 6.8 of one state-of-the-art method on Talking Face video). These results highlight the potential of using synthetic data to track non-rigid face in unconstrained poses.

International audience; This paper propose a new method for wide-rotation non-rigid face tracking that is still a challenging problem in computer vision community. Our method consists of training and tracking phases. In training, we propose to use a large off-line synthetic database to overcome the problem of data collection. The local appearance models are then trained using linear Support Vector Machine (SVM). In tracking, we propose a two-step approach: (i) The first step uses baseline matching for a good initialization. The matching strategy between the current frame and a set of adaptive keyframes is also involved to be recoverable in terms of failed tracking. (ii) The second step estimates the model parameters using a heuristic method via pose-wise SVMs. The combination makes our approach work robustly with wide rotation, up to 90∘ of vertical axis. In addition, our method appears to be robust even in the presence of fast movements thanks to baseline matching. Compared to state-of-the-art methods, our method shows a good compromise of rigid and non-rigid parameter accuracies. This study gives a promising perspective because of the good results in terms of pose estimation (average error is less than 4o on BUFT dataset) and landmark tracking precision (5.8 pixel error compared to 6.8 of one state-of-the-art method on Talking Face video. These results highlight the potential of using synthetic data to track non-rigid face in unconstrained poses

There is an amount of designed features (SIFT, SURF, or DAISY) which has been chosen in the standard implementation of some visual recognition and multimedia challenges. The power of these features lie on their invariance designed against rotation, scaling, and translation. Recent trends in deep learning, however, have pointed out that data-driven features learning performs better designed features in some tasks, since they can capture the global (via multi-layers network) or inter-local structures (convolutional network) of images. We argue that combining the two types of features can significantly improve visual object recognition performance. We propose in this paper a framework that uses sparse coding and the fusion of learned and designed features in order to build descriptive codewords. Evaluations on Caltech-101 and 15 Scenes validates our argument, with a better result compared with recent approaches.

International audience; This paper presents a new method to track both the face pose and the face animation with a monocular camera. The approach is based on the 3D face model CANDIDE and on the SIFT (Scale Invariant Feature Transform) descriptors, extracted around a few given landmarks (26 selected vertices of CANDIDE model) with a Bayesian approach. The training phase is performed on a synthetic database generated from the first video frame. At each current frame, the face pose and animation parameters are estimated via a Bayesian approach, with a Gaussian prior and a Gaussian likelihood function whose the mean and the covariance matrix eigenvalues are updated from the previous frame using eigen decomposition. Numerical results on pose estimation and landmark locations are reported using the Boston University Face Tracking (BUFT) database and Talking Face video. They show that our approach, compared to six other published algorithms, provides a very good compromise and presents a promising perspective due to the good results in terms of landmark localization.

This paper presents a new method to combine descriptors extracted from feature maps of Deconvolutional Networks and SIFT descriptors by converting them into histograms of local patterns, so the concatenation operation can be applied and ensure to increase the classification rate. We use K-means clustering algorithm to construct codebooks and compute Spatial Histograms to represent the distribution of local patterns in an image. Consequently, we can concatenate these histograms to make a new one that represents more local patterns than the originals. In the classification step, SVM associated with Histogram Intersection Kernel is utilized. In the experiments on Scene-15 Dataset containing 15 categories, the classification rates of our method are around 84% which outperforms Reconfigurable Bag-of-Words (RBoW), Sparse Covariance Patterns (SCP), Spatial Pyramid Matching (SPM), Spatial Pyramid Matching using Sparse Coding (ScSPM) and Visual Word Reweighting (VWR).

This paper considers the problem of channel estimation for space-time block code (STBC) systems with convolutional codes for error correction. We propose an iterative processing scheme which combines channel estimation and data detection to improve system performance and increase transmission efficiency. In the proposed scheme, pilot symbols are used for initial estimation of channel gains and data detection. The information bits detected during the data period are fed back to the channel estimator for re-estimating the channels and improving data detection. The proposed scheme allows to improve the BER performance and decrease the pilot length to 50% at the cost of only 2 processing iterations.

This paper presents a simple yet effective channel estimation method for transmit diversity systems. We propose to use the Alamoutipsilas space-time encoding scheme to create pilot symbols and insert into the transmit data symbol streams from the transmit antennas. The orthogonality of the pilot symbols from the two transmit antennas allows to eliminate the interference among transmit antennas. The proposed pilot symbol assisted (PSA) method thus does not require pilot symbol expansion as in the previous methods. We also propose to use a simple temporal average operation to process channel decision statistics to replace temporal filters or interpolators. For a 16 pilot symbols in a 100 data symbol frame transmitted using the Alamoutipsilas space-time block coding, the performance degradation of the estimated symbols is less than 0.2 dB compared with the case of perfect channel estimation.

De par leur richesse en ressources naturelles renouvelables, les zones inondables tropicales revetent un interet social et economique majeur pour les pays en developpement. Cependant, les fleuves tropicaux sont aujourd'hui de plus en plus amenages pour satisfaire les besoins lies a de nouvelles activites. Les zones jusque-la regulierement inondees par la crue annuelle se reduisent ou alors les rythmes de leur inondation sont profondement modifies. Les impacts de tels changements sont nombreux et portent atteinte a la biodiversite et a la durabilite des systemes d'exploitation. Il s'avere alors necessaire de definir de nouvelles approches de la gestion de l'eau, des espaces et des ressources vivantes, qui tout a la fois preservent les ecosystemes et prennent en consideration les besoins des differents usagers. Tel est l'objectif de cet ouvrage qui pose, dans un premier temps, la problematique societale autour de laquelle cette gestion doit etre repensee, en faisant apparaitre la diversite d'acteurs et d'institutions concernes. Il presente ensuite les acquis les plus recents de la recherche sur le fonctionnement de ces ecosystemes ainsi que sur les pratiques et strategies deployees par les populations qui les exploitent. Enfin est abordee la question des instruments a mettre en place pour assurer l'effectivite d'une gestion durable des zones inondables tropicales : apres avoir fait le point sur les apports de la recherche concernant les outils de traitement et de partage de l'information environnementale, l'ouvrage se termine par un debat sur les conditions de creation et de fonctionnement des institutions de suivi, de concertation et de decision. (Resume d'auteur)

International audience; The IRIM group is a consortium of French teams working on Multimedia Indexing and Retrieval. This paper describes its participation to the TRECVID 2012 semantic indexing and instance search tasks. For the semantic indexing task, our approach uses a six-stagesprocessing pipelines for computing scores for the likelihood of a video shot to contain a target concept. These scores are then used for producing a ranked list of images or shots that are the most likely to contain the target concept. The pipeline is composed of the following steps: descriptor extraction, descriptor optimization, classification, fusion of descriptor variants, higher-level fusion, and re-ranking. We evaluated a number of different descriptors and tried dierent fusion strategies. The best IRIM run has a Mean Inferred Average Precision of 0.2378, which ranked us 4th out of 16 participants.For the instance search task, our approach uses two steps. First individual methods of participants are used to compute similrity between an example image of instance and keyframes of a video clip. Then a two-step fusion method is used to combine these individual results and obtain a score for the likelihood of an instance to appear in a video clip. These scores are used to obtain a ranked list of clips the most likely to contain the queried instance. The best IRIM run has a MAP of 0.1192, which ranked us 29th on 79 fully automatic runs.