Search

International audience; -L'observation de la Terre a été largement facilitée depuis de nombreuses années grâce à l'utilisation des satellites imageurs radar à synthèse d'ouverture (SAR), qui offrent des capacités d'imagerie indépendantes des conditions météorologiques. Toutefois, l'interprétation de ces images SAR est complexe en raison de la présence de bruit inhérent à l'imagerie cohérente. En effet, des fluctuations apparaissent dans les images, notamment là où la réflectivité radar est élevée. Ainsi, de nombreuses méthodes ont été développées pour réduire le bruit présent dans ces images, notamment des méthodes neuronales particulièrement efficaces. Dans cet article, nous proposons d'étudier comment l'ajout d'une donnée auxiliaire comme une image optique peut améliorer la restauration de la réflectivité dans ce cadre.

Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection. However, hardware limitations of real-world systems constrain to smaller ensembles and lower-capacity networks, significantly deteriorating their performance and properties. We introduce Packed-Ensembles (PE), a strategy to design and train lightweight structured ensembles by carefully modulating the dimension of their encoding space. We leverage grouped convolutions to parallelize the ensemble into a single shared backbone and forward pass to improve training and inference speeds. PE is designed to operate within the memory limits of a standard neural network. Our extensive research indicates that PE accurately preserves the properties of DE, such as diversity, and performs equally well in terms of accuracy, calibration, out-of-distribution detection, and robustness to distribution shift. We make our code available at https://github.com/ENSTA-U2IS/torch-uncertainty., Comment: Published as a conference paper at ICLR 2023 (notable 25%)

International audience

In this paper we consider functional data with heterogeneity in time and in population. We propose a mixture model with segmentation of time to represent this heterogeneity while keeping the functional structure. Maximum likelihood estimator is considered, proved to be identifiable and consistent. In practice, an EM algorithm is used, combined with dynamic programming for the maximization step, to approximate the maximum likelihood estimator. The method is illustrated on a simulated dataset, and used on a real dataset of electricity consumption., Comment: 38 pages, 8 figures

Synthetic aperture radar (SAR) images are widely used for Earth observation to complement optical imaging. By combining information on the polarization and the phase shift of the radar echos, SAR images offer high sensitivity to the geometry and materials that compose a scene. This information richness comes with a drawback inherent to all coherent imaging modalities: a strong signal-dependent noise called "speckle". This paper addresses the mathematical issues of performing speckle reduction in a transformed domain: the matrix-log domain. Rather than directly estimating noiseless covariance matrices, recasting the denoising problem in terms of the matrix-log of the covariance matrices stabilizes noise fluctuations and makes it possible to apply off-the-shelf denoising algorithms. We refine the method MuLoG by replacing heuristic procedures with exact expressions and improving the estimation strategy. This corrects a bias of the original method and should facilitate and encourage the adaptation of general-purpose processing methods to SAR imaging.

Self-supervised learning (SSL) has allowed substantial progress in Automatic Speech Recognition (ASR) performance in low-resource settings. In this context, it has been demonstrated that larger self-supervised feature extractors are crucial for achieving lower downstream ASR error rates. Thus, better performance might be sanctioned with longer inferences. This article explores different approaches that may be deployed during the fine-tuning to reduce the computations needed in the SSL encoder, leading to faster inferences. We adapt a number of existing techniques to common ASR settings and benchmark them, displaying performance drops and gains in inference times. Interestingly, we found that given enough downstream data, a simple downsampling of the input sequences outperforms the other methods with both low performance drops and high computational savings, reducing computations by 61.3% with an WER increase of only 0.81. Finally, we analyze the robustness of the comparison to changes in dataset conditions, revealing sensitivity to dataset size., Comment: Submitted to ICASSP "Self-supervision in Audio, Speech and Beyond" workshop

Synthetic aperture radar tomographic imaging reconstructs the three-dimensional reflectivity of a scene from a set of coherent acquisitions performed in an interferometric configuration. In forest areas, a large number of elements backscatter the radar signal within each resolution cell. To reconstruct the vertical reflectivity profile, state-of-the-art techniques perform a regularized inversion implemented in the form of iterative minimization algorithms. We show that light-weight neural networks can be trained to perform the tomographic inversion with a single feed-forward pass, leading to fast reconstructions that could better scale to the amount of data provided by the future BIOMASS mission. We train our encoder-decoder network using simulated data and validate our technique on real L-band and P-band data., Comment: Submitted to IEEE Geoscience and Remote Sensing Letters, January 2023

Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood. It constructs an approximate posterior distribution by finding parameters for which the simulated data are close to the observations in terms of summary statistics. These statistics are defined beforehand and might induce a loss of information, which has been shown to deteriorate the quality of the approximation. To overcome this problem, Wasserstein-ABC has been recently proposed, and compares the datasets via the Wasserstein distance between their empirical distributions, but does not scale well to the dimension or the number of samples. We propose a new ABC technique, called Sliced-Wasserstein ABC and based on the Sliced-Wasserstein distance, which has better computational and statistical properties. We derive two theoretical results showing the asymptotical consistency of our approach, and we illustrate its advantages on synthetic data and an image denoising task., Comment: Accepted at ICASSP 2020 (publication and oral presentation)

We formulate the entropy of a quantized artificial neural network as a differentiable function that can be plugged as a regularization term into the cost function minimized by gradient descent. Our formulation scales efficiently beyond the first order and is agnostic of the quantization scheme. The network can then be trained to minimize the entropy of the quantized parameters, so that they can be optimally compressed via entropy coding. We experiment with our entropy formulation at quantizing and compressing well-known network architectures over multiple datasets. Our approach compares favorably over similar methods, enjoying the benefits of higher order entropy estimate, showing flexibility towards non-uniform quantization (we use Lloyd-max quantization), scalability towards any entropy order to be minimized and efficiency in terms of compression. We show that HEMP is able to work in synergy with other approaches aiming at pruning or quantizing the model itself, delivering significant benefits in terms of storage size compressibility without harming the model’s performance.

International audience; Conventional video coding and transmission systems are currently based on digital video compression (e.g., HEVC) on a suitable network protocol (802.11, 4G, or 5G) and rely on Shannon separation theorem. However, they suffer from some inherent limitations when the video content is transmitted over wireless error-prone networks. First, the coding choices (compression rate, channel coding rate) are decided a priori and at the transmitter and are the same for all the potential receivers. They could misfit with the actual channel conditions. Some user(s) with degraded channels may undergo digital cliff (glitches or freeze of the video) while other(s) may have a very good channel and yet not taking fully benefit of it since the design choices are based on more pessimistic hypotheses. Second, the traditional techniques require a permanent adaptation of the coding parameters by the transmitter relying on an estimate of the rate-distortion characteristic of the source and on an estimation of the channel characteristics, implying additional delay to perform this adaptation. Third, delay is introduced by the various buffers present at the encoder, within the network, and at the receiver. They are either required to smooth out variations of the encoding rate and of the channel characteristics, or due to the shared network infrastructure.Soft Video Delivery (SVD) architectures, pioneered by the SoftCast scheme, have demonstrated over the last decade a high potential to address/mitigate these issues. SVD architectures are joint source-channel video coding and transmission schemes that process pixels by successive linear operations (spatio-temporal decorrelation transform, power allocation, analog modulation) and directly transmitthe information without quantization or coding. SVD architectures deliver a single data stream that can be decoded by any receiver, even those experiencing bad channel quality. This data stream allows each receiver to decode a video quality commensurate with its channel quality, without requiring any feedback information, while avoiding the complex adaptation mechanisms of conventional schemes. Moreover, SVD architectures offer a relatively low and controlled latency that can be adjusted through the size of the temporal transform. This is a paradigm break with respect to traditional video transmission architectures, which has the potential of dramatically improving the quality of experience in wireless and latency-constrained scenarios.This tutorial will first introduce use cases where SVD architectures can make a difference compared to traditional schemes relying on conventional encoded video streams (e.g., HEVC) over a suitable network protocol (802.11, 4G, or 5G). Issues with conventional digital schemes will also be discussed (e.g., complex adaptation, cliff-effect, etc.), justifying the SVD approaches. Then, a block-by-block description of the components of the baseline SoftCast SVD scheme will be presented and visual examples provided to facilitate the understanding. A third part will be devoted to real implementations of SVD architectures, the dense modulation process and bandwidth computation will be detailed. Recent technical innovations and results from the literature will be presented and discussed. Finally, current research challenges related to the development of SVD architectures will be presented.

International audience; Ces dernières années, plusieurs efforts ont été consacrés à la réduction des différentes sources de latence dans la transmission vidéo, notamment l'acquisition, le codage et la transmission réseau. L'objectif est d'améliorer la qualité d'expérience dans les applications nécessitant une interaction en temps réel. Néanmoins, ces efforts sont fondamentalement contraints par des limites technologiques et physiques. Dans cet article, nous étudions une approche radicalement différente qui peut réduire arbitrairement la latence globale au moyen de l'extrapolation vidéo. Nous proposons deux schémas de compensation de latence dans lesquels l'extrapolation vidéo est effectuée soit au niveau de l'encodeur, soit au niveau du décodeur. Puisqu'une perte de fidélité est le prix à payer pour compenser arbitrairement la latence, nous étudions le compromis latence-fidélité en utilisant trois schémas de prédiction vidéo récents. Nos résultats préliminaires montrent qu'en acceptant une perte de qualité, nous pouvons compenser une latence typique de 100 ms avec une perte de 8 dB en PSNR avec le meilleur extrapolateur. Cette approche est prometteuse mais suggère également que des travaux supplémentaires doivent être réalisés dans le domaine de la prédiction vidéo afin de poursuivre la transmission vidéo sans latence.

Understanding generalization in modern machine learning settings has been one of the major challenges in statistical learning theory. In this context, recent years have witnessed the development of various generalization bounds suggesting different complexity notions such as the mutual information between the data sample and the algorithm output, compressibility of the hypothesis space, and the fractal dimension of the hypothesis space. While these bounds have illuminated the problem at hand from different angles, their suggested complexity notions might appear seemingly unrelated, thereby restricting their high-level impact. In this study, we prove novel generalization bounds through the lens of rate-distortion theory, and explicitly relate the concepts of mutual information, compressibility, and fractal dimensions in a single mathematical framework. Our approach consists of (i) defining a generalized notion of compressibility by using source coding concepts, and (ii) showing that the `compression error rate' can be linked to the generalization error both in expectation and with high probability. We show that in the `lossless compression' setting, we recover and improve existing mutual information-based bounds, whereas a `lossy compression' scheme allows us to link generalization to the rate-distortion dimension -- a particular notion of fractal dimension. Our results bring a more unified perspective on generalization and open up several future research directions., Accepted for presentation at the Conference on Learning Theory (COLT) 2022

International audience; Sliced probability metrics have become increasingly popular in machine learning, and they play a quintessential role in various applications, including statistical hypothesis testing and generative modeling. However, in a practical setting, the convergence behavior of the algorithms built upon these distances have not been well established, except for a few specific cases. In this paper, we introduce a new family of sliced probability metrics, namely Generalized Sliced Probability Metrics (GSPMs), based on the idea of slicing high-dimensional distributions into a set of their one-dimensional marginals. We show that GSPMs are true metrics, and they are related to the Maximum Mean Discrepancy (MMD). Exploiting this relationship, we consider GSPM-based gradient flows and show that, under mild assumptions, the gradient flow converges to the global optimum. Finally, we demonstrate that various choices of GSPMs lead to new positive definite kernels that could be used in the MMD formulation while providing a unique integral geometric interpretation. We illustrate the application of GSPMs in gradient flows.

Hedges play an important role in the management of conversational interaction. In peer tutoring, they are notably used by tutors in dyads (pairs of interlocutors) experiencing low rapport to tone down the impact of instructions and negative feedback. Pursuing the objective of building a tutoring agent that manages rapport with students in order to improve learning, we used a multimodal peer-tutoring dataset to construct a computational framework for identifying hedges. We compared approaches relying on pre-trained resources with others that integrate insights from the social science literature. Our best performance involved a hybrid approach that outperforms the existing baseline while being easier to interpret. We employ a model explainability tool to explore the features that characterize hedges in peer-tutoring conversations, and we identify some novel features, and the benefits of such a hybrid model approach., Published in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022

Speckle reduction is a key step in many remote sensing applications. By strongly affecting synthetic aperture radar (SAR) images, it makes them difficult to analyse. Due to the difficulty to model the spatial correlation of speckle, a deep learning algorithm with self-supervision is proposed in this paper: SAR2SAR. Multi-temporal time series are leveraged and the neural network learns to restore SAR images by only looking at noisy acquisitions. To this purpose, the recently proposed noise2noise framework has been employed. The strategy to adapt it to SAR despeckling is presented, based on a compensation of temporal changes and a loss function adapted to the statistics of speckle. A study with synthetic speckle noise is presented to compare the performances of the proposed method with other state-of-the-art filters. Then, results on real images are discussed, to show the potential of the proposed algorithm. The code is made available to allow testing and reproducible research in this field., The manuscript is the accepted version of IEEE STARS. Code is made available at https://gitlab.telecom-paris.fr/RING/SAR2SAR

John W. Tukey (1975) defined statistical data depth as a function that determines centrality of an arbitrary point with respect to a data cloud or to a probability measure. During the last decades, this seminal idea of data depth evolved into a powerful tool proving to be useful in various fields of science. Recently, extending the notion of data depth to the functional setting attracted a lot of attention among theoretical and applied statisticians. We go further and suggest a notion of data depth suitable for data represented as curves, or trajectories, which is independent of the parametrization. We show that our curve depth satisfies theoretical requirements of general depth functions that are meaningful for trajectories. We apply our methodology to diffusion tensor brain images and also to pattern recognition of hand written digits and letters. Supplementary Materials are available online.

Supervised deep learning approaches to underdetermined audio source separation achieve state-of-the-art performance but require a dataset of mixtures along with their corresponding isolated source signals. Such datasets can be extremely costly to obtain for musical mixtures. This raises a need for unsupervised methods. We propose a novel unsupervised model-based deep learning approach to musical source separation. Each source is modelled with a differentiable parametric source-filter model. A neural network is trained to reconstruct the observed mixture as a sum of the sources by estimating the source models' parameters given their fundamental frequencies. At test time, soft masks are obtained from the synthesized source signals. The experimental evaluation on a vocal ensemble separation task shows that the proposed method outperforms learning-free methods based on nonnegative matrix factorization and a supervised deep learning baseline. Integrating domain knowledge in the form of source models into a data-driven method leads to high data efficiency: the proposed approach achieves good separation quality even when trained on less than three minutes of audio. This work makes powerful deep learning based separation usable in scenarios where training data with ground truth is expensive or nonexistent., Revised version of the submission

International audience; Tournament procedures, recently introduced in Lugosi & Mendelson (2016), offer an appealing alternative, from a theoretical perspective at least, to the principle of Empirical Risk Minimization in machine learning. Statistical learning by Median-of-Means (MoM) basically consists in segmenting the training data into blocks of equal size and comparing the statistical performance of every pair of candidate decision rules on each data block: that with highest performance on the majority of the blocks is declared as the winner. In the context of nonparametric regression, functions having won all their duels have been shown to outperform empirical risk minimizers w.r.t. the mean squared error under minimal assumptions, while exhibiting robustness properties. It is the purpose of this paper to extend this approach, in order to address other learning problems in particular, for which the performance criterion takes the form of an expectation over pairs of observations rather than over one single observation, as may be the case in pairwise ranking, clustering or metric learning. Precisely, it is proved here that the bounds achieved by MoM are essentially conserved when the blocks are built by means of independent sampling without replacement schemes instead of a simple segmentation. These results are next extended to situations where the risk is related to a pairwise loss function and its empirical counterpart is of the form of a U-statistic. Beyond theoretical results guaranteeing the performance of the learning/estimation methods proposed, some numerical experiments provide empirical evidence of their relevance in practice.

We study the linear convergence of the primal-dual hybrid gradient method. After a review of current analyses, we show that they do not explain properly the behavior of the algorithm, even on the most simple problems. We thus introduce the quadratic error bound of the smoothed gap, a new regularity assumption that holds for a wide class of optimization problems. Equipped with this tool, we manage to prove tighter convergence rates. Then, we show that averaging and restarting the primal-dual hybrid gradient allows us to leverage better the regularity constant. Numerical experiments on linear and quadratic programs, ridge regression and image denoising illustrate the findings of the paper.

Contrastive Learning has shown impressive results on natural and medical images, without requiring annotated data. However, a particularity of medical images is the availability of meta-data (such as age or sex) that can be exploited for learning representations. Here, we show that the recently proposed contrastive y-Aware InfoNCE loss, that integrates multi-dimensional meta-data, asymptotically optimizes two properties: conditional alignment and global uniformity. Similarly to [Wang, 2020], conditional alignment means that similar samples should have similar features, but conditionally on the meta-data. Instead, global uniformity means that the (normalized) features should be uniformly distributed on the unit hyper-sphere, independently of the meta-data. Here, we propose to define conditional uniformity, relying on the meta-data, that repel only samples with dissimilar meta-data. We show that direct optimization of both conditional alignment and uniformity improves the representations, in terms of linear evaluation, on both CIFAR-100 and a brain MRI dataset., Comment: Accepted to MedNeurIPS 2021 (Oral)

International audience; To tackle interpretability in deep learning, we present a novel framework to jointly learn a predictive model and its associated interpretation model. The interpreter provides both local and global interpretability about the predictive model in terms of human-understandable high level attribute functions, with minimal loss of accuracy. This is achieved by a dedicated architecture and well chosen regularization penalties. We seek for a small-size dictionary of high level attribute functions that take as inputs the outputs of selected hidden layers and whose outputs feed a linear classifier. We impose strong conciseness on the activation of attributes with an entropy-based criterion while enforcing fidelity to both inputs and outputs of the predictive model. A detailed pipeline to visualize the learnt features is also developed. Moreover, besides generating interpretable models by design, our approach can be specialized to provide post-hoc interpretations for a pre-trained neural network. We validate our approach against several state-of-the-art methods on multiple datasets and show its efficacy on both kinds of tasks.

The Sliced-Wasserstein distance (SW) is being increasingly used in machine learning applications as an alternative to the Wasserstein distance and offers significant computational and statistical benefits. Since it is defined as an expectation over random projections, SW is commonly approximated by Monte Carlo. We adopt a new perspective to approximate SW by making use of the concentration of measure phenomenon: under mild assumptions, one-dimensional projections of a high-dimensional random vector are approximately Gaussian. Based on this observation, we develop a simple deterministic approximation for SW. Our method does not require sampling a number of random projections, and is therefore both accurate and easy to use compared to the usual Monte Carlo approximation. We derive nonasymptotical guarantees for our approach, and show that the approximation error goes to zero as the dimension increases, under a weak dependence condition on the data distribution. We validate our theoretical findings on synthetic datasets, and illustrate the proposed approximation on a generative modeling problem., Published at NeurIPS 2021

In this work, we consider the problem of blind source separation (BSS) by departing from the usual linear model and focusing on the linear-quadratic (LQ) model. We propose two provably robust and computationally tractable algorithms to tackle this problem under separability assumptions which require the sources to appear as samples in the data set. The first algorithm generalizes the successive nonnegative projection algorithm (SNPA), designed for linear BSS, and is referred to as SNPALQ. By explicitly modeling the product terms inherent to the LQ model along the iterations of the SNPA scheme, the nonlinear contributions of the mixing are mitigated, thus improving the separation quality. SNPALQ is shown to be able to recover the ground truth factors that generated the data, even in the presence of noise. The second algorithm is a brute-force (BF) algorithm, which is used as a post-processing step for SNPALQ. It enables to discard the spurious (mixed) samples extracted by SNPALQ, thus broadening its applicability. The BF is in turn shown to be robust to noise under easier-to-check and milder conditions than SNPALQ. We show that SNPALQ with and without the BF postprocessing is relevant in realistic numerical experiments., 23 pages + 24 pages of Appendix containing the proofs

In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by Davis and Drusvyatskiy. Such points lie on a manifold M where the function f has a direction of second-order negative curvature. Off this manifold, the norm of the Clarke subdifferential of f is lower-bounded. We require two conditions on f. The first assumption is a Verdier stratification condition, which is a refinement of the popular Whitney stratification. It allows us to establish a reinforced version of the projection formula of Bolte et.al. for Whitney stratifiable functions, and which is of independent interest. The second assumption, termed the angle condition, allows to control the distance of the iterates to M. When f is weakly convex, our assumptions are generic. Consequently, generically in the class of definable weakly convex functions, the SGD converges to a local minimizer.

International audience; Cover detection has gained sustained interest in the scientific community and has recently made significant progress both in terms of scalability and accuracy. However, most approaches are based on the estimation of harmonic and melodic features and neglect lyrics information although it is an important invariant across covers. In this work, we propose a novel approach leveraging lyrics without requiring access to full texts though the use of lyrics recognition on audio. Our approach relies on the fusion of a singing voice recognition framework and a more classic tonal-based cover detection method. To the best of our knowledge, this is the first time that lyrics estimation from audio has been explicitly used for cover detection. Furthermore, we exploit efficient string matching and an approximated nearest neighbors search algorithm which lead to a scalable system which is able to operate on very large databases. Extensive experiments on the largest publicly available cover detection dataset demonstrate the validity of using lyrics information for this task.

International audience; Recommending automatically a video given a music or a music given a video has become an important asset for the audiovisual industry-with user-generated or professional content. While both music and video have specific temporal organizations, most current works do not consider those and only focus on globally recommending a media. As a first step toward the improvement of these recommendation systems, we study in this paper the relationship between music and video temporal organization. We do this for the case of official music videos, with a quantitative and a qualitative approach. Our assumption is that the movement in the music are correlated to the ones in the video. To validate this, we first interview a set of internationally recognized music video experts. We then perform a largescale analysis of official music-video clips (which we manually annotated into video genres) using MIR description tools (downbeats and functional segments estimation) and Computer Vision tools (shot detection). Our study confirms that a "language of music-video clips" exists; i.e. editors favor the co-occurrence of music and video events using strategies such as anticipation. It also highlights that the amount of co-occurrence depends on the music and video genres.

International audience; Despite the success of end-to-end approaches, chroma (or pitch-class) features remain a useful mid-level representation of music audio recordings due to their direct interpretability. Since traditional chroma variants obtained with signal processing suffer from timbral artifacts such as overtones or vibrato, they do not directly reflect the pitch classes notated in the score. For this reason, training a chroma representation using deep learning ("deep chroma") has become an interesting strategy. Existing approaches involve the use of supervised learning with strongly aligned labels for which, however, only few datasets are available. Recently, the Connectionist Temporal Classification (CTC) loss, initially proposed for speech, has been adopted to learn monophonic (single-label) pitch-class features using weakly aligned labels based on corresponding score--audio segment pairs. To exploit this strategy for the polyphonic case, we propose the use of a multi-label variant of this CTC loss, the MCTC, and formalize this loss for the pitch-class scenario. Our experiments demonstrate that the weakly aligned approach achieves almost equivalent pitch-class estimates than training with strongly aligned annotations. We then study the sensitivity of our approach to segment duration and mismatch. Finally, we compare the learned features with other pitch-class representations and demonstrate their use for chord and local key recognition on classical music datasets.

Speckle fluctuations seriously limit the interpretability of synthetic aperture radar (SAR) images. Speckle reduction has thus been the subject of numerous works spanning at least four decades. Techniques based on deep neural networks have recently achieved a new level of performance in terms of SAR image restoration quality. Beyond the design of suitable network architectures or the selection of adequate loss functions, the construction of training sets is of uttermost importance. So far, most approaches have considered a supervised training strategy: the networks are trained to produce outputs as close as possible to speckle-free reference images. Speckle-free images are generally not available, which requires resorting to natural or optical images or the selection of stable areas in long time series to circumvent the lack of ground truth. Self-supervision, on the other hand, avoids the use of speckle-free images. We introduce a self-supervised strategy based on the separation of the real and imaginary parts of single-look complex SAR images, called MERLIN (coMplex sElf-supeRvised despeckLINg), and show that it offers a straightforward way to train all kinds of deep despeckling networks. Networks trained with MERLIN take into account the spatial correlations due to the SAR transfer function specific to a given sensor and imaging mode. By requiring only a single image, and possibly exploiting large archives, MERLIN opens the door to hassle-free as well as large-scale training of despeckling networks. The code of the trained models is made freely available at https://gitlab.telecom-paris.fr/RING/MERLIN., To appear on IEEE Transactions on Geoscience and Remote Sensing

Event logs are structured records of all kinds of activities taking place in a computer network. In particular, malicious actions taken by intruders are likely to leave a trace in the logs, making this data source useful for security monitoring and intrusion detection. However, the considerable volume of real-world event logs makes them difficult to analyze. This limitation has motivated a fair amount of research on malicious behavior detection through statistical methods. This thesis addresses some of the challenges that currently hinder the use of this approach in realistic settings. First of all, building an abstract representation of the data is nontrivial: event logs are complex and multi-faceted, making it difficult to capture all the relevant information they contain in a simple mathematical object. We take an interaction-centric approach to event log representation, motivated by the intuition that malicious events can often be seen as unexpected interactions between entities (users, hosts, etc.). While this representation preserves critical information, it also makes statistical modelling difficult. We thus build an ad hoc model and design a suitable inference procedure, using elements of latent space modelling, Bayesian filtering and multi-task learning.Another key challenge in event log analysis is that benign events account for a vast majority of the data, including a lot of unusual albeit legitimate events. Detecting individually anomalous events is thus not enough, and we also deal with spotting clusters of potentially malicious events. To that end, we leverage the concept of event graph and recast event-wise anomaly scores as a noisy graph-structured signal. This allows us to use graph signal processing tools to improve anomaly scores provided by statistical models.Finally, we propose scalable methods for anomalous cluster detection in node-valued signals defined over large graphs.; Les journaux d’événements sont des données structurées décrivant toutes sortes d’activités au sein d’un réseau informatique. En particulier, les comportements malveillants adoptés par d’éventuels attaquants sont susceptibles de laisser une trace dans ces journaux, rendant ces derniers utiles pour la supervision et la détection d’intrusion. Cependant, le volume considérable des journaux d’événements générés en production en rend l’analyse difficile. Cette problématique a suscité de nombreux travaux de recherche sur l’analyse statistique de journaux d’événements pour la détection d’intrusion.Cette thèse étudie certaines des principales difficultés rendant actuellement peu aisé le déploiementde telles approches. Tout d’abord, il n’est pas évident de construire une représentation abstraite des journaux d’événements : ces données sont complexes et peuvent être abordées sous de multiples perspectives, et il est donc difficile d’en capturer tout le sens dans un objet mathématique simple. Nous choisissons une approche centrée sur la notion d’interaction, motivée par l’idée que de nombreux événements malveillants peuvent être vus comme des interactions inattendues entre des entités (utilisateurs, hôtes, etc.). Tout en préservant les informations les plus cruciales, cette représentation rend cependant la modélisation statistique ardue. Nous proposons donc un modèle ad hoc ainsi que la procédure d’inférence associée, en nous inspirant de concepts tels que les modèles à espace d’états, le filtrage bayésien et l’apprentissage multitâche.Une autre caractéristique des journaux d’événements est qu’ils contiennent une large majorité d’événements bénins, dont certains sont incongrus bien que légitimes. Il n’est donc pas suffisant de détecter des événements anormaux, et nous étudions également la détection de clusters d’événementspotentiellement malveillants. Nous nous appuyons pour cela sur la notion de graphe d’événements afinde redéfinir les scores d’anormalité associés aux événements comme un signal structuré en graphe. Cela permet l’usage d’outils de traitement du signal afin de débruiter les scores d’anormalité produits par un modèle statistique. Enfin, nous proposons des méthodes efficaces pour la détection de cluster anormal dans un graphe de grande taille dont les sommets portent des observations scalaires.

National audience

Emergent states are behavioral, cognitive and affective processes appearing among the members of a group when they interact together. In the last decade, the development of computational approaches received a growing interest in building Human-Centered systems. Such a development is particularly difficult because some of these states have several dimensions interplaying somehow and somewhere over time. In this paper, we focus on cohesion, its dimensions and their interplay. Several definitions of cohesion exist, it can be simply defined as the tendency of a group to stick together to pursue goals and/or affective needs. This plethora of definitions resulted in many different cohesion dimensions. Social and Task dimensions are the most investigated both in Social Sciences and Computer Science since they both play an important role in a wide range of contexts and groups. To the best of our knowledge, however, no previous work on the prediction of cohesion dynamics focused on how these 2 dimensions interplay. We leverage Social Sciences to address this issue. In particular, we take advantage of the importance of Social cohesion for creating flexible and constructive relationships to reinforce Task cohesion. We describe a Deep Neural Network architecture (DNN) for predicting the dynamics of Task cohesion by applying transfer learning from a pre-trained model dedicated to the prediction of Social cohesion dynamics. Our architecture is evaluated against several baselines. Results show that it significantly improves the predictions of the Task cohesion dynamics, confirming the benefits of integrating Social Sciences insights into models architectures.

This document is an annex of the following paper: Achille Aknin and Roland Badeau.Stochastic reverberation model with a frequency dependent attenuation.IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2021 (submitted for publica-tion).It provides mathematical proofs of theorems that we use in the paper. More specifically, we will prove that the set P, of which we recall the definition in Section 1, is a matrix Lie group in Section 2 and that we can swap matrices of P with lower triangular Toeplitz matrices in Section 3.; International audience; In various audio signal processing applications, such as source separation and dereverberation, accurate mathematical modeling of both source signals and room reverberation is needed to properly describe the audio data. In a previous paper, we introduced a stochastic room impulse response model based on the image source principle, and we proposed an expectation-maximization algorithm that was able to efficiently estimate the model parameters in various experimental settings. This paper aims to extend the model in order to account for the dependency of the exponential decay over frequency, due to the walls usually absorbing less energy at low frequencies than at high frequencies. Our experimental results show that this refinement of the model is able to generate realistic room impulse responses, that are perceptively very close to the original ones.

International audience; Detecting the simultaneous activity of pitches in music audio recordings is a central task within music processing, commonly known as multi-pitch estimation or frame-wise polyphonic music transcription. Deep-learning approaches recently achieved major improvements for this task, but the lack of annotated, large-size datasets beyond the piano solo scenario is still a limitation for fully exploiting their potential. In this paper, we propose a strategy for training a CNN-based multi-pitch estimator on weakly aligned score--audio pairs of pieces in different instrumentations. To this end, we make use of a multi-label variant of the connectionist temporal classification loss (MCTC), recently proposed for image recognition tasks. We re-formalize the MCTC loss to be applicable for multi-pitch estimation and perform several systematic experiments to analyze its behavior and robustness to training conditions. Finally, we report on multi-pitch estimation results for common datasets using weakly aligned training with MCTC, which performs similar than systems trained on strongly aligned scores.

Understanding the scene around the ego-vehicle is key to assisted and autonomous driving. Nowadays, this is mostly conducted using cameras and laser scanners, despite their reduced performances in adverse weather conditions. Automotive radars are low-cost active sensors that measure properties of surrounding objects, including their relative speed, and have the key advantage of not being impacted by rain, snow or fog. However, they are seldom used for scene understanding due to the size and complexity of radar raw data and the lack of annotated datasets. Fortunately, recent open-sourced datasets have opened up research on classification, object detection and semantic segmentation with raw radar signals using end-to-end trainable models. In this work, we propose several novel architectures, and their associated losses, which analyse multiple "views" of the range-angle-Doppler radar tensor to segment it semantically. Experiments conducted on the recent CARRADA dataset demonstrate that our best model outperforms alternative models, derived either from the semantic segmentation of natural images or from radar scene understanding, while requiring significantly fewer parameters. Both our code and trained models are available at https://github.com/valeoai/MVRSS., Comment: 16 pages, 9 figures. Accepted at ICCV 2021

International audience; We propose a computationally efficient procedure for elevated mean detection on a connected subgraph of a network with node-related scalar observations. Our approach relies on two intuitions: first, a significant concentration of high observations in a connected subgraph implies that the subgraph induced by the nodes associated with the highest observations has a large connected component. Secondly, a greater detection power can be obtained in certain cases by denoising the observations using the network structure. Numerical experiments show that our procedure's detection performance and computational efficiency are both competitive.

International audience; The SoftCast scheme has been proposed as a promising alternative to traditional video broadcasting systems in wireless environments. In its current form, SoftCast performs image decoding at the receiver side by using a Linear Least Square Error (LLSE) estimator. Such approach maximizes the reconstructed quality in terms of Peak Signal-to-Noise Ratio (PSNR). However, we show that the LLSE induces an annoying blur effect at low Channel Signal-to-Noise Ratio (CSNR) quality. To cancel this artifact, we propose to replace the LLSE estimator by the Zero-Forcing (ZF) one. In order to better understand the perceived quality offered by these two estimators, a mathematical characterization as well as an objective and subjective studies are performed. Results show that the gains brought by the LLSE estimator, in terms of PSNR and Structural SIMiliraty (SSIM), are limited and quickly tend to null value as the CSNR increases. However, higher gains are obtained by the ZF estimator when considering the recent Video Multi-method Assessment Fusion (VMAF) metric proposed by Netflix, which evaluates the perceptual video quality. This result is confirmed by the subjective assessment.

This paper introduces Click to Move (C2M), a novel framework for video generation where the user can control the motion of the synthesized video through mouse clicks specifying simple object trajectories of the key objects in the scene. Our model receives as input an initial frame, its corresponding segmentation map and the sparse motion vectors encoding the input provided by the user. It outputs a plausible video sequence starting from the given frame and with a motion that is consistent with user input. Notably, our proposed deep architecture incorporates a Graph Convolution Network (GCN) modelling the movements of all the objects in the scene in a holistic manner and effectively combining the sparse user motion information and image features. Experimental results show that C2M outperforms existing methods on two publicly available datasets, thus demonstrating the effectiveness of our GCN framework at modelling object interactions. The source code is publicly available at https://github.com/PierfrancescoArdino/C2M., Comment: Accepted by International Conference on Computer Vision (ICCV 2021)

International audience

Cohesion is an affective group phenomenon. It has received a lot of attention from scholars both in Social Sciences and in Affective Computing that showed that cohesion and emotion influence each other, highlighting the need to jointly analyze them. This study presents 2 deep neural network architectures grounded on multitask learning to jointly predict cohesion and emotion. Inspired by 2 major Social Sciences approaches on group emotion (i.e., Top-down and Bottom-up), these architectures exploit cohesion and emotion interdependencies intending to improve the prediction of the dynamics (i.e. changes over time) of the Social and Task dimensions of cohesion. Emotion, here, is addressed in terms of its valence. Both architectures are evaluated against the performances of a similar model that only predicts the dynamics of both the Social and Task dimensions of cohesion, without integrating valence. Statistical analysis shows that only the deep model implementing the Bottom-up approach significantly improved the predictions of the Task cohesion’s dynamics. This result confirms the theoretical and practical benefits of multitasking as it takes full advantage of the inherent relationships between group emotion and cohesion to improve Task cohesion’s predictions.

A natural human-robot interaction (HRI) relies on the robot’s capacity to understand the users’ behaviors through psychological and sociological concepts. Users expect the robot to act in a realistic manner to create a more human-like relationship. In this context, trust is an essential concept as it determines the effectiveness of the system and its acceptance by users. The understanding of trust dynamics in HRI is still low and systematic studies of multimodal trust-related behaviors in HRI are relatively rare given the rising popularity of the topic. To bridge this gap, in this paper we present a novel coding system TURIN (Trust in hUman Robot INteraction) to study trust in HRI. A preliminary assessment of the coding system was carried out on the Vernissage dataset. Results show a significant agreement between expert annotators.

Traditional supervised learning with deep neural networks requires a tremendous amount of labelled data to converge to a good solution. For 3D medical images, it is often impractical to build a large homogeneous annotated dataset for a specific pathology. Self-supervised methods offer a new way to learn a representation of the images in an unsupervised manner with a neural network. In particular, contrastive learning has shown great promises by (almost) matching the performance of fully-supervised CNN on vision tasks. Nonetheless, this method does not take advantage of available meta-data, such as participant's age, viewed as prior knowledge. Here, we propose to leverage continuous proxy metadata, in the contrastive learning framework, by introducing a new loss called y-Aware InfoNCE loss. Specifically, we improve the positive sampling during pre-training by adding more positive examples with similar proxy meta-data with the anchor, assuming they share similar discriminative semantic features.With our method, a 3D CNN model pre-trained on $10^4$ multi-site healthy brain MRI scans can extract relevant features for three classification tasks: schizophrenia, bipolar diagnosis and Alzheimer's detection. When fine-tuned, it also outperforms 3D CNN trained from scratch on these tasks, as well as state-of-the-art self-supervised methods. Our code is made publicly available here., Comment: MICCAI 2021

International audience; The ability to collect and store ever more massive databases has been accompanied by the need to process them efficiently. In many cases, most observations have the same behavior, while a probable small proportion of these observations are abnormal. Detecting the latter, defined as outliers, is one of the major challenges for machine learning applications (e.g. in fraud detection or in predictive maintenance). In this paper, we propose a methodology addressing the problem of outlier detection, by learning a data-driven scoring function defined on the feature space which reflects the degree of abnormality of the observations. This scoring function is learnt through a well-designed binary classification problem whose empirical criterion takes the form of a two-sample linear rank statistics on which theoretical results are available. We illustrate our methodology with preliminary encouraging numerical experiments.

The spectral theory for weakly stationary processes valued in a separable Hilbert space has known renewed interest in the past decade. However, the recent literature on this topic is oftenbased on restrictive assumptions or lacks important insights. In this paper, we follow earlier approaches which fully exploit the normal Hilbert module property of the space of Hilbert-valued random variables. This approach clarifies and completes the isomorphic relationship between the modular spectral domain to the modular time domain provided by the Gramian-Cramér representation. We also discuss the general Bochner theorem and provide useful results on the composition and inversion of lag-invariant linear filters. Finally, we derive the Cramér-Karhunen-Loève decomposition and harmonic functional principal component analysis without relying on simplifying assumptions.

International audience; Estimating mixtures of damped chirp sinusoids in noise is a problem that affects audio analysis, coding, and synthesis applications. Phase-based non-stationary parameter estimators assume that sinusoids can be resolved in the Fourier transform domain, whereas high-resolution methods estimate superimposed components with accuracy close to the theoretical limits, but only for sinusoids with constant frequencies. We present a new method for estimating the parameters of superimposed damped chirps that has an accuracy competitive with existing non-stationary estimators but also has a high-resolution like subspace techniques. After providing the analytical expression for a Gaussian-windowed damped chirp signal's Fourier transform, we propose an efficient variational EM algorithm for nonlinear Bayesian regression that jointly estimates the amplitudes, phases, frequencies, chirp rates, and decay rates of multiple non-stationary components that may be obfuscated under the same local maximum in the frequency spectrum. Quantitative results show that the new method not only has an estimation accuracy that is close to the Cramér-Rao bound, but also a high resolution that outperforms the state-of-the-art.

International audience; Supervised source separation requires expensive synthetic datasets containing clean, ground truth-source signals, while unsupervised separation requires only data mixtures. Existing unsupervised methods still use supervision to avoid over-separation and compete with fully supervised methods. We present a new method of completely unsupervised single-channel blind source separation, based on variational auto-encoding, that automatically learns the correct number of sources in data mixtures and quantitatively outperforms the existing methods. A deep inference network disentangles (separates) data mixtures into low-dimensional latent source variables. A deep generative network individually decodes each latent source into its source signal, such that their sum represents the given mixture. Qualitative and quantitative results from separation experiments on pairs of randomly mixed MNIST handwritten digits and mixed audio spectrograms demonstrate that our method outperforms stateof-the-art unsupervised and semi-supervised methods, showing promise as a solution to this long-standing problem in computer vision and audition.

International audience; The analysis of load curves collected from smart meters is a key step for many energy management tasks ranging from consumption forecasting to customers characterization and load monitoring. In this contribution, we propose a model based on a functional formulation of nonnegative tensor factorization and derive updates for the corresponding optimization problem. We show on the concrete example of multi-sites load curves disaggregation how this formulation is helpful for 1) exhibiting smooth intraday consumption patterns and 2) taking into account external variables such as the outside temperature. The benefits are demonstrated on simulated and real data by exhibiting a meaningful clustering of the observed sites based on the obtained decomposition.

This paper proposes a novel energy-momentum conserving integration method. Adopting Projective Dynamics, the proposed method extends its unconstrained minimization for time integration into the constrained form with the position-based energy-momentum constraints. This resolves the well-known problem of unwanted dissipation of energy and momenta without compromising the real-time performance and simulation stability. The proposed method also enables users to directly control the energy and momenta so as to easily create the vivid deformable and global motions they want, which is a fascinating feature for many real-time applications such as virtual/augmented reality and games.

International audience

Recent advances in Transformer models allow for unprecedented sequence lengths, due to linear space and time complexity. In the meantime, relative positional encoding (RPE) was proposed as beneficial for classical Transformers and consists in exploiting lags instead of absolute positions for inference. Still, RPE is not available for the recent linear-variants of the Transformer, because it requires the explicit computation of the attention matrix, which is precisely what is avoided by such methods. In this paper, we bridge this gap and present Stochastic Positional Encoding as a way to generate PE that can be used as a replacement to the classical additive (sinusoidal) PE and provably behaves like RPE. The main theoretical contribution is to make a connection between positional encoding and cross-covariance structures of correlated Gaussian processes. We illustrate the performance of our approach on the Long-Range Arena benchmark and on music generation., Comment: ICML 2021 (long talk) camera-ready. 24 pages