Search

Training of large language models (LLMs) is typically distributed across a large number of accelerators to reduce training time. Since internal states and parameter gradients need to be exchanged at each and every single gradient step, all devices need to be co-located using low-latency high-bandwidth communication links to support the required high volume of exchanged bits. Recently, distributed algorithms like DiLoCo have relaxed such co-location constraint: accelerators can be grouped into ``workers'', where synchronizations between workers only occur infrequently. This in turn means that workers can afford being connected by lower bandwidth communication links without affecting learning quality. However, in these methods, communication across workers still requires the same peak bandwidth as before, as the synchronizations require all parameters to be exchanged across all workers. In this paper, we improve DiLoCo in three ways. First, we synchronize only subsets of parameters in sequence, rather than all at once, which greatly reduces peak bandwidth. Second, we allow workers to continue training while synchronizing, which decreases wall clock time. Third, we quantize the data exchanged by workers, which further reduces bandwidth across workers. By properly combining these modifications, we show experimentally that we can distribute training of billion-scale parameters and reach similar quality as before, but reducing required bandwidth by two orders of magnitude.

We propose the use of latent space generative world models to address the covariate shift problem in autonomous driving. A world model is a neural network capable of predicting an agent's next state given past states and actions. By leveraging a world model during training, the driving policy effectively mitigates covariate shift without requiring an excessive amount of training data. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations, so that at runtime it can recover from perturbations outside the training distribution. Additionally, we introduce a novel transformer-based perception encoder that employs multi-view cross-attention and a learned scene query. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing in the CARLA simulator, as well as showing the ability to handle perturbations in both CARLA and NVIDIA's DRIVE Sim., Comment: 7 pages, 6 figures, for ICRA 2025 conference, for associated video file, see https://youtu.be/fO7RZ57gVxk

Reinforcement learning from human feedback (RLHF) aligns large language models (LLMs) by encouraging their generations to have high rewards, using a reward model trained on human preferences. To prevent the forgetting of pre-trained knowledge, RLHF usually incorporates a KL regularization; this forces the policy to remain close to its supervised fine-tuned initialization, though it hinders the reward optimization. To tackle the trade-off between KL and reward, in this paper we introduce a novel alignment strategy named Weight Averaged Rewarded Policies (WARP). WARP merges policies in the weight space at three distinct stages. First, it uses the exponential moving average of the policy as a dynamic anchor in the KL regularization. Second, it applies spherical interpolation to merge independently fine-tuned policies into a new enhanced one. Third, it linearly interpolates between this merged model and the initialization, to recover features from pre-training. This procedure is then applied iteratively, with each iteration's final model used as an advanced initialization for the next, progressively refining the KL-reward Pareto front, achieving superior rewards at fixed KL. Experiments with GEMMA policies validate that WARP improves their quality and alignment, outperforming other open-source LLMs., Comment: 11 main pages (34 pages with Appendix)

Recent generative AI systems have demonstrated more advanced persuasive capabilities and are increasingly permeating areas of life where they can influence decision-making. Generative AI presents a new risk profile of persuasion due the opportunity for reciprocal exchange and prolonged interactions. This has led to growing concerns about harms from AI persuasion and how they can be mitigated, highlighting the need for a systematic study of AI persuasion. The current definitions of AI persuasion are unclear and related harms are insufficiently studied. Existing harm mitigation approaches prioritise harms from the outcome of persuasion over harms from the process of persuasion. In this paper, we lay the groundwork for the systematic study of AI persuasion. We first put forward definitions of persuasive generative AI. We distinguish between rationally persuasive generative AI, which relies on providing relevant facts, sound reasoning, or other forms of trustworthy evidence, and manipulative generative AI, which relies on taking advantage of cognitive biases and heuristics or misrepresenting information. We also put forward a map of harms from AI persuasion, including definitions and examples of economic, physical, environmental, psychological, sociocultural, political, privacy, and autonomy harm. We then introduce a map of mechanisms that contribute to harmful persuasion. Lastly, we provide an overview of approaches that can be used to mitigate against process harms of persuasion, including prompt engineering for manipulation classification and red teaming. Future work will operationalise these mitigations and study the interaction between different types of mechanisms of persuasion.

Progress in machine learning (ML) has been fueled by scaling neural network models. This scaling has been enabled by ever more heroic feats of engineering, necessary for accommodating ML approaches that require high bandwidth communication between devices working in parallel. In this work, we propose a co-designed modular architecture and training approach for ML models, dubbed DIstributed PAth COmposition (DiPaCo). During training, DiPaCo distributes computation by paths through a set of shared modules. Together with a Local-SGD inspired optimization (DiLoCo) that keeps modules in sync with drastically reduced communication, Our approach facilitates training across poorly connected and heterogeneous workers, with a design that ensures robustness to worker failures and preemptions. At inference time, only a single path needs to be executed for each input, without the need for any model compression. We consider this approach as a first prototype towards a new paradigm of large-scale learning, one that is less synchronous and more modular. Our experiments on the widely used C4 benchmark show that, for the same amount of training steps but less wall-clock time, DiPaCo exceeds the performance of a 1 billion-parameter dense transformer language model by choosing one of 256 possible paths, each with a size of 150 million parameters.

Local stochastic gradient descent (Local-SGD), also referred to as federated averaging, is an approach to distributed optimization where each device performs more than one SGD update per communication. This work presents an empirical study of {\it asynchronous} Local-SGD for training language models; that is, each worker updates the global parameters as soon as it has finished its SGD steps. We conduct a comprehensive investigation by examining how worker hardware heterogeneity, model size, number of workers, and optimizer could impact the learning performance. We find that with naive implementations, asynchronous Local-SGD takes more iterations to converge than its synchronous counterpart despite updating the (global) model parameters more frequently. We identify momentum acceleration on the global parameters when worker gradients are stale as a key challenge. We propose a novel method that utilizes a delayed Nesterov momentum update and adjusts the workers' local training steps based on their computation speed. This approach, evaluated with models up to 150M parameters on the C4 dataset, matches the performance of synchronous Local-SGD in terms of perplexity per update step, and significantly surpasses it in terms of wall clock time.

Large language models (LLM) have become a critical component in many applications of machine learning. However, standard approaches to training LLM require a large number of tightly interconnected accelerators, with devices exchanging gradients and other intermediate states at each optimization step. While it is difficult to build and maintain a single computing cluster hosting many accelerators, it might be easier to find several computing clusters each hosting a smaller number of devices. In this work, we propose a distributed optimization algorithm, Distributed Low-Communication (DiLoCo), that enables training of language models on islands of devices that are poorly connected. The approach is a variant of federated averaging, where the number of inner steps is large, the inner optimizer is AdamW, and the outer optimizer is Nesterov momentum. On the widely used C4 dataset, we show that DiLoCo on 8 workers performs as well as fully synchronous optimization while communicating 500 times less. DiLoCo exhibits great robustness to the data distribution of each worker. It is also robust to resources becoming unavailable over time, and vice versa, it can seamlessly leverage resources that become available during training.

The field of transfer learning is undergoing a significant shift with the introduction of large pretrained models which have demonstrated strong adaptability to a variety of downstream tasks. However, the high computational and memory requirements to finetune or use these models can be a hindrance to their widespread use. In this study, we present a solution to this issue by proposing a simple yet effective way to trade computational efficiency for asymptotic performance which we define as the performance a learning algorithm achieves as compute tends to infinity. Specifically, we argue that zero-shot structured pruning of pretrained models allows them to increase compute efficiency with minimal reduction in performance. We evaluate our method on the Nevis'22 continual learning benchmark that offers a diverse set of transfer scenarios. Our results show that pruning convolutional filters of pretrained models can lead to more than 20% performance improvement in low computational regimes.

Strong-field quantum electronics is emerging as a potential candidate in information processing but still coherence vs decoherence is a primary concern of the concept. Strong-field coherent processes in band gap materials have led during the last decade to the emergence of high harmonic generation in semiconductors, petahertz electronics, or strong-field quantum states. However, the coherent behavior of the sub-optical cycle-driven electrons has never been directly observed. We report here on the experimental evidence of coherent ultrashort emission of hot electrons from a nanostructured semiconductor. Our method uses sub-wavelength electric field enhancement to localize the electron emission within a nanometer-scale spot. We found similarities with the electron emission from metallic nanotips in the strong-field regime, a topic that has opened a vast domain of applications during the last decade. The electron spectra display both odd and even harmonic orders of the driving femtosecond laser frequency, a signature of the coherent nature of the electron emission and their attosecond timing. Our findings complete our knowledge of phenomena governing coherent strong-field processes in semiconductors and open perspectives for the generation of future quantum devices operating in the strong-field regime.

Continual learning for segmentation has recently seen increasing interest. However, all previous works focus on narrow semantic segmentation and disregard panoptic segmentation, an important task with real-world impacts. %a In this paper, we present the first continual learning model capable of operating on both semantic and panoptic segmentation. Inspired by recent transformer approaches that consider segmentation as a mask-classification problem, we design CoMFormer. Our method carefully exploits the properties of transformer architectures to learn new classes over time. Specifically, we propose a novel adaptive distillation loss along with a mask-based pseudo-labeling technique to effectively prevent forgetting. To evaluate our approach, we introduce a novel continual panoptic segmentation benchmark on the challenging ADE20K dataset. Our CoMFormer outperforms all the existing baselines by forgetting less old classes but also learning more effectively new classes. In addition, we also report an extensive evaluation in the large-scale continual semantic segmentation scenario showing that CoMFormer also significantly outperforms state-of-the-art methods., Comment: Under submission

A shared goal of several machine learning communities like continual learning, meta-learning and transfer learning, is to design algorithms and models that efficiently and robustly adapt to unseen tasks. An even more ambitious goal is to build models that never stop adapting, and that become increasingly more efficient through time by suitably transferring the accrued knowledge. Beyond the study of the actual learning algorithm and model architecture, there are several hurdles towards our quest to build such models, such as the choice of learning protocol, metric of success and data needed to validate research hypotheses. In this work, we introduce the Never-Ending VIsual-classification Stream (NEVIS'22), a benchmark consisting of a stream of over 100 visual classification tasks, sorted chronologically and extracted from papers sampled uniformly from computer vision proceedings spanning the last three decades. The resulting stream reflects what the research community thought was meaningful at any point in time, and it serves as an ideal test bed to assess how well models can adapt to new tasks, and do so better and more efficiently as time goes by. Despite being limited to classification, the resulting stream has a rich diversity of tasks from OCR, to texture analysis, scene recognition, and so forth. The diversity is also reflected in the wide range of dataset sizes, spanning over four orders of magnitude. Overall, NEVIS'22 poses an unprecedented challenge for current sequential learning approaches due to the scale and diversity of tasks, yet with a low entry barrier as it is limited to a single modality and well understood supervised learning problems. Moreover, we provide a reference implementation including strong baselines and an evaluation protocol to compare methods in terms of their trade-off between accuracy and compute.

Behavior prediction models have proliferated in recent years, especially in the popular real-world robotics application of autonomous driving, where representing the distribution over possible futures of moving agents is essential for safe and comfortable motion planning. In these models, the choice of coordinate frames to represent inputs and outputs has crucial trade offs which broadly fall into one of two categories. Agent-centric models transform inputs and perform inference in agent-centric coordinates. These models are intrinsically invariant to translation and rotation between scene elements, are best-performing on public leaderboards, but scale quadratically with the number of agents and scene elements. Scene-centric models use a fixed coordinate system to process all agents. This gives them the advantage of sharing representations among all agents, offering efficient amortized inference computation which scales linearly with the number of agents. However, these models have to learn invariance to translation and rotation between scene elements, and typically underperform agent-centric models. In this work, we develop knowledge distillation techniques between probabilistic motion forecasting models, and apply these techniques to close the gap in performance between agent-centric and scene-centric models. This improves scene-centric model performance by 13.2% on the public Argoverse benchmark, 7.8% on Waymo Open Dataset and up to 9.4% on a large In-House dataset. These improved scene-centric models rank highly in public leaderboards and are up to 15 times more efficient than their agent-centric teacher counterparts in busy scenes., Comment: Accepted at ICRA 2022

Rapid development of large-scale pre-training has resulted in foundation models that can act as effective feature extractors on a variety of downstream tasks and domains. Motivated by this, we study the efficacy of pre-trained vision models as a foundation for downstream continual learning (CL) scenarios. Our goal is twofold. First, we want to understand the compute-accuracy trade-off between CL in the raw-data space and in the latent space of pre-trained encoders. Second, we investigate how the characteristics of the encoder, the pre-training algorithm and data, as well as of the resulting latent space affect CL performance. For this, we compare the efficacy of various pre-trained models in large-scale benchmarking scenarios with a vanilla replay setting applied in the latent and in the raw-data space. Notably, this study shows how transfer, forgetting, task similarity and learning are dependent on the input data characteristics and not necessarily on the CL algorithms. First, we show that under some circumstances reasonable CL performance can readily be achieved with a non-parametric classifier at negligible compute. We then show how models pre-trained on broader data result in better performance for various replay sizes. We explain this with representational similarity and transfer properties of these representations. Finally, we show the effectiveness of self-supervised pre-training for downstream domains that are out-of-distribution as compared to the pre-training domain. We point out and validate several research directions that can further increase the efficacy of latent CL including representation ensembling. The diverse set of datasets used in this study can serve as a compute-efficient playground for further CL research. The codebase is available under https://github.com/oleksost/latent_CL.

Unsupervised Domain Adaptation (UDA) is a transfer learning task which aims at training on an unlabeled target domain by leveraging a labeled source domain. Beyond the traditional scope of UDA with a single source domain and a single target domain, real-world perception systems face a variety of scenarios to handle, from varying lighting conditions to many cities around the world. In this context, UDAs with several domains increase the challenges with the addition of distribution shifts within the different target domains. This work focuses on a novel framework for learning UDA, continuous UDA, in which models operate on multiple target domains discovered sequentially, without access to previous target domains. We propose MuHDi, for Multi-Head Distillation, a method that solves the catastrophic forgetting problem, inherent in continual learning tasks. MuHDi performs distillation at multiple levels from the previous model as well as an auxiliary target-specialist segmentation head. We report both extensive ablation and experiments on challenging multi-target UDA semantic segmentation benchmarks to validate the proposed learning scheme and architecture., Comment: Published at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022 Workshop on Continual Learning

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined, static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques -- common in other ML domains -- and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition and the Waymo Open Dataset Motion Prediction Challenge.

Deep network architectures struggle to continually learn new tasks without forgetting the previous tasks. A recent trend indicates that dynamic architectures based on an expansion of the parameters can reduce catastrophic forgetting efficiently in continual learning. However, existing approaches often require a task identifier at test-time, need complex tuning to balance the growing number of parameters, and barely share any information across tasks. As a result, they struggle to scale to a large number of tasks without significant overhead. In this paper, we propose a transformer architecture based on a dedicated encoder/decoder framework. Critically, the encoder and decoder are shared among all tasks. Through a dynamic expansion of special tokens, we specialize each forward of our decoder network on a task distribution. Our strategy scales to a large number of tasks while having negligible memory and time overheads due to strict control of the parameters expansion. Moreover, this efficient strategy doesn't need any hyperparameter tuning to control the network's expansion. Our model reaches excellent results on CIFAR100 and state-of-the-art performances on the large-scale ImageNet100 and ImageNet1000 while having less parameters than concurrent dynamic frameworks., Comment: CVPR 2022, Code at https://github.com/arthurdouillard/dytox

Deep learning approaches are nowadays ubiquitously used to tackle computer vision tasks such as semantic segmentation, requiring large datasets and substantial computational power. Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. However, continual learning methods are usually prone to catastrophic forgetting. This issue is further aggravated in CSS where, at each step, old classes from previous iterations are collapsed into the background. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships at feature level. Furthermore, we design an entropy-based pseudo-labelling of the background w.r.t. classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes. Finally, we introduce a novel rehearsal method that is particularly suited for segmentation. Our approach, called PLOP, significantly outperforms state-of-the-art methods in existing CSS scenarios, as well as in newly proposed challenging benchmarks., Comment: Under review at IEEE TPAMI, journal extension of arXiv:2011.11390

Device-to-device (D2D) and non-orthogonal multiple access (NOMA) are promising technologies to meet the challenges of the next generations of mobile communications in terms of network density and diversity for internet of things (IoT) services. This paper tackles the problem of maximizing the D2D sum-throughput in an IoT system underlaying a cellular network, through optimal channel and power allocation. NOMA is used to manage the interference between cellular users and full-duplex (FD) IoT devices. To this aim, mutual successive interference cancellation (SIC) conditions are identified to allow simultaneously the removal of the D2D devices interference at the level of the base station and the removal of the cellular users (CU) interference at the level of D2D devices. To optimally solve the joint channel and power allocation (PA) problem, a time-efficient solution of the PA problem in the FD context is elaborated. By means of graphical representation, the complex non-convex PA problem is efficiently solved in constant time complexity. This enables the global optimal resolution by successively solving the separate PA and channel assignment problems. The performance of the proposed strategy is compared against the classical state-of-the-art FD and HD scenarios, where SIC is not applied between CUs and IoT devices. The results show that important gains can be achieved by applying mutual SIC NOMA in the IoT-cellular context, in either HD or FD scenarios., Comment: Under minor revision for future publication in IEEE Internet of Things Journal

Continual learning is a machine learning sub-field specialized in settings with non-iid data. Hence, the training data distribution is not static and drifts through time. Those drifts might cause interferences in the trained model and knowledge learned on previous states of the data distribution might be forgotten. Continual learning's challenge is to create algorithms able to learn an ever-growing amount of knowledge while dealing with data distribution drifts. One implementation difficulty in these field is to create data loaders that simulate non-iid scenarios. Indeed, data loaders are a key component for continual algorithms. They should be carefully designed and reproducible. Small errors in data loaders have a critical impact on algorithm results, e.g. with bad preprocessing, wrong order of data or bad test set. Continuum is a simple and efficient framework with numerous data loaders that avoid researcher to spend time on designing data loader and eliminate time-consuming errors. Using our proposed framework, it is possible to directly focus on the model design by using the multiple scenarios and evaluation metrics implemented. Furthermore the framework is easily extendable to add novel settings for specific needs., Comment: Code: https://github.com/Continvvm/continuum

To achieve high data rates and better connectivity in future communication networks, the deployment of different types of access points (APs) is underway. In order to limit human intervention and reduce costs, the APs are expected to be equipped with self-organizing capabilities. Moreover, due to the spectrum crunch, frequency reuse among the deployed APs is inevitable, aggravating the problem of inter-cell interference (ICI). Therefore, ICI mitigation in self-organizing networks (SONs) is commonly identified as a key radio resource management mechanism to enhance performance in future communication networks. With the aim of reducing ICI in a SON, this paper proposes a novel solution for the uncoordinated channel and power allocation problems. Based on the multi-player multi-armed bandit (MAB) framework, the proposed technique does not require any communication or coordination between the APs. The case of varying channel rewards across APs is considered. In contrast to previous work on channel allocation using the MAB framework, APs are permitted to choose multiple channels for transmission. Moreover, non-orthogonal multiple access (NOMA) is used to allow multiple APs to access each channel simultaneously. This results in an MAB model with varying channel rewards, multiple plays and non-zero reward on collision. The proposed algorithm has an expected regret in the order of O(log^2 T ), which is validated by simulation results. Extensive numerical results also reveal that the proposed technique significantly outperforms the well-known upper confidence bound (UCB) algorithm, by achieving more than a twofold increase in the energy efficiency., Comment: Submitted to IEEE Transactions on Communications

Deep learning approaches are nowadays ubiquitously used to tackle computer vision tasks such as semantic segmentation, requiring large datasets and substantial computational power. Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. However, continual learning methods are usually prone to catastrophic forgetting. This issue is further aggravated in CSS where, at each step, old classes from previous iterations are collapsed into the background. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships at feature level. Furthermore, we design an entropy-based pseudo-labelling of the background w.r.t. classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes. Our approach, called PLOP, significantly outperforms state-of-the-art methods in existing CSS scenarios, as well as in newly proposed challenging benchmarks., Comment: Accepted at CVPR 2021, code: https://github.com/arthurdouillard/CVPR2021_PLOP

Developing deep networks that analyze fashion garments has many real-world applications. Among all fashion attributes, color is one of the most important yet challenging to detect. Existing approaches are classification-based and thus cannot go beyond the list of discrete predefined color names. In this paper, we handle color detection as a regression problem to predict the exact RGB values. That's why in addition to a first color classifier, we include a second regression stage for refinement in our newly proposed architecture. This second step combines two attention models: the first depends on the type of clothing, the second depends on the color previously detected by the classifier. Our final prediction is the weighted spatial pooling over the image pixels RGB values, where the illumination has been corrected. This architecture is modular and easily expanded to detect the RGBs of all colors in a multicolor garment. In our experiments, we show the benefits of each component of our architecture., Comment: 6 pages,3 figures,1 table

Hybrid plasmonic nanoemitters based on the combination of quantum dot emitters (QD) and plasmonic nanoantennas open up new perspectives in the control of light. However, precise positioning of any active medium at the nanoscale constitutes a challenge. Here, we report on the optimal overlap of antenna's near-field and active medium whose spatial distribution is controlled via a plasmon-triggered 2-photon polymerization of a photosensitive formulation containing QDs. Au nanoparticles of various geometries are considered. The response of these hybrid nano-emitters is shown to be highly sensitive to the light polarization. Different light emission states are evidenced by photoluminescence measurements. These states correspond to polarization-sensitive nanoscale overlap between the exciting local field and the active medium distribution. The decrease of the QD concentration within the monomer formulation allows trapping of a single quantum dot in the vicinity of the Au particle. The latter objects show polarization-dependent switching in the single-photon regime.

Continual learning aims to learn tasks sequentially, with (often severe) constraints on the storage of old learning samples, without suffering from catastrophic forgetting. In this work, we propose prescient continual learning, a novel experimental setting, to incorporate existing information about the classes, prior to any training data. Usually, each task in a traditional continual learning setting evaluates the model on present and past classes, the latter with a limited number of training samples. Our setting adds future classes, with no training samples at all. We introduce Ghost Model, a representation-learning-based model for continual learning using ideas from zero-shot learning. A generative model of the representation space in concert with a careful adjustment of the losses allows us to exploit insights from future classes to constraint the spatial arrangement of the past and current classes. Quantitative results on the AwA2 and aP\&Y datasets and detailed visualizations showcase the interest of this new setting and the method we propose to address it.

Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning. In this work, we propose PODNet, a model inspired by representation learning. By carefully balancing the compromise between remembering the old classes and learning new ones, PODNet fights catastrophic forgetting, even over very long runs of small incremental tasks --a setting so far unexplored by current works. PODNet innovates on existing art with an efficient spatial-based distillation-loss applied throughout the model and a representation comprising multiple proxy vectors for each class. We validate those innovations thoroughly, comparing PODNet with three state-of-the-art models on three datasets: CIFAR100, ImageNet100, and ImageNet1000. Our results showcase a significant advantage of PODNet over existing art, with accuracy gains of 12.10, 6.51, and 2.85 percentage points, respectively. Code is available at https://github.com/arthurdouillard/incremental_learning.pytorch, Comment: Accepted at ECCV 2020

The development of damage-resistant structural materials that can withstand harsh environments is a major issue in materials science and engineering. Bioinspired brick-and-mortar designs have recently demonstrated a range of interesting mechanical properties in proof-of-concept studies. However, reproducibility and scalability issues associated with the actual processing routes have impeded further developments and industrialization of such materials. Here we demonstrate a simple approach based on uniaxial pressing and field assisted sintering of commercially available raw materials to process bioinspired ceramic/ceramic composites of larger thickness than previous approaches, with a sample thickness up to 1 cm. The ceramic composite retains the strength typical of dense alumina ($430~\pm 30MPa$) while keeping the excellent damage resistance demonstrated previously at the millimeter scale with a crack initiation toughness of $6.6MPa.m^{1/2}$ and fracture toughness up to $17.6 MPa.m^{1/2}$. These results validate the potential of these all-ceramic composites, previously demonstrated at lab scale only, and could enable their optimization, scale-up, and industrialization., Comment: 7 pages, 4 figures

The two-photon luminescence (TPL) of gold nanoparticles (NP) was shown to result from the excitation of hot carriers, the plasmonic NP resonances playing an important role both for plasmon enhanced absorption and plasmon enhanced emission. However, the exact parameters enabling to control or optimize the NP nonlinear luminescence still need to be understood in detail. In this paper, we report the two-photon excited photoluminescence of single gold nanorods exhibiting identical aspect ratio (close to 4) and thus identical plasmonic resonances, but increasing volumes V (707

Vehicle taillight recognition is an important application for automated driving, especially for intent prediction of ado vehicles and trajectory planning of the ego vehicle. In this work, we propose an end-to-end deep learning framework to recognize taillights, i.e. rear turn and brake signals, from a sequence of images. The proposed method starts with a Convolutional Neural Network (CNN) to extract spatial features, and then applies a Long Short-Term Memory network (LSTM) to learn temporal dependencies. Furthermore, we integrate attention models in both spatial and temporal domains, where the attention models learn to selectively focus on both spatial and temporal features. Our method is able to outperform the state of the art in terms of accuracy on the UC Merced Vehicle Rear Signal Dataset, demonstrating the effectiveness of attention models for vehicle taillight recognition., Comment: Accepted by IV 2019

The effervescence and the stability of the ring of fine bubbles crowning the surface of a champagne glass, the "collar", constitute one of the hallmarks of Champagne. Defects in the stability of this collar are not well understood and account for a significant proportion of bottle return. This study aims to better understand the surface properties of champagne wine such that the foaming properties can be controlled more effectively. Early studies on Champagne foaming properties using the "Mosalux" measurement of the foam level formed by air flow in champagne through fritted glass pointed to a link between protein concentration and foam level [5] but no satisfactory correlation between protein content and foam stability was established. Later measurements were conducted with either ultra-filtrates or ultra-concentrates [4]. The authors demonstrated that macromolecule concentration was an essential parameter in the foam stability. The stability of bubbles is usually ascribed to the presence of an adsorption layer formed at the gas/liquid interface and its properties [3]. Thus surface properties of Champagne were analysed by ellipsometry and tensiometry. Measurements conducted on base wine, on ultra-filtered base wines and degassed champagne samples showed the presence of an adsorption layer formed at the air/champagne wine interface [6] and that adsorption layer being composed of macromolecules in a 104 to 105 molecular range [7]. Previous studies on champagne wine macromolecules had shown wine macromolecules to be mostly proteins and polysaccharides [8] with very little insight as to the chemical constitution. The present study describes the isolation and characterization of these macromolecules and their link with the adsorption layer., Comment: https://www.lavoisier.fr/livre/agro-alimentaire/macromolecules-and-secondary-metabolites-of-grapevine-and-wine-fruit-development-biotic-and-abiotic-stresses/jeandet/descriptif-9782743009656

The enhancement and control of non-linear phenomena at a nanometer scale has a wide range of applications in science and in industry. Among these phenomena, high-harmonic generation in solids is a recent focus of research to realize next generation petahertz optoelectronic devices or compact all solid state EUV sources. Here, we report on the realization of the first nanoscale high harmonic source. The strong field regime is reached by confining the electric field from a few nanojoules femtosecond laser in a single 3D semiconductor waveguide. We reveal a strong competition between enhancement of coherent harmonics and incoherent fluorescence favored by excitonic processes. However, far from the band edge, clear enhancement of the harmonic emission is reported with a robust sustainability offering a compact nanosource for applications. We illustrate the potential of our harmonic nano-device by performing a coherent diffractive imaging experiment. Ultra-compact UV/X-ray nanoprobes are foreseen to have other potential applications such as petahertz electronics, nano-tomography or nano-medicine.

This paper introduces new approaches for combining non-orthogonal multiple access with distributed antenna systems. The study targets a minimization of the total transmit power in each cell, under user rate and power multiplexing constraints. Several new suboptimal power allocation techniques are proposed. They are shown to yield very close performance to an optimal power allocation scheme. Also, a new approach based on mutual successive interference cancellation of paired users is proposed. Different techniques are designed for the joint allocation of subcarriers, antennas, and power, with a particular care given to maintain a moderate complexity. The coupling of non-orthogonal multiple access to distributed antenna systems is shown to greatly outperform any other combination of orthogonal/non-orthogonal multiple access schemes with distributed or centralized deployment scenarios., Comment: Paper accepted for publication in IEEE Trans. Vehicular Technology, 14 pages, 5 figures, 2 tables. arXiv admin note: substantial text overlap with arXiv:1710.06619

This paper proposes a new family of recursive systematic convolutional codes, defined in the non-binary domain over different Galois fields GF(q) and intended to be used as component codes for the design of non-binary turbo codes. A general framework for the design of the best codes over different GF(q) is described. The designed codes offer better performance than the non-binary convolutional codes found in the literature. They also outperform their binary counterparts when combined with their corresponding QAM modulation or with lower order modulations., Comment: Accepted for publication in IEEE-Globecom 2018 conference

In this paper, a weighted proportional fair (PF) scheduling method is proposed in the context of non-orthogonal multiple access (NOMA) with successive interference cancellation (SIC) at the receiver side. The new scheme introduces weights that adapt the classical PF metric to the NOMA scenario, improving performance indicators and enabling new services. The distinguishing value of the proposal resides in its ability to improve long term fairness and total system throughput while achieving a high level of fairness in every scheduling slot. Finally, it is shown that the additional complexity caused by the weight calculation has only a limited impact on the overall scheduler complexity while simulation results confirm the claimed improvements making the proposal an appealing alternative for resource allocation in a cellular downlink system., Comment: 11 pages, 8 figures, 2 tables. Submitted paper

This paper introduces new approaches for combining non-orthogonal multiple access (NOMA) with distributed base station (DBS) deployments. The purpose of the study is to unlock the true potentials of DBS systems in the NOMA context, since all previous works dealing with power minimization in NOMA are performed in the CBS (centralized base station) context. This work targets a minimization of the total transmit power in each cell, under user rate and power multiplexing constraints. Different techniques are designed for the joint allocation of subcarriers, antennas and power, with a particular care given to insuring a moderate complexity. Results show an important gain in the total transmit power obtained by the DBS-NOMA combination, with respect to both DBS-OMA (orthogonal multiple access) and CBS-NOMA deployment scenarios., Comment: Submitted paper