Search

Perception and prediction modules are critical components of autonomous driving systems, enabling vehicles to navigate safely through complex environments. The perception module is responsible for perceiving the environment, including static and dynamic objects, while the prediction module is responsible for predicting the future behavior of these objects. These modules are typically divided into three tasks: object detection, object tracking, and motion prediction. Traditionally, these tasks are developed and optimized independently, with outputs passed sequentially from one to the next. However, this approach has significant limitations: computational resources are not shared across tasks, the lack of joint optimization can amplify errors as they propagate throughout the pipeline, and uncertainty is rarely propagated between modules, resulting in significant information loss. To address these challenges, the joint perception and prediction paradigm has emerged, integrating perception and prediction into a unified model through multi-task learning. This strategy not only overcomes the limitations of previous methods, but also enables the three tasks to have direct access to raw sensor data, allowing richer and more nuanced environmental interpretations. This paper presents the first comprehensive survey of joint perception and prediction for autonomous driving. We propose a taxonomy that categorizes approaches based on input representation, scene context modeling, and output representation, highlighting their contributions and limitations. Additionally, we present a qualitative analysis and quantitative comparison of existing methods. Finally, we discuss future research directions based on identified gaps in the state-of-the-art., Comment: 24 pages, 5 sections, 7 figures, 7 tables. This work has been submitted to the IEEE Transactions on Intelligent Transportation Systems for possible publication

The development of Autonomous Driving (AD) systems in simulated environments like CARLA is crucial for advancing real-world automotive technologies. To drive innovation, CARLA introduced Leaderboard 2.0, significantly more challenging than its predecessor. However, current AD methods have struggled to achieve satisfactory outcomes due to a lack of sufficient ground truth data. Human driving logs provided by CARLA are insufficient, and previously successful expert agents like Autopilot and Roach, used for collecting datasets, have seen reduced effectiveness under these more demanding conditions. To overcome these data limitations, we introduce PRIBOOT, an expert agent that leverages limited human logs with privileged information. We have developed a novel BEV representation specifically tailored to meet the demands of this new benchmark and processed it as an RGB image to facilitate the application of transfer learning techniques, instead of using a set of masks. Additionally, we propose the Infraction Rate Score (IRS), a new evaluation metric designed to provide a more balanced assessment of driving performance over extended routes. PRIBOOT is the first model to achieve a Route Completion (RC) of 75% in Leaderboard 2.0, along with a Driving Score (DS) and IRS of 20% and 45%, respectively. With PRIBOOT, researchers can now generate extensive datasets, potentially solving the data availability issues that have hindered progress in this benchmark.

The non-thermal breakdown of a Mott insulator has been a topic of great theoretical and experimental interest with technological relevance. Recent experiments have found a sharp non-equilibrium insulator-to-metal transition that is accompanied by hysteresis, a negative differential conductance and lattice deformations. However, a thorough understanding of the underlying breakdown mechanism is still lacking. Here, we examine a scenario in which the breakdown is induced by chemical pressure in a paradigmatic model of interacting spinless fermions on a chain coupled to metallic reservoirs (leads). For the Markovian regime, at infinite bias, we qualitatively reproduce several established results. Beyond infinite bias, we find a rich phase diagram where the nature of the breakdown depends on the coupling strength as the bias voltage is tuned up, yielding different current-carrying non-equilibrium phases. For weak to intermediate coupling, we find a conducting CDW phase with a bias-dependent ordering wave vector. At large interaction strength, the breakdown connects the system to a charge-separated insulating phase. We find instances of hysteretic behavior, sharp current onset and negative differential conductance. Our results can help to shed light on recent experimental findings that address current-induced Mott breakdown., Comment: 10+9 pages, 4+8 figures

We study the effect of diluting a two-dimensional ferromagnetic insulator hosting a topological phase in the clean limit. By considering the ferromagnetic Heisenberg model in the honeycomb lattice with second nearest-neighbor Dzyanshikii-Moriya interaction, and working in the linear spin-wave approximation, we establish the topological phase diagram as a function of the fraction $p$ of diluted magnetic atoms. The topological phase with Chern number $C=1$ is robust up to a moderate dilution $p_{1}^{*}$, while above a higher dilution $p_{2}^{*}>p_{1}^{*}$ the system becomes trivial. Interestingly, both $p_{1}^{*}$ and $p_{2}^{*}$ are below the classical percolation threshold $p_{c}$ for the honeycomb lattice, which gives physical significance to the obtained phases. In the topological phase for $pp_{2}^{*}$, when all states become localized, the system shows $C=0$ as expected for a trivial phase. Replacing magnetic with non-magnetic atoms in a systems hosting a magnon Chern insulator in the clean limit puts all the three phases within experimental reach., Comment: 10 pages, 6 figures

Individuals and organizations cope with an always-growing amount of data, which is heterogeneous in its contents and formats. An adequate data management process yielding data quality and control over its lifecycle is a prerequisite to getting value out of this data and minimizing inherent risks related to multiple usages. Common data governance frameworks rely on people, policies, and processes that fall short of the overwhelming complexity of data. Yet, harnessing this complexity is necessary to achieve high-quality standards. The latter will condition any downstream data usage outcome, including generative artificial intelligence trained on this data. In this paper, we report our concrete experience establishing a simple, cost-efficient framework that enables metadata-driven, agile and (semi-)automated data governance (i.e. Data Governance 4.0). We explain how we implement and use this framework to integrate 25 years of clinical study data at an enterprise scale in a fully productive environment. The framework encompasses both methodologies and technologies leveraging semantic web principles. We built a knowledge graph describing avatars of data assets in their business context, including governance principles. Multiple ontologies articulated by an enterprise upper ontology enable key governance actions such as FAIRification, lifecycle management, definition of roles and responsibilities, lineage across transformations and provenance from source systems. This metadata model is the keystone to data governance 4.0: a semi-automatised data management process that considers the business context in an agile manner to adapt governance constraints to each use case and dynamically tune it based on business changes.

Despite the growing interest in innovative functionalities for collaborative robotics, volumetric detection remains indispensable for ensuring basic security. However, there is a lack of widely used volumetric detection frameworks specifically tailored to this domain, and existing evaluation metrics primarily focus on time and memory efficiency. To bridge this gap, the authors present a detailed comparison using a simulation environment, ground truth extraction, and automated evaluation metrics calculation. This enables the evaluation of state-of-the-art volumetric mapping algorithms, including OctoMap, SkiMap, and Voxblox, providing valuable insights and comparisons through the impact of qualitative and quantitative analyses. The study not only compares different frameworks but also explores various parameters within each framework, offering additional insights into their performance., Comment: 11 pages, 11 figures, 9 tables

Current approaches of Reinforcement Learning (RL) applied in urban Autonomous Driving (AD) focus on decoupling the perception training from the driving policy training. The main reason is to avoid training a convolution encoder alongside a policy network, which is known to have issues related to sample efficiency, degenerated feature representations, and catastrophic self-overfitting. However, this paradigm can lead to representations of the environment that are not aligned with the downstream task, which may result in suboptimal performances. To address this limitation, this paper proposes RLAD, the first Reinforcement Learning from Pixels (RLfP) method applied in the urban AD domain. We propose several techniques to enhance the performance of an RLfP algorithm in this domain, including: i) an image encoder that leverages both image augmentations and Adaptive Local Signal Mixing (A-LIX) layers; ii) WayConv1D, which is a waypoint encoder that harnesses the 2D geometrical information of the waypoints using 1D convolutions; and iii) an auxiliary loss to increase the significance of the traffic lights in the latent representation of the environment. Experimental results show that RLAD significantly outperforms all state-of-the-art RLfP methods on the NoCrash benchmark. We also present an infraction analysis on the NoCrash-regular benchmark, which indicates that RLAD performs better than all other methods in terms of both collision rate and red light infractions., Comment: in IEEE Transactions on Automation Science and Engineering

Collecting real-world data is often considered the bottleneck of Artificial Intelligence, stalling the research progress in several fields, one of which is camera localization. End-to-end camera localization methods are still outperformed by traditional methods, and we argue that the inconsistencies associated with the data collection techniques are restraining the potential of end-to-end methods. Inspired by the recent data-centric paradigm, we propose a framework that synthesizes large localization datasets based on realistic 3D reconstructions of the real world. Our framework, termed Synfeal: Synthetic from Real, is an open-source, data-driven simulator that synthesizes RGB images by moving a virtual camera through a realistic 3D textured mesh, while collecting the corresponding ground-truth camera poses. The results validate that the training of camera localization algorithms on datasets generated by Synfeal leads to better results when compared to datasets generated by state-of-the-art methods. Using Synfeal, we conducted the first analysis of the relationship between the size of the dataset and the performance of camera localization algorithms. Results show that the performance significantly increases with the dataset size. Our results also suggest that when a large localization dataset with high quality is available, training from scratch leads to better performances. Synfeal is publicly available at https://github.com/DanielCoelho112/synfeal.

Collaborative robotic industrial cells are workspaces where robots collaborate with human operators. In this context, safety is paramount, and for that a complete perception of the space where the collaborative robot is inserted is necessary. To ensure this, collaborative cells are equipped with a large set of sensors of multiple modalities, covering the entire work volume. However, the fusion of information from all these sensors requires an accurate extrinsic calibration. The calibration of such complex systems is challenging, due to the number of sensors and modalities, and also due to the small overlapping fields of view between the sensors, which are positioned to capture different viewpoints of the cell. This paper proposes a sensor to pattern methodology that can calibrate a complex system such as a collaborative cell in a single optimization procedure. Our methodology can tackle RGB and Depth cameras, as well as LiDARs. Results show that our methodology is able to accurately calibrate a collaborative cell containing three RGB cameras, a depth camera and three 3D LiDARs., Comment: Journal of Manufacturing Systems

In this work we generalize the MacDowell-Mansouri theory of gravity using strict 2-groups. To achieve this, we construct the categorical generalization of the ISO(4,1) group, which we call the de Sitter 2-group. We then proceed to generalize the MacDowell-Mansouri theory in two different ways. First, as a Yang-Mills-type theory, where the symmetry is explicitly broken to obtain an Einstein-Cartan theory coupled to Kalb-Ramond fields. And second, by using the categorical generalization of the BF theory, called 2-BF theory, which after the addition of some symmetry breaking and constraint terms gives the same Einstein-Cartan theory coupled to Kalb-Ramond fields plus some extra topological terms., Comment: 20 pages, no figures

Quantum information-based approaches, in particular the fidelity, have been flexible probes for phase boundaries of quantum matter. A major hurdle to a more widespread application of fidelity and other quantum information measures to strongly correlated quantum materials is the inaccessibility of the fidelity susceptibility to most state-of-the-art numerical methods. This is particularly apparent away from equilibrium where, at present, no general critical theory is available and many standard techniques fail. Motivated by the usefulness of quantum information based measures we show that a widely accessible quantity, the single-particle affinity, is able to serve as a versatile instrument to identify phase transitions beyond Landau's paradigm. We demonstrate that it not only is able to signal previously identified non-equilibrium phase transitions but also has the potential to detect hitherto unknown phases in models of quantum matter far from equilibrium., Comment: 8+4 pages, 3+5 figures