Search

Autonomous motorsports aim to replicate the human racecar driver with software and sensors. As in traditional motorsports, Autonomous Racing Vehicles (ARVs) are pushed to their handling limits in multi-agent scenarios at extremely high ($\geq 150mph$) speeds. This Operational Design Domain (ODD) presents unique challenges across the autonomy stack. The Indy Autonomous Challenge (IAC) is an international competition aiming to advance autonomous vehicle development through ARV competitions. While far from challenging what a human racecar driver can do, the IAC is pushing the state of the art by facilitating full-sized ARV competitions. This paper details the MIT-Pitt-RW Team's approach to autonomous racing in the IAC. In this work, we present our modular and fast approach to agent detection, motion planning and controls to create an autonomy stack. We also provide analysis of the performance of the software stack in single and multi-agent scenarios for rapid deployment in a fast-paced competition environment. We also cover what did and did not work when deployed on a physical system the Dallara AV-21 platform and potential improvements to address these shortcomings. Finally, we convey lessons learned and discuss limitations and future directions for improvement., Comment: Published in Journal of Field Robotics

Optical trapping is having ever-increasing impact in science $-$ particularly biophysics, photonics and most recently in quantum optomechanics $-$ owing to its superior capability for manipulating nanoscale structures and materials. However, essentially all experimental optical trapping studies in the optical dipole regime have, to date, been dominated by the interaction between a material's electric polarizability, $\alpha_{e}$, and the electric part of the incident electromagnetic field, and therefore described by electric field intensity gradient forces. Optical trapping based on optical magnetic light-matter interactions has not been experimentally addressed despite it's immediate extension of the boundaries of optical trapping research and applications. This paper addresses this long-standing deficiency through the realization of optical magnetic trapping of large index of refraction (i.e., Si) nanoparticles and also presents a formalism for quantitative understanding of the experimental findings. Our experimental optical trapping results require including optical magnetic polarizability, $\alpha_{m}$, and electric-magnetic scattering forces associated with the Photonic Hall effect that are qualitatively and quantitatively validated by Maxwell stress tensor calculations. Our findings bring new opportunities for nanoparticle manipulation, potentially relax the limitations Ashkin claimed based on the optical Earnshaw's theorem, motivate optical matter formation by optical magnetic interactions, and suggest new N-body effects and symmetry breaking to drive dynamics of optical matter systems.

With conventional silicon-based computing approaching its physical and efficiency limits, biocomputing emerges as a promising alternative. This approach utilises biomaterials such as DNA and neurons as an interesting alternative to data processing and storage. This study explores the potential of neuronal biocomputing to rival silicon-based systems. We explore neuronal logic gates and sequential circuits that mimic conventional computer architectures. Through mathematical modelling, optimisation, and computer simulation, we demonstrate the operational capabilities of neuronal sequential circuits. These circuits include a neuronal NAND gate, SR Latch flip-flop, and D flip-flop memory units. Our approach involves manipulating neuron communication, synaptic conductance, spike buffers, neuron types, and specific neuronal network topology designs. The experiments demonstrate the practicality of encoding binary information using patterns of neuronal activity and overcoming synchronization difficulties with neuronal buffers and inhibition strategies. Our results confirm the effectiveness and scalability of neuronal logic circuits, showing that they maintain a stable metabolic burden even in complex data storage configurations. Our study not only demonstrates the concept of embodied biocomputing by manipulating neuronal properties for digital signal processing but also establishes the foundation for cutting-edge biocomputing technologies. Our designs open up possibilities for using neurons as energy-efficient computing solutions. These solutions have the potential to become an alternate to silicon-based systems by providing a carbon-neutral, biologically feasible alternative.

With the rise of Embodied Foundation Models (EFMs), most notably Small Language Models (SLMs), adapting Transformers for edge applications has become a very active field of research. However, achieving end-to-end deployment of SLMs on microcontroller (MCU)-class chips without high-bandwidth off-chip main memory access is still an open challenge. In this paper, we demonstrate high-efficiency end-to-end SLM deployment on a multicore RISC-V (RV32) MCU augmented with ML instruction extensions and a hardware neural processing unit (NPU). To automate the exploration of the constrained, multi-dimensional memory vs. computation tradeoffs involved in aggressive SLM deployment on heterogeneous (multicore+NPU) resources, we introduce Deeploy, a novel Deep Neural Network (DNN) compiler, which generates highly-optimized C code requiring minimal runtime support. We demonstrate that Deeploy generates end-to-end code for executing SLMs, fully exploiting the RV32 cores' instruction extensions and the NPU: We achieve leading-edge energy and throughput of \SI{490}{\micro\joule \per Token}, at \SI{340}{Token \per \second} for an SLM trained on the TinyStories dataset, running for the first time on an MCU-class device without external memory., Comment: Accepted for publication at ESWEEK - CASES 2024

One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate an Attention-based model in a tinyML power envelope with an octa-core cluster coupled with an accelerator for quantized Attention. Our deployment flow enables an end-to-end 8-bit MobileBERT, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154 GOp/s at 32.5 Inf/s consuming 52.0 mW (0.65 V, 22 nm FD-SOI technology)., Comment: Pre-print manuscript submitted for review to the IEEE Design and Test Special Issue on tinyML

High-redshift radio(-loud) galaxies (H$z$RGs) are massive galaxies with powerful radio-loud active galactic nuclei (AGNs) and serve as beacons for protocluster identification. However, the interplay between H$z$RGs and the large-scale environment remains unclear. To understand the connection between H$z$RGs and the surrounding obscured star formation, we investigated the overdensity and spatial distribution of submillimeter-bright galaxies (SMGs) in the field of 4C\,23.56, a well-known H$z$RG at $z=2.48$. We used SCUBA-2 data ($\sigma\,{\sim}\,0.6$\,mJy) to estimate the $850\,{\rm \mu m}$ source number counts and examine the radial and azimuthal overdensities of the $850\,{\rm \mu m}$ sources in the vicinity of the H$z$RG. The angular distribution of SMGs is inhomogeneous around the H$z$RG 4C\,23.56, with fewer sources oriented along the radio jet. We also find a significant overdensity of bright SMGs (${\rm S}_{850\rm\,\mu m}\geq5\,$mJy). Faint and bright SMGs exhibit different spatial distributions. The former are concentrated in the core region, while the latter prefer the outskirts of the H$z$RG field. High-resolution observations show that the seven brightest SMGs in our sample are intrinsically bright, suggesting that the overdensity of bright SMGs is less likely due to the source multiplicity., Comment: 19 pages, 17 figures, 5 tables, accepted to A&A

In this paper, we introduce a variant of video object segmentation (VOS) that bridges interactive and semi-automatic approaches, termed Lazy Video Object Segmentation (ziVOS). In contrast, to both tasks, which handle video object segmentation in an off-line manner (i.e., pre-recorded sequences), we propose through ziVOS to target online recorded sequences. Here, we strive to strike a balance between performance and robustness for long-term scenarios by soliciting user feedback's on-the-fly during the segmentation process. Hence, we aim to maximize the tracking duration of an object of interest, while requiring minimal user corrections to maintain tracking over an extended period. We propose a competitive baseline, i.e., Lazy-XMem, as a reference for future works in ziVOS. Our proposed approach uses an uncertainty estimation of the tracking state to determine whether a user interaction is necessary to refine the model's prediction. To quantitatively assess the performance of our method and the user's workload, we introduce complementary metrics alongside those already established in the field. We evaluate our approach using the recently introduced LVOS dataset, which offers numerous long-term videos. Our code is publicly available at https://github.com/Vujas-Eteph/LazyXMem.

The growing demand for air travel requires technological advancements in air traffic management as well as mechanisms for monitoring and ensuring safe and efficient operations. In terminal airspaces, predictive models of future movements and traffic flows can help with proactive planning and efficient coordination; however, varying airport topologies, and interactions with other agents, among other factors, make accurate predictions challenging. Data-driven predictive models have shown promise for handling numerous variables to enable various downstream tasks, including collision risk assessment, taxi-out time prediction, departure metering, and emission estimations. While data-driven methods have shown improvements in these tasks, prior works lack large-scale curated surface movement datasets within the public domain and the development of generalizable trajectory forecasting models. In response to this, we propose two contributions: (1) Amelia-48, a large surface movement dataset collected using the System Wide Information Management (SWIM) Surface Movement Event Service (SMES). With data collection beginning in Dec 2022, the dataset provides more than a year's worth of SMES data (~30TB) and covers 48 airports within the US National Airspace System. In addition to releasing this data in the public domain, we also provide post-processing scripts and associated airport maps to enable research in the forecasting domain and beyond. (2) Amelia-TF model, a transformer-based next-token-prediction large multi-agent multi-airport trajectory forecasting model trained on 292 days or 9.4 billion tokens of position data encompassing 10 different airports with varying topology. The open-sourced model is validated on unseen airports with experiments showcasing the different prediction horizon lengths, ego-agent selection strategies, and training recipes to demonstrate the generalization capabilities., Comment: 24 pages, 9 figures, 8 tables

Prime-Cam is a first-generation science instrument for the CCAT Observatory's six-meter aperture Fred Young Submillimeter Telescope (FYST). FYST's crossed-Dragone design provides high optical throughput to take advantage of its unique site at 5600 m on Cerro Chajnantor in Chile's Atacama Desert to reach mapping speeds over ten times greater than current and near-term submillimeter experiments. Housing up to seven independent instrument modules in its 1.8-meter diameter cryostat, Prime-Cam will combine broadband polarization-sensitive modules and spectrometer modules designed for observations in several frequency windows between 210 GHz and 850 GHz to study a wide range of astrophysical questions from Big Bang cosmology to the formation of stars and galaxies in the Epoch of Reionization and beyond. In order to cover this range of frequencies and observation modes, each of the modules contains a set of cold reimaging optics that is optimized for the science goals of that module. These optical setups include several filters, three or four anti-reflection-coated silicon lenses, and a Lyot stop to control the field of view and illumination of the primary mirror, satisfy a series of mechanical constraints, and maximize optical performance within each passband. We summarize the design considerations and trade-offs for the optics in these modules and provide a status update on the fabrication of the Prime-Cam receiver and the design of its 1 K and 100 mK thermal BUSs., Comment: 10 pages, 6 figures, presented at SPIE Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy XII

Interaction-induced charge orders with electronic origin occur as states of spontaneously broken symmetry in several materials platforms. An electronic mechanism for charge order requires an attractive component in the effective charge vertex. We put forward such a mechanism for the formation of unconventional charge density waves in a metal. These states result from the condensation of particle-hole pairs with finite wave vector and non-zero angular momentum and correspond to bond or loop current order on a lattice. The mechanism we describe can be viewed as Kohn Luttinger analysis in the particle-hole channel with finite transferred momentum. It incorporates one-loop spin and pairing correctionsn, which are then used as an input for a summation in the charge channel triggering an instability. We extend our analysis to a spin-fluctuation approach, where the effective charge interaction is dressed by the particle-hole ladder with exchanged momentum. We argue that this mechanism works for weakly-interacting metals with nested Fermi surface and a large number of fermion flavors. We apply the Kohn-Luttinger-like approach to square- and triangular-lattice Hubbard models with SU($N_f$) flavour symmetry and show that it leads to different types of $p$-wave charge density waves. We also study effects beyond weak coupling at and away from Van Hove filling in terms of a phenomenological model with additional exchange interaction. In the vicinity of Van Hove filling, we obtain $d$-wave charge density waves with wave vectors determined by nesting as leading instabilities. In addition, we find another charge density wave with wave vector $K/4$ on the triangular lattice on both sides of Van Hove filling. We demonstrate that this $K/4$ instability can win the competition against pairing for $N_f=4$ via an unbiased functional renormalisation group calculation., Comment: 12+5 pages, 8+1 figures

The Simons Observatory (SO) is a cosmic microwave background experiment composed of three 0.42 m Small Aperture Telescopes (SATs) and one 6 m Large Aperture Telescope (LAT) in the Atacama Desert of Chile. The Large Aperture Telescope Receiver (LATR) was integrated into the LAT in August 2023; however, because mirrors were not yet installed, the LATR optical chain was capped at the 4K stage. In this dark configuration we are able to characterize many elements of the instrument without contributions from atmospheric noise. Here we show this noise is below the required upper limit and its features are well described with a simple noise model. Maps produced using this noise model have properties that are in good agreement with the white noise levels of our dark data. Additionally, we show that our nominal scan strategy has a minimal effect on the noise when compared to the noise when the telescope is stationary

Purpose: Federated training is often hindered by heterogeneous datasets due to divergent data storage options, inconsistent naming schemes, varied annotation procedures, and disparities in label quality. This is particularly evident in the emerging multi-modal learning paradigms, where dataset harmonization including a uniform data representation and filtering options are of paramount importance. Methods: DICOM structured reports enable the standardized linkage of arbitrary information beyond the imaging domain and can be used within Python deep learning pipelines with highdicom. Building on this, we developed an open platform for data integration and interactive filtering capabilities that simplifies the process of assembling multi-modal datasets. Results: In this study, we extend our prior work by showing its applicability to more and divergent data types, as well as streamlining datasets for federated training within an established consortium of eight university hospitals in Germany. We prove its concurrent filtering ability by creating harmonized multi-modal datasets across all locations for predicting the outcome after minimally invasive heart valve replacement. The data includes DICOM data (i.e. computed tomography images, electrocardiography scans) as well as annotations (i.e. calcification segmentations, pointsets and pacemaker dependency), and metadata (i.e. prosthesis and diagnoses). Conclusion: Structured reports bridge the traditional gap between imaging systems and information systems. Utilizing the inherent DICOM reference system arbitrary data types can be queried concurrently to create meaningful cohorts for clinical studies. The graphical interface as well as example structured report templates will be made publicly available.

Top-down Bird's Eye View (BEV) maps are a popular representation for ground robot navigation due to their richness and flexibility for downstream tasks. While recent methods have shown promise for predicting BEV maps from First-Person View (FPV) images, their generalizability is limited to small regions captured by current autonomous vehicle-based datasets. In this context, we show that a more scalable approach towards generalizable map prediction can be enabled by using two large-scale crowd-sourced mapping platforms, Mapillary for FPV images and OpenStreetMap for BEV semantic maps. We introduce Map It Anywhere (MIA), a data engine that enables seamless curation and modeling of labeled map prediction data from existing open-source map platforms. Using our MIA data engine, we display the ease of automatically collecting a dataset of 1.2 million pairs of FPV images & BEV maps encompassing diverse geographies, landscapes, environmental factors, camera models & capture scenarios. We further train a simple camera model-agnostic model on this data for BEV map prediction. Extensive evaluations using established benchmarks and our dataset show that the data curated by MIA enables effective pretraining for generalizable BEV map prediction, with zero-shot performance far exceeding baselines trained on existing datasets by 35%. Our analysis highlights the promise of using large-scale public maps for developing & testing generalizable BEV perception, paving the way for more robust autonomous navigation.

Traversability estimation in rugged, unstructured environments remains a challenging problem in field robotics. Often, the need for precise, accurate traversability estimation is in direct opposition to the limited sensing and compute capability present on affordable, small-scale mobile robots. To address this issue, we present a novel method to learn [u]ncertainty-aware [n]avigation features from high-fidelity scans of [real]-world environments (UNRealNet). This network can be deployed on-robot to predict these high-fidelity features using input from lower-quality sensors. UNRealNet predicts dense, metric-space features directly from single-frame lidar scans, thus reducing the effects of occlusion and odometry error. Our approach is label-free, and is able to produce traversability estimates that are robot-agnostic. Additionally, we can leverage UNRealNet's predictive uncertainty to both produce risk-aware traversability estimates, and refine our feature predictions over time. We find that our method outperforms traditional local mapping and inpainting baselines by up to 40%, and demonstrate its efficacy on multiple legged platforms.

Federated learning (FL) is a renowned technique for utilizing decentralized data while preserving privacy. However, real-world applications often involve inherent challenges such as partially labeled datasets, where not all clients possess expert annotations of all labels of interest, leaving large portions of unlabeled data unused. In this study, we conduct the largest federated cardiac CT imaging analysis to date, focusing on partially labeled datasets ($n=8,124$) of Transcatheter Aortic Valve Implantation (TAVI) patients over eight hospital clients. Transformer architectures, which are the major building blocks of current foundation models, have shown superior performance when trained on larger cohorts than traditional CNNs. However, when trained on small task-specific labeled sample sizes, it is currently not feasible to exploit their underlying attention mechanism for improved performance. Therefore, we developed a two-stage semi-supervised learning strategy that distills knowledge from several task-specific CNNs (landmark detection and segmentation of calcification) into a single transformer model by utilizing large amounts of unlabeled data typically residing unused in hospitals to mitigate these issues. This method not only improves the predictive accuracy and generalizability of transformer-based architectures but also facilitates the simultaneous learning of all partial labels within a single transformer model across the federation. Additionally, we show that our transformer-based model extracts more meaningful features for further downstream tasks than the UNet-based one by only training the last layer to also solve segmentation of coronary arteries. We make the code and weights of the final model openly available, which can serve as a foundation model for further research in cardiac CT imaging.

Aerial manipulation has gained interest in completing high-altitude tasks that are challenging for human workers, such as contact inspection and defect detection, etc. Previous research has focused on maintaining static contact points or forces. This letter addresses a more general and dynamic task: simultaneously tracking time-varying contact force in the surface normal direction and motion trajectories on tangential surfaces. We propose a pipeline that includes a contact-aware trajectory planner to generate dynamically feasible trajectories, and a hybrid motion-force controller to track such trajectories. We demonstrate the approach in an aerial calligraphy task using a novel sponge pen design as the end-effector, whose stroke width is proportional to the contact force. Additionally, we develop a touchscreen interface for flexible user input. Experiments show our method can effectively draw diverse letters, achieving an IoU of 0.59 and an end-effector position (force) tracking RMSE of 2.9 cm (0.7 N). Website: https://xiaofeng-guo.github.io/flying-calligrapher/, Comment: 8 pages, 9 figures, 1 table

Uncrewed Aerial Vehicles (UAVs) are a leading choice of platforms for a variety of information-gathering applications. Sensor planning can enhance the efficiency and success of these types of missions when coupled with a higher-level informative path-planning algorithm. This paper aims to address these data acquisition challenges by developing an informative non-myopic sensor planning framework for a single-axis gimbal coupled with an informative path planner to maximize information gain over a prior information map. This is done by finding reduced sensor sweep bounds over a planning horizon such that regions of higher confidence are prioritized. This novel sensor planning framework is evaluated against a predefined sensor sweep and no sensor planning baselines as well as validated in two simulation environments. In our results, we observe an improvement in the performance by 21.88% and 13.34% for the no sensor planning and predefined sensor sweep baselines respectively., Comment: 7 pages, 6 figures, CASE 2024

We present a high-significance cross-correlation of CMB lensing maps from the Atacama Cosmology Telescope (ACT) Data Release 6 (DR6) with spectroscopically calibrated luminous red galaxies (LRGs) from the Dark Energy Spectroscopic Instrument (DESI). We detect this cross-correlation at a significance of 38$\sigma$; combining our measurement with the Planck Public Release 4 (PR4) lensing map, we detect the cross-correlation at 50$\sigma$. Fitting this jointly with the galaxy auto-correlation power spectrum to break the galaxy bias degeneracy with $\sigma_8$, we perform a tomographic analysis in four LRG redshift bins spanning $0.4 \le z \le 1.0$ to constrain the amplitude of matter density fluctuations through the parameter combination $S_8^\times = \sigma_8 \left(\Omega_m / 0.3\right)^{0.4}$. Prior to unblinding, we confirm with extragalactic simulations that foreground biases are negligible and carry out a comprehensive suite of null and consistency tests. Using a hybrid effective field theory (HEFT) model that allows scales as small as $k_{\rm max}=0.6$ $h/{\rm Mpc}$, we obtain a 3.3% constraint on $S_8^\times = \sigma_8 \left(\Omega_m / 0.3\right)^{0.4} = 0.792^{+0.024}_{-0.028}$ from ACT data, as well as constraints on $S_8^\times(z)$ that probe structure formation over cosmic time. Our result is consistent with the early-universe extrapolation from primary CMB anisotropies measured by Planck PR4 within 1.2$\sigma$. Jointly fitting ACT and Planck lensing cross-correlations we obtain a 2.7% constraint of $S_8^\times = 0.776^{+0.019}_{-0.021}$, which is consistent with the Planck early-universe extrapolation within 2.1$\sigma$, with the lowest redshift bin showing the largest difference in mean. The latter may motivate further CMB lensing tomography analyses at $z<0.6$ to assess the impact of potential systematics or the consistency of the $\Lambda$CDM model over cosmic time., Comment: Prepared for submission to JCAP (47 pages, 13 figures)

Submillimeter and millimeter wavelengths provide a unique view of the Universe, from the gas and dust that fills and surrounds galaxies to the chromosphere of our own Sun. Current single-dish facilities have presented a tantalising view of the brightest (sub-)mm sources, and interferometers have provided the exquisite resolution necessary to analyse the details in small fields, but there are still many open questions that cannot be answered with current facilities. In this report we summarise the science that is guiding the design of the Atacama Large Aperture Submillimeter Telescope (AtLAST). We demonstrate how tranformational advances in topics including star formation in high redshift galaxies, the diffuse circumgalactic medium, Galactic ecology, cometary compositions and solar flares motivate the need for a 50m, single-dish telescope with a 1-2 degree field of view and a new generation of highly multiplexed continuum and spectral cameras. AtLAST will have the resolution to drastically lower the confusion limit compared to current single-dish facilities, whilst also being able to rapidly map large areas of the sky and detect extended, diffuse structures. Its high sensitivity and large field of view will open up the field of submillimeter transient science by increasing the probability of serendipitous detections. Finally, the science cases listed here motivate the need for a highly flexible operations model capable of short observations of individual targets, large surveys, monitoring programmes, target of opportunity observations and coordinated observations with other observatories. AtLAST aims to be a sustainable, upgradeable, multipurpose facility that will deliver orders of magnitude increases in sensitivity and mapping speeds over current and planned submillimeter observatories., Comment: 47 pages, 12 figures. For further details on AtLAST see https://atlast.uio.no

We investigated the effectiveness of a sustained and spiraled content literacy intervention that emphasizes building domain and topic knowledge schemas and vocabulary for elementary-grade students. The Model of Reading Engagement (MORE) intervention underscores thematic lessons that provide an intellectual structure for helping students connect new learning to a general schema in Grade 1 (animal survival), Grade 2 (scientific investigation of past events like dinosaur mass extinctions), and Grade 3 (scientific investigation of living systems). A total of 30 elementary schools (N = 2,870 students) were randomized to a treatment or control condition. In the treatment condition (i.e., full spiral curriculum), students participated in content literacy lessons from Grades 1 to 3 during the school year and wide reading of thematically related informational texts in the summer following Grades 1 and 2. In the control condition (i.e., partial spiral curriculum), students participated in lessons in only Grade 3. The Grade 3 lessons for both conditions were implemented online during the COVID-19 pandemic school year. Results reveal that treatment students outperformed control students on science vocabulary knowledge across all three grades. Furthermore, intent-to-treat analyses revealed positive transfer effects on Grade 3 science reading (ES = 0.14), domain-general reading comprehension (ES = 0.11), and mathematics achievement (ES = 0.12). Treatment impacts were sustained at 14-month follow-up on Grade 4 reading comprehension (ES = 0.12) and mathematics achievement (ES = 0.16). Findings indicate that a content literacy intervention that spirals topics and vocabulary across grades can improve students' long-term academic achievement outcomes.

The development of low-frequency radio astronomy experiments for detecting 21-cm line emission from hydrogen presents new opportunities for creative solutions to the challenge of characterizing an antenna beam pattern. The Array of Long Baseline Antennas for Taking Radio Observations from the Seventy-ninth parallel (ALBATROS) is a new radio interferometer sited in the Canadian high Arctic that aims to map Galactic foregrounds at frequencies below $\sim$30 MHz. We present PteroSoar, a custom-built hexacopter outfitted with a transmitter, that will be used to characterize the beam patterns of ALBATROS and other experiments. The PteroSoar drone hardware is motivated by the need for user-servicing at remote sites and environmental factors that are unique to the high Arctic. In particular, magnetic heading is unreliable because the magnetic field lines near the north pole are almost vertical. We therefore implement moving baseline real time kinematic (RTK) positioning with two GPS units to obtain heading solutions with $\sim$1$^\circ$ accuracy. We present a preliminary beam map of an ALBATROS antenna, thus demonstrating successful PteroSoar operation in the high Arctic.

Vertical take-off and landing (VTOL) aircraft do not require a prolonged runway, thus allowing them to land almost anywhere. In recent years, their flexibility has made them popular in development, research, and operation. When compared to traditional fixed-wing aircraft and rotorcraft, VTOLs bring unique challenges as they combine many maneuvers from both types of aircraft. Pilot workload is a critical factor for safe and efficient operation of VTOLs. In this work, we conduct a user study to collect multimodal data from 28 pilots while they perform a variety of VTOL flight tasks. We analyze and interpolate behavioral patterns related to their performance and perceived workload. Finally, we build machine learning models to estimate their workload from the collected data. Our results are promising, suggesting that quantitative and accurate VTOL pilot workload monitoring is viable. Such assistive tools would help the research field understand VTOL operations and serve as a stepping stone for the industry to ensure VTOL safe operations and further remote operations., Comment: 8 pages, 7 figures

The Simons Observatory is a new ground-based cosmic microwave background experiment, which is currently being commissioned in Chile's Atacama Desert. During its survey, the observatory's small aperture telescopes will map 10% of the sky in bands centered at frequencies ranging from 27 to 280 GHz to constrain cosmic inflation models, and its large aperture telescope will map 40% of the sky in the same bands to constrain cosmological parameters and use weak lensing to study large-scale structure. To achieve these science goals, the Simons Observatory is deploying these telescopes' receivers with 60,000 state-of-the-art superconducting transition-edge sensor bolometers for its first five year survey. Reading out this unprecedented number of cryogenic sensors, however, required the development of a novel readout system. The SMuRF electronics were developed to enable high-density readout of superconducting sensors using cryogenic microwave SQUID multiplexing technology. The commissioning of the SMuRF systems at the Simons Observatory is the largest deployment to date of microwave multiplexing technology for transition-edge sensors. In this paper, we show that a significant fraction of the systems deployed so far to the Simons Observatory's large aperture telescope meet baseline specifications for detector yield and readout noise in this early phase of commissioning., Comment: 10 pages, 5 figures, 1 table. To be presented at SPIE Astronomical Telescopes + Instrumentation 2024

Sub-mm and mm wavelengths provide a unique view of the Universe, from the gas and dust that fills and surrounds galaxies to the chromosphere of our own Sun. Current single-dish facilities have presented a tantalising view of the brightest (sub-)mm sources, and interferometers have provided the exquisite resolution necessary to analyse the details in small fields, but there are still many open questions that cannot be answered with current facilities: Where are all the baryons? How do structures interact with their environments? What does the time-varying (sub-)mm sky look like? In order to make major advances on these questions and others, what is needed now is a facility capable of rapidly mapping the sky spatially, spectrally, and temporally, which can only be done by a high throughput, single-dish observatory. An extensive design study for this new facility is currently being undertaken. In this paper, we focus on the key science drivers and the requirements they place on the observatory. As a 50m single dish telescope with a 1-2{\deg} field of view, the strength of the Atacama Large Aperture Submillimeter Telescope (AtLAST) is in science where a large field of view, highly multiplexed instrumentation and sensitivity to faint large-scale structure is important. AtLAST aims to be a sustainable, upgradeable, multipurpose facility that will deliver orders of magnitude increases in sensitivity and mapping speeds over current and planned telescopes., Comment: 12 pages, Conference proceedings paper for the 2024 SPIE Astronomical Telescopes + Instrumentation meeting

Ternary neural networks (TNNs) offer a superior accuracy-energy trade-off compared to binary neural networks. However, until now, they have required specialized accelerators to realize their efficiency potential, which has hindered widespread adoption. To address this, we present xTern, a lightweight extension of the RISC-V instruction set architecture (ISA) targeted at accelerating TNN inference on general-purpose cores. To complement the ISA extension, we developed a set of optimized kernels leveraging xTern, achieving 67% higher throughput than their 2-bit equivalents. Power consumption is only marginally increased by 5.2%, resulting in an energy efficiency improvement by 57.1%. We demonstrate that the proposed xTern extension, integrated into an octa-core compute cluster, incurs a minimal silicon area overhead of 0.9% with no impact on timing. In end-to-end benchmarks, we demonstrate that xTern enables the deployment of TNNs achieving up to 1.6 percentage points higher CIFAR-10 classification accuracy than 2-bit networks at equal inference latency. Our results show that xTern enables RISC-V-based ultra-low-power edge AI platforms to benefit from the efficiency potential of TNNs., Comment: Accepted for publication at IEEE ASAP 2024

In an isotropic background comprised of free charges, the transverse and longitudinal modes of the photon acquire large corrections to their dispersion relations, described by the in-medium photon self-energy. Previous work has developed simple approximations that describe the propagation of on-shell photons in plasmas of varying temperatures and densities. However, off-shell excitations can also receive large medium-induced corrections, and the on-shell approximations have often been used in an effort to capture these effects. In this work we show that the off-shell self-energy can be qualitatively very different than the on-shell case. We develop analytic approximations that are accurate everywhere in phase space, especially in classical and degenerate plasmas. From these, we recover the on-shell expressions in the appropriate limit. Our expressions also reproduce the well-known Lindhard response function from solid-state physics for the longitudinal mode., Comment: 13 pages, 4 figures

Fault-tolerant quantum computation using two-dimensional topological quantum error correcting codes can benefit from multi-qubit long-range operations. By using simple commutation rules, a quantum circuit can be transpiled into a sequence of solely non-Clifford multi-qubit gates. Prior work on fault-tolerant compilation avoids optimal scheduling of such gates since they reduce the parallelizability of the circuit. We observe that the reduced parallelization potential is outweighed by the significant reduction in the number of gates. We therefore devise a method for scheduling multi-qubit lattice surgery using an earliest-available-first policy, solving the associated forest packing problem using a representation of the multi-qubit gates as Steiner trees. Our extensive testing on random and application-inspired circuits demonstrates the method's scalability and performance. We show that the transpilation significantly reduces the circuit length on the set of circuits tested, and that the resulting circuit of multi-qubit gates has a further reduction in the expected circuit execution time compared to serial execution., Comment: 23 pages, 7 figures, 4 tables

The problem of path planning for autonomously searching and tracking multiple objects is important to reconnaissance, surveillance, and many other data-gathering applications. Due to the inherent competing objectives of searching for new objects while maintaining tracks for found objects, most current approaches rely on multi-objective planning methods, leaving it up to the user to tune parameters to balance between the two objectives, usually based on heuristics or trial and error. In this paper, we introduce $\textit{UniSaT}$ ($\textit{Unified Search and Track}$), a unified-objective formulation for the search and track problem based on Random Finite Sets (RFS). This is done by modeling both the unknown and known objects through a combined generalized labeled multi-Bernoulli (GLMB) filter. For the unseen objects, we can leverage both cardinality and spatial prior distributions, which means $\textit{UniSaT}$ does not rely on knowing the exact count of the expected number of objects in the space. The planner maximizes the mutual information of this unified belief model, creating balanced search and tracking behaviors. We demonstrate our work in a simulated environment and show both qualitative results as well as quantitative improvements over a multi-objective method., Comment: 13 pages, 5 figures, 1 table

Modern motion planners for autonomous driving frequently use imitation learning (IL) to draw from expert driving logs. Although IL benefits from its ability to glean nuanced and multi-modal human driving behaviors from large datasets, the resulting planners often struggle with out-of-distribution (OOD) scenarios and with traffic rule compliance. On the other hand, classical rule-based planners, by design, can generate safe traffic rule compliant behaviors while being robust to OOD scenarios, but these planners fail to capture nuances in agent-to-agent interactions and human drivers' intent. RuleFuser, an evidential framework, combines IL planners with classical rule-based planners to draw on the complementary benefits of both, thereby striking a balance between imitation and safety. Our approach, tested on the real-world nuPlan dataset, combines the IL planner's high performance in in-distribution (ID) scenarios with the rule-based planners' enhanced safety in out-of-distribution (OOD) scenarios, achieving a 38.43% average improvement on safety metrics over the IL planner without much detriment to imitation metrics in OOD scenarios., Comment: 17 pages, 5 figures, 3 tables

In the realm of robotics, the quest for achieving real-world autonomy, capable of executing large-scale and long-term operations, has positioned place recognition (PR) as a cornerstone technology. Despite the PR community's remarkable strides over the past two decades, garnering attention from fields like computer vision and robotics, the development of PR methods that sufficiently support real-world robotic systems remains a challenge. This paper aims to bridge this gap by highlighting the crucial role of PR within the framework of Simultaneous Localization and Mapping (SLAM) 2.0. This new phase in robotic navigation calls for scalable, adaptable, and efficient PR solutions by integrating advanced artificial intelligence (AI) technologies. For this goal, we provide a comprehensive review of the current state-of-the-art (SOTA) advancements in PR, alongside the remaining challenges, and underscore its broad applications in robotics. This paper begins with an exploration of PR's formulation and key research challenges. We extensively review literature, focusing on related methods on place representation and solutions to various PR challenges. Applications showcasing PR's potential in robotics, key PR datasets, and open-source libraries are discussed. We also emphasizes our open-source package, aimed at new development and benchmark for general PR. We conclude with a discussion on PR's future directions, accompanied by a summary of the literature covered and access to our open-source library, available to the robotics community at: https://github.com/MetaSLAM/GPRS., Comment: 20 pages, 12 figures, under review

Multi-view stereo omnidirectional distance estimation usually needs to build a cost volume with many hypothetical distance candidates. The cost volume building process is often computationally heavy considering the limited resources a mobile robot has. We propose a new geometry-informed way of distance candidates selection method which enables the use of a very small number of candidates and reduces the computational cost. We demonstrate the use of the geometry-informed candidates in a set of model variants. We find that by adjusting the candidates during robot deployment, our geometry-informed distance candidates also improve a pre-trained model's accuracy if the extrinsics or the number of cameras changes. Without any re-training or fine-tuning, our models outperform models trained with evenly distributed distance candidates. Models are also released as hardware-accelerated versions with a new dedicated large-scale dataset. The project page, code, and dataset can be found at https://theairlab.org/gicandidates/ .

The study of variational quantum algorithms (VQCs) has received significant attention from the quantum computing community in recent years. These hybrid algorithms, utilizing both classical and quantum components, are well-suited for noisy intermediate-scale quantum devices. Though estimating exact gradients using the parameter-shift rule to optimize the VQCs is realizable in NISQ devices, they do not scale well for larger problem sizes. The computational complexity, in terms of the number of circuit evaluations required for gradient estimation by the parameter-shift rule, scales linearly with the number of parameters in VQCs. On the other hand, techniques that approximate the gradients of the VQCs, such as the simultaneous perturbation stochastic approximation (SPSA), do not scale with the number of parameters but struggle with instability and often attain suboptimal solutions. In this work, we introduce a novel gradient estimation approach called Guided-SPSA, which meaningfully combines the parameter-shift rule and SPSA-based gradient approximation. The Guided-SPSA results in a 15% to 25% reduction in the number of circuit evaluations required during training for a similar or better optimality of the solution found compared to the parameter-shift rule. The Guided-SPSA outperforms standard SPSA in all scenarios and outperforms the parameter-shift rule in scenarios such as suboptimal initialization of the parameters. We demonstrate numerically the performance of Guided-SPSA on different paradigms of quantum machine learning, such as regression, classification, and reinforcement learning., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

This paper presents a deep reinforcement learning approach for synthesizing unitaries into quantum circuits. Unitary synthesis aims to identify a quantum circuit that represents a given unitary while minimizing circuit depth, total gate count, a specific gate count, or a combination of these factors. While past research has focused predominantly on continuous gate sets, synthesizing unitaries from the parameter-free Clifford+T gate set remains a challenge. Although the time complexity of this task will inevitably remain exponential in the number of qubits for general unitaries, reducing the runtime for simple problem instances still poses a significant challenge. In this study, we apply the tree-search method Gumbel AlphaZero to solve the problem for a subset of exactly synthesizable Clifford+T unitaries. Our method effectively synthesizes circuits for up to five qubits generated from randomized circuits with up to 60 gates, outperforming existing tools like QuantumCircuitOpt and MIN-T-SYNTH in terms of synthesis time for larger qubit counts. Furthermore, it surpasses Synthetiq in successfully synthesizing random, exactly synthesizable unitaries. These results establish a strong baseline for future unitary synthesis algorithms., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. 12 pages, 6 figures, 1 table

The study of transient and variable events, including novae, active galactic nuclei, and black hole binaries, has historically been a fruitful path for elucidating the evolutionary mechanisms of our universe. The study of such events in the millimeter and submillimeter is, however, still in its infancy. Submillimeter observations probe a variety of materials, such as optically thick dust, which are hard to study in other wavelengths. Submillimeter observations are sensitive to a number of emission mechanisms, from the aforementioned cold dust, to hot free-free emission, and synchrotron emission from energetic particles. Study of these phenomena has been hampered by a lack of prompt, high sensitivity submillimeter follow-up, as well as by a lack of high-sky-coverage submillimeter surveys. In this paper, we describe how the proposed Atacama Large Aperture Submillimeter Telescope (AtLAST) could fill in these gaps in our understanding of the transient universe. We discuss a number of science cases that would benefit from AtLAST observations, and detail how AtLAST is uniquely suited to contributing to them. In particular, AtLAST's large field of view will enable serendipitous detections of transient events, while its anticipated ability to get on source quickly and observe simultaneously in multiple bands make it also ideally suited for transient follow-up. We make theoretical predictions for the instrumental and observatory properties required to significantly contribute to these science cases, and compare them to the projected AtLAST capabilities. Finally, we consider the unique ways in which transient science cases constrain the observational strategies of AtLAST, and make prescriptions for how AtLAST should observe in order to maximize its transient science output without impinging on other science cases., Comment: 19 pages, 5 figures

In this work we define the plasma dispersion function for a suprathermal plasma described with a regularized kappa distribution. As is known from Maxwellian as well as (standard) kappa plasmas, the respective Fried-Conte and the modified plasma dispersion functions are valuable tools for various analytical studies. For the latter is has been proven very useful to know about the mathematical properties, as analytical continuation, series expansions as well as asymptotic expressions. Given the growing popularity of the regularized kappa distribution, as indicated by its increasing number of applications to various problems related to suprathermal plasmas, we extend its theoretical treatment here by providing the corresponding plasma dispersion function along with various of its properties., Comment: Accepted for publication in Physics of Plasmas

Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments, laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256x256-dimensional linear system, indicating its practical robustness., Comment: Accepted to the IEEE International Conference on Quantum Computing and Engineering (QCE 2024), Montr\'eal, Qu\'ebec, Canada. 9 pages, 6 figures, 1 table

It is notorious that quantum mechanics (QM) cannot predict well-defined values for all physical quantities. Less well-known, however, is the fact that QM is unable to furnish probabilistic predictions even in emblematic scenarios such as the double-slit experiment. In contrast, equipped with postulate trajectories, Bohmian mechanics (BM) has inherited more predictive power. It follows that, contrary to common belief, QM and BM are not just different interpretations but distinct theories. This work formalizes the aforementioned assertions and illustrates them through three case studies: (i) free particle, (ii) free fall under a uniform gravitational field, and (iii) the double-slit experiment. Specifically, we introduce a prescription for constructing a flight-time probability distribution within generic trajectory-equipped theories. We then apply our formalism to BM and derive probability distributions that are unreachable by QM. Our results can, in principle, be tested against real experiments, thereby assessing the validity of Bohmian trajectories., Comment: 8 pages, 4 figures. Comments are welcome

Linear systems of equations can be found in various mathematical domains, as well as in the field of machine learning. By employing noisy intermediate-scale quantum devices, variational solvers promise to accelerate finding solutions for large systems. Although there is a wealth of theoretical research on these algorithms, only fragmentary implementations exist. To fill this gap, we have developed the variational-lse-solver framework, which realizes existing approaches in literature, and introduces several enhancements. The user-friendly interface is designed for researchers that work at the abstraction level of identifying and developing end-to-end applications., Comment: Accepted to the 2nd International Workshop on Quantum Machine Learning: From Research to Practice (QML@QCE 2024), Montr\'eal, Qu\'ebec, Canada. 4 pages, 2 figures, 1 table

Quantum computer simulation software is an integral tool for the research efforts in the quantum computing community. An important aspect is the efficiency of respective frameworks, especially for training variational quantum algorithms. Focusing on the widely used Qiskit software environment, we develop the qiskit-torch-module. It improves runtime performance by two orders of magnitude over comparable libraries, while facilitating low-overhead integration with existing codebases. Moreover, the framework provides advanced tools for integrating quantum neural networks with PyTorch. The pipeline is tailored for single-machine compute systems, which constitute a widely employed setup in day-to-day research efforts., Comment: Accepted to the IEEE International Conference on Quantum Computing and Engineering (QCE 2024), Montr\'eal, Qu\'ebec, Canada. 7 pages, 4 figures, 3 tables

Quantum computing is a promising technology to address combinatorial optimization problems, for example via the quantum approximate optimization algorithm (QAOA). Its potential, however, hinges on scaling toy problems to sizes relevant for industry. In this study, we address this challenge by an elaborate combination of two decomposition methods, namely graph shrinking and circuit cutting. Graph shrinking reduces the problem size before encoding into QAOA circuits, while circuit cutting decomposes quantum circuits into fragments for execution on medium-scale quantum computers. Our shrinking method adaptively reduces the problem such that the resulting QAOA circuits are particularly well-suited for circuit cutting. Moreover, we integrate two cutting techniques which allows us to run the resulting circuit fragments sequentially on the same device. We demonstrate the utility of our method by successfully applying it to the archetypical traveling salesperson problem (TSP) which often occurs as a sub-problem in practically relevant vehicle routing applications. For a TSP with seven cities, we are able to retrieve an optimum solution by consecutively running two 7-qubit QAOA circuits. Without decomposition methods, we would require five times as many qubits. Our results offer insights into the performance of algorithms for combinatorial optimization problems within the constraints of current quantum technology.

Few-shot object detection has drawn increasing attention in the field of robotic exploration, where robots are required to find unseen objects with a few online provided examples. Despite recent efforts have been made to yield online processing capabilities, slow inference speeds of low-powered robots fail to meet the demands of real-time detection-making them impractical for autonomous exploration. Existing methods still face performance and efficiency challenges, mainly due to unreliable features and exhaustive class loops. In this work, we propose a new paradigm AirShot, and discover that, by fully exploiting the valuable correlation map, AirShot can result in a more robust and faster few-shot object detection system, which is more applicable to robotics community. The core module Top Prediction Filter (TPF) can operate on multi-scale correlation maps in both the training and inference stages. During training, TPF supervises the generation of a more representative correlation map, while during inference, it reduces looping iterations by selecting top-ranked classes, thus cutting down on computational costs with better performance. Surprisingly, this dual functionality exhibits general effectiveness and efficiency on various off-the-shelf models. Exhaustive experiments on COCO2017, VOC2014, and SubT datasets demonstrate that TPF can significantly boost the efficacy and efficiency of most off-the-shelf models, achieving up to 36.4% precision improvements along with 56.3% faster inference speed. Code and Data are at: https://github.com/ImNotPrepared/AirShot.

With the increasing sophistication and capability of quantum hardware, its integration, and employment in high performance computing (HPC) infrastructure becomes relevant. This opens largely unexplored access models and scheduling questions in such quantum-classical computing environments, going beyond the current cloud access model. SCIM MILQ is a scheduler for quantum tasks in HPC infrastructure. It combines well-established scheduling techniques with methods unique to quantum computing, such as circuit cutting. SCIM MILQ can schedule tasks while minimizing the makespan, i.e., the time that elapses from the start of work to the end, improving on average by 25%. Additionally, it reduces the noise in the circuit by up to 10%, increasing the outcome's reliability. We compare it against an existing baseline and show its viability in an HPC environment., Comment: 9 pages, 2 figures, 3 tables, 1 algorithm, submission to Quantum Week 2024

Observing and filming a group of moving actors with a team of aerial robots is a challenging problem that combines elements of multi-robot coordination, coverage, and view planning. A single camera may observe multiple actors at once, and the robot team may observe individual actors from multiple views. As actors move about, groups may split, merge, and reform, and robots filming these actors should be able to adapt smoothly to such changes in actor formations. Rather than adopt an approach based on explicit formations or assignments, we propose an approach based on optimizing views directly. We model actors as moving polyhedra and compute approximate pixel densities for each face and camera view. Then, we propose an objective that exhibits diminishing returns as pixel densities increase from repeated observation. This gives rise to a multi-robot perception planning problem which we solve via a combination of value iteration and greedy submodular maximization. %using a combination of value iteration to optimize views for individual robots and sequential submodular maximization methods to coordinate the team. We evaluate our approach on challenging scenarios modeled after various kinds of social behaviors and featuring different numbers of robots and actors and observe that robot assignments and formations arise implicitly based on the movements of groups of actors. Simulation results demonstrate that our approach consistently outperforms baselines, and in addition to performing well with the planner's approximation of pixel densities our approach also performs comparably for evaluation based on rendered views. Overall, the multi-round variant of the sequential planner we propose meets (within 1%) or exceeds the formation and assignment baselines in all scenarios we consider., Comment: 10 pages, 5 figures, submitted to CDC 2024

Transformer networks are rapidly becoming SotA in many fields, such as NLP and CV. Similarly to CNN, there is a strong push for deploying Transformer models at the extreme edge, ultimately fitting the tiny power budget and memory footprint of MCUs. However, the early approaches in this direction are mostly ad-hoc, platform, and model-specific. This work aims to enable and optimize the flexible, multi-platform deployment of encoder Tiny Transformers on commercial MCUs. We propose a complete framework to perform end-to-end deployment of Transformer models onto single and multi-core MCUs. Our framework provides an optimized library of kernels to maximize data reuse and avoid unnecessary data marshaling operations into the crucial attention block. A novel MHSA inference schedule, named Fused-Weight Self-Attention, is introduced, fusing the linear projection weights offline to further reduce the number of operations and parameters. Furthermore, to mitigate the memory peak reached by the computation of the attention map, we present a Depth-First Tiling scheme for MHSA. We evaluate our framework on three different MCU classes exploiting ARM and RISC-V ISA, namely the STM32H7, the STM32L4, and GAP9 (RV32IMC-XpulpV2). We reach an average of 4.79x and 2.0x lower latency compared to SotA libraries CMSIS-NN (ARM) and PULP-NN (RISC-V), respectively. Moreover, we show that our MHSA depth-first tiling scheme reduces the memory peak by up to 6.19x, while the fused-weight attention can reduce the runtime by 1.53x, and number of parameters by 25%. We report significant improvements across several Tiny Transformers: for instance, when executing a transformer block for the task of radar-based hand-gesture recognition on GAP9, we achieve a latency of 0.14ms and energy consumption of 4.92 micro-joules, 2.32x lower than the SotA PULP-NN library on the same platform., Comment: Pre-print manuscript submitted for review to the IEEE Transactions on Computers

We propose a novel method to improve estimation of asset returns for portfolio optimization. This approach first performs a monthly directional market forecast using an online decision tree. The decision tree is trained on a novel set of features engineered from portfolio theory: the efficient frontier functional coefficients. Efficient frontiers can be decomposed to their functional form, a square-root second-order polynomial, and the coefficients of this function captures the information of all the constituents that compose the market in the current time period. To make these forecasts actionable, these directional forecasts are integrated to a portfolio optimization framework using expected returns conditional on the market forecast as an estimate for the return vector. This conditional expectation is calculated using the inverse Mills ratio, and the Capital Asset Pricing Model is used to translate the market forecast to individual asset forecasts. This novel method outperforms baseline portfolios, as well as other feature sets including technical indicators and the Fama-French factors. To empirically validate the proposed model, we employ a set of market sector ETFs.

We present a simple and effective way to account for non-convex costs and constraints~in~state feedback synthesis, and an interpretation for the variables in which state feedback synthesis is typically convex. We achieve this by deriving the controller design using moment matrices of state and input. It turns out that this approach allows the consideration of non-convex constraints by relaxing them as expectation constraints, and that the variables in which state feedback synthesis is typically convexified can be identified with blocks of these moment matrices., Comment: Preprent to be submitted to IEEE Conference on Decision and Control

The limited sensing resolution of resource-constrained off-road vehicles poses significant challenges towards reliable off-road autonomy. To overcome this limitation, we propose a general framework based on fusing the future information (i.e. future fusion) for self-supervision. Recent approaches exploit this future information alongside the hand-crafted heuristics to directly supervise the targeted downstream tasks (e.g. traversability estimation). However, in this paper, we opt for a more general line of development - time-efficient completion of the highest resolution (i.e. 2cm per pixel) BEV map in a self-supervised manner via future fusion, which can be used for any downstream tasks for better longer range prediction. To this end, first, we create a high-resolution future-fusion dataset containing pairs of (RGB / height) raw sparse and noisy inputs and map-based dense labels. Next, to accommodate the noise and sparsity of the sensory information, especially in the distal regions, we design an efficient realization of the Bayes filter onto the vanilla convolutional network via the recurrent mechanism. Equipped with the ideas from SOTA generative models, our Bayesian structure effectively predicts high-quality BEV maps in the distal regions. Extensive evaluation on both the quality of completion and downstream task on our future-fusion dataset demonstrates the potential of our approach.