Search

Contemporary radio access networks employ link adaption (LA) algorithms to optimize the modulation and coding schemes to adapt to the prevailing propagation conditions and are near-optimal in terms of the achieved spectral efficiency. LA is a challenging task in the presence of mobility, fast fading, and imperfect channel quality information and limited knowledge of the receiver characteristics at the transmitter, which render model-based LA algorithms complex and suboptimal. Model-based LA is especially difficult as connected user equipment devices become increasingly heterogeneous in terms of receiver capabilities, antenna configurations and hardware characteristics. Recognizing these difficulties, previous works have proposed reinforcement learning (RL) for LA, which faces deployment difficulties due to their potential negative impacts on live performance. To address this challenge, this paper considers offline RL to learn LA policies from data acquired in live networks with minimal or no intrusive effects on the network operation. We propose three LA designs based on batch-constrained deep Q-learning, conservative Q-learning, and decision transformers, showing that offline RL algorithms can achieve performance of state-of-the-art online RL methods when data is collected with a proper behavioral policy.

Blockchain technology is booming up the digital world in recent days and thus paved a way for creating separate blockchain network for various industries. This technology is characterized by its distributed, decentralized, and immutable ledger system which serves as a fundamental platform for managing smart contract transactions (SCTs). However, these self-executing codes implemented using blockchains undergo sequential validation within a block which introduces performance bottlenecks. In response, this paper introduces a framework called the Multi-Bin Parallel Scheduler (MBPS) designed for parallelizing blockchain smart contract transactions to leverage the capabilities of multicore systems. Our proposed framework facilitates concurrent execution of SCTs, enhancing performance by allowing non-conflicting transactions to be processed simultaneously while preserving deterministic order. The framework comprises of three vital stages: conflict detection, bin creation and execution. We conducted an evaluation of our MBPS framework in Hyperledger Sawtooth v1.2.6, revealing substantial performance enhancements compared to existing parallel SCT execution frameworks across various smart contract applications. This research contributes to the ongoing optimization efforts in blockchain technology demonstrating its potential for scalability and efficiency in real-world scenarios.

Recent advances in generative modeling with diffusion processes (DPs) enabled breakthroughs in image synthesis. Despite impressive image quality, these models have various prompt compliance problems, including low recall in generating multiple objects, difficulty in generating text in images, and meeting constraints like object locations and pose. For fine-grained editing and manipulation, they also require fine-grained semantic or instance maps that are tedious to produce manually. While prompt compliance can be enhanced by addition of loss functions at inference, this is time consuming and does not scale to complex scenes. To overcome these limitations, this work introduces a new family of \textit{Factor Graph Diffusion Models} (FG-DMs) that models the joint distribution of images and conditioning variables, such as semantic, sketch, depth or normal maps via a factor graph decomposition. This joint structure has several advantages, including support for efficient sampling based prompt compliance schemes, which produce images of high object recall, semi-automated fine-grained editing, text-based editing of conditions with noise inversion, explainability at intermediate levels, ability to produce labeled datasets for the training of downstream models such as segmentation or depth, training with missing data, and continual learning where new conditioning variables can be added with minimal or no modifications to the existing structure. We propose an implementation of FG-DMs by adapting a pre-trained Stable Diffusion (SD) model to implement all FG-DM factors, using only COCO dataset, and show that it is effective in generating images with 15\% higher recall than SD while retaining its generalization ability. We introduce an attention distillation loss that encourages consistency among the attention maps of all factors, improving the fidelity of the generated conditions and image., Comment: Accepted to NeurIPS 2024 conference. Project Page: https://deepaksridhar.github.io/factorgraphdiffusion.github.io/

The ability for optically active media to rotate the polarization of light is the basis of polarimetry, an illustrious technique responsible for many breakthroughs in fields as varied as astronomy, medicine and material science. Here, we recast the primary mechanism for spin readout in semiconductor-based quantum computers, Pauli spin-blockade (PSB), as the natural extension of polarimetry to the third dimension. We perform polarimetry with spins through a silicon quantum dot exchanging a hole with a boron acceptor, demonstrating the role of spin-orbit coupling in creating spin misalignment. Perfect spin alignment may be recovered by means of rotating the applied magnetic-field orientation. This work shows how spin misalignment sets a fundamental upper limit for the spin readout fidelity in quantum-computing systems based on PSB.

Designing topological materials with specific topological indices is a complex inverse problem, traditionally tackled through manual, intuition-driven methods that are neither scalable nor efficient for exploring the vast space of possible material configurations. In this work, we develop an algorithm that leverages the covariance matrix adaptation evolution strategy to optimize the Fourier representation of the periodic functions shaping the designer material's characteristics. This includes mass profiles or dielectric tensors for phononic and photonic crystals, respectively, as much as synthetic potentials applicable to electronic and ultra-cold atomic systems. We demonstrate our methodology with a detailed characterization of a class of topological bands known as "fragile topological", showcasing the algorithm's capability to address both topological characteristics and spectral quality. This automation not only streamlines the design process but also significantly expands the potential for identifying and constructing high quality designer topological materials across the wide range of platforms, and is readily extendable to other setups, including higher-dimensional and non-linear systems., Comment: 4 pages + 45 pages SI, 4 + 49 figures

In recent years, serverless computing, especially Function as a Service (FaaS), is rapidly growing in popularity as a cloud programming model. The serverless computing model provides an intuitive interface for developing cloud-based applications, where the development and deployment of scalable microservices has become easier and cost-effective. An increasing number of batch-processing applications are deployed as pipelines that comprise a sequence of functions that must meet their deadline targets to be practical. In this paper, we present our Hybrid Cloud Scheduler (HCS) for orchestrating the execution of serverless batch-processing pipelines deployed over heterogeneous infrastructures. Our framework enables developers to (i) automatically schedule and execute batch-processing applications in heterogeneous environments such as the private edge and public cloud serverless infrastructures, (ii) benefit from cost reduction through the utilization of their own resources in a private cluster, and (iii) significantly improves the probability of meeting the deadline requirements of their applications. Our experimental evaluation demonstrates the efficiency and benefits of our approach.

Serverless computing, also referred to as Function-as-a-Service (FaaS), is a cloud computing model that has attracted significant attention and has been widely adopted in recent years. The serverless computing model offers an intuitive, event-based interface that makes the development and deployment of scalable cloud-based applications easier and cost-effective. An important aspect that has not been examined in these systems is their energy consumption during the application execution. One way to deal with this issue is to schedule the function invocations in an energy-efficient way. However, efficient scheduling of applications in a multi-tenant environment, like FaaS systems, poses significant challenges. The trade-off between the server's energy usage and the hosted functions' performance requirements needs to be taken into consideration. In this work, we propose an Energy Efficient Scheduler for orchestrating the execution of serverless functions so that it minimizes energy consumption while it satisfies the applications' performance demands. Our approach considers real-time performance measurements and historical data and applies a novel DVFS technique to minimize energy consumption. Our detailed experimental evaluation using realistic workloads on our local cluster illustrates the working and benefits of our approach.

Autonomous driving and advanced driver assistance systems (ADAS) rely on cameras to control the driving. In many prior approaches an attacker aiming to stop the vehicle had to send messages on the specialized and better-defended CAN bus. We suggest an easier alternative: manipulate the IP-based network communication between the camera and the ADAS logic, inject fake images of stop signs or red lights into the video stream, and let the ADAS stop the car safely. We created an attack tool that successfully exploits the GigE Vision protocol. Then we analyze two classes of passive anomaly detectors to identify such attacks: protocol-based detectors and video-based detectors. We implemented multiple detectors of both classes and evaluated them on data collected from our test vehicle and also on data from the public BDD corpus. Our results show that such detectors are effective against naive adversaries, but sophisticated adversaries can evade detection. Finally, we propose a novel class of active defense mechanisms that randomly adjust camera parameters during the video transmission, and verify that the received images obey the requested adjustments. Within this class we focus on a specific implementation, the width-varying defense, which randomly modifies the width of every frame. Beyond its function as an anomaly detector, this defense is also a protective measure against certain attacks: by distorting injected image patches it prevents their recognition by the ADAS logic. We demonstrate the effectiveness of the width-varying defense through theoretical analysis and by an extensive evaluation of several types of attack in a wide range of realistic road driving conditions. The best the attack was able to achieve against this defense was injecting a stop sign for a duration of 0.2 seconds, with a success probability of 0.2%, whereas stopping a vehicle requires about 2.5 seconds.

We reframe scene flow as the task of estimating a continuous space-time ODE that describes motion for an entire observation sequence, represented with a neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via pure self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, EulerFlow produces high quality flow estimates on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsupervised method by more than 2.5x, and even exceeding the next-best supervised method by over 10%., Comment: Project page at https://vedder.io/eulerflow

This research presents a comprehensive framework for analyzing liquidity in financial markets, particularly in the context of high-frequency trading. By leveraging advanced machine learning classification techniques, including Logistic Regression, Support Vector Machine, and Random Forest, the study aims to predict minute-level price movements using an extensive set of liquidity metrics derived from the Trade and Quote (TAQ) data. The findings reveal that employing a broad spectrum of liquidity measures yields higher predictive accuracy compared to models utilizing a reduced subset of features. Key liquidity metrics, such as Liquidity Ratio, Flow Ratio, and Turnover, consistently emerged as significant predictors across all models, with the Random Forest algorithm demonstrating superior accuracy. This study not only underscores the critical role of liquidity in market stability and transaction costs but also highlights the complexities involved in short-interval market predictions. The research suggests that a comprehensive set of liquidity measures is essential for accurate prediction, and proposes future work to validate these findings across different stock datasets to assess their generalizability.

Drones provide a versatile platform for remote sensing and atmospheric studies. However, strict payload mass limits and intense vibrations have proven obstacles to adoption for astronomy. We present a concept for system-level testing of a long-baseline CubeSat space interferometer using drones, taking advantage of their cm-level xyz station-keeping, 6-dof freedom of movement, large operational environment, access to guide stars for end-to-end testing of optical train and control algorithms, and comparable mass and power requirements. We have purchased two different drone platforms (Aurelia X6 Pro, Freefly Alta X) and present characterization studies of vibrations, flight stability, gps positioning precision, and more. We also describe our progress in sub-system development, including inter-drone laser metrology, realtime gimbal control, and LED beacon tracking. Lastly, we explore whether custom-built drone-borne telescopes could be used for interferometry of bright objects over km-level baselines using vibration-isolation platforms and a small fast delay for fringe-tracking., Comment: submitted to SPIE 2024 (Yokohama)

A quantum system interacting with a time-periodic excitation creates a ladder of hybrid eigenstates in which the system is mixed with an increasing number of photons. This mechanism, referred to as dressing, has been observed in the context of light-matter interaction in systems as varied as atoms, molecules and solid-state qubits. In this work, we demonstrate state dressing in a semiconductor quantum dot tunnel-coupled to a charge reservoir. We observe the emergence of a Floquet ladder of states in the system's high-frequency electrical response, manifesting as interference fringes at the multiphoton resonances despite the system lacking an avoided crossing. We study the dressed quantum dot while changing reservoir temperature, charge lifetime, and excitation amplitude and reveal the fundamental nature of the mechanism by developing a theory based on the quantum dynamics of the Floquet ladder, which is in excellent agreement with the data. Furthermore, we show how the technique finds applications in the accurate electrostatic characterisation of semiconductor quantum dots., Comment: 8 Pages, 4 Figures. Supplementary: 4 Pages, 2 Figures

Motion planning is crucial for safe navigation in complex urban environments. Historically, motion planners (MPs) have been evaluated with procedurally-generated simulators like CARLA. However, such synthetic benchmarks do not capture real-world multi-agent interactions. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic, effectively turning the fixed dataset into a reactive simulator. We analyze the characteristics of nuPlan's recorded logs and find that each city has its own unique driving behaviors, suggesting that robust planners must adapt to different environments. We learn to model such unique behaviors with BehaviorNet, a graph convolutional neural network (GCNN) that predicts reactive agent behaviors using features derived from recently-observed agent histories; intuitively, some aggressive agents may tailgate lead vehicles, while others may not. To model such phenomena, BehaviorNet predicts the parameters of an agent's motion controller rather than directly predicting its spacetime trajectory (as most forecasters do). Finally, we present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions. Our extensive experiments demonstrate that AdaptiveDriver achieves state-of-the-art results on the nuPlan closed-loop planning benchmark, improving over prior work by 2% on Test-14 Hard R-CLS, and generalizes even when evaluated on never-before-seen cities., Comment: Project Page: https://arunbalajeev.github.io/world_models_planning/world_model_paper.html

State-of-the-art 3D object detectors are often trained on massive labeled datasets. However, annotating 3D bounding boxes remains prohibitively expensive and time-consuming, particularly for LiDAR. Instead, recent works demonstrate that self-supervised pre-training with unlabeled data can improve detection accuracy with limited labels. Contemporary methods adapt best-practices for self-supervised learning from the image domain to point clouds (such as contrastive learning). However, publicly available 3D datasets are considerably smaller and less diverse than those used for image-based self-supervised learning, limiting their effectiveness. We do note, however, that such 3D data is naturally collected in a multimodal fashion, often paired with images. Rather than pre-training with only self-supervised objectives, we argue that it is better to bootstrap point cloud representations using image-based foundation models trained on internet-scale data. Specifically, we propose a shelf-supervised approach (e.g. supervised with off-the-shelf image foundation models) for generating zero-shot 3D bounding boxes from paired RGB and LiDAR data. Pre-training 3D detectors with such pseudo-labels yields significantly better semi-supervised detection accuracy than prior self-supervised pretext tasks. Importantly, we show that image-based shelf-supervision is helpful for training LiDAR-only, RGB-only and multi-modal (RGB + LiDAR) detectors. We demonstrate the effectiveness of our approach on nuScenes and WOD, significantly improving over prior work in limited data settings. Our code is available at https://github.com/meharkhurana03/cm3d, Comment: The first two authors contributed equally. This work has been accepted to the Conference on Robot Learning (CoRL) 2024

The SWMR atomic register is a fundamental building block in shared memory distributed systems and implementing it from SWSR atomic registers is an important problem. While this problem has been solved in crash-prone systems, it has received less attention in Byzantine systems. Recently, Hu and Toueg gave such an implementation of the SWMR register from SWSR registers. While their definition of register linearizability is consistent with the definition of Byzantine linearizability of a concurrent history of Cohen and Keidar, it has these drawbacks. (1) If the writer is Byzantine, the register is linearizable no matter what values the correct readers return. (2) It ignores values written consistently by a Byzantine writer. We need a stronger notion of a {\em correct write operation}. (3) It allows a value written to just one or a few readers' SWSR registers to be returned, thereby not validating the intention of the writer to write that value honestly. (4) Its notion of a ``current'' value returned by a correct reader is not related to the most recent value written by a correct write operation of a Byzantine writer. We need a more up to date version of the value that can be returned by a correct reader. In this paper, we give a stronger definition of a Byzantine linearizable register that overcomes the above drawbacks. Then we give a construction of a Byzantine linearizable SWMR atomic register from SWSR registers that meets our stronger definition. The construction is correct when $n>3f$, where $n$ is the number of readers, $f$ is the maximum number of Byzantine readers, and the writer can also be Byzantine. The construction relies on a public-key infrastructure., Comment: 18 pages

Extensive research has explored the optical properties of topological insulating materials, driven by their inherent stability and potential applications. In this study, we unveil a novel functionality of three-dimensional integer quantum Hall (3D IQH) states as broad-band filters for circularly polarized light, particularly effective in the terahertz (THz) frequency range under realistic system parameters. We also investigate the impact of practical imperfections, demonstrating the resilience of this filtering effect. Our findings reveal that this phenomenon is independent of the microscopic origin of the 3D IQH state, prompting discussions on its feasibility across diverse candidate materials. These results contribute to our understanding of fundamental optical properties and hold promise for practical applications in optical technologies., Comment: 5 pages, 3 figures

Integrated Speech and Large Language Models (SLMs) that can follow speech instructions and generate relevant text responses have gained popularity lately. However, the safety and robustness of these models remains largely unclear. In this work, we investigate the potential vulnerabilities of such instruction-following speech-language models to adversarial attacks and jailbreaking. Specifically, we design algorithms that can generate adversarial examples to jailbreak SLMs in both white-box and black-box attack settings without human involvement. Additionally, we propose countermeasures to thwart such jailbreaking attacks. Our models, trained on dialog data with speech instructions, achieve state-of-the-art performance on spoken question-answering task, scoring over 80% on both safety and helpfulness metrics. Despite safety guardrails, experiments on jailbreaking demonstrate the vulnerability of SLMs to adversarial perturbations and transfer attacks, with average attack success rates of 90% and 10% respectively when evaluated on a dataset of carefully designed harmful questions spanning 12 different toxic categories. However, we demonstrate that our proposed countermeasures reduce the attack success significantly., Comment: 9+6 pages, Submitted to ACL 2024

Reinforcement Learning is a promising tool for learning complex policies even in fast-moving and object-interactive domains where human teleoperation or hard-coded policies might fail. To effectively reflect this challenging category of tasks, we introduce a dynamic, interactive RL testbed based on robot air hockey. By augmenting air hockey with a large family of tasks ranging from easy tasks like reaching, to challenging ones like pushing a block by hitting it with a puck, as well as goal-based and human-interactive tasks, our testbed allows a varied assessment of RL capabilities. The robot air hockey testbed also supports sim-to-real transfer with three domains: two simulators of increasing fidelity and a real robot system. Using a dataset of demonstration data gathered through two teleoperation systems: a virtualized control environment, and human shadowing, we assess the testbed with behavior cloning, offline RL, and RL from scratch.

Visual control policies can encounter significant performance degradation when visual conditions like lighting or camera position differ from those seen during training -- often exhibiting sharp declines in capability even for minor differences. In this work, we examine robustness to a suite of these types of visual changes for RGB-D and point cloud based visual control policies. To perform these experiments on both model-free and model-based reinforcement learners, we introduce a novel Point Cloud World Model (PCWM) and point cloud based control policies. Our experiments show that policies that explicitly encode point clouds are significantly more robust than their RGB-D counterparts. Further, we find our proposed PCWM significantly outperforms prior works in terms of sample efficiency during training. Taken together, these results suggest reasoning about the 3D scene through point clouds can improve performance, reduce learning time, and increase robustness for robotic learners. Project Webpage: https://pvskand.github.io/projects/PCWM, Comment: Accepted at International Conference on Robotics and Automation, 2024

In this paper, we study the partitioning of a context-aware shared memory data structure so that it can be implemented as a distributed data structure running on multiple machines. By context-aware data structures, we mean that the result of an operation not only depends upon the value of the shared data but also upon the previous operations performed by the same client. While there is substantial work on designing distributed data structures, designing distributed context-aware data structures has not received much attention. We focus on singly-linked lists as a case study of the context-aware data structure. We start with a shared memory context-aware lock-free singly-linked list and show how it can be transformed into a distributed lock-free context-aware singly-linked list. The main challenge in such a transformation is to preserve properties of client-visible operations of the underlying data structure. We present two protocols that preserve these properties of client-visible operations of the linked list. In the first protocol, the distribution is done in the background as a low priority task, while in the second protocol the client-visible operations help the task of distribution without affecting client latency. In both protocols, the client-visible operations remain lock-free. Also, our transformation approach does not utilize any hardware primitives (except a compare-and-swap operation on a single word). We note that our transformation is generic and can be used for other lock-free context-aware data structures that can be constructed from singly-linked lists.

We propose the SAL (Segment Anything in Lidar) method consisting of a text-promptable zero-shot model for segmenting and classifying any object in Lidar, and a pseudo-labeling engine that facilitates model training without manual supervision. While the established paradigm for Lidar Panoptic Segmentation (LPS) relies on manual supervision for a handful of object classes defined a priori, we utilize 2D vision foundation models to generate 3D supervision ``for free''. Our pseudo-labels consist of instance masks and corresponding CLIP tokens, which we lift to Lidar using calibrated multi-modal data. By training our model on these labels, we distill the 2D foundation models into our Lidar SAL model. Even without manual labels, our model achieves $91\%$ in terms of class-agnostic segmentation and $54\%$ in terms of zero-shot Lidar Panoptic Segmentation of the fully supervised state-of-the-art. Furthermore, we outperform several baselines that do not distill but only lift image features to 3D. More importantly, we demonstrate that SAL supports arbitrary class prompts, can be easily extended to new datasets, and shows significant potential to improve with increasing amounts of self-labeled data. Code and models are available at this $\href{https://github.com/nv-dvl/segment-anything-lidar}{URL}$., Comment: Accepted to ECCV 2024

In semiconductor nanostructures, spin blockade (SB) is the most scalable mechanism for electrical spin readout requiring only two bound spins for its implementation which, in conjunction with charge sensing techniques, has led to high-fidelity readout of spins in semiconductor-based quantum processors. However, various mechanisms may lift SB, such as strong spin-orbit coupling (SOC) or low-lying excited states, hence posing challenges to perform spin readout at scale and with high fidelity in such systems. Here, we present a method, based on the dependence of the two-spin system polarizability on energy detuning, to perform spin state readout even when SB lifting mechanisms are dominant. It leverages SB lifting as a resource to detect different spin measurement outcomes selectively and positively. We demonstrate the method using a hybrid system formed by a quantum dot (QD) and a Boron acceptor in a silicon p-type transistor and show spin selective and positive readout of different spin states under SB lifting conditions due to (i) SOC and (ii) low-lying orbital states in the QD. We further use the method to determine the detuning-dependent spin relaxation time of 0.1-8~$\mu$s. Our method should help perform high-fidelity projective spin measurements in systems subject to strong SOC and may facilitate quantum tomography and state leakage studies., Comment: 13 pages, 10 figures

Current scene flow methods broadly fail to describe motion on small objects, and current scene flow evaluation protocols hide this failure by averaging over many points, with most drawn larger objects. To fix this evaluation failure, we propose a new evaluation protocol, Bucket Normalized EPE, which is class-aware and speed-normalized, enabling contextualized error comparisons between object types that move at vastly different speeds. To highlight current method failures, we propose a frustratingly simple supervised scene flow baseline, TrackFlow, built by bolting a high-quality pretrained detector (trained using many class rebalancing techniques) onto a simple tracker, that produces state-of-the-art performance on current standard evaluations and large improvements over prior art on our new evaluation. Our results make it clear that all scene flow evaluations must be class and speed aware, and supervised scene flow methods must address point class imbalances. We release the evaluation code publicly at https://github.com/kylevedder/BucketedSceneFlowEval., Comment: Accepted to ECCV 2024. Project page at https://vedder.io/trackflow

The era of vision-language models (VLMs) trained on web-scale datasets challenges conventional formulations of "open-world" perception. In this work, we revisit the task of few-shot object detection (FSOD) in the context of recent foundational VLMs. First, we point out that zero-shot predictions from VLMs such as GroundingDINO significantly outperform state-of-the-art few-shot detectors (48 vs. 33 AP) on COCO. Despite their strong zero-shot performance, such foundation models may still be sub-optimal. For example, trucks on the web may be defined differently from trucks for a target application such as autonomous vehicle perception. We argue that the task of few-shot recognition can be reformulated as aligning foundation models to target concepts using a few examples. Interestingly, such examples can be multi-modal, using both text and visual cues, mimicking instructions that are often given to human annotators when defining a target concept of interest. Concretely, we propose Foundational FSOD, a new benchmark protocol that evaluates detectors pre-trained on any external data and fine-tuned on multi-modal (text and visual) K-shot examples per target class. We repurpose nuImages for Foundational FSOD, benchmark several popular open-source VLMs, and provide an empirical analysis of state-of-the-art methods. Lastly, we discuss our recent CVPR 2024 Foundational FSOD competition and share insights from the community. Notably, the winning team significantly outperforms our baseline by 23.3 mAP! Our code and dataset splits are available at https://github.com/anishmadan23/foundational_fsod, Comment: The first two authors contributed equally. This work has been accepted to the Neural Information Processing Systems (NeurIPS) 2024 Datasets & Benchmark Track

Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors, particularly on large-scale multi-modal (LiDAR + RGB) data. Surprisingly, although semantic class labels naturally follow a long-tailed distribution, existing benchmarks only focus on a few common classes (e.g., pedestrian and car) and neglect many rare but crucial classes (e.g., emergency vehicle and stroller). However, AVs must reliably detect both common and rare classes for safe operation in the open world. We address this challenge by formally studying the problem of Long-Tailed 3D Detection (LT3D), which evaluates all annotated classes, including those in-the-tail. We address LT3D with hierarchical losses that promote feature sharing across classes, and introduce diagnostic metrics that award partial credit to ``reasonable'' mistakes with respect to the semantic hierarchy (e.g., mistaking a child for an adult). Further, we point out that rare-class accuracy is particularly improved via multi-modal late fusion (MMLF) of independently trained uni-modal LiDAR and RGB detectors. Importantly, such an MMLF framework allows us to leverage large-scale uni-modal datasets (with more examples for rare classes) to train better uni-modal detectors, unlike prevailing end-to-end trained multi-modal detectors that require paired multi-modal data. Finally, we examine three critical components of our simple MMLF approach from first principles and investigate whether to train 2D or 3D RGB detectors for fusion, whether to match RGB and LiDAR detections in 3D or the projected 2D image plane, and how to fuse matched detections. Our proposed MMLF approach significantly improves LT3D performance over prior work, particularly improving rare class performance from 12.8 to 20.0 mAP!, Comment: The first two authors contributed equally. Project page: https://mayechi.github.io/lt3d-lf-io/

Modelling the electrical response of multi-level quantum systems at finite frequency has been typically performed in the context of two incomplete paradigms: (i) input-output theory, which is valid at any frequency but neglects dynamic losses, and (ii) semiclassical theory, which captures well dynamic dissipation effects but is only accurate at low frequencies. Here, we develop a unifying theory, valid for arbitrary frequencies, that captures both the quantum behaviour and the non-unitary effects introduced by relaxation and dephasing. The theory allows a multi-level system to be described by a universal small-signal equivalent circuit model, a resonant RLC circuit, whose topology only depends on the number of energy levels. We apply our model to a double quantum-dot charge qubit and a Majorana qubit, showing the capability to continuously describe the systems from adiabatic to resonant and from coherent to incoherent, suggesting new and realistic experiments for improved quantum state readout. Our model will facilitate the design of hybrid quantum-classical circuits and the simulation of qubit control and quantum state readout.

Graphs are arguably one of the most fundamental data-structure used in many domains such as block-chain, networks etc. Theoretically and practically, improving Graph performance is one of the most studied and omnipresent research problems. In this paper, we have implemented a dynamic unbounded concurrent graph which can perform the add, delete or lookup operations on vertices and edges concurrently. All these methods are lock-free and linearizable. On top of this, we have also implemented the wait-free graph snapshot algorithm. To the best of knowledge this is first wait-free implementation of snapshot on concurrent graphs. We have used the snapshot of the algorithm to calculate the diameter and between centrality. We have compared our implementation with its counterparts and outperformed them by a good margin. This illustrates the efficiency of our snapshot method which is a generic method and can be used to perform other useful graph analytics operations.

The COVID 19 pandemic has paused many ongoing research projects and unified researchers' attention to focus on COVID 19 related issues. Our project traces 712294 scientists' publications related to COVID 19 for two years, from January 2020 to December 2021, to detect the dynamic evolution patterns of the COVID 19 collaboration network over time. By studying the collaboration network of COVID 19 scientists, we observe how a new scientific community has been built in preparation for a sudden shock. The number of newcomers grows incrementally, and the connectivity of the collaboration network shifts from loose to tight promptly. Even though every scientist has an equal opportunity to start a study, collaboration disparity still exists. Following the scale-free distribution, only a few top authors are highly connected with other authors. These top authors are more likely to attract newcomers and work with each other. As the collaboration network evolves, the increase rate in the probability of attracting newcomers for authors with higher degrees increases, whereas the increase rates in the likelihood of forming new links among authors with higher degrees decreases. This highlights the interesting trend that the COVID pandemic alters the research collaboration trends that star scientists are starting to collaborate more with newcomers but less with existing collaborators, which, in a certain way, reduces the collaboration disparity.

The experimental characterization of quantum spin liquids poses significant challenges due to the absence of long-range magnetic order, even at absolute zero temperature. The identification of these states of matter often relies on the analysis of their excitations. In this paper, we propose a method for detecting the signatures of the fractionalized excitations in quantum spin liquids using a tunneling spectroscopy setup. Inspired by the recent development of the quantum twisting microscope, we consider a planar tunneling junction, in which a candidate quantum spin liquid material is placed between two graphene layers. By tuning the relative twist angle and voltage bias between the leads, we can extract the dynamical spin structure factor of the tunneling barrier with momentum and energy resolution. Our proposal presents a promising tool for experimentally characterizing quantum spin liquids in two-dimensional materials., Comment: 18 pages, 3 figures

Non-blocking search data structures offer scalability with a progress guarantee on high-performance multi-core architectures. In the recent past, "learned queries" have gained remarkable attention. It refers to predicting the rank of a key computed by machine learning models trained to infer the cumulative distribution function of an ordered dataset. A line of works exhibits the superiority of learned queries over classical query algorithms. Yet, to our knowledge, no existing non-blocking search data structure employs them. In this paper, we introduce \textbf{Kanva}, a framework for learned non-blocking search. Kanva has an intuitive yet non-trivial design: traverse down a shallow hierarchy of lightweight linear models to reach the "non-blocking bins," which are dynamic ordered search structures. The proposed approach significantly outperforms the current state-of-the-art -- non-blocking interpolation search trees and elimination (a,b) trees -- in many workload and data distributions. Kanva is provably linearizable.

Interpolation models are critical for a wide range of applications, from numerical optimization to artificial intelligence. The reliability of the provided interpolated value is of utmost importance, and it is crucial to avoid the insurgence of spurious noise. Noise sources can be prevented using proper countermeasures when the training set is designed, but the data sparsity is inevitable in some cases. A typical example is represented by the application of an optimization algorithm: the area where the minimum or maximum of the objective function is assumed to be present is where new data is abundantly added, but other areas of design variable space are significantly neglected. In these cases, a regularization of the interpolation model becomes absolutely crucial. In this paper we are presenting an approach for the regularization of an interpolator based on the control of its kernel function via the condition number of the self-correlation matrix.

LiDAR-based 3D detection plays a vital role in autonomous navigation. Surprisingly, although autonomous vehicles (AVs) must detect both near-field objects (for collision avoidance) and far-field objects (for longer-term planning), contemporary benchmarks focus only on near-field 3D detection. However, AVs must detect far-field objects for safe navigation. In this paper, we present an empirical analysis of far-field 3D detection using the long-range detection dataset Argoverse 2.0 to better understand the problem, and share the following insight: near-field LiDAR measurements are dense and optimally encoded by small voxels, while far-field measurements are sparse and are better encoded with large voxels. We exploit this observation to build a collection of range experts tuned for near-vs-far field detection, and propose simple techniques to efficiently ensemble models for long-range detection that improve efficiency by 33% and boost accuracy by 3.2% CDS., Comment: Accepted to ICCV 2023 Workshop - Robustness and Reliability of Autonomous Vehicles in the Open-World

Semiconductor quantum dots operated dynamically are the basis of many quantum technologies such as quantum sensors and computers. Hence, modelling their electrical properties at microwave frequencies becomes essential to simulate their performance in larger electronic circuits. Here, we develop a self-consistent quantum master equation formalism to obtain the admittance of a quantum dot tunnel-coupled to a charge reservoir under the effect of a coherent photon bath. We find a general expression for the admittance that captures the well-known semiclassical (thermal) limit, along with the transition to lifetime and power broadening regimes due to the increased coupling to the reservoir and amplitude of the photonic drive, respectively. Furthermore, we describe two new photon-mediated regimes: Floquet broadening, determined by the dressing of the QD states, and broadening determined by photon loss in the system. Our results provide a method to simulate the high-frequency behaviour of QDs in a wide range of limits, describe past experiments, and propose novel explorations of QD-photon interactions.

CRUD operations, along with range queries make a highly useful abstract data type (ADT), employed by many dynamic analytics tasks. Despite its wide applications, to our knowledge, no fully wait-free data structure is known to support this ADT. In this paper, we introduce Uruv, a proactive linearizable and practical wait-free concurrent data structure that implements the ADT mentioned above. Structurally, Uruv installs a balanced search index on the nodes of a linked list. Uruv is the first wait-free and proactive solution for concurrent B+tree. Experiments show that Uruv significantly outperforms previously proposed lock-free B+trees for dictionary operations and a recently proposed lock-free method to implement the ADT mentioned above.

Despite its broad practical applications such as in fraud prevention, open-set speaker identification (OSI) has received less attention in the speaker recognition community compared to speaker verification (SV). OSI deals with determining if a test speech sample belongs to a speaker from a set of pre-enrolled individuals (in-set) or if it is from an out-of-set speaker. In addition to the typical challenges associated with speech variability, OSI is prone to the "false-alarm problem"; as the size of the in-set speaker population (a.k.a watchlist) grows, the out-of-set scores become larger, leading to increased false alarm rates. This is in particular challenging for applications in financial institutions and border security where the watchlist size is typically of the order of several thousand speakers. Therefore, it is important to systematically quantify the false-alarm problem, and develop techniques that alleviate the impact of watchlist size on detection performance. Prior studies on this problem are sparse, and lack a common benchmark for systematic evaluations. In this paper, we present the first public benchmark for OSI, developed using the VoxCeleb dataset. We quantify the effect of the watchlist size and speech duration on the watchlist-based speaker detection task using three strong neural network based systems. In contrast to the findings from prior research, we show that the commonly adopted adaptive score normalization is not guaranteed to improve the performance for this task. On the other hand, we show that score calibration and score fusion, two other commonly used techniques in SV, result in significant improvements in OSI performance., Comment: 8 pages

Scene flow estimation is the task of describing the 3D motion field between temporally successive point clouds. State-of-the-art methods use strong priors and test-time optimization techniques, but require on the order of tens of seconds to process full-size point clouds, making them unusable as computer vision primitives for real-time applications such as open world object detection. Feedforward methods are considerably faster, running on the order of tens to hundreds of milliseconds for full-size point clouds, but require expensive human supervision. To address both limitations, we propose Scene Flow via Distillation, a simple, scalable distillation framework that uses a label-free optimization method to produce pseudo-labels to supervise a feedforward model. Our instantiation of this framework, ZeroFlow, achieves state-of-the-art performance on the Argoverse 2 Self-Supervised Scene Flow Challenge while using zero human labels by simply training on large-scale, diverse unlabeled data. At test-time, ZeroFlow is over 1000x faster than label-free state-of-the-art optimization-based methods on full-size point clouds (34 FPS vs 0.028 FPS) and over 1000x cheaper to train on unlabeled data compared to the cost of human annotation (\$394 vs ~\$750,000). To facilitate further research, we release our code, trained model weights, and high quality pseudo-labels for the Argoverse 2 and Waymo Open datasets at https://vedder.io/zeroflow.html, Comment: Accepted to ICLR 2024. 9 pages, 4 pages of citations, 6 pages of Supplemental. Project page with data releases is at http://vedder.io/zeroflow.html

Superconducting stiffness $\rho_s$ and coherence length $\xi$ are usually determined by measuring the penetration depth $\lambda$ of a magnetic field and the upper critical field $H_{c2}$ of a superconductor (SC), respectively. However, in magnetic SC, e.g. some of the iron-based, this could lead to erroneous results since the internal field could be very different from the applied one. To overcome this problem in Fe$_{1+y}$Se$_x$Te$_{1-x}$ with $x \sim 0.5$ and $y \sim 0$ (FST), we measure both quantities with the Stiffnessometer technique. In this technique, one applies a rotor-free vector potential $\textbf{A}$ to a superconducting ring and measures the current density $\textbf{j}$ via the ring's magnetic moment $\textbf{m}$. $\rho_s$ and $\xi$ are determined from London's equation $\textbf{j}=-\rho_s\textbf{A}$ and its range of validity. This method is particularly accurate at temperatures close to the critical temperature $T_c$. We find weaker $\rho_s$ and longer $\xi$ than existing literature reports, and critical exponents which agree better with expectations based on the Ginzburg-Landau theory., Comment: 10 pages, 10 figures

As computing hardware becomes more specialized, designing environmentally sustainable computing systems requires accounting for both hardware and software parameters. Our goal is to design low carbon computing systems while maintaining a competitive level of performance and operational efficiency. Despite previous carbon modeling efforts for computing systems, there is a distinct lack of holistic design strategies to simultaneously optimize for carbon, performance, power and energy. In this work, we take a data-driven approach to characterize the carbon impact (quantified in units of CO2e) of various artificial intelligence (AI) and extended reality (XR) production-level hardware and application use-cases. We propose a holistic design exploration framework to optimize and design for carbon-efficient computing systems and hardware. Our frameworks identifies significant opportunities for carbon efficiency improvements in application-specific and general purpose hardware design and optimization. Using our framework, we demonstrate 10$\times$ carbon efficiency improvement for specialized AI and XR accelerators (quantified by a key metric, tCDP: the product of total CO2e and total application execution time), up to 21% total life cycle carbon savings for existing general-purpose hardware and applications due to hardware over-provisioning, and up to 7.86$\times$ carbon efficiency improvement using advanced 3D integration techniques for resource-constrained XR systems.

In recent years, supervised learning has become the dominant paradigm for training deep-learning based methods for 3D object detection. Lately, the academic community has studied 3D object detection in the context of autonomous vehicles (AVs) using publicly available datasets such as nuScenes and Argoverse 2.0. However, these datasets may have incomplete annotations, often only labeling a small subset of objects in a scene. Although commercial services exists for 3D bounding box annotation, these are often prohibitively expensive. To address these limitations, we propose ReBound, an open-source 3D visualization and dataset re-annotation tool that works across different datasets. In this paper, we detail the design of our tool and present survey results that highlight the usability of our software. Further, we show that ReBound is effective for exploratory data analysis and can facilitate active-learning. Our code and documentation is available at https://github.com/ajedgley/ReBound, Comment: Accepted to CHI 2023 Workshop - Intervening, Teaming, Delegating: Creating Engaging Automation Experiences (AutomationXP)

Blockchain technology is a distributed, decentralized, and immutable ledger system. It is the platform of choice for managing smart contract transactions (SCTs). Smart contracts are self-executing codes of agreement between interested parties commonly implemented using blockchains. A block contains a set of transactions representing changes to the system and a hash of the previous block. The SCTs are executed multiple times during the block production and validation phases across the network. The execution is sequential in most blockchain technologies. In this work, we incorporate a direct acyclic graph (DAG) based parallel scheduler framework for concurrent execution of SCTs. The dependencies among a block's transactions are represented through a concurrent DAG data structure that assists in throughput optimization. We have created a DAG scheduler module that can be incorporated into blockchain platforms for concurrent execution with ease. We have also formally established the safety and liveness properties of the DAG scheduler. For evaluation, our framework is implemented in Hyperledger Sawtooth V1.2.6. The performance across multiple smart contract applications is measured for various scheduler types. Experimental analysis shows that the proposed framework achieves notable performance improvements over the parallel SCT execution frameworks.

Person recognition at a distance entails recognizing the identity of an individual appearing in images or videos collected by long-range imaging systems such as drones or surveillance cameras. Despite recent advances in deep convolutional neural networks (DCNNs), this remains challenging. Images or videos collected by long-range cameras often suffer from atmospheric turbulence, blur, low-resolution, unconstrained poses, and poor illumination. In this paper, we provide a brief survey of recent advances in person recognition at a distance. In particular, we review recent work in multi-spectral face verification, person re-identification, and gait-based analysis techniques. Furthermore, we discuss the merits and drawbacks of existing approaches and identify important, yet under explored challenges for deploying remote person recognition systems in-the-wild., Comment: This work has been accepted to the IEEE Asilomar Conference on Signals, Systems, and Computers (ACSSC) 2022

The diagnosis of primary bone tumors is challenging, as the initial complaints are often non-specific. Early detection of bone cancer is crucial for a favorable prognosis. Incidentally, lesions may be found on radiographs obtained for other reasons. However, these early indications are often missed. In this work, we propose an automatic algorithm to detect bone lesions in conventional radiographs to facilitate early diagnosis. Detecting lesions in such radiographs is challenging: first, the prevalence of bone cancer is very low; any method must show high precision to avoid a prohibitive number of false alarms. Second, radiographs taken in health maintenance organizations (HMOs) or emergency departments (EDs) suffer from inherent diversity due to different X-ray machines, technicians and imaging protocols. This diversity poses a major challenge to any automatic analysis method. We propose to train an off-the-shelf object detection algorithm to detect lesions in radiographs. The novelty of our approach stems from a dedicated preprocessing stage that directly addresses the diversity of the data. The preprocessing consists of self-supervised region-of-interest detection using vision transformer (ViT), and a foreground-based histogram equalization for contrast enhancement to relevant regions only. We evaluate our method via a retrospective study that analyzes bone tumors on radiographs acquired from January 2003 to December 2018 under diverse acquisition protocols. Our method obtains 82.43% sensitivity at 1.5% false-positive rate and surpasses existing preprocessing methods. For lesion detection, our method achieves 82.5% accuracy and an IoU of 0.69. The proposed preprocessing method enables to effectively cope with the inherent diversity of radiographs acquired in HMOs and EDs., Comment: Citation: Tal Zimbalist, Ronnie Rosen, Keren Peri-Hanania, Yaron Caspi, Bar Rinott, Carmel Zeltser-Dekel, Eyal Bercovich, Yonina C. Eldar, Shai Bagon, Detecting bone lesions in X-ray under diverse acquisition conditions, J. Med. Imag. 11(2), 024502 (2024), doi: 10.1117/1.JMI.11.2.024502

Silicon offers the enticing opportunity to integrate hybrid quantum-classical computing systems on a single platform. For qubit control and readout, high-frequency signals are required. Therefore, devices that can facilitate its generation are needed. Here, we present a quantum dot-based radiofrequency multiplier operated at cryogenic temperatures. The device is based on the non-linear capacitance-voltage characteristics of quantum dot systems arising from their low-dimensional density of states. We implement the multiplier in a multi-gate silicon nanowire transistor using two complementary device configurations: a single quantum dot coupled to a charge reservoir and a coupled double quantum dot. We study the harmonic voltage conversion as a function of energy detuning, multiplication factor and harmonic phase noise and find near ideal performance up to a multiplication factor of 10. Our results demonstrate a method for high-frequency conversion that could be readily integrated into silicon-based quantum computing systems and be applied to other semiconductors., Comment: 17 pages, 16 figures

Contemporary autonomous vehicle (AV) benchmarks have advanced techniques for training 3D detectors, particularly on large-scale lidar data. Surprisingly, although semantic class labels naturally follow a long-tailed distribution, contemporary benchmarks focus on only a few common classes (e.g., pedestrian and car) and neglect many rare classes in-the-tail (e.g., debris and stroller). However, AVs must still detect rare classes to ensure safe operation. Moreover, semantic classes are often organized within a hierarchy, e.g., tail classes such as child and construction-worker are arguably subclasses of pedestrian. However, such hierarchical relationships are often ignored, which may lead to misleading estimates of performance and missed opportunities for algorithmic innovation. We address these challenges by formally studying the problem of Long-Tailed 3D Detection (LT3D), which evaluates on all classes, including those in-the-tail. We evaluate and innovate upon popular 3D detection codebases, such as CenterPoint and PointPillars, adapting them for LT3D. We develop hierarchical losses that promote feature sharing across common-vs-rare classes, as well as improved detection metrics that award partial credit to "reasonable" mistakes respecting the hierarchy (e.g., mistaking a child for an adult). Finally, we point out that fine-grained tail class accuracy is particularly improved via multimodal fusion of RGB images with LiDAR; simply put, small fine-grained classes are challenging to identify from sparse (lidar) geometry alone, suggesting that multimodal cues are crucial to long-tailed 3D detection. Our modifications improve accuracy by 5% AP on average for all classes, and dramatically improve AP for rare classes (e.g., stroller AP improves from 3.6 to 31.6)! Our code is available at https://github.com/neeharperi/LT3D, Comment: This work has been accepted to the Conference on Robot Learning (CoRL) 2022

Domain adaptation for sensor-based activity learning is of utmost importance in remote health monitoring research. However, many domain adaptation algorithms suffer with failure to operate adaptation in presence of target domain heterogeneity (which is always present in reality) and presence of multiple inhabitants dramatically hinders their generalizability producing unsatisfactory results for semi-supervised and unseen activity learning tasks. We propose \emph{AEDA}, a novel deep auto-encoder-based model to enable semi-supervised domain adaptation in the existence of target domain heterogeneity and how to incorporate it to empower heterogeneity to any homogeneous deep domain adaptation architecture for cross-domain activity learning. Experimental evaluation on 18 different heterogeneous and multi-inhabitants use-cases of 8 different domains created from 2 publicly available human activity datasets (wearable and ambient smart homes) shows that \emph{AEDA} outperforms (max. 12.8\% and 8.9\% improvements for ambient smart home and wearables) over existing domain adaptation techniques for both seen and unseen activity learning in a heterogeneous setting.

Usually, clinicians assess the correct hemodynamic behavior and fetal well-being during the gestational age thanks to their professional expertise, with the support of some indices defined for Doppler fetal waveforms. Although this approach has demonstrated to be satisfactory in the most of the cases, it can be largely improved with the aid of more advanced techniques, i.e. numerical analysis and simulation. Another key aspect limiting the analysis is that clinicians rely on a limited number of Doppler waveforms observed during the clinical examination. Moreover, the use of simple velocimetric indicators for deriving possible malfunctions of the fetal cardiovascular system can be misleading, being the fetal assessment based on a mere statistical analysis (comparison with physiological ranges), without any deep physio-pathological interpretations of the observed hemodynamic changes. The use of a lumped mathematical model, properly describing the entire fetal cardiovascular system, would be absolutely helpful in this context: by targeting physiological model parameters on the clinical reliefs, we could gain deep insights of the full system. The calibration of model parameters may also help in formulating patient-specific early diagnosis of fetal pathologies. In the present work, we develop a robust parameter estimation algorithm based on two different optimization methods using synthetic data. In particular, we deal with the inverse problem of recognizing the most significant parameters of a lumped fetal circulation model by using time tracings of fetal blood flows and pressures obtained by the model. This represents a first methodological work for the assessment of the accuracy in the identification of model parameters of an algorithm based on closed-loop mathematical model of fetal circulation and opens the way to the application of the algorithm to clinical data., Comment: 33 pages, 6 figures

Currently the UK Electric market is guided by load (demand) forecasts published every thirty minutes by the regulator. A key factor in predicting demand is weather conditions, with forecasts published every hour. We present HYENA: a hybrid predictive model that combines feature engineering (selection of the candidate predictor features), mobile-window predictors and finally LSTM encoder-decoders to achieve higher accuracy with respect to mainstream models from the literature. HYENA decreased MAPE loss by 16\% and RMSE loss by 10\% over the best available benchmark model, thus establishing a new state of the art for the UK electric load (and price) forecasting.

Semiconductor nanostructures hold great promise for high-efficiency waste heat recovery exploiting thermoelectric energy conversion, a technological breakthrough that could significantly contribute to providing environmentally friendly energy sources as well as in enabling the realization of self-powered biomedical and wearable devices. A crucial requirement in this field is the reduction of the thermal conductivity of the thermoelectric material without detrimentally affecting its electrical transport properties. In this work we demonstrate a drastic reduction of thermal conductivity in III-V semiconductor nanowires due to the presence of intentionally realized periodic crystal lattice twin planes. The electrical and thermal transport of these nanostructures, known as twinning superlattice nanowires, have been probed and compared with their twin-free counterparts, showing a one order of magnitude decrease of thermal conductivity while maintaining unaltered electrical transport properties, thus yielding a factor ten enhancement of the thermoelectric figure of merit, ZT. Our study reports for the first time the experimental measurement of electrical and thermal properties in twinning superlattice nanowires, which emerge as a novel class of nanomaterials for high efficiency thermoelectric energy harvesting.

Many existing privacy-enhanced speech emotion recognition (SER) frameworks focus on perturbing the original speech data through adversarial training within a centralized machine learning setup. However, this privacy protection scheme can fail since the adversary can still access the perturbed data. In recent years, distributed learning algorithms, especially federated learning (FL), have gained popularity to protect privacy in machine learning applications. While FL provides good intuition to safeguard privacy by keeping the data on local devices, prior work has shown that privacy attacks, such as attribute inference attacks, are achievable for SER systems trained using FL. In this work, we propose to evaluate the user-level differential privacy (UDP) in mitigating the privacy leaks of the SER system in FL. UDP provides theoretical privacy guarantees with privacy parameters $\epsilon$ and $\delta$. Our results show that the UDP can effectively decrease attribute information leakage while keeping the utility of the SER system with the adversary accessing one model update. However, the efficacy of the UDP suffers when the FL system leaks more model updates to the adversary. We make the code publicly available to reproduce the results in https://github.com/usc-sail/fed-ser-leakage.

Object detection and forecasting are fundamental components of embodied perception. These two problems, however, are largely studied in isolation by the community. In this paper, we propose an end-to-end approach for detection and motion forecasting based on raw sensor measurement as opposed to ground truth tracks. Instead of predicting the current frame locations and forecasting forward in time, we directly predict future object locations and backcast to determine where each trajectory began. Our approach not only improves overall accuracy compared to other modular or end-to-end baselines, it also prompts us to rethink the role of explicit tracking for embodied perception. Additionally, by linking future and current locations in a many-to-one manner, our approach is able to reason about multiple futures, a capability that was previously considered difficult for end-to-end approaches. We conduct extensive experiments on the popular nuScenes dataset and demonstrate the empirical effectiveness of our approach. In addition, we investigate the appropriateness of reusing standard forecasting metrics for an end-to-end setup, and find a number of limitations which allow us to build simple baselines to game these metrics. We address this issue with a novel set of joint forecasting and detection metrics that extend the commonly used AP metrics from the detection community to measuring forecasting accuracy. Our code is available at https://github.com/neeharperi/FutureDet, Comment: This work has been accepted to Computer Vision and Pattern Recognition (CVPR) 2022