Search

Synthetic data (SIM) drawn from simulators have emerged as a popular alternative for training models where acquiring annotated real-world images is difficult. However, transferring models trained on synthetic images to real-world applications can be challenging due to appearance disparities. A commonly employed solution to counter this SIM2REAL gap is unsupervised domain adaptation, where models are trained using labeled SIM data and unlabeled REAL data. Mispredictions made by such SIM2REAL adapted models are often associated with miscalibration - stemming from overconfident predictions on real data. In this paper, we introduce AUGCAL, a simple training-time patch for unsupervised adaptation that improves SIM2REAL adapted models by - (1) reducing overall miscalibration, (2) reducing overconfidence in incorrect predictions and (3) improving confidence score reliability by better guiding misclassification detection - all while retaining or improving SIM2REAL performance. Given a base SIM2REAL adaptation algorithm, at training time, AUGCAL involves replacing vanilla SIM images with strongly augmented views (AUG intervention) and additionally optimizing for a training time calibration loss on augmented SIM predictions (CAL intervention). We motivate AUGCAL using a brief analytical justification of how to reduce miscalibration on unlabeled REAL data. Through our experiments, we empirically show the efficacy of AUGCAL across multiple adaptation methods, backbones, tasks and shifts., Comment: Published at ICLR 2024

We present the results of a time-dependent search for neutrino flares in data collected by IceCube between May 2011 and 2021. This data set contains cascade-like events originating from charged-current electron neutrino and tau neutrino interactions and all-flavor neutral-current interactions. IceCube's previous all-sky searches for neutrino flares used data sets consisting of track-like events originating from charged-current muon neutrino interactions. The cascade data sets are statistically independent of the track data sets and provide a new opportunity to observe the transient all-sky landscape. This search uses the spatial, temporal, and energy information of the cascade-like events to conduct searches for the most statistically significant neutrino flares in the northern and southern skies. No statistically significant time-dependent neutrino emission was observed. For the most statistically significant location in the northern sky, $p_\mathrm{global} =$ 0.71, and in the southern sky, $p_\mathrm{global} =$ 0.51. These results are compatible with the background hypothesis. Assuming an E$^{-2.53}$ spectrum from the diffuse astrophysical neutrino flux as measured with cascades, these results are used to calculate upper limits at the 90\% confidence level on neutrino flares of varying duration and constrain the contribution of these flares to the diffuse astrophysical neutrino flux. These constraints are independent of a specified class of astrophysical objects and show that multiple unresolved transient sources may contribute to the diffuse astrophysical neutrino flux., Comment: Submitted to The Astrophysical Journal

Large-scale automatic speech translation systems today lack key features that help machine-mediated communication feel seamless when compared to human-to-human dialogue. In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion. First, we contribute an improved version of the massively multilingual and multimodal SeamlessM4T model-SeamlessM4T v2. This newer model, incorporating an updated UnitY2 framework, was trained on more low-resource language data. SeamlessM4T v2 provides the foundation on which our next two models are initiated. SeamlessExpressive enables translation that preserves vocal styles and prosody. Compared to previous efforts in expressive speech research, our work addresses certain underexplored aspects of prosody, such as speech rate and pauses, while also preserving the style of one's voice. As for SeamlessStreaming, our model leverages the Efficient Monotonic Multihead Attention mechanism to generate low-latency target translations without waiting for complete source utterances. As the first of its kind, SeamlessStreaming enables simultaneous speech-to-speech/text translation for multiple source and target languages. To ensure that our models can be used safely and responsibly, we implemented the first known red-teaming effort for multimodal machine translation, a system for the detection and mitigation of added toxicity, a systematic evaluation of gender bias, and an inaudible localized watermarking mechanism designed to dampen the impact of deepfakes. Consequently, we bring major components from SeamlessExpressive and SeamlessStreaming together to form Seamless, the first publicly available system that unlocks expressive cross-lingual communication in real-time. The contributions to this work are publicly released and accessible at https://github.com/facebookresearch/seamless_communication

This paper presents the modeling and numerical analysis of the Kangaroo lower body prototype, a novel bipedal humanoid robot developed and manufactured by PAL Robotics. Kangaroo features high-power linear electric actuators combined with unique serial-parallel hybrid chains, which allow for the positioning of all the leg actuators near the base of the robot in order to improve the overall mass distribution. To model and analyze such complex nonlinear mechanisms, we employ a constrained formulation that is extended to account for floating-base systems in contact with the environment. A comparison is made to demonstrate the significant improvements achieved with TALOS, another humanoid bipedal robot designed by PAL Robotics, in terms of equivalent Cartesian inertia at the feet and centroidal angular momentum. Finally, the paper includes numerical experiments conducted through simulation and preliminary tests performed on the actual Kangaroo platform.

This brief shares learnings from Building Equitable Learning Environments (BELE) district partnerships around the sixth Essential Action: Measure What Matters. This Essential Action focuses on achieving equitable learning environments through the routine collection and review of relevant student feedback data to co-design new practices and policies to improve learning environments. Although the following paper highlights a specific student feedback tool (Elevate), the authors believe these learnings are applicable to other student-centered data systems used by schools and districts alike to measure what matters. As a result of working directly with district and school partners, in addition to focus groups with BELE Network coaches and leaders, the authors found five key actions as necessary for supporting effective implementation of student-centered data systems to improve learning conditions and experiences at the classroom and school level: (1) center and share power with students through student experience data; (2) clearly and frequently communicate the purpose for student data systems; (3) use data to cultivate and model a culture of learning and growth for adults and students; (4) build on and nurture foundational trust and safety through practices that demonstrate respect and professional regard; and (5) demonstrate and facilitate effective implementation through district- and school-level support. Each section of this paper centers on a key action, interwoven with field examples ("In Action") of the condition's practical application in partner districts. [This report was produced with the National Equity Project, part of the Building Equitable Learning Environments (BELE) Network.]

The SEL Innovations series aims to help the field imagine new, more expansive and equitable approaches to SEL and wellness to ensure that all children, adolescents, and adults feel safe, supported, and seen so that they can thrive. This report focuses on the conditions and guiding actions to foster authentic school-family partnerships. Research confirms that school-family partnerships are crucial. While much attention has been given to the core competencies, our new innovations series digs deeper into the "rings" of CASEL's SEL Framework where young people live and learn, beginning with school-family partnerships. The report offers case studies illustrating four guiding actions in practice, featuring Communities in Schools of Michigan, The Allegheny County Family Center Network's Parent Advisory Council, Village of Wisdom, and Minneapolis Public Schools Parent Participatory Evaluation Initiative. The report includes these guiding actions for school staff: (1) Begin with family priorities, interests, concerns, knowledge, and resources to create opportunities for building trust and genuine relationships; (2) Transform power dynamics by increasing the capacity of both families and staff around knowledge and confidence; (3) Build reciprocity and agency between school staff and families by setting up systems and structures that foster a sense of camaraderie and collaboration; and (4) Undertake change as collective inquiry together using the relationships that are built as a way to collaboratively problem-solve.

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge. Ego-Exo4D centers around simultaneously-captured egocentric and exocentric video of skilled human activities (e.g., sports, music, dance, bike repair). 740 participants from 13 cities worldwide performed these activities in 123 different natural scene contexts, yielding long-form captures from 1 to 42 minutes each and 1,286 hours of video combined. The multimodal nature of the dataset is unprecedented: the video is accompanied by multichannel audio, eye gaze, 3D point clouds, camera poses, IMU, and multiple paired language descriptions -- including a novel "expert commentary" done by coaches and teachers and tailored to the skilled-activity domain. To push the frontier of first-person video understanding of skilled human activity, we also present a suite of benchmark tasks and their annotations, including fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand/body pose. All resources are open sourced to fuel new research in the community. Project page: http://ego-exo4d-data.org/, Comment: Expanded manuscript (compared to arxiv v1 from Nov 2023 and CVPR 2024 paper from June 2024) for more comprehensive dataset and benchmark presentation, plus new results on v2 data release

Accurate atmospheric 3D wind observations are a high priority in the science community. To address this requirement and to support researchers' needs to acquire and analyze wind data from multiple sources, the System for Analysis of Wind Collocations (SAWC) was jointly developed by NOAA/NESDIS/STAR, UMD/ESSIC/CISESS, and UW-Madison/CIMSS. SAWC encompasses a multi-year archive of global 3D winds observed by Aeolus, sondes, aircraft, stratospheric superpressure balloons, and satellite-derived atmospheric motion vectors, archived and uniformly formatted in netCDF for public consumption; identified pairings between select datasets collocated in space and time; and a downloadable software application developed for users to interactively collocate and statistically compare wind observations based on their research needs. The utility of SAWC is demonstrated by conducting a one-year (September 2019-August 2020) evaluation of Aeolus level-2B (L2B) winds (Baseline 11 L2B processor version). Observations from four archived conventional wind datasets are collocated with Aeolus. Recommended quality controls are applied. Wind comparisons are assessed using the SAWC collocation application. Comparison statistics are stratified by season, geographic region, and Aeolus observing mode. The results highlight the value of SAWC's capabilities, from product validation through intercomparison studies to the evaluation of data usage in applications and advances in the global Earth observing architecture, Comment: 20 pages, 13 figures, 6 tables

Large language models (LLMs) solve problems more accurately and interpretably when instructed to work out the answer step by step using a ``chain-of-thought'' (CoT) prompt. One can also improve LLMs' performance on a specific task by supervised fine-tuning, i.e., by using gradient ascent on some tunable parameters to maximize the average log-likelihood of correct answers from a labeled training set. Naively combining CoT with supervised tuning requires supervision not just of the correct answers, but also of detailed rationales that lead to those answers; these rationales are expensive to produce by hand. Instead, we propose a fine-tuning strategy that tries to maximize the \emph{marginal} log-likelihood of generating a correct answer using CoT prompting, approximately averaging over all possible rationales. The core challenge is sampling from the posterior over rationales conditioned on the correct answer; we address it using a simple Markov-chain Monte Carlo (MCMC) expectation-maximization (EM) algorithm inspired by the self-taught reasoner (STaR), memoized wake-sleep, Markovian score climbing, and persistent contrastive divergence. This algorithm also admits a novel control-variate technique that drives the variance of our gradient estimates to zero as the model improves. Applying our technique to GSM8K and the tasks in BIG-Bench Hard, we find that this MCMC-EM fine-tuning technique typically improves the model's accuracy on held-out examples more than STaR or prompt-tuning with or without CoT., Comment: 23 pages, 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

The Axion Dark Matter eXperiment (ADMX) has previously excluded Dine-Fischler-Srednicki-Zhitnisky (DFSZ) axions between 680-790 MHz under the assumption that the dark matter is described by the isothermal halo model. However, the precise nature of the velocity distribution of dark matter is still unknown, and alternative models have been proposed. We report the results of a non-virialized axion search over the mass range 2.81-3.31 {\mu}eV, corresponding to the frequency range 680-800 MHz. This analysis marks the most sensitive search for non-virialized axions sensitive to Doppler effects in the Milky Way Halo to date. Accounting for frequency shifts due to the detector's motion through the Galaxy, we exclude cold flow relic axions with a velocity dispersion of order 10^-7 c with 95% confidence.

Window attention, position embeddings, and high resolution finetuning are core concepts in the modern transformer era of computer vision. However, we find that naively combining these near ubiquitous components can have a detrimental effect on performance. The issue is simple: interpolating position embeddings while using window attention is wrong. We study two state-of-the-art methods that have these three components, namely Hiera and ViTDet, and find that both do indeed suffer from this bug. To fix it, we introduce a simple absolute window position embedding strategy, which solves the bug outright in Hiera and allows us to increase both speed and performance of the model in ViTDet. We finally combine the two to obtain HieraDet, which achieves 61.7 box mAP on COCO, making it state-of-the-art for models that only use ImageNet-1k pretraining. This all stems from what is essentially a 3 line bug fix, which we name "absolute win"., Comment: Preprint. Code release will be coming in the future