Search

May 17, 2024 marks the 70th anniversary of Brown v. Board of Education, the landmark Supreme Court decision that ruled segregated schools were "inherently unequal." At the time, North Carolina was one of 17 states that enforced de jure segregation, that is, segregation by law. The state of North Carolina and the school districts within the state have played prominent roles in our nation's history of school desegregation. North Carolina's public school enrollment is increasingly multiracial, and the expansion of school choice means that a growing share of students attends charters and private schools, both of which tend to be more segregated than traditional public schools. On the cusp of this important anniversary, the authors assess where North Carolina schools are now in terms of school desegregation, as segregated schools are systematically linked to unequal educational opportunities and outcomes, while desegregated schools are associated with numerous short-term, long-term, academic, and nonacademic outcomes for individuals and society.

Young and forming planetesimals experience impacts from particles present in a protostellar disk. Using crater scaling laws, we integrate ejecta distributions for oblique impacts. For impacts at 10 to 65 m/s, expected for impacts associated with a disk wind, we estimate the erosion rate and torque exerted on the planetesimal. We find that the mechanism for angular momentum drain proposed by Dobrovolskis and Burns (1984) for asteroids could operate in the low velocity regime of a disk wind. Though spin-down associated with impacts can facilitate planetesimal collapse, we find that the process is inefficient. We find that angular momentum drain via impacts operates in the gravitational focusing regime, though even less efficiently than for lower mass planetesimals. The angular momentum transfer is most effective when the wind speed is low, the projectile density is high compared to the bulk planetesimal density, and the planetesimal is composed of low-strength material. Due to its inefficiency, we find that angular momentum drain due to impacts within a pebble cloud does not by itself facilitate collapse of single planetesimals., Comment: 20 Pages, 10 Figures, Submitted to Icarus. Jupyter Notebook containing the code is included as an Ancillary file

Motion control is crucial for generating expressive and compelling video content; however, most existing video generation models rely mainly on text prompts for control, which struggle to capture the nuances of dynamic actions and temporal compositions. To this end, we train a video generation model conditioned on spatio-temporally sparse or dense motion trajectories. In contrast to prior motion conditioning work, this flexible representation can encode any number of trajectories, object-specific or global scene motion, and temporally sparse motion; due to its flexibility we refer to this conditioning as motion prompts. While users may directly specify sparse trajectories, we also show how to translate high-level user requests into detailed, semi-dense motion prompts, a process we term motion prompt expansion. We demonstrate the versatility of our approach through various applications, including camera and object motion control, "interacting" with an image, motion transfer, and image editing. Our results showcase emergent behaviors, such as realistic physics, suggesting the potential of motion prompts for probing video models and interacting with future generative world models. Finally, we evaluate quantitatively, conduct a human study, and demonstrate strong performance. Video results are available on our webpage: https://motion-prompting.github.io/, Comment: Project page: https://motion-prompting.github.io/

From cell development to space rockets, the mechanical stability of thin shells is crucial across many industrial and natural processes. However, predicting shells' failure properties remains an open challenge, owing to their sensitivity to imperfections inevitably present in real structures. Existing predictive methods are informed by classical stability analyses of idealized shells, which associate shell buckling with a spontaneously emerging delocalized buckling mode. Here, we show through experimental and numerical observations how realistic geometric imperfections instead result in a universal localized buckling mode. The profile of this mode is robust across shells despite their differing underlying defect structures. Further contrasting with classical predictions, a real shell's equilibrium surface deformations grow in the shape of this universal localized mode even at stable loads well before failure. Our findings show that these complex nonlinear dynamics for loads below buckling are described by a 1-D relationship for structures buckling from a saddle-node bifurcation instability. This breakthrough provides a simplified yet general understanding of buckling mechanisms, leading to improved non-destructive predictive methods. To showcase these applications, we experimentally implement a lateral-probing technique that predicts the buckling loads of several unmodified shells to within 1% of the correct value.

In experimental and observational data settings, researchers often have limited knowledge of the reasons for censored or missing outcomes. To address this uncertainty, we propose bounds on causal effects for censored outcomes, accommodating the scenario where censoring is an unobserved mixture of informative and non-informative components. Within this mixed censoring framework, we explore several assumptions to derive bounds on causal effects, including bounds expressed as a function of user-specified sensitivity parameters. We develop influence-function based estimators of these bounds to enable flexible, non-parametric, and machine learning based estimation, achieving root-n convergence rates and asymptotic normality under relatively mild conditions. We further consider the identification and estimation of bounds for other causal quantities that remain meaningful when informative censoring reflects a competing risk, such as death. We conduct extensive simulation studies and illustrate our methodology with a study on the causal effect of antipsychotic drugs on diabetes risk using a health insurance dataset.

We investigate prophet inequalities with competitive ratios approaching $1$, seeking to generalize $k$-uniform matroids. We first show that large girth does not suffice: for all $k$, there exists a matroid of girth $\geq k$ and a prophet inequality instance on that matroid whose optimal competitive ratio is $\frac{1}{2}$. Next, we show $k$-fold matroid unions do suffice: we provide a prophet inequality with competitive ratio $1-O(\sqrt{\frac{\log k}{k}})$ for any $k$-fold matroid union. Our prophet inequality follows from an online contention resolution scheme. The key technical ingredient in our online contention resolution scheme is a novel bicriterion concentration inequality for arbitrary monotone $1$-Lipschitz functions over independent items which may be of independent interest. Applied to our particular setting, our bicriterion concentration inequality yields "Chernoff-strength" concentration for a $1$-Lipschitz function that is not (approximately) self-bounding., Comment: To appear in ITCS 2025

Convex geometry and complex geometry have long had fascinating interactions. This survey offers a tour of a few.

This article initiates the study of discrete Okounkov bodies and higher-dimensional Weierstrass gap phenomena, with applications to asymptotic analysis of stability and global log canonical thresholds.

We introduce the concept of a generative infinite game, a video game that transcends the traditional boundaries of finite, hard-coded systems by using generative models. Inspired by James P. Carse's distinction between finite and infinite games, we leverage recent advances in generative AI to create Unbounded: a game of character life simulation that is fully encapsulated in generative models. Specifically, Unbounded draws inspiration from sandbox life simulations and allows you to interact with your autonomous virtual character in a virtual world by feeding, playing with and guiding it - with open-ended mechanics generated by an LLM, some of which can be emergent. In order to develop Unbounded, we propose technical innovations in both the LLM and visual generation domains. Specifically, we present: (1) a specialized, distilled large language model (LLM) that dynamically generates game mechanics, narratives, and character interactions in real-time, and (2) a new dynamic regional image prompt Adapter (IP-Adapter) for vision models that ensures consistent yet flexible visual generation of a character across multiple environments. We evaluate our system through both qualitative and quantitative analysis, showing significant improvements in character life simulation, user instruction following, narrative coherence, and visual consistency for both characters and the environments compared to traditional related approaches., Comment: Project page: https://generative-infinite-game.github.io/

Scaling up autoregressive models in vision has not proven as beneficial as in large language models. In this work, we investigate this scaling problem in the context of text-to-image generation, focusing on two critical factors: whether models use discrete or continuous tokens, and whether tokens are generated in a random or fixed raster order using BERT- or GPT-like transformer architectures. Our empirical results show that, while all models scale effectively in terms of validation loss, their evaluation performance -- measured by FID, GenEval score, and visual quality -- follows different trends. Models based on continuous tokens achieve significantly better visual quality than those using discrete tokens. Furthermore, the generation order and attention mechanisms significantly affect the GenEval score: random-order models achieve notably better GenEval scores compared to raster-order models. Inspired by these findings, we train Fluid, a random-order autoregressive model on continuous tokens. Fluid 10.5B model achieves a new state-of-the-art zero-shot FID of 6.16 on MS-COCO 30K, and 0.69 overall score on the GenEval benchmark. We hope our findings and results will encourage future efforts to further bridge the scaling gap between vision and language models., Comment: Tech report

Panoramic image stitching provides a unified, wide-angle view of a scene that extends beyond the camera's field of view. Stitching frames of a panning video into a panoramic photograph is a well-understood problem for stationary scenes, but when objects are moving, a still panorama cannot capture the scene. We present a method for synthesizing a panoramic video from a casually-captured panning video, as if the original video were captured with a wide-angle camera. We pose panorama synthesis as a space-time outpainting problem, where we aim to create a full panoramic video of the same length as the input video. Consistent completion of the space-time volume requires a powerful, realistic prior over video content and motion, for which we adapt generative video models. Existing generative models do not, however, immediately extend to panorama completion, as we show. We instead apply video generation as a component of our panorama synthesis system, and demonstrate how to exploit the strengths of the models while minimizing their limitations. Our system can create video panoramas for a range of in-the-wild scenes including people, vehicles, and flowing water, as well as stationary background features., Comment: Project page at https://vidpanos.github.io/. To appear at SIGGRAPH Asia 2024 (conference track)

In the trace reconstruction problem, one attempts to reconstruct a fixed but unknown string $x$ of length $n$ from a given number of traces $\tilde{x}$ drawn iid from the application of a noisy process (such as the deletion channel) to $x$. The best known algorithm for the trace reconstruction from the deletion channel is due to Chase, and recovers the input string whp given $\exp(\tilde{O}(n^{1/5}))$ traces [Cha21b]. The main component in Chase's algorithm is a procedure for k-mer estimation, which, for any marker $w$ in $\{0, 1\}^k$ of length $k$, computes a "smoothed" distribution of its appearances in the input string $x$ [CGL+23, MS24]. Current k-mer estimation algorithms fail when the deletion probability is above $1/2$, requiring a more complex analysis for Chase's algorithm. Moreover, the only known extension of these approaches beyond the deletion channels is based on numerically estimating high-order differentials of a multivariate polynomial, making it highly impractical [Rub23]. In this paper, we construct a simple Monte Carlo method for k-mer estimation which can be easily applied to a much wider variety of channels. In particular, we solve k-mer estimation for any combination of insertion, deletion, and bit-flip channels, even in the high deletion probability regime, allowing us to directly apply Chase's algorithm for this wider class of channels. To accomplish this, we utilize an approach from the field of quantum error mitigation (the process of using many measurements from noisy quantum computers to simulate a clean quantum computer), called the quasi-probability method (also known as probabilistic error cancellation) [TBG17, PSW22]. We derive a completely classical version of this technique, and use it to construct a k-mer estimation algorithm. No background in quantum computing is needed to understand this paper., Comment: 26 pages, 0 figures

Despite the recent progress, existing frame interpolation methods still struggle with processing extremely high resolution input and handling challenging cases such as repetitive textures, thin objects, and large motion. To address these issues, we introduce a patch-based cascaded pixel diffusion model for frame interpolation, HiFI, that excels in these scenarios while achieving competitive performance on standard benchmarks. Cascades, which generate a series of images from low- to high-resolution, can help significantly with large or complex motion that require both global context for a coarse solution and detailed context for high resolution output. However, contrary to prior work on cascaded diffusion models which perform diffusion on increasingly large resolutions, we use a single model that always performs diffusion at the same resolution and upsamples by processing patches of the inputs and the prior solution. We show that this technique drastically reduces memory usage at inference time and also allows us to use a single model at test time, solving both frame interpolation and spatial up-sampling, saving training cost. We show that HiFI helps significantly with high resolution and complex repeated textures that require global context. HiFI demonstrates comparable or beyond state-of-the-art performance on multiple benchmarks (Vimeo, Xiph, X-Test, SEPE-8K). On our newly introduced dataset that focuses on particularly challenging cases, HiFI also significantly outperforms other baselines on these cases. Please visit our project page for video results: https://hifi-diffusion.github.io, Comment: Project page: https://hifi-diffusion.github.io/

It has recently been discovered that the conclusions of many highly influential econometrics studies can be overturned by removing a very small fraction of their samples (often less than $0.5\%$). These conclusions are typically based on the results of one or more Ordinary Least Squares (OLS) regressions, raising the question: given a dataset, can we certify the robustness of an OLS fit on this dataset to the removal of a given number of samples? Brute-force techniques quickly break down even on small datasets. Existing approaches which go beyond brute force either can only find candidate small subsets to remove (but cannot certify their non-existence) [BGM20, KZC21], are computationally intractable beyond low dimensional settings [MR22], or require very strong assumptions on the data distribution and too many samples to give reasonable bounds in practice [BP21, FH23]. We present an efficient algorithm for certifying the robustness of linear regressions to removals of samples. We implement our algorithm and run it on several landmark econometrics datasets with hundreds of dimensions and tens of thousands of samples, giving the first non-trivial certificates of robustness to sample removal for datasets of dimension $4$ or greater. We prove that under distributional assumptions on a dataset, the bounds produced by our algorithm are tight up to a $1 + o(1)$ multiplicative factor., Comment: 65 pages, 2 figures

The composition of protoplanetary disks, and hence the initial conditions of planet formation, may be strongly influenced by the infall and thermal processing of material during the protostellar phase. Composition of dust and ice in protostellar envelopes, shaped by energetic processes driven by the protostar, serves as the fundamental building material for planets and complex organic molecules. As part of the JWST GO program, "Investigating Protostellar Accretion" (IPA), we observed an intermediate-mass protostar HOPS 370 (OMC2-FIR3) using NIRSpec/IFU and MIRI/MRS. This study presents the gas and ice phase chemical inventory revealed with the JWST in the spectral range of $\sim$2.9 to 28 $\mu$m and explores the spatial variation of volatile ice species in the protostellar envelope. We find evidence for thermal processing of ice species throughout the inner envelope. We present the first high-spatial resolution ($\sim 80$ au) maps of key volatile ice species H$_{2}$O, CO$_{2}$, $^{13}$CO$_2$, CO, and OCN$^-$, which reveal a highly structured and inhomogeneous density distribution of the protostellar envelope, with a deficiency of ice column density that coincides with the jet/outflow shocked knots. Further, we observe high relative crystallinity of H$_{2}$O ice around the shocked knot seen in the H$_2$ and OH wind/outflow, which can be explained by a lack of outer colder material in the envelope along the line of sight due to the irregular structure of the envelope. These observations show clear evidence of thermal processing of the ices in the inner envelope, close to the outflow cavity walls, heated by the luminous protostar., Comment: Submitted to ApJ. Main text:16 pages with 11 figures

Training a diverse ensemble of models has several practical applications such as providing candidates for model selection with better out-of-distribution (OOD) generalization, and enabling the detection of OOD samples via Bayesian principles. An existing approach to diverse ensemble training encourages the models to disagree on provided OOD samples. However, the approach is computationally expensive and it requires well-separated ID and OOD examples, such that it has only been demonstrated in small-scale settings. $\textbf{Method.}$ This work presents a method for Scalable Ensemble Diversification (SED) applicable to large-scale settings (e.g. ImageNet) that does not require OOD samples. Instead, SED identifies hard training samples on the fly and encourages the ensemble members to disagree on these. To improve scaling, we show how to avoid the expensive computations in existing methods of exhaustive pairwise disagreements across models. $\textbf{Results.}$ We evaluate the benefits of diversification with experiments on ImageNet. First, for OOD generalization, we observe large benefits from the diversification in multiple settings including output-space (classical) ensembles and weight-space ensembles (model soups). Second, for OOD detection, we turn the diversity of ensemble hypotheses into a novel uncertainty score estimator that surpasses a large number of OOD detection baselines. Code is available here: https://github.com/AlexanderRubinstein/diverse-universe-public., Comment: Under review

Given a so called ''Sperner coloring'' of a triangulation of the $D$-dimensional simplex, Sperner's lemma guarantees the existence of a rainbow simplex, i.e. a simplex colored by all $D+1$ colors. However, finding a rainbow simplex was the first problem to be proven $\mathsf{PPAD}$-complete in Papadimitriou's classical paper introducing the class $\mathsf{PPAD}$ (1994). In this paper, we prove that the problem does not become easier if we relax ''all $D+1$ colors'' to allow some fraction of missing colors: in fact, for any constant $D$, finding even a simplex with just three colors remains $\mathsf{PPAD}$-complete! Our result has an interesting application for the envy-free cake cutting from fair division. It is known that if agents value pieces of cake using general continuous functions satisfying a simple boundary condition (''a non-empty piece is better than an empty piece of cake''), there exists an envy-free allocation with connected pieces. We show that for any constant number of agents it is $\mathsf{PPAD}$-complete to find an allocation -- even using any constant number of possibly disconnected pieces -- that makes just three agents envy-free. Our results extend to super-constant dimension, number of agents, and number of pieces, as long as they are asymptotically bounded by any $\log^{1-\Omega(1)}(\epsilon)$, where $\epsilon$ is the precision parameter (side length for Sperner and approximate envy-free for cake cutting)., Comment: To appear at FOCS 2024

Advances in CAD and CAM have enabled engineers and design teams to digitally design parts with unprecedented ease. Software solutions now come with a range of modules for optimizing designs for performance requirements, generating instructions for manufacturing, and digitally tracking the entire process from design to procurement in the form of product life-cycle management tools. However, existing solutions force design teams and corporations to take a primarily serial approach where manufacturing and procurement decisions are largely contingent on design, rather than being an integral part of the design process. In this work, we propose a new approach to part making where design, manufacturing, and supply chain requirements and resources can be jointly considered and optimized. We present the Generative Manufacturing compiler that accepts as input the following: 1) An engineering part requirements specification that includes quantities such as loads, domain envelope, mass, and compliance, 2) A business part requirements specification that includes production volume, cost, and lead time, 3) Contextual knowledge about the current manufacturing state such as availability of relevant manufacturing equipment, materials, and workforce, both locally and through the supply chain. Based on these factors, the compiler generates and evaluates manufacturing process alternatives and the optimal derivative designs that are implied by each process, and enables a user guided iterative exploration of the design space. As part of our initial implementation of this compiler, we demonstrate the effectiveness of our approach on examples of a cantilever beam problem and a rocket engine mount problem and showcase its utility in creating and selecting optimal solutions according to the requirements and resources.

The multipole expansion, a ubiquitous tool in a variety of physical problems, can be formulated in spherical and Cartesian coordinates. By constructing an explicit map linking both formulations in isotropic media, we discover a lack of equivalence between both approaches in anisotropic media. In isotropic media, the Cartesian multipole tensor can be reduced to a spherical tensor containing significantly fewer entries. In anisotropic media, however, the loss of propagation symmetry prevents this reduction. In turn, non-harmonic sources radiate fields that can be projected onto a finite set of Cartesian multipole moments but require (possibly infinitely) many spherical moments. For harmonic sources, the link between both approaches presents a systematic way to construct the spherical multipole expansion from the Cartesian one. The lack of equivalence between both approaches results in physically significant effects wherever the field propagation includes the Laplace operator. We illustrate this issue on an electromagnetic radiation inverse problem in anisotropic media, including an analysis of a large-anisotropy regime. We show that the use of the Cartesian approach significantly increases the efficiency and interpretability of the model. The proposed approach opens the door to wider applications of the multipole expansion in anisotropic media, whose importance is rising in multiple physical systems., Comment: 16 pages, 5 figures

We show how to use parallelization to speed up sampling from an arbitrary distribution $\mu$ on a product space $[q]^n$, given oracle access to counting queries: $\mathbb{P}_{X\sim \mu}[X_S=\sigma_S]$ for any $S\subseteq [n]$ and $\sigma_S \in [q]^S$. Our algorithm takes $O({n^{2/3}\cdot \operatorname{polylog}(n,q)})$ parallel time, to the best of our knowledge, the first sublinear in $n$ runtime for arbitrary distributions. Our results have implications for sampling in autoregressive models. Our algorithm directly works with an equivalent oracle that answers conditional marginal queries $\mathbb{P}_{X\sim \mu}[X_i=\sigma_i\;\vert\; X_S=\sigma_S]$, whose role is played by a trained neural network in autoregressive models. This suggests a roughly $n^{1/3}$-factor speedup is possible for sampling in any-order autoregressive models. We complement our positive result by showing a lower bound of $\widetilde{\Omega}(n^{1/3})$ for the runtime of any parallel sampling algorithm making at most $\operatorname{poly}(n)$ queries to the counting oracle, even for $q=2$.

Motivated by the study of state opioid policies, we propose a novel approach that uses autoregressive models for causal effect estimation in settings with panel data and staggered treatment adoption. Specifically, we seek to estimate of the impact of key opioid-related policies by quantifying the effects of must access prescription drug monitoring programs (PDMPs), naloxone access laws (NALs), and medical marijuana laws on opioid prescribing. Existing methods, such as differences-in-differences and synthetic controls, are challenging to apply in these types of dynamic policy landscapes where multiple policies are implemented over time and sample sizes are small. Autoregressive models are an alternative strategy that have been used to estimate policy effects in similar settings, but until this paper have lacked formal justification. We outline a set of assumptions that tie these models to causal effects, and we study biases of estimates based on this approach when key causal assumptions are violated. In a set of simulation studies that mirror the structure of our application, we also show that our proposed estimators frequently outperform existing estimators. In short, we justify the use of autoregressive models to provide robust evidence on the effectiveness of four state policies in combating the opioid crisis.

With the growing integration of AI in daily life, ensuring the robustness of systems to inference-time attacks is crucial. Among the approaches for certifying robustness to such adversarial examples, randomized smoothing has emerged as highly promising due to its nature as a wrapper around arbitrary black-box models. Previous work on randomized smoothing in natural language processing has primarily focused on specific subsets of edit distance operations, such as synonym substitution or word insertion, without exploring the certification of all edit operations. In this paper, we adapt Randomized Deletion (Huang et al., 2023) and propose, CERTified Edit Distance defense (CERT-ED) for natural language classification. Through comprehensive experiments, we demonstrate that CERT-ED outperforms the existing Hamming distance method RanMASK (Zeng et al., 2023) in 4 out of 5 datasets in terms of both accuracy and the cardinality of the certificate. By covering various threat models, including 5 direct and 5 transfer attacks, our method improves empirical robustness in 38 out of 50 settings., Comment: 22 pages, 3 figures, 12 tables. Include 11 pages of appendices

A new method recovers the phase difference of interfering wavefronts from a pattern of interference fringes, avoiding the phase discontinuity problem. The method relies on the numerical solution of one-dimensional first-order ordinary differential equations. The solution of each equation allows to compute a corresponding cross section of the two-dimensional phase, leading to the effective recovery of the phase surface. The unwrapping procedure can be performed in each orthogonal direction., Comment: 4 pages 3 figures

Let $\Lambda_X(s)=\det(I-sX^{\dagger})$ be the characteristic polynomial of the unitary matrix $X$. It is believed that the distribution of values of $\Lambda_X(s)$ model the distribution of values of the Riemann zeta-function $\zeta(s)$. This principle motivates many avenues of study. Of particular interest is the behavior of $\Lambda_X'(s)$ and the distribution of its zeros (all of which lie inside or on the unit circle). In this article we present several identities for the moments of $\Lambda_X'(s)$ averaged over $U(N)$, for $x \in \mathbb{C}$ as well as specialized to $|x|=1$. Additionally, we prove, for positive integer $k$, that the polynomial $\int_{U(N)} |\Lambda_X(1)|^{2k} dX$ of degree $k^2$ in $N$ divides the polynomial $\int_{U(N)} |\Lambda_X'(1)|^{2k} dX$ which is of degree $k^2+2k$ in $N$ and that the ratio, $f(N,k)$, of these moments factors into linear factors modulo $4k-1$ if $4k-1$ is prime. We also discuss the relationship of these moments to a solution of a second order non-linear Painl\'{e}ve differential equation. Finally we give some formulas in terms of the $_3F_2$ hypergeometric series for the moments in the simplest case when $N=2$, and also study the radial distribution of the zeros of $\Lambda_X'(s)$ in that case.

Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight $\textit{Spatial Adapters}$ that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on $\textit{"frozen videos"}$ (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel $\textit{Motion Adapter}$ module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model., Comment: Webpage: https://still-moving.github.io/ | Video: https://www.youtube.com/watch?v=U7UuV_VIjnA

We present Magic Insert, a method for dragging-and-dropping subjects from a user-provided image into a target image of a different style in a physically plausible manner while matching the style of the target image. This work formalizes the problem of style-aware drag-and-drop and presents a method for tackling it by addressing two sub-problems: style-aware personalization and realistic object insertion in stylized images. For style-aware personalization, our method first fine-tunes a pretrained text-to-image diffusion model using LoRA and learned text tokens on the subject image, and then infuses it with a CLIP representation of the target style. For object insertion, we use Bootstrapped Domain Adaption to adapt a domain-specific photorealistic object insertion model to the domain of diverse artistic styles. Overall, the method significantly outperforms traditional approaches such as inpainting. Finally, we present a dataset, SubjectPlop, to facilitate evaluation and future progress in this area. Project page: https://magicinsert.github.io/, Comment: Project page: https://magicinsert.github.io/

This article addresses a combinatorial problem with applications to algebraic geometry. To a convex lattice polytope $P$ and each of its integer dilations $kP$ one may associate the barycenter of its lattice points. This sequence of $k$-quantized barycenters converge to the (classical) barycenter of the polytope considered as a convex body. A basic question arises: is there a complete asymptotic expansion for this sequence? If so, what are its terms? This article initiates the study of this question. First, we establish the existence of such an expansion as well as determine the first two terms. Second, for Delzant lattice polytopes we use toric algebra to determine all terms using mixed volumes of virtual rooftop polytopes, or alternatively in terms of higher Donaldson--Futaki invariants. Third, for reflexive polytopes we show the quantized barycenters are colinear to first order, and actually colinear in the case of polygons. The proofs use Ehrhart theory, convexity arguments, and toric algebra. As applications we derive the complete asymptotic expansion of the Fujita--Odaka stability thresholds $\delta_k$ on arbitrary polarizations on (possibly singular) toric varieties. In fact, we show they are rational functions of $k$ for sufficiently large $k$. This gives the first general result on Tian's stabilization problem for $\delta_k$-invariants for (possibly singular) toric Fanos: $\delta_k$ stabilize in $k$ if and only if they are all equal to $1$, and when smooth if and only if asymptotically Chow semistable. We also relate the asymptotic expansions to higher Donaldson--Futaki invariants of test configurations motivated by Ehrhart theory, and unify in passing previous results of Donaldson, Ono, Futaki, and Rubinstein--Tian--Zhang., Comment: with an appendix by Yaxiong Liu

We classify the localizing tensor ideals of the derived categories of mixed Tate motives over certain algebraically closed fields. More precisely, we prove that these categories are stratified in the sense of Barthel, Heard and Sanders. A key ingredient in the proof is the development of a new technique for transporting stratification between categories by means of Brown--Adams representability, which may be of independent interest.

We study algorithms for estimating the size of maximum matching. This problem has been subject to extensive research. For $n$-vertex graphs, Bhattacharya, Kiss, and Saranurak [FOCS'23] (BKS) showed that an estimate that is within $\varepsilon n$ of the optimal solution can be achieved in $n^{2-\Omega_\varepsilon(1)}$ time, where $n$ is the number of vertices. While this is subquadratic in $n$ for any fixed $\varepsilon > 0$, it gets closer and closer to the trivial $\Theta(n^2)$ time algorithm that reads the entire input as $\varepsilon$ is made smaller and smaller. In this work, we close this gap and show that the algorithm of BKS is close to optimal. In particular, we prove that for any fixed $\delta > 0$, there is another fixed $\varepsilon = \varepsilon(\delta) > 0$ such that estimating the size of maximum matching within an additive error of $\varepsilon n$ requires $\Omega(n^{2-\delta})$ time in the adjacency list model.

We study the iteration complexity of decentralized learning of approximate correlated equilibria in incomplete information games. On the negative side, we prove that in $\mathit{extensive}$-$\mathit{form}$ $\mathit{games}$, assuming $\mathsf{PPAD} \not\subset \mathsf{TIME}(n^{\mathsf{polylog}(n)})$, any polynomial-time learning algorithms must take at least $2^{\log_2^{1-o(1)}(|\mathcal{I}|)}$ iterations to converge to the set of $\epsilon$-approximate correlated equilibrium, where $|\mathcal{I}|$ is the number of nodes in the game and $\epsilon > 0$ is an absolute constant. This nearly matches, up to the $o(1)$ term, the algorithms of [PR'24, DDFG'24] for learning $\epsilon$-approximate correlated equilibrium, and resolves an open question of Anagnostides, Kalavasis, Sandholm, and Zampetakis [AKSZ'24]. Our lower bound holds even for the easier solution concept of $\epsilon$-approximate $\mathit{coarse}$ correlated equilibrium On the positive side, we give uncoupled dynamics that reach $\epsilon$-approximate correlated equilibria of a $\mathit{Bayesian}$ $\mathit{game}$ in polylogarithmic iterations, without any dependence of the number of types. This demonstrates a separation between Bayesian games and extensive-form games.

Reuse of data in adaptive workflows poses challenges regarding overfitting and the statistical validity of results. Previous work has demonstrated that interacting with data via differentially private algorithms can mitigate overfitting, achieving worst-case generalization guarantees with asymptotically optimal data requirements. However, such past work assumes data is static and cannot accommodate situations where data grows over time. In this paper we address this gap, presenting the first generalization bounds for adaptive analysis in the dynamic data setting. We allow the analyst to adaptively schedule their queries conditioned on the current size of the data, in addition to previous queries and responses. We also incorporate time-varying empirical accuracy bounds and mechanisms, allowing for tighter guarantees as data accumulates. In a batched query setting, the asymptotic data requirements of our bound grows with the square-root of the number of adaptive queries, matching prior works' improvement over data splitting for the static setting. We instantiate our bound for statistical queries with the clipped Gaussian mechanism, where it empirically outperforms baselines composed from static bounds.

Modern NLP models are often trained on public datasets drawn from diverse sources, rendering them vulnerable to data poisoning attacks. These attacks can manipulate the model's behavior in ways engineered by the attacker. One such tactic involves the implantation of backdoors, achieved by poisoning specific training instances with a textual trigger and a target class label. Several strategies have been proposed to mitigate the risks associated with backdoor attacks by identifying and removing suspected poisoned examples. However, we observe that these strategies fail to offer effective protection against several advanced backdoor attacks. To remedy this deficiency, we propose a novel defensive mechanism that first exploits training dynamics to identify poisoned samples with high precision, followed by a label propagation step to improve recall and thus remove the majority of poisoned instances. Compared with recent advanced defense methods, our method considerably reduces the success rates of several backdoor attacks while maintaining high classification accuracy on clean test sets., Comment: accepted to TACL

Randomized smoothing has shown promising certified robustness against adversaries in classification tasks. Despite such success with only zeroth-order access to base models, randomized smoothing has not been extended to a general form of regression. By defining robustness in regression tasks flexibly through probabilities, we demonstrate how to establish upper bounds on input data point perturbation (using the $\ell_2$ norm) for a user-specified probability of observing valid outputs. Furthermore, we showcase the asymptotic property of a basic averaging function in scenarios where the regression model operates without any constraint. We then derive a certified upper bound of the input perturbations when dealing with a family of regression models where the outputs are bounded. Our simulations verify the validity of the theoretical results and reveal the advantages and limitations of simple smoothing functions, i.e., averaging, in regression tasks. The code is publicly available at \url{https://github.com/arekavandi/Certified_Robust_Regression}.

In recent years, the challenge of extracting information from business documents has emerged as a critical task, finding applications across numerous domains. This effort has attracted substantial interest from both industry and academy, highlighting its significance in the current technological landscape. Most datasets in this area are primarily focused on Key Information Extraction (KIE), where the extraction process revolves around extracting information using a specific, predefined set of keys. Unlike most existing datasets and benchmarks, our focus is on discovering key-value pairs (KVPs) without relying on predefined keys, navigating through an array of diverse templates and complex layouts. This task presents unique challenges, primarily due to the absence of comprehensive datasets and benchmarks tailored for non-predetermined KVP extraction. To address this gap, we introduce KVP10k , a new dataset and benchmark specifically designed for KVP extraction. The dataset contains 10707 richly annotated images. In our benchmark, we also introduce a new challenging task that combines elements of KIE as well as KVP in a single task. KVP10k sets itself apart with its extensive diversity in data and richly detailed annotations, paving the way for advancements in the field of information extraction from complex business documents., Comment: accepted ICDAR2024

The implications of backdoor attacks on English-centric large language models (LLMs) have been widely examined - such attacks can be achieved by embedding malicious behaviors during training and activated under specific conditions that trigger malicious outputs. Despite the increasing support for multilingual capabilities in open-source and proprietary LLMs, the impact of backdoor attacks on these systems remains largely under-explored. Our research focuses on cross-lingual backdoor attacks against multilingual LLMs, particularly investigating how poisoning the instruction-tuning data for one or two languages can affect the outputs for languages whose instruction-tuning data were not poisoned. Despite its simplicity, our empirical analysis reveals that our method exhibits remarkable efficacy in models like mT5 and GPT-4o, with high attack success rates, surpassing 90% in more than 7 out of 12 languages across various scenarios. Our findings also indicate that more powerful models show increased susceptibility to transferable cross-lingual backdoor attacks, which also applies to LLMs predominantly pre-trained on English data, such as Llama2, Llama3, and Gemma. Moreover, our experiments demonstrate 1) High Transferability: the backdoor mechanism operates successfully in cross-lingual response scenarios across 26 languages, achieving an average attack success rate of 99%, and 2) Robustness: the proposed attack remains effective even after defenses are applied. These findings expose critical security vulnerabilities in multilingual LLMs and highlight the urgent need for more robust, targeted defense strategies to address the unique challenges posed by cross-lingual backdoor transfer., Comment: work in progress

Retrieval-augmented generation (RAG) mitigates many problems of fully parametric language models, such as temporal degradation, hallucinations, and lack of grounding. In RAG, the model's knowledge can be updated from documents provided in context. This leads to cases of conflict between the model's parametric knowledge and the contextual information, where the model may not always update its knowledge. Previous work studied context-memory knowledge conflicts by creating synthetic documents that contradict the model's correct parametric answers. We present a framework for studying such knowledge conflicts in a realistic setup. We update incorrect parametric knowledge using real conflicting documents. This reflects how knowledge conflicts arise in practice. In this realistic scenario, we find that knowledge updates fail less often than previously reported. In cases where the models still fail to update their answers, we find a parametric bias: the incorrect parametric answer appearing in context makes the knowledge update likelier to fail. These results suggest that the factual parametric knowledge of LLMs can negatively influence their reading abilities and behaviors. Our code is available at https://github.com/kortukov/realistic_knowledge_conflicts/ .

This work aims to utilize the increased sensitivity and resolution of the JWST to quantify the HDO/H$_{2}$O ratio in ices toward young stellar objects (YSOs) and to determine if the HDO/H$_{2}$O ratios measured in the gas phase toward massive YSOs (MYSOs) are representative of the ratios in their ice envelopes. Two protostars observed in the Investigating Protostellar Accretion (IPA) program using JWST NIRSpec were analyzed: HOPS 370, an intermediate-mass YSO (IMYSO), and IRAS 20126+4104, a MYSO. The HDO ice toward these sources was detected above the 3$\sigma$ level and quantified via its 4.1 $\mu$m band. The contributions from the CH$_{3}$OH combination modes to the observed optical depth in this spectral region were constrained via the CH$_{3}$OH 3.53 $\mu$m band to ensure that the integrated optical depth of the HDO feature was not overestimated. H$_{2}$O ice was quantified via its 3 $\mu$m band. From these fits, ice HDO/H$_{2}$O abundance ratios of 4.6$\pm$1.8$\times$10$^{-3}$ and 2.6$\pm$1.2$\times$10$^{-3}$ are obtained for HOPS 370 and IRAS 20126+4104, respectively. The simultaneous detections of both crystalline HDO and crystalline H$_{2}$O corroborate the assignment of the observed feature at 4.1 $\mu$m to HDO ice. The ice HDO/H$_{2}$O ratios are similar to the highest reported gas HDO/H$_{2}$O ratios measured toward MYSOs as well as the hot inner regions of isolated low-mass protostars, suggesting that at least some of the gas HDO/H$_{2}$O ratios measured toward massive hot cores are representative of the HDO/H$_{2}$O ratios in ices. The need for an H$_{2}$O-rich CH$_{3}$OH component in the CH$_{3}$OH ice analysis supports recent experimental and observational results that indicate that some CH$_{3}$OH ice may form prior to the CO freeze-out stage in H$_{2}$O-rich ice layers., Comment: Accepted for publication in A&A. 23 pages, 17 figures, 10 tables

Polymers are an effective test-bed for studying topological constraints in condensed matter due to a wide array of synthetically-available chain topologies. When linear and ring polymers are blended together, emergent rheological properties are observed as the blend can be more viscous than either of the individual components. This emergent behavior arises since ring-linear blends can form long-lived topological constraints as the linear polymers thread the ring polymers. Here, we demonstrate how the Gauss linking integral can be used to efficiently evaluate the relaxation of topological constraints in ring-linear polymer blends. For majority-linear blends, the relaxation rate of topological constraints depends primarily on reptation of the linear polymers, resulting in the diffusive time $\tau_{d,R}$ for rings of length $N_R$ blended with linear chains of length $N_l$ to scale as $\tau_{d,R}\sim N_R^2N_L^{3.4}$., Comment: 6 pages, 6 figures

A new method recovers phase difference of interfering wavefronts from a pattern of interference fringes, avoiding discontinuity problem. The continuous phase is a solution of the first order differential equation of the interferogram function computed from the fringe intensity profile selected along the pathway over the interferogram., Comment: 4 pages, 1 figure

The recent development of high average, high peak power lasers has revived the effort of using lasers as a potential tool to influence natural lightning. Although impressive, the current progress in laser lightning control technology may only be the beginning of a new area involving a positive feedback between powerful laser development and atmospheric research. In this review paper, we critically evaluate the past, present and future of Laser Lightning Control (LLC), considering both its technological and scientific significance in atmospheric research., Comment: Revised version after first round of peer-review. Restructuration, adding more physics contextuality and pushed some tables in a newly created Appendix

Artificial intelligence (AI) and autonomous edge computing in space are emerging areas of interest to augment capabilities of nanosatellites, where modern sensors generate orders of magnitude more data than can typically be transmitted to mission control. Here, we present the hardware and software design of an onboard AI subsystem hosted on SpIRIT. The system is optimised for on-board computer vision experiments based on visible light and long wave infrared cameras. This paper highlights the key design choices made to maximise the robustness of the system in harsh space conditions, and their motivation relative to key mission requirements, such as limited compute resources, resilience to cosmic radiation, extreme temperature variations, distribution shifts, and very low transmission bandwidths. The payload, called Loris, consists of six visible light cameras, three infrared cameras, a camera control board and a Graphics Processing Unit (GPU) system-on-module. Loris enables the execution of AI models with on-orbit fine-tuning as well as a next-generation image compression algorithm, including progressive coding. This innovative approach not only enhances the data processing capabilities of nanosatellites but also lays the groundwork for broader applications to remote sensing from space., Comment: AI4Space 2024, 3rd Workshop on AI for Space, CVPR 2024

Recent advances in large-scale pretraining have yielded visual foundation models with strong capabilities. Not only can recent models generalize to arbitrary images for their training task, their intermediate representations are useful for other visual tasks such as detection and segmentation. Given that such models can classify, delineate, and localize objects in 2D, we ask whether they also represent their 3D structure? In this work, we analyze the 3D awareness of visual foundation models. We posit that 3D awareness implies that representations (1) encode the 3D structure of the scene and (2) consistently represent the surface across views. We conduct a series of experiments using task-specific probes and zero-shot inference procedures on frozen features. Our experiments reveal several limitations of the current models. Our code and analysis can be found at https://github.com/mbanani/probe3d., Comment: Accepted to CVPR 2024. Project page: https://github.com/mbanani/probe3d

We propose a set of causal estimands that we call the "mediated probabilities of causation." These estimands quantify the probabilities that an observed negative outcome was induced via a mediating pathway versus a direct pathway in a stylized setting involving a binary exposure or intervention, a single binary mediator, and a binary outcome. We outline a set of conditions sufficient to identify these effects given observed data, and propose a doubly-robust projection based estimation strategy that allows for the use of flexible non-parametric and machine learning methods for estimation. We argue that these effects may be more relevant than the probability of causation, particularly in settings where we observe both some negative outcome and negative mediating event, and we wish to distinguish between settings where the outcome was induced via the exposure inducing the mediator versus the exposure inducing the outcome directly. We motivate these estimands by discussing applications to legal and medical questions of causal attribution.

Using the MIRI/MRS spectrometer on JWST, we have detected pure rotational, suprathermal OH emissions from the vicinity of the intermediate-mass protostar HOPS 370 (OMC2/FIR3). These emissions are observed from shocked knots in a jet/outflow, and originate in states of rotational quantum number as high as 46 that possess excitation energies as large as $E_U/k = 4.65 \times 10^4$ K. The relative strengths of the observed OH lines provide a powerful diagnostic of the ultraviolet radiation field in a heavily-extinguished region ($A_V \sim 10 - 20$) where direct UV observations are impossible. To high precision, the OH line strengths are consistent with a picture in which the suprathermal OH states are populated following the photodissociation of water in its $\tilde B - X$ band by ultraviolet radiation produced by fast ($\sim 80\,\rm km\,s^{-1}$) shocks along the jet. The observed dominance of emission from symmetric ($A^\prime$) OH states over that from antisymmetric ($A^{\prime\prime}$) states provides a distinctive signature of this particular population mechanism. Moreover, the variation of intensity with rotational quantum number suggests specifically that Ly$\alpha$ radiation is responsible for the photodissociation of water, an alternative model with photodissociation by a 10$^4$ K blackbody being disfavored at a high level of significance. Using measurements of the Br$\alpha$ flux to estimate the Ly$\alpha$ production rate, we find that $\sim 4\%$ of the Ly$\alpha$ photons are absorbed by water. Combined with direct measurements of water emissions in the $\nu_2 = 1 -0$ band, the OH observations promise to provide key constraints on future models for the diffusion of Ly$\alpha$ photons in the vicinity of a shock front., Comment: 30 pages, 7 figures. Accepted for publication in ApJ Letters

While multilingual machine translation (MNMT) systems hold substantial promise, they also have security vulnerabilities. Our research highlights that MNMT systems can be susceptible to a particularly devious style of backdoor attack, whereby an attacker injects poisoned data into a low-resource language pair to cause malicious translations in other languages, including high-resource languages. Our experimental results reveal that injecting less than 0.01% poisoned data into a low-resource language pair can achieve an average 20% attack success rate in attacking high-resource language pairs. This type of attack is of particular concern, given the larger attack surface of languages inherent to low-resource settings. Our aim is to bring attention to these vulnerabilities within MNMT systems with the hope of encouraging the community to address security concerns in machine translation, especially in the context of low-resource languages., Comment: NAACL main long paper