Search

Addressing Lidar Panoptic Segmentation (LPS ) is crucial for safe deployment of autonomous vehicles. LPS aims to recognize and segment lidar points w.r.t. a pre-defined vocabulary of semantic classes, including thing classes of countable objects (e.g., pedestrians and vehicles) and stuff classes of amorphous regions (e.g., vegetation and road). Importantly, LPS requires segmenting individual thing instances (e.g., every single vehicle). Current LPS methods make an unrealistic assumption that the semantic class vocabulary is fixed in the real open world, but in fact, class ontologies usually evolve over time as robots encounter instances of novel classes that are considered to be unknowns w.r.t. the pre-defined class vocabulary. To address this unrealistic assumption, we study LPS in the Open World (LiPSOW): we train models on a dataset with a pre-defined semantic class vocabulary and study their generalization to a larger dataset where novel instances of thing and stuff classes can appear. This experimental setting leads to interesting conclusions. While prior art train class-specific instance segmentation methods and obtain state-of-the-art results on known classes, methods based on class-agnostic bottom-up grouping perform favorably on classes outside of the initial class vocabulary (i.e., unknown classes). Unfortunately, these methods do not perform on-par with fully data-driven methods on known classes. Our work suggests a middle ground: we perform class-agnostic point clustering and over-segment the input cloud in a hierarchical fashion, followed by binary point segment classification, akin to Region Proposal Network [1]. We obtain the final point cloud segmentation by computing a cut in the weighted hierarchical tree of point segments, independently of semantic classification. Remarkably, this unified approach leads to strong performance on both known and unknown classes., Comment: Pre-print. Accepted in the International Journal of Computer Vision, 19 Sept 2024. Code available at https://github.com/g-meghana-reddy/open-world-panoptic-segmentation

Antarctic ice shelves play a vital role in preserving the physical conditions of the Antarctic cryosphere and the Southern Ocean, and beyond. By serving as a buttressing force, ice shelves prevent sea-level rise by restraining the flow of continental ice and glaciers to the sea. Sea-level rise impacts the global environment in multiple ways, including flooding habitats, eroding coastlines, and contaminating soil and groundwater. It is therefore essential to monitor the stability of Antarctic ice shelves, for which a variety of complementary data sources is required. We have developed RIS-Vis, a novel data visualization platform to monitor Antarctic ice shelves. Although focused on the Ross Ice Shelf (RIS), RIS-Vis could be readily scaled to monitor other ice shelves around Antarctica, and elsewhere. Currently, RIS-Vis is capable of analyzing and visualizing seismic, geodetic, and weather data to provide meaningful information for Antarctic cryosphere research. RIS-Vis was built using Python libraries including Obspy, APScheduler, and the Plotly Dash framework, and uses SQLite as the backing database. Visualizations developed on RIS-Vis include filtered seismic waveforms, spectrograms, and power spectral densities, geodetic-based ice-shelf flow, and meteorological variables such as atmospheric temperature and pressure. The dashboard visualization platform abstracts away the time-intensive analysis process of raw data and allows scientists to better concentrate on RIS science.

In this paper, we discuss a longstanding conjecture of Greenberg in the Iwasawa theory of elliptic curves. Greenberg's conjecture states that if $E/\mathbb{Q}$ is an elliptic curve with good ordinary reduction at $p$, and $E[p]$ is irreducible as a Galois module, then the Selmer group of $E$ over the cyclotomic $\mathbb{Z}_p$ extension of $\mathbb{Q}$ has $\mu$-invariant zero. We prove that if $E$ is an elliptic curve over $\mathbb{Q}$, then we have $\mu=0$ for all but finitely many primes $p$ of good ordinary reduction., Comment: Comments welcome! arXiv admin note: text overlap with arXiv:2405.05871

The systematic evaluation and understanding of computer vision models under varying conditions require large amounts of data with comprehensive and customized labels, which real-world vision datasets rarely satisfy. While current synthetic data generators offer a promising alternative, particularly for embodied AI tasks, they often fall short for computer vision tasks due to low asset and rendering quality, limited diversity, and unrealistic physical properties. We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models, based on the newly developed embodied AI benchmark, BEHAVIOR-1K. BVS supports a large number of adjustable parameters at the scene level (e.g., lighting, object placement), the object level (e.g., joint configuration, attributes such as "filled" and "folded"), and the camera level (e.g., field of view, focal length). Researchers can arbitrarily vary these parameters during data generation to perform controlled experiments. We showcase three example application scenarios: systematically evaluating the robustness of models across different continuous axes of domain shift, evaluating scene understanding models on the same set of images, and training and evaluating simulation-to-real transfer for a novel vision task: unary and binary state prediction. Project website: https://behavior-vision-suite.github.io/, Comment: CVPR 2024 (Highlight). Project website: https://behavior-vision-suite.github.io/

This paper is about the Iwasawa theory of elliptic curves over the cyclotomic $\mathbb{Z}_p$-extension $\mathbb{Q}^{\text{cyc}}$ of $\mathbb{Q}$. We discuss a deep conjecture of Greenberg that if $E/\mathbb{Q}$ is an elliptic curve with good ordinary reduction at $p$, and $E[p]$ is irreducible as a Galois module, then the Selmer group of $E$ over $\mathbb{Q}^{\text{cyc}}$ has $\mu$-invariant zero. We prove new cases of Greenberg's conjecture for some elliptic curves of analytic rank $0$. The proof involves studying the $p$-adic $L$-function of $E$. The crucial input is a new technique using the Rankin-Selberg method., Comment: Comments welcome!

We propose a novel, brain-inspired deep neural network model known as the Deep Oscillatory Neural Network (DONN). Deep neural networks like the Recurrent Neural Networks indeed possess sequence processing capabilities but the internal states of the network are not designed to exhibit brain-like oscillatory activity. With this motivation, the DONN is designed to have oscillatory internal dynamics. Neurons of the DONN are either nonlinear neural oscillators or traditional neurons with sigmoidal or ReLU activation. The neural oscillator used in the model is the Hopf oscillator, with the dynamics described in the complex domain. Input can be presented to the neural oscillator in three possible modes. The sigmoid and ReLU neurons also use complex-valued extensions. All the weight stages are also complex-valued. Training follows the general principle of weight change by minimizing the output error and therefore has an overall resemblance to complex backpropagation. A generalization of DONN to convolutional networks known as the Oscillatory Convolutional Neural Network is also proposed. The two proposed oscillatory networks are applied to a variety of benchmark problems in signal and image/video processing. The performance of the proposed models is either comparable or superior to published results on the same data sets.

While large language models (LLMs) have shown remarkable capability to generate convincing text across diverse domains, concerns around its potential risks have highlighted the importance of understanding the rationale behind text generation. We present LLM Attributor, a Python library that provides interactive visualizations for training data attribution of an LLM's text generation. Our library offers a new way to quickly attribute an LLM's text generation to training data points to inspect model behaviors, enhance its trustworthiness, and compare model-generated text with user-provided text. We describe the visual and interactive design of our tool and highlight usage scenarios for LLaMA2 models fine-tuned with two different datasets: online articles about recent disasters and finance-related question-answer pairs. Thanks to LLM Attributor's broad support for computational notebooks, users can easily integrate it into their workflow to interactively visualize attributions of their models. For easier access and extensibility, we open-source LLM Attributor at https://github.com/poloclub/ LLM-Attribution. The video demo is available at https://youtu.be/mIG2MDQKQxM., Comment: 8 pages, 3 figures, For a video demo, see https://youtu.be/mIG2MDQKQxM

We give a complete and self-contained exposition of the $J$-tame inflation lemma: Given any tame almost complex structure $J$ on a symplectic $4$-manifold $(M,\omega)$, and given any compact, embedded, $J$-holomorphic submanifold $Z$, it is always possible to construct a deformation of symplectic forms $\omega_t$ in classes $[\omega_t]=[\omega]+t\mathrm{PD}{Z}$, for $0\leq t$ less than an upper bound $0

Tables convey factual and quantitative data with implicit conventions created by humans that are often challenging for machines to parse. Prior work on table recognition (TR) has mainly centered around complex task-specific combinations of available inputs and tools. We present UniTable, a training framework that unifies both the training paradigm and training objective of TR. Its training paradigm combines the simplicity of purely pixel-level inputs with the effectiveness and scalability empowered by self-supervised pretraining from diverse unannotated tabular images. Our framework unifies the training objectives of all three TR tasks - extracting table structure, cell content, and cell bounding box - into a unified task-agnostic training objective: language modeling. Extensive quantitative and qualitative analyses highlight UniTable's state-of-the-art (SOTA) performance on four of the largest TR datasets. UniTable's table parsing capability has surpassed both existing TR methods and general large vision-language models, e.g., GPT-4o, GPT-4-turbo with vision, and LLaVA. Our code is publicly available at https://github.com/poloclub/unitable, featuring a Jupyter Notebook that includes the complete inference pipeline, fine-tuned across multiple TR datasets, supporting all three TR tasks.

Large language models (LLMs) require well-crafted prompts for effective use. Prompt engineering, the process of designing prompts, is challenging, particularly for non-experts who are less familiar with AI technologies. While researchers have proposed techniques and tools to assist LLM users in prompt design, these works primarily target AI application developers rather than non-experts. To address this research gap, we propose social prompt engineering, a novel paradigm that leverages social computing techniques to facilitate collaborative prompt design. To investigate social prompt engineering, we introduce Wordflow, an open-source and social text editor that enables everyday users to easily create, run, share, and discover LLM prompts. Additionally, by leveraging modern web technologies, Wordflow allows users to run LLMs locally and privately in their browsers. Two usage scenarios highlight how social prompt engineering and our tool can enhance laypeople's interaction with LLMs. Wordflow is publicly accessible at https://poloclub.github.io/wordflow., Comment: 8 pages, 7 figures. Wordflow is available at: https://poloclub.github.io/wordflow. The code is available at: https://github.com/poloclub/wordflow/. For a demo video, see: https://youtu.be/3dOcVuofGVo

We propose a novel Chaotic Associative Memory model using a network of chaotic Rossler systems and investigate the storage capacity and retrieval capabilities of this model as a function of increasing periodicity and chaos. In early models of associate memory networks, memories were modeled as fixed points, which may be mathematically convenient but has poor neurobiological plausibility. Since brain dynamics is inherently oscillatory, attempts have been made to construct associative memories using nonlinear oscillatory networks. However, oscillatory associative memories are plagued by the problem of poor storage capacity, though efforts have been made to improve capacity by adding higher order oscillatory modes. The chaotic associative memory proposed here exploits the continuous spectrum of chaotic elements and has higher storage capacity than previously described oscillatory associate memories., Comment: 10 pages, 8 Figures, Submitted to "Chaos: An Interdisciplinary Journal of Nonlinear Science"

In this paper, we address the problem of predicting collision for objects moving on the surface of a spherical manifold. Toward this end, we develop the notion of a collision triangle on such manifolds. We use this to determine analytical conditions governing the speed ratios and direcions of motion of objects that lead to collisions on the sphere. We first develop these conditions for point objects and subsequently extend this to the case of circular patch shaped objects moving on the sphere., Comment: 14 pages

In a paper published in 2023, Wagner introduced and studied Jacobi forms with complex multiplication, and gave several applications. One such application was in constructing a new doubly-infinite family of partition-theoretic objects, called $(k,j)$-coloured overpartitions and labelled by $\overline{p}_{k,j}$, and using the Jacobi forms to construct crank functions which explain the Ramanujan-type congruences satisfied by $\overline{p}_{k,j}$. In this note, we give an asymptotic formula for the number of $(k,j)$-coloured overpartitions and prove that any crank constructed by Wagner is asymptotically equidistributed on arithmetic progressions, following several recent papers in the literature., Comment: 11 pages, 1 table. Comments welcome!

With the evolution of Large Language Models (LLMs) we can solve increasingly more complex NLP tasks across various domains, including spreadsheets. This work investigates whether LLMs can generate code (Excel OfficeScripts, a TypeScript API for executing many tasks in Excel) that solves Excel specific tasks provided via natural language user instructions. To do so we introduce a new large-scale benchmark, InstructExcel, created by leveraging the 'Automate' feature in Excel to automatically generate OfficeScripts from users' actions. Our benchmark includes over 10k samples covering 170+ Excel operations across 2,000 publicly available Excel spreadsheets. Experiments across various zero-shot and few-shot settings show that InstructExcel is a hard benchmark for state of the art models like GPT-4. We observe that (1) using GPT-4 over GPT-3.5, (2) providing more in-context examples, and (3) dynamic prompting can help improve performance on this benchmark., Comment: Findings of EMNLP 2023, 18 pages

Users are increasingly being warned to check AI-generated content for correctness. Still, as LLMs (and other generative models) generate more complex output, such as summaries, tables, or code, it becomes harder for the user to audit or evaluate the output for quality or correctness. Hence, we are seeing the emergence of tool-assisted experiences to help the user double-check a piece of AI-generated content. We refer to these as co-audit tools. Co-audit tools complement prompt engineering techniques: one helps the user construct the input prompt, while the other helps them check the output response. As a specific example, this paper describes recent research on co-audit tools for spreadsheet computations powered by generative models. We explain why co-audit experiences are essential for any application of generative AI where quality is important and errors are consequential (as is common in spreadsheet computations). We propose a preliminary list of principles for co-audit, and outline research challenges.

This paper explores a multi-agent containment problem, where a fast evader, modeled having constant speed and using constant heading, attempts to escape a circular containment region that is orbited by a slower pursuer with a nonzero capture radius. The pursuer is constrained to move along the edge of the containment region and seeks to capture the evader. This paper presents an in-depth analysis of this pursuer-evader containment scenario. First, multiple types of capture conditions for a single-pursuer case are analyzed defining the worst-case initial position for the pursuer. Second, a parametric study is performed to demonstrate the effects of speed ratio, capture radius, and initial location of the evader. Finally, a reachability analysis is performed to investigate the viable escape headings and reachable regions by the evader. This work provides a foundation for the analysis of escape under more general evader inputs as well as a multiple-pursuer version of the scenario.

We contribute to the classification of modular categories $\mathcal{C}$ with $\operatorname{FPdim}(\mathcal{C})\equiv 2 \pmod 4$. We prove that such categories have group of invertibles of even order, and that they factorize as $\mathcal C\cong \widetilde{\mathcal C} \boxtimes \operatorname{sem}$, where $\widetilde{\mathcal C}$ is an odd-dimensional modular category and $\operatorname{sem}$ is the rank 2 pointed modular category. This reduces the classification of these categories to the classification of odd-dimensional modular categories. It follows that modular categories $\mathcal C$ with $\operatorname{FPdim}(\mathcal{C})\equiv 2 \pmod 4$ of rank up to 46 are pointed. More generally, we prove that if $\mathcal C$ is a weakly integral MTC and $p$ is an odd prime dividing the order of the group of invertibles that has multiplicity one in $\operatorname{FPdim}(\mathcal C)$, then we have a factorization $\mathcal C \cong \widetilde{\mathcal C} \boxtimes \operatorname{Vec}_{\mathbb Z_p}^{\chi},$ for $\widetilde{\mathcal C}$ an MTC with dimension not divisible by $p$., Comment: Theorem 3.13 has been generalized