Search

Exploring scientific datasets with billions of samples in real-time visualization presents a challenge - balancing high-fidelity rendering with speed. This work introduces a novel renderer - Neural Accelerated Renderer (NAR), that uses the neural deferred rendering framework to visualize large-scale scientific point cloud data. NAR augments a real-time point cloud rendering pipeline with high-quality neural post-processing, making the approach ideal for interactive visualization at scale. Specifically, we train a neural network to learn the point cloud geometry from a high-performance multi-stream rasterizer and capture the desired postprocessing effects from a conventional high-quality renderer. We demonstrate the effectiveness of NAR by visualizing complex multidimensional Lagrangian flow fields and photometric scans of a large terrain and compare the renderings against the state-of-the-art high-quality renderers. Through extensive evaluation, we demonstrate that NAR prioritizes speed and scalability while retaining high visual fidelity. We achieve competitive frame rates of $>$ 126 fps for interactive rendering of $>$ 350M points (i.e., an effective throughput of $>$ 44 billion points per second) using $\sim$12 GB of memory on RTX 2080 Ti GPU. Furthermore, we show that NAR is generalizable across different point clouds with similar visualization needs and the desired post-processing effects could be obtained with substantial high quality even at lower resolutions of the original point cloud, further reducing the memory requirements.

Biological cells exhibit a hierarchical spatial organization, where various compartments harbor condensates that form by phase separation. Cells can control the emergence of these condensates by affecting compartment size, the amount of the involved molecules, and their physical interactions. While physical interactions directly affect compartment binding and phase separation, they can also cause oligomerization, which has been suggested as a control mechanism. Analyzing an equilibrium model, we illustrate that oligomerization amplifies compartment binding and phase separation, which reinforce each other. This nonlinear interplay can also induce multistability, which provides additional potential for control. Our work forms the basis for deriving thermodynamically consistent kinetic models to understand how biological cells can regulate phase separation in their compartments., Comment: 6 pages, 5 figures

These lecture notes describe a basic theory of chemically active droplets, which are droplets kept away from equilibrium by driven chemical reactions. The notes assume a basic familiarity with equilibrium thermodynamics of phase separation, and thus focus on three separate themes, which were discussed in three separate lectures: (i) The kinetics of phase separation, including the early-stage dynamics of spinodal decomposition and the late-stage dynamics of Ostwald ripening. (ii) Transition state theory as a simple, thermodynamically-consistent kinetic theory of chemical reactions, which permits explicit driving in open systems. (iii) The combination of phase separation and reactions, leading to active droplets. We discuss the two fundamental classes of internally-maintained and externally-maintained droplets. A simple version of externally-maintained droplets permits an effective electrostatic analogy, which indicates how the reaction-diffusion system mediates long-ranged interactions. All these aspects are discussed in the context of biomolecular condensates., Comment: Lecture Notes for the Boulder Summer School 2024; 29 pages, 9 figures

The numba-mpi package offers access to the Message Passing Interface (MPI) routines from Python code that uses the Numba just-in-time (JIT) compiler. As a result, high-performance and multi-threaded Python code may utilize MPI communication facilities without leaving the JIT-compiled code blocks, which is not possible with the mpi4py package, a higher-level Python interface to MPI. For debugging purposes, numba-mpi retains full functionality of the code even if the JIT compilation is disabled. The numba-mpi API constitutes a thin wrapper around the C API of MPI and is built around Numpy arrays including handling of non-contiguous views over array slices. Project development is hosted at GitHub leveraging the mpi4py/setup-mpi workflow enabling continuous integration tests on Linux (MPICH, OpenMPI & Intel MPI), macOS (MPICH & OpenMPI) and Windows (MS MPI). The paper covers an overview of the package features, architecture and performance. As of v1.0, the following MPI routines are exposed and covered by unit tests: size/rank, [i]send/[i]recv, wait[all|any], test[all|any], allreduce, bcast, barrier, scatter/[all]gather & wtime. The package is implemented in pure Python and depends on numpy, numba and mpi4py (the latter used at initialization and as a source of utility routines only). The performance advantage of using numba-mpi compared to mpi4py is depicted with a simple example, with entirety of the code included in listings discussed in the text. Application of numba-mpi for handling domain decomposition in numerical solvers for partial differential equations is presented using two external packages that depend on numba-mpi: py-pde and PyMPDATA-MPI.

Recent advancements in novel view synthesis have enabled real-time rendering speeds and high reconstruction accuracy. 3D Gaussian Splatting (3D-GS), a foundational point-based parametric 3D scene representation, models scenes as large sets of 3D Gaussians. Complex scenes can comprise of millions of Gaussians, amounting to large storage and memory requirements that limit the viability of 3D-GS on devices with limited resources. Current techniques for compressing these pretrained models by pruning Gaussians rely on combining heuristics to determine which ones to remove. In this paper, we propose a principled spatial sensitivity pruning score that outperforms these approaches. It is computed as a second-order approximation of the reconstruction error on the training views with respect to the spatial parameters of each Gaussian. Additionally, we propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing the training pipeline. After pruning 88.44% of the Gaussians, we observe that our PUP 3D-GS pipeline increases the average rendering speed of 3D-GS by 2.65$\times$ while retaining more salient foreground information and achieving higher image quality metrics than previous pruning techniques on scenes from the Mip-NeRF 360, Tanks & Temples, and Deep Blending datasets.

Mixtures with many components can segregate into coexisting phases, e.g., in biological cells and synthetic materials such as metallic glass. The interactions between components dictate what phases form in equilibrium, but quantifying this relationship has proven difficult. We derive scaling relations for the number of coexisting phases in multicomponent liquids with random interactions and compositions, which we verify numerically. Our results indicate that interactions only need to increase logarithmically with the number of components for the liquid to segregate into many phases. In contrast, a stability analysis of the homogeneous state predicts a power-law scaling. This discrepancy implies an enormous parameter regime where the number of coexisting phases exceeds the number of unstable modes, generalizing the nucleation and growth regime of binary mixtures to many components., Comment: 15 pages, 10 figures, including appendix

Texture plays a vital role in enhancing visual richness in both real photographs and computer-generated imagery. However, the process of editing textures often involves laborious and repetitive manual adjustments of textons, which are the small, recurring local patterns that define textures. In this work, we introduce a fully unsupervised approach for representing textures using a compositional neural model that captures individual textons. We represent each texton as a 2D Gaussian function whose spatial support approximates its shape, and an associated feature that encodes its detailed appearance. By modeling a texture as a discrete composition of Gaussian textons, the representation offers both expressiveness and ease of editing. Textures can be edited by modifying the compositional Gaussians within the latent space, and new textures can be efficiently synthesized by feeding the modified Gaussians through a generator network in a feed-forward manner. This approach enables a wide range of applications, including transferring appearance from an image texture to another image, diversifying textures, texture interpolation, revealing/modifying texture variations, edit propagation, texture animation, and direct texton manipulation. The proposed approach contributes to advancing texture analysis, modeling, and editing techniques, and opens up new possibilities for creating visually appealing images with controllable textures.

Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text. One approach to fine-tuning, called reinforcement learning from human feedback, learns from humans' expressed preferences over multiple outputs. Another approach is constitutional AI, in which the input from humans is a list of high-level principles. But how do we deal with potentially diverging input from humans? How can we aggregate the input into consistent data about "collective" preferences or otherwise use it to make collective choices about model behavior? In this paper, we argue that the field of social choice is well positioned to address these questions, and we discuss ways forward for this agenda, drawing on discussions in a recent workshop on Social Choice for AI Ethics and Safety held in Berkeley, CA, USA in December 2023., Comment: 15 pages, 4 figures

In this paper, we present GaNI, a Global and Near-field Illumination-aware neural inverse rendering technique that can reconstruct geometry, albedo, and roughness parameters from images of a scene captured with co-located light and camera. Existing inverse rendering techniques with co-located light-camera focus on single objects only, without modeling global illumination and near-field lighting more prominent in scenes with multiple objects. We introduce a system that solves this problem in two stages; we first reconstruct the geometry powered by neural volumetric rendering NeuS, followed by inverse neural radiosity that uses the previously predicted geometry to estimate albedo and roughness. However, such a naive combination fails and we propose multiple technical contributions that enable this two-stage approach. We observe that NeuS fails to handle near-field illumination and strong specular reflections from the flashlight in a scene. We propose to implicitly model the effects of near-field illumination and introduce a surface angle loss function to handle specular reflections. Similarly, we observe that invNeRad assumes constant illumination throughout the capture and cannot handle moving flashlights during capture. We propose a light position-aware radiance cache network and additional smoothness priors on roughness to reconstruct reflectance. Experimental evaluation on synthetic and real data shows that our method outperforms the existing co-located light-camera-based inverse rendering techniques. Our approach produces significantly better reflectance and slightly better geometry than capture strategies that do not require a dark room.

Droplets are essential for spatially controlling biomolecules in cells. To work properly, cells need to control the emergence and morphology of droplets. On the one hand, driven chemical reactions can affect droplets profoundly. For instance, reactions can control how droplets nucleate and how large they grow. On the other hand, droplets coexist with various organelles and other structures inside cells, which could affect their nucleation and morphology. To understand the interplay of these two aspects, we study a continuous field theory of active phase separation. Our numerical simulations reveal that reactions suppress nucleation while attractive walls enhance it. Intriguingly, these two effects are coupled, leading to shapes that deviate substantially from the spherical caps predicted for passive systems. These distortions result from anisotropic fluxes responding to the boundary conditions dictated by the Young-Dupr\'e equation. Interestingly, an electrostatic analogy of chemical reactions confirms these effects. We thus demonstrate how driven chemical reactions affect the emergence and morphology of droplets, which could be crucial for understanding biological cells and improving technical applications, e.g., in chemical engineering., Comment: 9 pages, 8 figures

Phase separation, crucial for spatially segregating biomolecules in cells, is well-understood in the simple case of a few components with pairwise interactions. Yet, biological cells challenge the simple picture in at least two ways: First, biomolecules, like proteins and nucleic acids, exhibit complex, higher-order interactions, where a single molecule may interact with multiple others simultaneously. Second, cells comprise a myriad of different components that form various droplets. Such multicomponent phase separation has been studied in the simple case of pairwise interactions, but an analysis of higher-order interactions is lacking. We propose such a theory and study the corresponding phase diagrams numerically. We find that interactions between three components are similar to pairwise interactions, whereas composition-dependent higher-order interactions between two components can oppose phase separation. This surprising result can only be revealed from the equilibrium phase diagrams, implying that the often-used stability analysis of homogeneous states is inadequate to study these systems. We thus show that higher-order interactions could play a crucial role in forming droplets in cells, and their manipulation could offer novel approaches to controlling multicomponent phase separation., Comment: 12 pages, 8 figures, and Appendix

In fair division of a connected graph $G = (V, E)$, each of $n$ agents receives a share of $G$'s vertex set $V$. These shares partition $V$, with each share required to induce a connected subgraph. Agents use their own valuation functions to determine the non-negative numerical values of the shares, which determine whether the allocation is fair in some specified sense. We introduce forbidden substructures called graph cutsets, which block divisions that are fair in the EF1 (envy-free up to one item) sense by cutting the graph into "too many pieces". Two parameters - gap and valence - determine blocked values of $n$. If $G$ guarantees connected EF1 allocations for $n$ agents with valuations that are CA (common and additive), then $G$ contains no elementary cutset of gap $k \ge 2$ and valence in the interval $\[n - k + 1, n - 1\]$. If $G$ guarantees connected EF1 allocations for $n$ agents with valuations in the broader CM (common and monotone) class, then $G$ contains no cutset of gap $k \ge 2$ and valence in the interval $\[n - k + 1, n - 1\]$. These results rule out the existence of connected EF1 allocations in a variety of situations. For some graphs $G$ we can, with help from some new positive results, pin down $G$'s spectrum - the list of exactly which values of $n$ do/do not guarantee connected EF1 allocations. Examples suggest a conjectured common spectral pattern for all graphs. Further, we show that it is NP-hard to determine whether a graph admits a cutset. We also provide an example of a (non-traceable) graph on eight vertices that has no cutsets of gap $\ge 2$ at all, yet fails to guarantee connected EF1 allocations for three agents with CA preferences., Comment: Extended abstract accepted at AAMAS'24

Video matting has broad applications, from adding interesting effects to casually captured movies to assisting video production professionals. Matting with associated effects such as shadows and reflections has also attracted increasing research activity, and methods like Omnimatte have been proposed to separate dynamic foreground objects of interest into their own layers. However, prior works represent video backgrounds as 2D image layers, limiting their capacity to express more complicated scenes, thus hindering application to real-world videos. In this paper, we propose a novel video matting method, OmnimatteRF, that combines dynamic 2D foreground layers and a 3D background model. The 2D layers preserve the details of the subjects, while the 3D background robustly reconstructs scenes in real-world videos. Extensive experiments demonstrate that our method reconstructs scenes with better quality on various videos., Comment: ICCV 2023. Project page: https://omnimatte-rf.github.io/

A number of citation indices have been proposed for measuring and ranking the research publication records of scholars. Some of the best known indices, such as those proposed by Hirsch and Woeginger, are designed to reward most highly those records that strike some balance between productivity (number of papers published), and impact (frequency with which those papers are cited). A large number of rarely cited publications will not score well, nor will a very small number of heavily cited papers. We discuss three new citation indices, one of which was independently proposed in \cite{FHLB}. Each rests on the notion of scale invariance, fundamental to John Nash's solution of the two-person bargaining problem. Our main focus is on one of these -- a scale invariant version of the Hirsch index. We argue that it has advantages over the original; it produces fairer rankings within subdisciplines, is more decisive (discriminates more finely, yielding fewer ties) and more dynamic (growing over time via more frequent, smaller increments), and exhibits enhanced centrality and tail balancedness. Simulations suggest that scale invariance improves robustness under Poisson noise, with increased decisiveness having no cost in terms of the number of ``accidental" reversals, wherein random irregularities cause researcher A to receive a lower index value than B, although A's productivity and impact are both slightly higher than B's. Moreover, we provide an axiomatic characterization of the scale invariant Hirsch index, via axioms that bear a close relationship, in discrete analogue, to those used by Nash in \cite{Nas50}. This argues for the mathematical naturality of the new index. An earlier version was presented at the 5th World Congress of the Game Theory Society, Maastricht, Netherlands in 2016., Comment: 44 pages, 8 figures

We present a robust methodology for identifying photometric binaries in star clusters. Using Gaia DR3, Pan-STARRS and 2MASS data, we self-consistently define the cluster parameters and binary demographics for the open clusters (OCs) NGC 2168 (M35), NGC 7789, NGC 6819, NGC 2682 (M67), NGC 188, and NGC 6791. These clusters span in age from ~200 Myr (NGC 2168) to more than ~8 Gyr (NGC 6791) and have all been extensively studied in the literature. We use the Bayesian Analysis of Stellar Evolution software suite (BASE-9) to derive the age, distance, reddening, metallicity, binary fraction, and binary mass-ratio posterior distributions for each cluster. We perform a careful analysis of our completeness and also compare our results to previous spectroscopic surveys. For our sample of main-sequence stars with masses between 0.6 - 1 M_Sun, we find that these OCs have similar binary fractions that are also broadly consistent with the field multiplicity fraction. Within the clusters, the binary fraction increases dramatically toward the cluster centers, likely a result of mass segregation. Furthermore nearly all clusters show evidence of mass segregation within the single and binary populations, respectively. The OC binary fraction increases significantly with cluster age in our sample, possibly due to a combination of mass-segregation and cluster dissolution processes. We also find a hint of an anti-correlation between binary fraction and cluster central density as well as total cluster mass, possibly due to an increasing frequency of higher energy close stellar encounters that inhibit long-period binary survival and/or formation., Comment: 20 pages, 11 figures, and 5 tables. Accepted for publication in ApJ

We search for mass segregation in the intermediate-aged open cluster NGC 6819 within a carefully identified sample of probable cluster members. Using photometry from the Gaia, 2MASS, and Pan-STARRS surveys as inputs for a Bayesian statistics software suite, BASE-9, we identify a rich population of (photometric) binaries and derive posterior distributions for the cluster age, distance, metallicity and reddening as well as star-by-star photometric membership probabilities, masses and mass ratios (for binaries). Within our entire sample, we find 2632 cluster members and 777 binaries. We then select a main-sequence 'primary sample' with 14.85 < G < 19.5 containing 1342 cluster members and 250 binaries with mass ratios q > 0.5, to investigate for mass segregation. Within this primary sample, we find the binary radial distribution is significantly shifted toward the cluster center as compared to the single stars, resulting in a binary fraction that increases significantly toward the cluster core. Furthermore, we find that within the binary sample, more massive binaries have more centrally concentrated radial distributions than less massive binaries. The same is true for the single stars. We verify the expectation of mass segregation for this stellar sample in NGC 6819 through both relaxation time arguments and by investigating a sophisticated N-body model of the cluster. Importantly, this is the first study to investigate mass segregation of the binaries in the open cluster NGC 6819., Comment: 10 pages, 8 figures, Published in ApJ

M35 is a young open cluster and home to an extensive binary population. Using Gaia DR3, Pan-STARRS, and 2MASS photometry with the Bayesian statistical software, BASE-9, we derive precise cluster parameters, identify single and binary cluster members, and extract their masses. We identify 571 binaries down to Gaia G = 20.3 and a lower-limit on the binary frequency of f_b = 0.41 +/- 0.02. We extend the binary demographics by many magnitudes faint-ward of previous (radial-velocity) studies of this cluster and further away from the cluster center (1.78-degrees, roughly 10 core radii). We find the binary stars to be more centrally concentrated than the single stars in the cluster. Furthermore, we find strong evidence for mass segregation within the binary population itself, with progressively more massive binary samples becoming more and more centrally concentrated. For the single stars, we find weaker evidence for mass segregation; only the most massive single stars (> 2.5MSun) appear more centrally concentrated. Given the cluster age of ~200 Myr, and our derived half-mass relaxation time for the cluster of 230 +/- 84 Myr, we estimate ~47% of the binary stars and ~12% of single stars in the cluster have had time to become dynamically mass segregated. Importantly, when we investigate only stars with mass segregation timescales greater than the cluster age, we still find the binaries to be more centrally concentrated than the singles, suggesting the binaries may have formed with a primordially different spatial distribution than the single stars., Comment: 8 pages, 5 figures, published in ApJL

Elastic microphase separation refers to equilibrium patterns that form by phase separation in elastic gels. Recent experiments revealed a continuous phase transition from the homogeneous phase to a regularly patterned phase, whose period decreased for stiffer systems. We here propose a model that captures these observations. The model combines a continuous field of the elastic component to describe phase separation with nonlocal elasticity theory to capture the gel's microstructure. Analytical approximations unveil that the pattern period is determined by the geometric mean between the elasto-capillary length and a microscopic length scale of the gel. Our theory highlights the importance of nonlocal elasticity in soft matter systems, reveals the mechanism of elastic microphase separation, and will improve the engineering of such systems., Comment: 9 pages, 6 figures

Spatiotemporal patterns are often modeled using reaction-diffusion equations, which combine complex reactions between constituents with ideal diffusive motion. Such descriptions neglect physical interactions between constituents, which might affect resulting patterns. To overcome this, we study how physical interactions affect cyclic dominant reactions, like the seminal rock-paper-scissors game, which exhibits spiral waves for ideal diffusion. Generalizing diffusion to incorporate physical interactions, we find that weak interactions change the length- and time-scales of spiral waves, consistent with a mapping to the complex Ginzburg-Landau equation. In contrast, strong repulsive interactions typically generate oscillating lattices, and strong attraction leads to an interplay of phase separation and chemical oscillations, like droplets co-locating with cores of spiral waves. Our work suggests that physical interactions are relevant for forming spatiotemporal patterns in nature, and it might shed light on how biodiversity is maintained in ecological settings., Comment: 9 pages, 4 figures, and appendix

Inverse rendering methods that account for global illumination are becoming more popular, but current methods require evaluating and automatically differentiating millions of path integrals by tracing multiple light bounces, which remains expensive and prone to noise. Instead, this paper proposes a radiometric prior as a simple alternative to building complete path integrals in a traditional differentiable path tracer, while still correctly accounting for global illumination. Inspired by the Neural Radiosity technique, we use a neural network as a radiance function, and we introduce a prior consisting of the norm of the residual of the rendering equation in the inverse rendering loss. We train our radiance network and optimize scene parameters simultaneously using a loss consisting of both a photometric term between renderings and the multi-view input images, and our radiometric prior (the residual term). This residual term enforces a physical constraint on the optimization that ensures that the radiance field accounts for global illumination. We compare our method to a vanilla differentiable path tracer, and more advanced techniques such as Path Replay Backpropagation. Despite the simplicity of our approach, we can recover scene parameters with comparable and in some cases better quality, at considerably lower computation times., Comment: Homepage: https://inverse-neural-radiosity.github.io

We propose PAniC-3D, a system to reconstruct stylized 3D character heads directly from illustrated (p)ortraits of (ani)me (c)haracters. Our anime-style domain poses unique challenges to single-view reconstruction; compared to natural images of human heads, character portrait illustrations have hair and accessories with more complex and diverse geometry, and are shaded with non-photorealistic contour lines. In addition, there is a lack of both 3D model and portrait illustration data suitable to train and evaluate this ambiguous stylized reconstruction task. Facing these challenges, our proposed PAniC-3D architecture crosses the illustration-to-3D domain gap with a line-filling model, and represents sophisticated geometries with a volumetric radiance field. We train our system with two large new datasets (11.2k Vroid 3D models, 1k Vtuber portrait illustrations), and evaluate on a novel AnimeRecon benchmark of illustration-to-3D pairs. PAniC-3D significantly outperforms baseline methods, and provides data to establish the task of stylized reconstruction from portrait illustrations., Comment: CVPR 2023, code release: https://github.com/ShuhongChen/panic3d-anime-reconstruction

Turing's mechanism is often invoked to explain periodic patterns in nature, although direct experimental support is scarce. Turing patterns form in reaction-diffusion systems when the activating species diffuse much slower than the inhibiting species, and the involved reactions are highly non-linear. Such reactions can originate from co-operativity, whose physical interactions should also affect diffusion. We here take direct interactions into account and show that they strongly affect Turing patterns. We find that weak repulsion between the activator and inhibitor can substantially lower the required differential diffusivity and reaction non-linearity. In contrast, strong interactions can induce phase separation, but the resulting length scale is still typically governed by the fundamental reaction-diffusion length scale. Taken together, our theory connects traditional Turing patterns with chemically active phase separation, thus describing a wider range of systems. Moreover, we demonstrate that even weak interactions affect patterns substantially, so they should be incorporated when modeling realistic systems., Comment: 5 pages, 5 figures, and appendix

The olfactory sense measures the chemical composition of the environment using a diverse array of olfactory receptors. In vertebrates, the olfactory receptors reside in a mucus layer in the nasal cavity and can thus only detect odorants that are inhaled with the airflow and dissolved in mucus. These physical processes fundamentally affect how many odorant molecules contact the receptors. We hypothesize that the olfactory system works efficiently by optimizing the placement of receptors for maximal information transmission. Using a simplified model, we capture all relevant physical processes and show that odorant concentrations generally exhibit an exponential distribution. Combining this result with information theory, we further show that receptors separated into distinct spatial zones maximize the transmitted information. Our results are consistent with experimentally observed receptors zones and might help to improve artificial smell sensors., Comment: 6 pages, 4 figures

Driven chemical reactions can control the macroscopic properties of droplets, like their size. Such active droplets are critical in structuring the interior of biological cells. Cells also need to control where and when droplets appear, so they need to control droplet nucleation. Our numerical simulations demonstrate that reactions generally suppress nucleation if they stabilize the homogeneous state. An equilibrium surrogate model reveals that reactions increase the effective energy barrier of nucleation, enabling quantitative predictions of the increased nucleation times. Moreover, the surrogate model allows us to construct a phase diagram, which summarizes how reactions affect the stability of the homogeneous phase and the droplet state. This simple picture provides accurate predictions of how driven reactions delay nucleation, which is relevant for understanding droplets in biological cells and chemical engineering., Comment: 7 pages, 3 figures

Biological flow networks adapt their network morphology to optimise flow while being exposed to external stimuli from different spatial locations in their environment. These adaptive flow networks retain a memory of the stimulus location in the network morphology. Yet, what limits this memory and how many stimuli can be stored is unknown. Here, we study a numerical model of adaptive flow networks by applying multiple stimuli subsequently. We find strong memory signals for stimuli imprinted for a long time into young networks. Consequently, networks can store many stimuli for intermediate stimulus duration, which balance imprinting and ageing., Comment: 7 pages, 4 figures, 9 pages of appendix