Search

A prominent self-supervised learning paradigm is to model the representations as clusters, or more generally as a mixture model. Learning to map the data samples to compact representations and fitting the mixture model simultaneously leads to the representation collapse problem. Regularizing the distribution of data points over the clusters is the prevalent strategy to avoid this issue. While this is sufficient to prevent full representation collapse, we show that a partial prototype collapse problem still exists in the DINO family of methods, that leads to significant redundancies in the prototypes. Such prototype redundancies serve as shortcuts for the method to achieve a marginal latent class distribution that matches the prescribed prior. We show that by encouraging the model to use diverse prototypes, the partial prototype collapse can be mitigated. Effective utilization of the prototypes enables the methods to learn more fine-grained clusters, encouraging more informative representations. We demonstrate that this is especially beneficial when pre-training on a long-tailed fine-grained dataset., Comment: First version of the paper appeared in OpenReview on 22 Sep 2023. Accepted to BMVC 2024

Landau Yang theorem is well known for the past several decades. It prohibits decay of a massive spin 1 particle to two photons. This emerges simply from the representation theory of the Poincare group and Bose Statistics. It does not require any action or Lagrangian. We generalize this theorem to theories with supersymmetry (SUSY) which disallows even decay to two photinos (Majorana fermions). We will prove that if the photon has a mass, howsoever small, this theorem can be evaded. We also show that the supersymmetric selection rule above can also be evaded through the Stueckelberg mass term. Further interesting implications are also pointed out., Comment: 17 pages

Jaynes-Cummings (JC) lattices can be constructed by connecting quantum two-level systems with cavities and have been widely studied for polariton many-body states and multipartite entanglement. Although adiabatic evolution has been studied for the generation of many-body states in this system, its reliance on long timescales can lead to serious decoherence. Here we present a scheme that utilizes local counterdiabatic (CD) driving to provide fast and high-fidelity state preparation in JC lattices. The exact CD Hamiltonian for this system contains nonlocal couplings between qubits and cavities at different and distant sites, which causes a challenge in the implementation. Leveraging the symmetries of the eigenstates under both periodic and open boundary conditions, we derive a local CD Hamiltonian that generates the same dynamics as the exact CD Hamiltonian and our numerical simulations confirm this result. We also show that a multipartite W-state can be prepared with high fidelity using this method. The implementation and decoherence of this scheme with superconducting quantum devices are also discussed., Comment: 6 pages, 4 figures

We report the existence of capillary waves propagating along the boundary between miscible co-flowing fluids. We unveil that the interplay between transient interfacial stresses and confinement drives the progressive transition from the well-known inertial regime, characterized by a wavenumber independent of frequency, $k\sim\omega^{0}$, to an unexpected capillary-wave scaling, $k\sim\omega^{2/3}$. This allows us to measure the effective interfacial tension between miscible fluids and its rapid decay on time scales never probed so far, which we rationalize with a model going beyond square gradient theories., Comment: 20 pages, 11 figures

We present Lagrangian Hashing, a representation for neural fields combining the characteristics of fast training NeRF methods that rely on Eulerian grids (i.e.~InstantNGP), with those that employ points equipped with features as a way to represent information (e.g. 3D Gaussian Splatting or PointNeRF). We achieve this by incorporating a point-based representation into the high-resolution layers of the hierarchical hash tables of an InstantNGP representation. As our points are equipped with a field of influence, our representation can be interpreted as a mixture of Gaussians stored within the hash table. We propose a loss that encourages the movement of our Gaussians towards regions that require more representation budget to be sufficiently well represented. Our main finding is that our representation allows the reconstruction of signals using a more compact representation without compromising quality., Comment: Project page: https://theialab.github.io/laghashes/

Our experiments on a sphere falling under gravity in Stokes flow show significant history effects. We observe an algebraic, not exponential, relaxation rate to the terminal velocity, validating the solution to the Basset-Boussinesq-Oseen equation. Unlike in steady Stokes theory, our experiments and theory reveal a vortex ring forming around the sphere and drifting away. As the Reynolds number nears unity, the vortex ring lags behind the sphere, departing from Stokesian theory, though the sphere's algebraic response persists. These findings are critical for interactions in the Stokes limit.

We report that conventional saturable periodic structures, in sharp contrast to the conventional systems with different nonlinearities which exhibit the typical S- shaped optical bi- and multi-stable states, reveal some unusual and unique nonlinear dynamics. These include the onset of ramp-like optical bistability (OB) and optical multistability (OM) curves which further transit into mixed OM states combining both ramp-like states followed by the S-shaped multistable curves. We also extend this study to another domain of physics, namely parity-time ($\mathcal{PT}$)- symmetry, by including equal amount of gain and loss into the system which then establishes additional degree of freedom by enabling the investigation into additional two domains which are the unbroken and broken $\mathcal{PT}$- symmetric regimes. Although these bi- and multi-stable states are unusual and unique, when the frequency detuning is introduced, the revival of S-shaped stable states is possible but only in the presence of unbroken $\mathcal{PT}$- symmetry. Conversely, the broken $\mathcal{PT}$- symmetry which usually generates ramp-like multistable states, gives rise to the birth of novel multistable states with a vortex like envelope, (the curve that features simultaneous increase in the critical switch-up and switch-down powers with an increase in the input power) causing a novel structure which has not been reported in the existing literature of different physical systems manifesting multi-stable states., Comment: 17 pages, 10 figures. arXiv admin note: substantial text overlap with arXiv:2404.13724, arXiv:2404.13828

A novel proposal linking the origin of supermassive blackhole formation and ultralight dark matter. Ultralight bosons of mass $10^{-22}-10^{-24}$ eV which is one of the front running proposal for dark matter could also linked with the formation and evolution of supermassive blackholes at the center of all the galaxies., Comment: Essay written for the GRF 2024 essay competition: selected for honorable mention

The variations between in-group and out-group speech (intergroup bias) are subtle and could underlie many social phenomena like stereotype perpetuation and implicit bias. In this paper, we model the intergroup bias as a tagging task on English sports comments from forums dedicated to fandom for NFL teams. We curate a unique dataset of over 6 million game-time comments from opposing perspectives (the teams in the game), each comment grounded in a non-linguistic description of the events that precipitated these comments (live win probabilities for each team). Expert and crowd annotations justify modeling the bias through tagging of implicit and explicit referring expressions and reveal the rich, contextual understanding of language and the world required for this task. For large-scale analysis of intergroup variation, we use LLMs for automated tagging, and discover that some LLMs perform best when prompted with linguistic descriptions of the win probability at the time of the comment, rather than numerical probability. Further, large-scale tagging of comments using LLMs uncovers linear variations in the form of referent across win probabilities that distinguish in-group and out-group utterances. Code and data are available at https://github.com/venkatasg/intergroup-nfl .

Particles in pressure-driven channel flow are often inhomogeneously distributed. Two modes of low-Reynolds number instability, absent in Poiseuille flow of clean fluid, are created by inhomogeneous particle loading, and their mechanism is worked out here. Two distinct classes of behaviour are seen: when the critical layer of the dominant perturbation overlaps with variations in particle concentration, the new instabilities arise, which we term overlap modes. But when the layers are distinct, only the traditional Tollmien-Schlichting mode of instability occurs. We derive the dominant critical layer balance equations in this flow along the lines done classically for clean fluid. These reveal how concentration variations within the critical layer cause two the particle-driven instabilities. As a result of these variations, disturbance kinetic energy production is qualitatively and majorly altered. Surprisingly the two overlap modes, though completely different in the symmetry of the eigenstructure and regime of exponential growth, show practically identical energy budgets, highlighting the relevance of variations within the critical layer. The wall layer is shown to be unimportant. We derive a minimal composite theory comprising all terms in the complete equation which are dominant somewhere in the flow, and show that it contains the essential physics. When particles are infinitely dense relative to the fluid, the volume fraction is negligible. But for finite density ratios, the volume fraction of particles causes a profile of effective viscosity. This is shown to be uniformly stabilizing in the present flow. Gravity is neglected here, and will be important to study in future. So will transient growth of perturbations due to non-normality of the stability operator, in a quest for the mechanism of transition to turbulence.

Dual pixels contain disparity cues arising from the defocus blur. This disparity information is useful for many vision tasks ranging from autonomous driving to 3D creative realism. However, directly estimating disparity from dual pixels is less accurate. This work hypothesizes that distilling high-precision dark stereo knowledge, implicitly or explicitly, to efficient dual-pixel student networks enables faithful reconstructions. This dark knowledge distillation should also alleviate stereo-synchronization setup and calibration costs while dramatically increasing parameter and inference time efficiency. We collect the first and largest 3-view dual-pixel video dataset, dpMV, to validate our explicit dark knowledge distillation hypothesis. We show that these methods outperform purely monocular solutions, especially in challenging foreground-background separation regions using faithful guidance from dual pixels. Finally, we demonstrate an unconventional use case unlocked by dpMV and implicit dark knowledge distillation from an ensemble of teachers for Light Field (LF) video reconstruction. Our LF video reconstruction method is the fastest and most temporally consistent to date. It remains competitive in reconstruction fidelity while offering many other essential properties like high parameter efficiency, implicit disocclusion handling, zero-shot cross-dataset transfer, geometrically consistent inference on higher spatial-angular resolutions, and adaptive baseline control. All source code is available at the anonymous repository https://github.com/Aryan-Garg., Comment: International Conference of Computational Photography (ICCP 2024), 11 pages and 12 figures

Self-distillation methods using Siamese networks are popular for self-supervised pre-training. DINO is one such method based on a cross-entropy loss between $K$-dimensional probability vectors, obtained by applying a softmax function to the dot product between representations and learnt prototypes. Given the fact that the learned representations are $L^2$-normalized, we show that DINO and its derivatives, such as iBOT, can be interpreted as a mixture model of von Mises-Fisher components. With this interpretation, DINO assumes equal precision for all components when the prototypes are also $L^2$-normalized. Using this insight we propose DINO-vMF, that adds appropriate normalization constants when computing the cluster assignment probabilities. Unlike DINO, DINO-vMF is stable also for the larger ViT-Base model with unnormalized prototypes. We show that the added flexibility of the mixture model is beneficial in terms of better image representations. The DINO-vMF pre-trained model consistently performs better than DINO on a range of downstream tasks. We obtain similar improvements for iBOT-vMF vs iBOT and thereby show the relevance of our proposed modification also for other methods derived from DINO., Comment: Accepted to ICLR 2023

In this article, we investigate the implications of the unsupervised learning rule known as Input-Correlations (ICO) learning in the nonlinear dynamics of two linearly coupled PT-symmetric Li\'enard oscillators. The fixed points of the oscillator have been evaluated analytically and the Jacobian linearization is employed to study their stability. We find that on increasing the amplitude of the external periodic drive, the system exhibits period-doubling cascade to chaos within a specific parametric regime wherein we observe emergent chaotic dynamics. We further notice that the system indicates an intermittency route to chaos in the chaotic regime. Finally, in the period-4 regime of our bifurcation analysis, we predict the emergence of transient chaos which eventually settles down to a period-2 oscillator response which has been further validated by both the maximal Finite-Time Lyapunov Exponent (FTLE) using the well-known Gram-Schmidt orthogonalization technique and the Hilbert Transform of the time-series. In the transiently chaotic regime, we deploy the ICO learning to analyze the time-series from which we identify that when the chaotic evolution transforms into periodic dynamics, the synaptic weight associated with the time-series of the loss oscillator exhibits stationary temporal evolution. This signifies that in the periodic regime, there is no overlap between the filtered signals obtained from the time-series of the coupled PT-symmetric oscillators. In addition, the temporal evolution of the weight associated with the stimulus mimics the behaviour of the Hilbert transform of the time-series., Comment: 9 pages, 12 figures

Small heavy particles cannot get attracted into a region of closed streamlines in a non-accelerating frame (Sapsis & Haller 2010). In a rotating system, however, particles can get trapped (Angilella 2010) near vortices. We perform numerical simulations examining trapping of inertial particles in a prototypical rotating flow: an identical pair of rotating Lamb-Oseen vortices, without gravity. Our parameter space includes the particle Stokes number $St$, measuring the particle's inertia, and a density parameter $R$, measuring the particle-to-fluid relative density. We focus on inertial particles that are finitely denser than the fluid. Particles can get indefinitely trapped near the vortices and display extreme clustering into smaller dimensional objects: attracting fixed-points, limit cycles and chaotic attractors. As $St$ increases for a given $R$, we may have an incomplete or complete period-doubling route to chaos, as well as an unusual period-halving route back to a fixed-point attractor. The fraction of trapped particles can vary non-monotonically with $St$. We may even have windows in $St$ for which no particle trapping occurs. At $St$ larger than a critical value, beyond no trapping occurs, significant fractions of particles can spend long but finite times in the vortex vicinity. The inclusion of the Basset-Boussinesq history (BBH) force is imperative in our study due to particle's finite density. BBH force significantly increases the basin of attraction as well as the range of $St$ where trapping can occur. Extreme clustering can be physically significant in planetesimal formation by dust aggregation in protoplanetary disks, phytoplankton aggregation in oceans, etc.

In this work, we observe that the $\mathcal{PT}$-symmetric fiber Bragg gratings (PTFBGs) with saturable nonlinearity (SNL) exhibit ramp-like, mixed, optical multistability (OM) in the broken regime. The interplay between nonlinearity and detuning parameter plays a central role in transforming the characteristics of the hysteresis curves and facilitates the realization of different OM curves. Also, it plays a crucial role in reducing the switch-up and down intensities of various stable branches of an OM curve. In a mixed OM curve, either the ramp-like hysteresis curves or S-like hysteresis curves can appear predominantly depending on the magnitude of the detuning parameter. An increase in the device length or nonlinearity increases the number of stable states for fixed values of input intensity. Under a reversal in the direction of light incidence, the ramp-like OM and mixed OM curves assume an unusual vortex-like envelope at lower intensities. Numerical simulations reveal that the switch-up and down intensities of different stable branches of a ramp-like OM and mixed OM curves drift towards the higher and lower intensity sides, respectively (opposite direction). The drift is severe to the extent that an intermediate hysteresis curve features switch-down action at near-zero switching intensities. Also, the input intensities required to realize ramp-like, and mixed OM curves reduce dramatically under a reversal in the direction of light incidence., Comment: 15 pages, 12 figures

In this work, we predict that periodic structures without gain and loss do not exhibit an S-shaped hysteresis curve in the presence of saturable nonlinearity (SNL). Instead, the input-output characteristics of the system admit ramp-like optical bistability (OB) and multistability (OM) curves that are unprecedented in the context of conventional periodic structures in the literature. An increase in the nonlinearity (NL) or the gain-loss parameter increases the switch-up and down intensities of different stable branches in a ramp-like OM curve. Revival of the typical S-shaped hysteresis curve requires the device to work under the combined influence of frequency detuning and $\mathcal{PT}$-symmetry. An increase in the detuning, NL and gain-loss parameters reduces the switching intensities of the S-shaped OB (OM) curves. During the process, mixed OM curves that feature a fusion between ramp-like and S-shaped OM curves emanate at low values of the detuning parameter in the input-output characteristics. The detuning parameter values for which ramp-like, S-shaped, and mixed OM appear varies with the NL coefficient. For a given range of input intensities, the number of stable states admitted by the system increases with the device length or NL. When the laser light enters the device from the opposite end of the grating, nonreciprocal switching occurs at ultra-low intensities via an interplay between NL, detuning, and gain-loss parameters., Comment: 17 pages, 8 figures

Largely due to their implicit nature, neural fields lack a direct mechanism for filtering, as Fourier analysis from discrete signal processing is not directly applicable to these representations. Effective filtering of neural fields is critical to enable level-of-detail processing in downstream applications, and support operations that involve sampling the field on regular grids (e.g. marching cubes). Existing methods that attempt to decompose neural fields in the frequency domain either resort to heuristics or require extensive modifications to the neural field architecture. We show that via a simple modification, one can obtain neural fields that are low-pass filtered, and in turn show how this can be exploited to obtain a frequency decomposition of the entire signal. We demonstrate the validity of our technique by investigating level-of-detail reconstruction, and showing how coarser representations can be computed effectively., Comment: Project Page: https://theialab.github.io/banf

We present a specialized procedural model for generating synthetic agricultural scenes, focusing on soybean crops, along with various weeds. This model is capable of simulating distinct growth stages of these plants, diverse soil conditions, and randomized field arrangements under varying lighting conditions. The integration of real-world textures and environmental factors into the procedural generation process enhances the photorealism and applicability of the synthetic data. Our dataset includes 12,000 images with semantic labels, offering a comprehensive resource for computer vision tasks in precision agriculture, such as semantic segmentation for autonomous weed control. We validate our model's effectiveness by comparing the synthetic data against real agricultural images, demonstrating its potential to significantly augment training data for machine learning models in agriculture. This approach not only provides a cost-effective solution for generating high-quality, diverse data but also addresses specific needs in agricultural vision tasks that are not fully covered by general-purpose models., Comment: 10 pages, 8 figures, 3 tables

Fluid inertia imparts rich dynamics to leaves falling from trees. We show that bodies with two planes of symmetry can display a range of behaviours even without inertia. Any such body supports a conserved quantity in its dynamics, and is either a settler, a drifter or a flutterer, depending only on its shape. At large time, settlers and drifters respectively fall vertically and obliquely, while flutterers rotate forever while executing intricate patterns. The dynamics of flutterers decouples into a periodic and a Floquet part with different time scales, giving periodicity or quasiperiodicity. We design a set of bodies and use the boundary integral method to show that settlers, drifters and flutterers, all lie in this set.

We present a unification and generalization of what is known in the literature as sequentially and hierarchically semi-separable (SSS and HSS) representations for matrices. Describing rank-structured representations of (inverses of) sparse matrices whose adjacency graph is a tree, it is shown that these so-called tree quasi-separable (TQS) matrices inherit all the favorable algebraic properties of SSS and HSS under addition, products, and inversion. To arrive at these properties, we prove a key result that characterizes the conversion of any dense matrix into a TQS representation. Here, we specifically show through an explicit construction procedure that the generator sizes are dictated by the ranks of certain Hankel blocks of the matrix. Analogous to SSS and HSS, TQS matrices admit fast matrix-vector products and direct solvers provided the generator sizes are small. A sketch of the associated algorithms is provided., Comment: 20 pages

In noisy quantum systems, achieving high-fidelity state preparation using the adiabatic approach faces a dilemma: either extending the evolution time to reduce diabatic transitions or shortening it to mitigate decoherence effects. Here, we present a quantum shortcut to adiabaticity for state preparation in a finite-sized Jaynes-Cummings lattice by applying counter-diabatic (CD) driving along given adiabatic trajectories. Leveraging the symmetry of eigenstates in our system, we convert the CD driving to an implementable Hamiltonian that only involves local qubit-cavity couplings for a two-site lattice with one polariton excitation. Additionally, we derive a partial analytical form of the CD driving for the lattice with two excitations. Our numerical results demonstrate that circuit errors and environmental noise have negligible effects on our scheme under practical parameters. We also show that our scheme can be characterized through the detection of qubit operators. This approach can lead to a promising pathway to high-fidelity state preparation in a significantly reduced timescale when compared to conventional adiabatic methods., Comment: 12 pages, 7 figures

Bandyapadhyay et al. introduced the generalized minimum-membership geometric set cover (GMMGSC) problem [SoCG, 2023], which is defined as follows. We are given two sets $P$ and $P'$ of points in $\mathbb{R}^{2}$, $n=\max(|P|, |P'|)$, and a set $\mathcal{S}$ of $m$ axis-parallel unit squares. The goal is to find a subset $\mathcal{S}^{*}\subseteq \mathcal{S}$ that covers all the points in $P$ while minimizing $\mathsf{memb}(P', \mathcal{S}^{*})$, where $\mathsf{memb}(P', \mathcal{S}^{*})=\max_{p\in P'}|\{s\in \mathcal{S}^{*}: p\in s\}|$. We study GMMGSC problem and give a $16$-approximation algorithm that runs in $O(m^2\log m + m^2n)$ time. Our result is a significant improvement to the $144$-approximation given by Bandyapadhyay et al. that runs in $\tilde{O}(nm)$ time. GMMGSC problem is a generalization of another well-studied problem called Minimum Ply Geometric Set Cover (MPGSC), in which the goal is to minimize the ply of $\mathcal{S}^{*}$, where the ply is the maximum cardinality of a subset of the unit squares that have a non-empty intersection. The best-known result for the MPGSC problem is an $8$-approximation algorithm by Durocher et al. that runs in $O(n + m^{8}k^{4}\log k + m^{8}\log m\log k)$ time, where $k$ is the optimal ply value [WALCOM, 2023]., Comment: To appear in CALDAM 2024

Head Non-contrast computed tomography (NCCT) scan remain the preferred primary imaging modality due to their widespread availability and speed. However, the current standard for manual annotations of abnormal brain tissue on head NCCT scans involves significant disadvantages like lack of cutoff standardization and degeneration identification. The recent advancement of deep learning-based computer-aided diagnostic (CAD) models in the multidisciplinary domain has created vast opportunities in neurological medical imaging. Significant literature has been published earlier in the automated identification of brain tissue on different imaging modalities. However, determining Intracranial hemorrhage (ICH) and infarct can be challenging due to image texture, volume size, and scan quality variability. This retrospective validation study evaluated a DL-based algorithm identifying ICH and infarct from head-NCCT scans. The head-NCCT scans dataset was collected consecutively from multiple diagnostic imaging centers across India. The study exhibits the potential and limitations of such DL-based software for introduction in routine workflow in extensive healthcare facilities.

Techniques in causal analysis of language models illuminate how linguistic information is organized in LLMs. We use one such technique, AlterRep, a method of counterfactual probing, to explore the internal structure of multilingual models (mBERT and XLM-R). We train a linear classifier on a binary language identity task, to classify tokens between Language X and Language Y. Applying a counterfactual probing procedure, we use the classifier weights to project the embeddings into the null space and push the resulting embeddings either in the direction of Language X or Language Y. Then we evaluate on a masked language modeling task. We find that, given a template in Language X, pushing towards Language Y systematically increases the probability of Language Y words, above and beyond a third-party control language. But it does not specifically push the model towards translation-equivalent words in Language Y. Pushing towards Language X (the same direction as the template) has a minimal effect, but somewhat degrades these models. Overall, we take these results as further evidence of the rich structure of massive multilingual language models, which include both a language-specific and language-general component. And we show that counterfactual probing can be fruitfully applied to multilingual models., Comment: 12 pages, 5 figures, MRL Workshop @ EMNLP 2023