Search

In this work, we introduce Mask-JEPA, a self-supervised learning framework tailored for mask classification architectures (MCA), to overcome the traditional constraints associated with training segmentation models. Mask-JEPA combines a Joint Embedding Predictive Architecture with MCA to adeptly capture intricate semantics and precise object boundaries. Our approach addresses two critical challenges in self-supervised learning: 1) extracting comprehensive representations for universal image segmentation from a pixel decoder, and 2) effectively training the transformer decoder. The use of the transformer decoder as a predictor within the JEPA framework allows proficient training in universal image segmentation tasks. Through rigorous evaluations on datasets such as ADE20K, Cityscapes and COCO, Mask-JEPA demonstrates not only competitive results but also exceptional adaptability and robustness across various training scenarios. The architecture-agnostic nature of Mask-JEPA further underscores its versatility, allowing seamless adaptation to various mask classification family., Comment: 27 pages, 5 figures

Domain shift occurs when training (source) and test (target) data diverge in their distribution. Source-Free Domain Adaptation (SFDA) addresses this domain shift problem, aiming to adopt a trained model on the source domain to the target domain in a scenario where only a well-trained source model and unlabeled target data are available. In this scenario, handling false labels in the target domain is crucial because they negatively impact the model performance. To deal with this problem, we propose to update cluster prototypes (i.e., centroid of each sample cluster) and their structure in the target domain formulated by the source model in online manners. In the feature space, samples in different regions have different pseudo-label distribution characteristics affected by the cluster prototypes, and we adopt distinct training strategies for these samples by defining clean and noisy regions: we selectively train the target with clean pseudo-labels in the clean region, whereas we introduce mix-up inputs representing intermediate features between clean and noisy regions to increase the compactness of the cluster. We conducted extensive experiments on multiple datasets in online/offline SFDA settings, whose results demonstrate that our method, CNG-SFDA, achieves state-of-the-art for most cases. Code is available at https://github.com/hyeonwoocho7/CNG-SFDA., Comment: 14 pages, 5 figures, ACCV 2024 Camera-Ready Version

We investigate quantum phase transitions in the transverse field Ising chain with algebraically decaying long-range (LR) antiferromagnetic interactions using the variational Monte Carlo method with the restricted Boltzmann machine employed as a trial wave function ansatz. First, we measure the critical exponents and the central charge through the finite-size scaling analysis, verifying the contrasting observations in the previous tensor network studies. The correlation function exponent and the central charge deviate from the short-range (SR) Ising values at a small decay exponent $\alpha_\mathrm{LR}$, while the other critical exponents examined are very close to the SR Ising exponents regardless of $\alpha_\mathrm{LR}$ examined. However, in the further test of the critical Binder ratio, we find that the universal ratio of the SR limit does not hold for $\alpha_\mathrm{LR} < 2$, implying a deviation in the criticality. On the other hand, we find evidence of the conformal invariance breakdown in the conformal field theory (CFT) test of the correlation function. The deviation from the CFT description becomes more pronounced as $\alpha_\mathrm{LR}$ decreases, although a precise breakdown threshold is yet to be determined.

We present a perturbation method to compute the out-of-time-ordered correlator in the strongly disordered Heisenberg XXZ model in the deep many-body localized regime. We characterize the discrete structure of the information propagation across the eigenstates, revealing a highly structured light cone confined by the strictly logarithmic upper and lower bounds representing the slowest and fastest scrambling available in this system. We explain these bounds by deriving the closed-form expression of the effective interaction for the slowest scrambling and by constructing the effective model of a half length for the fastest scrambling. We extend our lowest-order perturbation formulations to the higher dimensions, proposing that the logarithmic upper and lower light cones may persist in a finite two-dimensional system in the limit of strong disorder and weak hopping.

We study the logarithmic correction to the scaling of the first Lee-Yang (LY) zero in the classical $XY$ model on square lattices by using tensor renormalization group methods. In comparing the higher-order tensor renormalization group (HOTRG) and the loop-optimized tensor network renormalization (LoopTNR), we find that the entanglement filtering in LoopTNR is crucial to gaining high accuracy for the characterization of the logarithmic correction, while HOTRG still proposes approximate bounds for the zero location associated with two different bond-merging algorithms of the higher-order singular value decomposition and the oblique projectors. Using the LoopTNR data computed up to the system size of $L=1024$ in the $L \times L$ lattices, we estimate the logarithmic correction exponent $r = -0.0643(9)$ from the extrapolation of the finite-size effective exponent, which is comparable to the renormalization group prediction of $r = -1/16$.

We investigate the connection between the supervised learning of the binary phase classification in the ferromagnetic Ising model and the standard finite-size-scaling theory of the second-order phase transition. Proposing a minimal one-free-parameter neural network model, we analytically formulate the supervised learning problem for the canonical ensemble being used as a training data set. We show that just one free parameter is capable enough to describe the data-driven emergence of the universal finite-size-scaling function in the network output that is observed in a large neural network, theoretically validating its critical point prediction for unseen test data from different underlying lattices yet in the same universality class of the Ising criticality. We also numerically demonstrate the interpretation with the proposed one-parameter model by providing an example of finding a critical point with the learning of the Landau mean-field free energy being applied to the real data set from the uncorrelated random scale-free graph with a large degree exponent.

In this paper, we propose a confidence-calibration method for predicting the winner of a famous multiplayer online battle arena (MOBA) game, League of Legends. In MOBA games, the dataset may contain a large amount of input-dependent noise; not all of such noise is observable. Hence, it is desirable to attempt a confidence-calibrated prediction. Unfortunately, most existing confidence calibration methods are pertaining to image and document classification tasks where consideration on uncertainty is not crucial. In this paper, we propose a novel calibration method that takes data uncertainty into consideration. The proposed method achieves an outstanding expected calibration error (ECE) (0.57%) mainly owing to data uncertainty consideration, compared to a conventional temperature scaling method of which ECE value is 1.11%., Comment: Submitted to IEEE Conference on Games(CoG) 2020

We investigate the finite-size-scaling (FSS) behavior of the leading Fisher zero of the partition function in the complex temperature plane in the $p$-state clock models of $p=5$ and $6$. We derive the logarithmic finite-size corrections to the scaling of the leading zeros which we numerically verify by performing the higher-order tensor renormalization group (HOTRG) calculations in the square lattices of a size up to $128 \times 128$ sites. The necessity of the deterministic HOTRG method in the clock models is noted by the extreme vulnerability of the numerical leading zero identification against stochastic noises that are hard to be avoided in the Monte-Carlo approaches. We characterize the system-size dependence of the numerical vulnerability of the zero identification by the type of phase transition, suggesting that the two transitions in the clock models are not of an ordinary first- or second-order type. In the direct FSS analysis of the leading zeros in the clock models, we find that their FSS behaviors show excellent agreement with our predictions of the logarithmic corrections to the Berezinskii-Kosterlitz-Thouless ansatz at both of the high- and low-temperature transitions.

Learning with an artificial neural network encodes the system behavior in a feed-forward function with a number of parameters optimized by data-driven training. An open question is whether one can minimize the network complexity without loss of performance to reveal how and why it works. Here we investigate the learning of the phase transition in the Ising model and find that having two hidden neurons can be enough for an accurate prediction of critical temperature. We show that the networks learn the scaling dimension of the order parameter while being trained as a phase classifier, demonstrating how the machine learning exploits the Ising universality to work for different lattices of the same criticality within a single set of trainings in one lattice geometry.

We investigate the first-order transition in the spin-1 two-dimensional Blume-Capel model in square lattices by revisiting the transfer-matrix method. With large strip widths increased up to the size of 18 sites, we construct the detailed phase coexistence curve which shows excellent quantitative agreement with the recent advanced Monte Carlo results. In the deep first-order area, we observe the exponential system-size scaling of the spectral gap of the transfer matrix from which linearly increasing interfacial tension is deduced with decreasing temperature. We find that the first-order signature at low temperatures is strongly pronounced with much suppressed finite-size influence in the examined thermodynamic properties of entropy, non-zero spin population, and specific heat. It turns out that the jump at the transition becomes increasingly sharp as it goes deep into the first-order area, which is in contrast to the Wang-Landau results where finite-size smoothing gets more severe at lower temperatures.

We investigate the partition function zeros of the two-dimensional $p$-state clock model in the complex temperature plane by using the Wang-Landau method. For $p=5$, $6$, $8$, and $10$, we propose a modified energy representation to enumerate exact irregular energy levels for the density of states without any binning artifacts. Comparing the leading zeros between different $p$'s, we provide strong evidence that the upper transition at $p=6$ is indeed of the Berezinskii-Kosterlitz-Thouless (BKT) type in contrast to the claim of the previous Fisher zero study [Phys. Rev. E \textbf{80}, 042103 (2009)]. We find that the leading zeros of $p=6$ at the upper transition collapse onto the zero trajectories of the larger $p$'s including the $XY$ limit while the finite-size behavior of $p=5$ differs from the converged behavior of $p \ge 6$ within the system sizes examined. In addition, we argue that the nondivergent specific heat in the BKT transition is responsible for the small partition function magnitude that decreases exponentially with increasing system size near the leading zero, fundamentally limiting access to large systems in search for zeros with an estimator under finite statistical fluctuations.

We investigate the Gorkov--Melik-Barkhudarov (GM) correction to superfluid transition temperature in two-dimensional Fermi gases with Rashba spin-orbit coupling (SOC) across the SOC-driven BCS-BEC crossover. In the calculation of the induced interaction, we find that the spin-component mixing due to SOC can induce both of the conventional screening and additional antiscreening contributions that interplay significantly in the strong SOC regime. While the GM correction generally lowers the estimate of transition temperature, it turns out that at a fixed weak interaction, the correction effect exhibits a crossover behavior where the ratio between the estimates without and with the correction first decreases with SOC and then becomes insensitive to SOC when it goes into the strong SOC regime. We demonstrate the applicability of the GM correction by comparing the zero-temperature condensate fraction with the recent quantum Monte Carlo results.