Search

We introduce a new framework, dubbed Cerberus, for attribute-based person re-identification (reID). Our approach leverages person attribute labels to learn local and global person representations that encode specific traits, such as gender and clothing style. To achieve this, we define semantic IDs (SIDs) by combining attribute labels, and use a semantic guidance loss to align the person representations with the prototypical features of corresponding SIDs, encouraging the representations to encode the relevant semantics. Simultaneously, we enforce the representations of the same person to be embedded closely, enabling recognizing subtle differences in appearance to discriminate persons sharing the same attribute labels. To increase the generalization ability on unseen data, we also propose a regularization method that takes advantage of the relationships between SID prototypes. Our framework performs individual comparisons of local and global person representations between query and gallery images for attribute-based reID. By exploiting the SID prototypes aligned with the corresponding representations, it can also perform person attribute recognition (PAR) and attribute-based person search (APS) without bells and whistles. Experimental results on standard benchmarks on attribute-based person reID, Market-1501 and DukeMTMC, demonstrate the superiority of our model compared to the state of the art.

Many cellular processes involve information processing and decision making. We can probe these processes at increasing molecular detail. The analysis of heterogeneous data remains a challenge that requires new ways of thinking about cells in quantitative, predictive, and mechanistic ways. We discuss the role of mathematical models in the context of cell-fate decision making systems across the tree of life. Complex multi-cellular organisms have been a particular focus, but single celled organisms also have to sense and respond to their environment. We center our discussion around the idea of design principles which we can learn from observations and modeling, and exploit in order to (re)-design or guide cellular behavior., Comment: 23 pages, 3 figures

Despite the growing interest in Mamba architecture as a potential replacement for Transformer architecture, parameter-efficient fine-tuning (PEFT) approaches for Mamba remain largely unexplored. In our study, we introduce two key insights-driven strategies for PEFT in Mamba architecture: (1) While state-space models (SSMs) have been regarded as the cornerstone of Mamba architecture, then expected to play a primary role in transfer learning, our findings reveal that Projectors -- not SSMs -- are the predominant contributors to transfer learning, and (2) Based on our observation that adapting pretrained Projectors to new tasks can be effectively approximated through a near-diagonal linear transformation, we propose a novel PEFT method specialized to Mamba architecture: Projector-targeted Diagonal-centric Linear Transformation (ProDiaL). ProDiaL focuses on optimizing only diagonal-centric linear transformation matrices, without directly fine-tuning the pretrained Projector weights. This targeted approach allows efficient task adaptation, utilizing less than 1% of the total parameters, and exhibits strong performance across both vision and language Mamba models, highlighting its versatility and effectiveness.

Soft metals like lithium and sodium play a critical role in battery technology owing to their high energy density. Texture formation by grain selection growth of soft metals during electrochemical processes is a crucial factor affecting power and safety. Developing a framework to understand and control grain growth is a multifaceted challenge. Here, a general thermodynamic theory and phase-field model are formulated to study grain selection growth of soft metals. Our study focuses on the interplay between surface energy and atomic mobility-related intrinsic strain energy in grain selection growth. Differences in grain selection growth arise from the anisotropy in surface energy and diffusion barrier of soft metal atoms. Our findings highlight the kinetic limitations of solid-state Li metal batteries, which originate from load stress-induced surface energy anisotropy. These insights lead to the development of an amorphous LixSi1-x (0.50

Zero-shot anomaly detection (ZSAD) is crucial for detecting anomalous patterns in target datasets without using training samples, specifically in scenarios where there are distributional differences between the target domain and training data or where data scarcity arises because of restricted access. Although recently pretrained vision-language models demonstrate strong zero-shot performance across various visual tasks, they focus on learning class semantics, which makes their direct application to ZSAD challenging. To address this scenario, we propose GlocalCLIP, which uniquely separates global and local prompts and jointly optimizes them. This approach enables the object-agnostic glocal semantic prompt to effectively capture general normal and anomalous patterns without dependency on specific objects in the image. We refine the text prompts for more precise adjustments by utilizing deep-text prompt tuning in the text encoder. In the vision encoder, we apply V-V attention layers to capture detailed local image features. Finally, we introduce glocal contrastive learning to improve the complementary learning of global and local prompts, effectively detecting anomalous patterns across various domains. The generalization performance of GlocalCLIP in ZSAD was demonstrated on 15 real-world datasets from both the industrial and medical domains, achieving superior performance compared to existing methods. Code will be made available at https://github.com/YUL-git/GlocalCLIP., Comment: 29 pages, 36 figures

We propose new methods for multivariate linear regression when the regression coefficient matrix is sparse and the error covariance matrix is dense. We assume that the error covariance matrix has equicorrelation across the response variables. Two procedures are proposed: one is based on constant marginal response variance (compound symmetry), and the other is based on general varying marginal response variance. Two approximate procedures are also developed for high dimensions. We propose an approximation to the Gaussian validation likelihood for tuning parameter selection. Extensive numerical experiments illustrate when our procedures outperform relevant competitors as well as their robustness to model misspecification.

Deep Generative Models (DGMs) have rapidly advanced in recent years, becoming essential tools in various fields due to their ability to learn complex data distributions and generate synthetic data. Their importance in transportation research is increasingly recognized, particularly for applications like traffic data generation, prediction, and feature extraction. This paper offers a comprehensive introduction and tutorial on DGMs, with a focus on their applications in transportation. It begins with an overview of generative models, followed by detailed explanations of fundamental models, a systematic review of the literature, and practical tutorial code to aid implementation. The paper also discusses current challenges and opportunities, highlighting how these models can be effectively utilized and further developed in transportation research. This paper serves as a valuable reference, guiding researchers and practitioners from foundational knowledge to advanced applications of DGMs in transportation research., Comment: 64 pages, 21 figures, 4 tables

Quantum entanglement between paired photons is the foundation of optical quantum computing, quantum sensing, and quantum networks. Traditionally, quantum information science has focused on the particle nature of photons at the microscopic scale, often neglecting the phase information of single photons, even for the bipartite quantum entanglement. Recently, a coherence-based approach has been explored to understand the so-called quantum mystery of nonlocal intensity fringes emerging from local randomness. Here, a pure coherence method is presented to create macroscopic quantum features using conventional laser light via linear optics-based measurement modifications. To achieve this, a polarization-basis control of the laser light is conducted to generate indistinguishable characteristics between orthogonally polarized light pairs. Using projection measurements of the polarization-controlled light pairs, we derive coherence solutions of local randomness and nonlocal correlations between independently controlled local parameters, where a fixed relative phase relationship between paired lights is an essential condition to determine the corresponding Bell states., Comment: 6 pages, 1 figure

According to the WHO, in 2021, about 32% of pregnant women in sub-Saharan Africa were infected with malaria during pregnancy. Malaria infection during pregnancy can cause various adverse birth outcomes such as low birthweight. Over the past two decades, while some sub-Saharan African areas have experienced a large reduction in malaria prevalence due to improved malaria control and treatments, others have observed little change. Individual-level interventional studies have shown that preventing malaria infection during pregnancy can improve birth outcomes such as birthweight; however, it is still unclear whether natural reductions in malaria prevalence may help improve community-level birth outcomes. We conduct an observational study using 203,141 children's records in 18 sub-Saharan African countries from 2000 to 2018. Using heterogeneity of changes in malaria prevalence, we propose and apply a novel pair-of-pairs design via two-stage bipartite and non-bipartite matching to conduct a difference-in-differences study with a continuous measure of malaria prevalence, namely the Plasmodium falciparum parasite rate among children aged 2 to 10 ($\text{PfPR}_{2-10}$). The proposed novel statistical methodology allows us to apply difference-in-differences without dichotomizing $\text{PfPR}_{2-10}$, which can substantially increase the effective sample size, improve covariate balance, and facilitate the dose-response relationship during analysis. Our outcome analysis finds that among the pairs of clusters we study, the largest reduction in $\text{PfPR}_{2-10}$ over early and late years is estimated to increase the average birthweight by 98.899 grams (95% CI: $[39.002, 158.796]$), which is associated with reduced risks of several adverse birth or life-course outcomes. The proposed novel statistical methodology can be replicated in many other disease areas.

Polarization-entangled photon pairs generated from second-order nonlinear optical media have been extensively studied for both fundamental research and potential applications of quantum information. In spontaneous parametric down-conversion (SPDC), quantum entanglement between paired photons, often regarded as mysterious, has been demonstrated for local randomness and nonlocal correlation through polarization-basis projections using linear optics (Phys. Rev. A 60, R773 (1999)). This paper presents a coherence analysis of these established quantum phenomena with polarization control of the paired photons and their projection measurements. First, we analyze the quantum superposition of photon pairs generated randomly from cross-sandwiched nonlinear media, focusing on local randomness, which depends on the incoherence among measured events. Second, we investigate coincidence detection between paired photons to understand the nonlocal correlation arising from independently controlled remote parameters, resulting in an inseparable product-basis relationship. This coherence-based approach sheds light on a deterministic perspective on quantum features, emphasizing the significance of phase information intrinsic to the wave nature of photons., Comment: 7 pages, 2 figures

Understanding how biological constraints shape neural computation is a central goal of computational neuroscience. Spatially embedded recurrent neural networks provide a promising avenue to study how modelled constraints shape the combined structural and functional organisation of networks over learning. Prior work has shown that spatially embedded systems like this can combine structure and function into single artificial models during learning. But it remains unclear precisely how, in general, structural constraints bound the range of attainable configurations. In this work, we show that it is possible to study these restrictions through entropic measures of the neural weights and eigenspectrum, across both rate and spiking neural networks. Spatial embedding, in contrast to baseline models, leads to networks with a highly specific low entropy modularity where connectivity is readily interpretable given the known spatial and communication constraints acting on them. Crucially, these networks also demonstrate systematically modulated spectral dynamics, revealing how they exploit heterogeneity in their function to overcome the constraints imposed on their structure. This work deepens our understanding of constrained learning in neural networks, across coding schemes and tasks, where solutions to simultaneous structural and functional objectives must be accomplished in tandem., Comment: 19 pages, 5 figures

The combination of machine learning and physical laws has shown immense potential for solving scientific problems driven by partial differential equations (PDEs) with the promise of fast inference, zero-shot generalisation, and the ability to discover new physics. Examples include the use of fundamental physical laws as inductive bias to machine learning algorithms, also referred to as physics-driven machine learning, and the application of machine learning to represent features not represented in the differential equations such as closures for unresolved spatiotemporal scales. However, the simulation of complex physical systems by coupling advanced numerics for PDEs with state-of-the-art machine learning demands the composition of specialist PDE solving frameworks with industry-standard machine learning tools. Hand-rolling either the PDE solver or the neural net will not cut it. In this work, we introduce a generic differentiable programming abstraction that provides scientists and engineers with a highly productive way of specifying end-to-end differentiable models coupling machine learning and PDE-based components, while relying on code generation for high performance. Our interface automates the coupling of arbitrary PDE-based systems and machine learning models and unlocks new applications that could not hitherto be tackled, while only requiring trivial changes to existing code. Our framework has been adopted in the Firedrake finite-element library and supports the PyTorch and JAX ecosystems, as well as downstream libraries.

We address the problem of person re-identification (reID), that is, retrieving person images from a large dataset, given a query image of the person of interest. A key challenge is to learn person representations robust to intra-class variations, as different persons could have the same attribute, and persons' appearances look different, e.g., with viewpoint changes. Recent reID methods focus on learning person features discriminative only for a particular factor of variations (e.g., human pose), which also requires corresponding supervisory signals (e.g., pose annotations). To tackle this problem, we propose to factorize person images into identity-related and unrelated features. Identity-related features contain information useful for specifying a particular person (e.g., clothing), while identity-unrelated ones hold other factors (e.g., human pose). To this end, we propose a new generative adversarial network, dubbed identity shuffle GAN (IS-GAN). It disentangles identity-related and unrelated features from person images through an identity-shuffling technique that exploits identification labels alone without any auxiliary supervisory signals. We restrict the distribution of identity-unrelated features or encourage the identity-related and unrelated features to be uncorrelated, facilitating the disentanglement process. Experimental results validate the effectiveness of IS-GAN, showing state-of-the-art performance on standard reID benchmarks, including Market-1501, CUHK03, and DukeMTMC-reID. We further demonstrate the advantages of disentangling person representations on a long-term reID task, setting a new state of the art on a Celeb-reID dataset., Comment: arXiv admin note: substantial text overlap with arXiv:1910.12003

Precision measurement has been an important research area in sensing and metrology. In classical physics, the Fisher information determines the maximum extractable information from statistically unknown signals, based on a joint probability density function of independently and identically distributed random variables. The Cramer-Rao lower bound (CRLB) indicates the minimum error of the Fisher information, generally known as the shot-noise limit. On the other hand, coherence has pushed the resolution limit further overcoming the diffraction limit using many-wave interference strictly confined to the first-order intensity correlation. However, practical implementation is limited by the lithographic constraints in, e.g., optical gratings. Recently, a coherence technique of superresolution has been introduced to overcome the diffraction limit in phase sensitivity using higher-order intensity correlations of a phase-controlled output field from an interferometer. Here, the superresolution is adopted for precision metrology in an optical spectrometer, whose enhanced frequency resolution is linearly proportional to the intensity-product order, overcoming CRLB. Unlike quantum sensing using entangled photons, this technique is purely classical and offers robust performance against environmental noises, benefiting from the interferometer scanning mode for fringe counting., Comment: 9 pages, 3 figures

In this work, we introduce the demonstration of symbolic music generation, focusing on providing short musical motifs that serve as the central theme of the narrative. For the generation, we adopt an autoregressive model which takes musical metadata as inputs and generates 4 bars of multitrack MIDI sequences. During training, we randomly drop tokens from the musical metadata to guarantee flexible control. It provides users with the freedom to select input types while maintaining generative performance, enabling greater flexibility in music composition. We validate the effectiveness of the strategy through experiments in terms of model capacity, musical fidelity, diversity, and controllability. Additionally, we scale up the model and compare it with other music generation model through a subjective test. Our results indicate its superiority in both control and music quality. We provide a URL link https://www.youtube.com/watch?v=-0drPrFJdMQ to our demonstration video.

Network quantization generally converts full-precision weights and/or activations into low-bit fixed-point values in order to accelerate an inference process. Recent approaches to network quantization further discretize the gradients into low-bit fixed-point values, enabling an efficient training. They typically set a quantization interval using a min-max range of the gradients or adjust the interval such that the quantization error for entire gradients is minimized. In this paper, we analyze the quantization error of gradients for the low-bit fixed-point training, and show that lowering the error for large-magnitude gradients boosts the quantization performance significantly. Based on this, we derive an upper bound of quantization error for the large gradients in terms of the quantization interval, and obtain an optimal condition for the interval minimizing the quantization error for large gradients. We also introduce an interval update algorithm that adjusts the quantization interval adaptively to maintain a small quantization error for large gradients. Experimental results demonstrate the effectiveness of our quantization method for various combinations of network architectures and bit-widths on various tasks, including image classification, object detection, and super-resolution., Comment: Accepted to ECCV 2024

Dataset distillation synthesizes a small set of images from a large-scale real dataset such that synthetic and real images share similar behavioral properties (e.g, distributions of gradients or features) during a training process. Through extensive analyses on current methods and real datasets, together with empirical observations, we provide in this paper two important things to share for dataset distillation. First, object parts that appear on one side of a real image are highly likely to appear on the opposite side of another image within a dataset, which we call the bilateral equivalence. Second, the bilateral equivalence enforces synthetic images to duplicate discriminative parts of objects on both the left and right sides of the images, limiting the recognition of subtle differences between objects. To address this problem, we introduce a surprisingly simple yet effective technique for dataset distillation, dubbed FYI, that enables distilling rich semantics of real images into synthetic ones. To this end, FYI embeds a horizontal flipping technique into distillation processes, mitigating the influence of the bilateral equivalence, while capturing more details of objects. Experiments on CIFAR-10/100, Tiny-ImageNet, and ImageNet demonstrate that FYI can be seamlessly integrated into several state-of-the-art methods, without modifying training objectives and network architectures, and it improves the performance remarkably., Comment: Accepted to ECCV 2024

3D building reconstruction from imaging data is an important task for many applications ranging from urban planning to reconnaissance. Modern Novel View synthesis (NVS) methods like NeRF and Gaussian Splatting offer powerful techniques for developing 3D models from natural 2D imagery in an unsupervised fashion. These algorithms generally require input training views surrounding the scene of interest, which, in the case of large buildings, is typically not available across all camera elevations. In particular, the most readily available camera viewpoints at scale across most buildings are at near-ground (e.g., with mobile phones) and aerial (drones) elevations. However, due to the significant difference in viewpoint between drone and ground image sets, camera registration - a necessary step for NVS algorithms - fails. In this work we propose a method, DRAGON, that can take drone and ground building imagery as input and produce a 3D NVS model. The key insight of DRAGON is that intermediate elevation imagery may be extrapolated by an NVS algorithm itself in an iterative procedure with perceptual regularization, thereby bridging the visual feature gap between the two elevations and enabling registration. We compiled a semi-synthetic dataset of 9 large building scenes using Google Earth Studio, and quantitatively and qualitatively demonstrate that DRAGON can generate compelling renderings on this dataset compared to baseline strategies., Comment: 12 pages, 9 figures, accepted to ICCP 2024

We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in developing their sovereign LLMs., Comment: 44 pages; updated authors list and fixed author names

The consequence of a change in school leadership (e.g., principal turnover) on student achievement has important implications for education policy. The impact of such an event can be estimated via the popular Difference in Difference (DiD) estimator, where those schools with a turnover event are compared to a selected set of schools that did not have such an event. The strength of this comparison depends on the plausibility of the "parallel trends" assumption that the "treated group" of those schools which had leadership turnover, absent such turnover, would have changed "similarly" to those which did not. To bolster such a claim, one might generate a comparison group, via matching, that is similar to the treated group with respect to pre-treatment outcomes and/or pre-treatment covariates. Unfortunately, as has been previously pointed out, this intuitively appealing approach also has a cost in terms of bias. To assess the tradeoffs of matching in our application, we first characterize the bias of matching prior to a DiD analysis under a linear structural model that allows for time-invariant observed and unobserved confounders with time-varying effects on the outcome. Given our framework, we verify that matching on baseline covariates generally reduces bias. We further show how additionally matching on pre-treatment outcomes has both cost and benefit. First, matching on pre-treatment outcomes partially balances unobserved confounders, which mitigates some bias. This reduction is proportional to the outcome's reliability, a measure of how coupled the outcomes are with the latent covariates. Offsetting these gains, matching also injects bias into the final estimate by undermining the second difference in the DiD via a regression-to-the-mean effect. Consequently, we provide heuristic guidelines for determining to what degree the bias reduction of matching is likely to outweigh the bias cost. We illustrate our guidelines by reanalyzing a principal turnover study that used matching prior to a DiD analysis and find that matching on both the pre-treatment outcomes and observed covariates makes the estimated treatment effect more credible. [This paper was published in "The Annals of Applied Statistics" v18 n3 2024.]

As DNNs are widely adopted in various application domains while demanding increasingly higher compute and memory requirements, designing efficient and performant NPUs (Neural Processing Units) is becoming more important. However, existing architectural NPU simulators lack support for high-speed simulation, multi-core modeling, multi-tenant scenarios, detailed DRAM/NoC modeling, and/or different deep learning frameworks. To address these limitations, this work proposes ONNXim, a fast cycle-level simulator for multi-core NPUs in DNN serving systems. It takes DNN models represented in the ONNX graph format generated from various deep learning frameworks for ease of simulation. In addition, based on the observation that typical NPU cores process tensor tiles from on-chip scratchpad memory with deterministic compute latency, we forgo a detailed modeling for the computation while still preserving simulation accuracy. ONNXim also preserves dependencies between compute and tile DMAs. Meanwhile, the DRAM and NoC are modeled in cycle-level to properly model contention among multiple cores that can execute different DNN models for multi-tenancy. Consequently, ONNXim is significantly faster than existing simulators (e.g., by up to 384x over Accel-sim) and enables various case studies, such as multi-tenant NPUs, that were previously impractical due to slow speed and/or lack of functionalities. ONNXim is publicly available at https://github.com/PSAL-POSTECH/ONNXim.

We present Diffusion Model Patching (DMP), a simple method to boost the performance of pre-trained diffusion models that have already reached convergence, with a negligible increase in parameters. DMP inserts a small, learnable set of prompts into the model's input space while keeping the original model frozen. The effectiveness of DMP is not merely due to the addition of parameters but stems from its dynamic gating mechanism, which selects and combines a subset of learnable prompts at every step of the generative process (e.g., reverse denoising steps). This strategy, which we term "mixture-of-prompts", enables the model to draw on the distinct expertise of each prompt, essentially "patching" the model's functionality at every step with minimal yet specialized parameters. Uniquely, DMP enhances the model by further training on the same dataset on which it was originally trained, even in a scenario where significant improvements are typically not expected due to model convergence. Experiments show that DMP significantly enhances the converged FID of DiT-L/2 on FFHQ 256x256 by 10.38%, achieved with only a 1.43% parameter increase and 50K additional training iterations., Comment: Project page: https://sangminwoo.github.io/DMP/

We present personalized residuals and localized attention-guided sampling for efficient concept-driven generation using text-to-image diffusion models. Our method first represents concepts by freezing the weights of a pretrained text-conditioned diffusion model and learning low-rank residuals for a small subset of the model's layers. The residual-based approach then directly enables application of our proposed sampling technique, which applies the learned residuals only in areas where the concept is localized via cross-attention and applies the original diffusion weights in all other regions. Localized sampling therefore combines the learned identity of the concept with the existing generative prior of the underlying diffusion model. We show that personalized residuals effectively capture the identity of a concept in ~3 minutes on a single GPU without the use of regularization images and with fewer parameters than previous models, and localized sampling allows using the original model as strong prior for large parts of the image., Comment: CVPR 2024. Project page at https://cusuh.github.io/personalized-residuals

Coherence spectroscopy has been intensively studied over the last several decades for various applications in science and engineering. The Rayleigh criterion defines the resolution limit of an interferometer, where many-wave interference beats the resolution limit of a two-slit system. On the other hand, the diffraction angle in a slit is reduced by the Kth power of the measured signal, resulting in the shot-noise limit. Here, the Kth power of the measured signal in an N-slit interferometer is studied for enhanced coherence spectroscopy to overcome the resolution limit of the original system. The Kth power to the individual intensities of the N-slit interferometer is numerically demonstrated for enhanced resolution satisfying the shot-noise limit. As a result, the Kth power of the intensity beats the resolution limit of the N-slit interferometer, in which the out-of-shelf spectrometer or wavelength meter can be a primary beneficiary of this technique. Due to the same resolution of the Heisenberg limit in quantum sensing as in the N-slit interference fringe, the proposed Kth power technique also beats the superresolution in quantum metrology., Comment: 7 pages, 5 figures

Superresolution is a unique quantum feature generated by N00N states or phase-controlled coherent photons via projection measurements in a Mach-Zehnder interferometer (MZI). Superresolution has no direct relation with supersensitivity in quantum sensing and has a potential application for the precision measurement of an unknown signal frequency. Recently, phase-controlled quantum erasers have been demonstrated for superresolution using classical light of a continuous-wave laser to overcome the diffraction limit in classical physics and to solve the limited scalability in N00N state-based quantum sensing. Here, a quantum spectrometer is presented for the macroscopic superresolution using phase-controlled spatial light modulators (SLMs) in MZI. For validity, a general solution of the superresolution is derived from the SLM-based projection measurements and an unprecedented resolution is numerically confirmed for an unknown frequency of light., Comment: 9 pages, 4 figures

Recently, many studies have shown the efficiency of using Bidirectional Encoder Representations from Transformers (BERT) in various Natural Language Processing (NLP) tasks. Specifically, English spelling correction task that uses Encoder-Decoder architecture and takes advantage of BERT has achieved state-of-the-art result. However, to our knowledge, there is no implementation in Vietnamese yet. Therefore, in this study, a combination of Transformer architecture (state-of-the-art for Encoder-Decoder model) and BERT was proposed to deal with Vietnamese spelling correction. The experiment results have shown that our model outperforms other approaches as well as the Google Docs Spell Checking tool, achieves an 86.24 BLEU score on this task., Comment: 13 pages

Emerging Compute Express Link (CXL) enables cost-efficient memory expansion beyond the local DRAM of processors. While its CXL$.$mem protocol provides minimal latency overhead through an optimized protocol stack, frequent CXL memory accesses can result in significant slowdowns for memory-bound applications whether they are latency-sensitive or bandwidth-intensive. The near-data processing (NDP) in the CXL controller promises to overcome such limitations of passive CXL memory. However, prior work on NDP in CXL memory proposes application-specific units that are not suitable for practical CXL memory-based systems that should support various applications. On the other hand, existing CPU or GPU cores are not cost-effective for NDP because they are not optimized for memory-bound applications. In addition, the communication between the host processor and CXL controller for NDP offloading should achieve low latency, but existing CXL$.$io/PCIe-based mechanisms incur $\mu$s-scale latency and are not suitable for fine-grained NDP. To achieve high-performance NDP end-to-end, we propose a low-overhead general-purpose NDP architecture for CXL memory referred to as Memory-Mapped NDP (M$^2$NDP), which comprises memory-mapped functions (M$^2$func) and memory-mapped $\mu$threading (M$^2\mu$thread). M$^2$func is a CXL$.$mem-compatible low-overhead communication mechanism between the host processor and NDP controller in CXL memory. M$^2\mu$thread enables low-cost, general-purpose NDP unit design by introducing lightweight $\mu$threads that support highly concurrent execution of kernels with minimal resource wastage. Combining them, M$^2$NDP achieves significant speedups for various workloads by up to 128x (14.5x overall) and reduces energy by up to 87.9% (80.3% overall) compared to baseline CPU/GPU hosts with passive CXL memory., Comment: Accepted at the 57th IEEE/ACM International Symposium on Microarchitecture (MICRO), 2024

Quantization-aware training (QAT) simulates a quantization process during training to lower bit-precision of weights/activations. It learns quantized weights indirectly by updating latent weights, i.e., full-precision inputs to a quantizer, using gradient-based optimizers. We claim that coupling a user-defined learning rate (LR) with these optimizers is sub-optimal for QAT. Quantized weights transit discrete levels of a quantizer, only if corresponding latent weights pass transition points, where the quantizer changes discrete states. This suggests that the changes of quantized weights are affected by both the LR for latent weights and their distributions. It is thus difficult to control the degree of changes for quantized weights by scheduling the LR manually. We conjecture that the degree of parameter changes in QAT is related to the number of quantized weights transiting discrete levels. Based on this, we introduce a transition rate (TR) scheduling technique that controls the number of transitions of quantized weights explicitly. Instead of scheduling a LR for latent weights, we schedule a target TR of quantized weights, and update the latent weights with a novel transition-adaptive LR (TALR), enabling considering the degree of changes for the quantized weights during QAT. Experimental results demonstrate the effectiveness of our approach on standard benchmarks.

X-ray grating interferometry allows for the simultaneous acquisition of attenuation, differential-phase contrast, and dark-field images, resulting from X-ray attenuation, refraction, and small-angle scattering, respectively. The modulated phase grating (MPG) interferometer is a recently developed grating interferometry system capable of generating a directly resolvable interference pattern using a relatively large period grating envelope function that is sampled at a pitch that allows for X-ray spatial coherence using a microfocus X-ray source or by use of a source G0 grating that follows the Lau condition. We present the theory of the MPG interferometry system for a 2-dimensional staggered grating, derived using Fourier optics, and we compare the theoretical predictions with experiments we have performed with a microfocus X-ray system at Pennington Biomedical Research Center, LSU. The theoretical and experimental fringe visibility is evaluated as a function of grating-to-detector distance. Quantitative experiments are performed with porous carbon and alumina samples, and qualitative analysis of attenuation and dark-field images of a dried anchovy are shown., Comment: 22 pages, 13 figures. This is a pre-print

We investigate the estimation of treatment effects from a sample that is stratified on the binary treatment status. In the case of unconfounded assignment where the potential outcomes are independent of the treatment given covariates, we show that standard estimators of the average treatment effect are inconsistent. In the case of an endogenous treatment and a binary instrument, we show that the IV estimator is inconsistent for the local average treatment effect. In both cases, we propose simple alternative estimators that are consistent in stratified samples, assuming that the fraction treated in the population is known or can be estimated.

Post-training quantization (PTQ) is an efficient model compression technique that quantizes a pretrained full-precision model using only a small calibration set of unlabeled samples without retraining. PTQ methods for convolutional neural networks (CNNs) provide quantization results comparable to full-precision counterparts. Directly applying them to vision transformers (ViTs), however, incurs severe performance degradation, mainly due to the differences in architectures between CNNs and ViTs. In particular, the distribution of activations for each channel vary drastically according to input instances, making PTQ methods for CNNs inappropriate for ViTs. To address this, we introduce instance-aware group quantization for ViTs (IGQ-ViT). To this end, we propose to split the channels of activation maps into multiple groups dynamically for each input instance, such that activations within each group share similar statistical properties. We also extend our scheme to quantize softmax attentions across tokens. In addition, the number of groups for each layer is adjusted to minimize the discrepancies between predictions from quantized and full-precision models, under a bit-operation (BOP) constraint. We show extensive experimental results on image classification, object detection, and instance segmentation, with various transformer architectures, demonstrating the effectiveness of our approach., Comment: CVPR 2024