Search

We demonstrate theoretically and experimentally how the diffraction and interferometric resolution limit for single-mode coherent cw laser light can be overcome by multi-photon interference. By use of a Mach-Zehnder interferometer, operated in the single input and single or double output port geometries, we observe a fringe width reduction of the conventional interference pattern, predicted by the wave or single photon quantum theory, by a factor of up to $1/\sqrt{2N}$ through coincident detection of $N=2,3,4$ photons. Our scheme does not require squeezed or entangled light to overcome the standard quantum limit and greatly facilitates precision interferometry experiments.

Training-free network architecture search (NAS) aims to discover high-performing networks with zero-cost proxies, capturing network characteristics related to the final performance. However, network rankings estimated by previous training-free NAS methods have shown weak correlations with the performance. To address this issue, we propose AZ-NAS, a novel approach that leverages the ensemble of various zero-cost proxies to enhance the correlation between a predicted ranking of networks and the ground truth substantially in terms of the performance. To achieve this, we introduce four novel zero-cost proxies that are complementary to each other, analyzing distinct traits of architectures in the views of expressivity, progressivity, trainability, and complexity. The proxy scores can be obtained simultaneously within a single forward and backward pass, making an overall NAS process highly efficient. In order to integrate the rankings predicted by our proxies effectively, we introduce a non-linear ranking aggregation method that highlights the networks highly-ranked consistently across all the proxies. Experimental results conclusively demonstrate the efficacy and efficiency of AZ-NAS, outperforming state-of-the-art methods on standard benchmarks, all while maintaining a reasonable runtime cost., Comment: CVPR 2024

We consider the problem of causal inference based on observational data (or the related missing data problem) with a binary or discrete treatment variable. In that context we study counterfactual density estimation, which provides more nuanced information than counterfactual mean estimation (i.e., the average treatment effect). We impose the shape-constraint of log-concavity (a unimodality constraint) on the counterfactual densities, and then develop doubly robust estimators of the log-concave counterfactual density (based on an augmented inverse-probability weighted pseudo-outcome), and show the consistency in various global metrics of that estimator. Based on that estimator we also develop asymptotically valid pointwise confidence intervals for the counterfactual density.

The classically defined minimum uncertainty of the optical phase is known as the standard quantum limit or shot-noise limit (SNL) originating in the uncertainty principle of quantum mechanics. Based on SNL, the phase sensitivity is inversely proportional to the square root K, where K is the number of interfering photons or statistically measured events. Thus, using a high-power laser is advantageous to enhance sensitivity due to the square root K gain in the signal-to-noise ratio. In a typical interferometer, however, the resolution remains in the diffraction limit of the K=1 case unless the interfering photons are resolved as in quantum sensing. Here, a projection-measurement method in quantum sensing is adapted for an interferometer to achieve an additional square root K gain in resolution. For the projection measurement, the interference fringe of an interferometer can be Kth-powered to replace the Kth-order intensity product. To understand many-wave interference-caused enhanced resolution, several types of interferometers are numerically compared to draw corresponding resolution parameters. As a result, the achieved resolution by the Kth power to an N-slit interferometer exceeds the diffraction limit and the Heisenberg limit in quantum sensing., Comment: 9 pages, 7 figures

Diffusion models have achieved remarkable success across a range of generative tasks. Recent efforts to enhance diffusion model architectures have reimagined them as a form of multi-task learning, where each task corresponds to a denoising task at a specific noise level. While these efforts have focused on parameter isolation and task routing, they fall short of capturing detailed inter-task relationships and risk losing semantic information, respectively. In response, we introduce Switch Diffusion Transformer (Switch-DiT), which establishes inter-task relationships between conflicting tasks without compromising semantic information. To achieve this, we employ a sparse mixture-of-experts within each transformer block to utilize semantic information and facilitate handling conflicts in tasks through parameter isolation. Additionally, we propose a diffusion prior loss, encouraging similar tasks to share their denoising paths while isolating conflicting ones. Through these, each transformer block contains a shared expert across all tasks, where the common and task-specific denoising paths enable the diffusion model to construct its beneficial way of synergizing denoising tasks. Extensive experiments validate the effectiveness of our approach in improving both image quality and convergence rate, and further analysis demonstrates that Switch-DiT constructs tailored denoising paths across various generation scenarios., Comment: Project Page: https://byeongjun-park.github.io/Switch-DiT/

Modern transformer-based Large Language Models (LLMs) are constructed with a series of decoder blocks. Each block comprises three key components: (1) QKV generation, (2) multi-head attention, and (3) feed-forward networks. In batched processing, QKV generation and feed-forward networks involve compute-intensive matrix-matrix multiplications (GEMM), while multi-head attention requires bandwidth-heavy matrix-vector multiplications (GEMV). Machine learning accelerators like TPUs or NPUs are proficient in handling GEMM but are less efficient for GEMV computations. Conversely, Processing-in-Memory (PIM) technology is tailored for efficient GEMV computation, while it lacks the computational power to handle GEMM effectively. Inspired by this insight, we propose NeuPIMs, a heterogeneous acceleration system that jointly exploits a conventional GEMM-focused NPU and GEMV-optimized PIM devices. The main challenge in efficiently integrating NPU and PIM lies in enabling concurrent operations on both platforms, each addressing a specific kernel type. First, existing PIMs typically operate in a "blocked" mode, allowing only either NPU or PIM to be active at any given time. Second, the inherent dependencies between GEMM and GEMV in LLMs restrict their parallel processing. To tackle these challenges, NeuPIMs is equipped with dual row buffers in each bank, facilitating the simultaneous management of memory read/write operations and PIM commands. Further, NeuPIMs employs a runtime sub-batch interleaving technique to maximize concurrent execution, leveraging batch parallelism to allow two independent sub-batches to be pipelined within a single NeuPIMs device. Our evaluation demonstrates that compared to GPU-only, NPU-only, and a na\"ive NPU+PIM integrated acceleration approaches, NeuPIMs achieves 3$\times$, 2.4$\times$ and 1.6$\times$ throughput improvement, respectively., Comment: 16 pages, 15 figures

Two-dimensional (2D) materials have gained increasing prominence not only in fundamental research but also in daily applications. However, to fully harness their potential, it is crucial to optimize their properties with an external parameter and track the electronic structure simultaneously. Magnetotransport over a wide magnetic field range is a powerful method to probe the electronic structure and, for metallic 2D materials, quantum oscillations superimposed on the transport signals encode Fermi surface parameters. In this manuscript, we utilize biaxial strain as an external tuning parameter and investigate the effects of strain on the electronic properties of two quasi-2D superconductors, MoTe$_2$ and RbV$_3$Sb$_5$, by measuring their magnetoresistance in pulsed magnetic fields up to 60 T. With a careful selection of insulating substrates, we demonstrate the possibility of both the compressive and tensile biaxial strain, imposed on MoTe$_2$ and RbV$_3$Sb$_5$, respectively. For both systems, the applied strain has led to superconducting critical temperature enhancement compared to their free-standing counterparts, proving the effectiveness of this biaxial strain method at cryogenic temperatures. Clear quantum oscillations in the magnetoresistance -- the Shubnikov-de Haas (SdH) effect -- are obtained in both samples. In strained MoTe$_2$, the magnetoresistance exhibits a nearly quadratic dependence on the magnetic field and remains non-saturating even at the highest field. Whereas in strained RbV$_3$Sb$_5$, two SdH frequencies showed a substantial enhancement in effective mass values, hinting at a possible enhancement of charge fluctuations. Our results demonstrate that combining biaxial strain and pulsed magnetic field paves the way for studying 2D materials under unprecedented conditions., Comment: 6 pages, 4 figures

Recently, the delayed-choice quantum eraser has been applied for coherently excited superresolution using phase-controlled projection measurements of laser light to overcome the diffraction limit in classical physics as well as to solve the limited photon number of the N00N state in quantum physics. Unlike other methods of phase-controlled superresolution in a noninterferometric system, the proposed method is for the intensity products between phase-controlled quantum erasers, resulting in superresolution compatible with the most conventional sensing metrologies. Here, a general scheme of the phase-controlled quantum eraser-based superresolution is proposed and its general solution is derived for an arbitrary Nth-order intensity correlation, where the superresolution shows the photonic de Broglie wave-like quantum feature. Furthermore, phase quantization of the superresolution is discussed to better understand quantum mechanics., Comment: 9 pages, 4 figures

In this work, we introduce a new algorithm for N-to-M checkpointing in finite element simulations. This new algorithm allows efficient saving/loading of functions representing physical quantities associated with the mesh representing the physical domain. Specifically, the algorithm allows for using different numbers of parallel processes for saving and loading, allowing for restarting and post-processing on the process count appropriate to the given phase of the simulation and other conditions. For demonstration, we implemented this algorithm in PETSc, the Portable, Extensible Toolkit for Scientific Computation, and added a convenient high-level interface into Firedrake, a system for solving partial differential equations using finite element methods. We evaluated our new implementation by saving and loading data involving 8.2 billion finite element degrees of freedom using 8,192 parallel processes on ARCHER2, the UK National Supercomputing Service., Comment: author accepted manuscript

This paper studies the effects of differential exposure to COVID-19 on educational outcomes in Guatemala. The government adopted a warning index (ranging from 0 to 10) to classify municipalities by infection rates in 2020, which was then used by the Ministry of Education in 2021 to establish a "stoplight" system for in-person instruction. Using administrative panel data for all students in Guatemala, the study employs a difference-in-differences strategy that leverages municipal differences over time in the warning index to estimate the effects of the pandemic on dropout, promotion, and school switching. The results show that municipalities with a higher warning index had significantly larger dropout, lower promotion rates, and a greater share of students switching from private to public schools. These effects were more pronounced during the first year of the pandemic. The findings show differential effects by the level of instruction, with greater losses for younger children in initial and primary education. The results are robust to specification choice, multiple hypothesis adjustments, and placebo experiments, suggesting that the pandemic has had heterogeneous consequences. [This paper is a product of the Education Global Practice.]

It is a public secret that doing email securely is fraught with challenges. We found a vulnerability present at many email providers, allowing us to spoof email on behalf of many organisations. As email vulnerabilities are ten a penny, instead of focusing on yet another email vulnerability we ask a different question: how do organisations react to the disclosure of such a security issue in the wild? We specifically focus on organisations from the public and critical infrastructure sector who are required to respond to such notifications by law. We find that many organisations are difficult to reach when it concerns security issues, even if they have a security contact point. Additionally, our findings show that having policy in place improves the response and resolution rate, but that even with a policy in place, half of our reports remain unanswered and unsolved after 90~days. Based on these findings we provide recommendations to organisations and bodies such as ENISA to improve future coordinated vulnerability disclosure processes., Comment: 15 pages, 15 figures

We report on the melting dynamics of ice suspended in fresh water and subject to natural convective flows. Using direct numerical simulations we investigate the melt rate of ellipsoidal objects for $2.32\times 10^4 \leq \text{Ra} \leq 7.61\times 10^8$, where \text{Ra} is the Rayleigh number defined with the temperature difference between the ice and the surrounding water. We reveal that the system exhibits non-monotonic behavior in three control parameters. As a function of the aspect ratio of the ellipsoidal, the melting time shows a distinct minimum that is different from a disk which has the minimum perimeter. Furthermore, also with \text{Ra} the system shows a non-monotonic trend, since for large \text{Ra} and large aspect ratio the flow separates, leading to distinctly different dynamics. Lastly, since the density of water is non-monotonic with temperature, the melt rate depends non-monotonically also on the ambient temperature, as for intermediate temperatures ($\unit{4}{\celsius}$--$\unit{7}{\celsius}$) the flow is (partially) reversed. In general, the shape which melts the slowest is quite distinct from that of a disk.

Super-resolution overcoming the standard quantum limit has been intensively studied for quantum sensing applications of precision target detection over the last decades. Not only higher-order entangled photons but also phase-controlled coherent photons have been used to demonstrate the super-resolution. Due to the extreme inefficiency of higher-order entangled photon-pair generation and ultralow signal-to-noise ratio, however, quantum sensing has been severely limited. Here, we report observations of coherently excited super-resolution using phase-controlled coherent photons in a delayed-choice quantum eraser scheme. Using phase manipulations of the quantum erasers, super-resolution has been observed for higher-order intensity correlations between them, satisfying the Heisenberg limit in phase resolution. This new type of precision phase-detection technique opens the door to practical applications of quantum sensing compatible with current technologies based on coherence optics., Comment: 11 pages, 3 figures

Understanding and manipulating the interactions between foreign bodies and cell membranes during endo- and phagocytosis is of paramount importance, not only for the fate of living cells but also for numerous biomedical applications. This study aims to elucidate the role of variables such as anisotropic particle shape, curvature, orientation, membrane tension, and adhesive strength in this essential process, using a minimal experimental biomimetic system comprising giant unilamellar vesicles and rod-like particles with different curvatures and aspect ratios. We find that the particle wrapping process is dictated by the balance between the elastic energy penalty and adhesion energy gain, leading to two distinct engulfment pathways, tip-first and side-first, emphasizing the significance of the particle orientation in determining the pathway. Moreover, our experimental results are consistent with theoretical predictions in a state diagram, showcasing how to control the wrapping pathway from surfing to partial to complete wrapping by the interplay between membrane tension and adhesive strength. At moderate particle concentrations, we observed the formation of rod clusters, which exhibited cooperative and sequential wrapping. Our study not only contributes to a comprehensive understanding of the mechanistic intricacies of endocytosis by highlighting how the interplay between the anisotropic particle shape, curvature, orientation, membrane tension, and adhesive strength can influence the engulfment pathway but also provides a foundational base for future research in the field.

To apply the latest computer vision techniques that require a large computational cost in real industrial applications, knowledge distillation methods (KDs) are essential. Existing logit-based KDs apply the constant temperature scaling to all samples in dataset, limiting the utilization of knowledge inherent in each sample individually. In our approach, we classify the dataset into two categories (i.e., low energy and high energy samples) based on their energy score. Through experiments, we have confirmed that low energy samples exhibit high confidence scores, indicating certain predictions, while high energy samples yield low confidence scores, meaning uncertain predictions. To distill optimal knowledge by adjusting non-target class predictions, we apply a higher temperature to low energy samples to create smoother distributions and a lower temperature to high energy samples to achieve sharper distributions. When compared to previous logit-based and feature-based methods, our energy-based KD (Energy KD) achieves better performance on various datasets. Especially, Energy KD shows significant improvements on CIFAR-100-LT and ImageNet datasets, which contain many challenging samples. Furthermore, we propose high energy-based data augmentation (HE-DA) for further improving the performance. We demonstrate that meaningful performance improvement could be achieved by augmenting only 20-50% of dataset, suggesting that it can be employed on resource-limited devices. To the best of our knowledge, this paper represents the first attempt to make use of energy function in knowledge distillation and data augmentation, and we believe it will greatly contribute to future research., Comment: 33 pages, 7 figures. This work has been submitted to the Elsevier for possible publication

Previous logits-based Knowledge Distillation (KD) have utilized predictions about multiple categories within each sample (i.e., class predictions) and have employed Kullback-Leibler (KL) divergence to reduce the discrepancy between the student and teacher predictions. Despite the proliferation of KD techniques, the student model continues to fall short of achieving a similar level as teachers. In response, we introduce a novel and effective KD method capable of achieving results on par with or superior to the teacher models performance. We utilize teacher and student predictions about multiple samples for each category (i.e., batch predictions) and apply cosine similarity, a commonly used technique in Natural Language Processing (NLP) for measuring the resemblance between text embeddings. This metric's inherent scale-invariance property, which relies solely on vector direction and not magnitude, allows the student to dynamically learn from the teacher's knowledge, rather than being bound by a fixed distribution of the teacher's knowledge. Furthermore, we propose a method called cosine similarity weighted temperature (CSWT) to improve the performance. CSWT reduces the temperature scaling in KD when the cosine similarity between the student and teacher models is high, and conversely, it increases the temperature scaling when the cosine similarity is low. This adjustment optimizes the transfer of information from the teacher to the student model. Extensive experimental results show that our proposed method serves as a viable alternative to existing methods. We anticipate that this approach will offer valuable insights for future research on model compression., Comment: 13 pages, 5 figures

The improvement in the performance of efficient and lightweight models (i.e., the student model) is achieved through knowledge distillation (KD), which involves transferring knowledge from more complex models (i.e., the teacher model). However, most existing KD techniques rely on Kullback-Leibler (KL) divergence, which has certain limitations. First, if the teacher distribution has high entropy, the KL divergence's mode-averaging nature hinders the transfer of sufficient target information. Second, when the teacher distribution has low entropy, the KL divergence tends to excessively focus on specific modes, which fails to convey an abundant amount of valuable knowledge to the student. Consequently, when dealing with datasets that contain numerous confounding or challenging samples, student models may struggle to acquire sufficient knowledge, resulting in subpar performance. Furthermore, in previous KD approaches, we observed that data augmentation, a technique aimed at enhancing a model's generalization, can have an adverse impact. Therefore, we propose a Robustness-Reinforced Knowledge Distillation (R2KD) that leverages correlation distance and network pruning. This approach enables KD to effectively incorporate data augmentation for performance improvement. Extensive experiments on various datasets, including CIFAR-100, FGVR, TinyImagenet, and ImageNet, demonstrate our method's superiority over current state-of-the-art methods., Comment: 11 pages, 7 figures. This work has been submitted to the IEEE for possible publication

The set of canonical decompositions of a cusped hyperbolic 3-manifold is a complete topological invariant. However, there are only a handful of infinite families for which canonical decompositions are known. In this paper, we find canonical triangulations for Dehn fillings of the Borromean rings link complement and two related manifolds obtained by half-twists. The underlying triangulations were shown to be geometric by Ham and Purcell. To obtain canonical triangulations, we show a local convexity result at every face of the geometric triangulation., Comment: 24 pages, 12 figures

While multi-exit neural networks are regarded as a promising solution for making efficient inference via early exits, combating adversarial attacks remains a challenging problem. In multi-exit networks, due to the high dependency among different submodels, an adversarial example targeting a specific exit not only degrades the performance of the target exit but also reduces the performance of all other exits concurrently. This makes multi-exit networks highly vulnerable to simple adversarial attacks. In this paper, we propose NEO-KD, a knowledge-distillation-based adversarial training strategy that tackles this fundamental challenge based on two key contributions. NEO-KD first resorts to neighbor knowledge distillation to guide the output of the adversarial examples to tend to the ensemble outputs of neighbor exits of clean data. NEO-KD also employs exit-wise orthogonal knowledge distillation for reducing adversarial transferability across different submodels. The result is a significantly improved robustness against adversarial attacks. Experimental results on various datasets/models show that our method achieves the best adversarial accuracy with reduced computation budgets, compared to the baselines relying on existing adversarial training or knowledge distillation techniques for multi-exit networks., Comment: 10 pages, 4 figures, accepted by 37th Conference on Neural Information Processing Systems (NeurIPS 2023)