Search

In this article, the non-Hermitian characteristics of three-dimensional PT-symmetric coupled electronic resonators are theoretically analyzed. First, the concept of non-Hermitian PT symmetry is illustrated in the context of electronics using a pair of coupled electronic resonators. Two typical configurations of parallel-coupled PT-symmetric electronic trimers are then analyzed. The results indicate that, for the planar configuration, the system can exhibit two phase transitions as the coupling coefficient or gain-loss parameter changes, different from the linear configuration. By comparing system equations based on coupled-mode theory and circuit theory, it is shown that high dimensionality alone is not a sufficient condition for the existence of a higher-order exceptional point; an approximation condition is also required. A modified exceptional point is proposed, and the approximation conditions for the mean deviation $D$ for the real part of the three eigenfrequencies, satisfying $D \leq 1\%$ and $D \leq 0.1\%$, are discussed, respectively. The theoretical results presented in this paper not only reveal the unique non-Hermitian characteristics of high-dimensional PT-symmetric electronic systems but also offer theoretical support for wireless power transmission and wireless sensing technologies.

With the availability of large pre-trained models, a modern workflow for building real-world machine learning solutions is to fine-tune such models on a downstream task with a relatively small domain-specific dataset. In such applications, one major challenge is that the small fine-tuning dataset does not have sufficient coverage of the distribution encountered when the model is deployed. It is thus important to design fine-tuning methods that are robust to out-of-distribution (OOD) data that are under-represented by the training data. This paper compares common fine-tuning methods to investigate their OOD performance and demonstrates that standard methods will result in a significant change to the pre-trained model so that the fine-tuned features overfit the fine-tuning dataset. However, this causes deteriorated OOD performance. To overcome this issue, we show that protecting pre-trained features leads to a fine-tuned model more robust to OOD generalization. We validate the feature protection methods with extensive experiments of fine-tuning CLIP on ImageNet and DomainNet., Comment: arXiv admin note: substantial text overlap with arXiv:2309.06256

Invariance learning methods aim to learn invariant features in the hope that they generalize under distributional shifts. Although many tasks are naturally characterized by continuous domains, current invariance learning techniques generally assume categorically indexed domains. For example, auto-scaling in cloud computing often needs a CPU utilization prediction model that generalizes across different times (e.g., time of a day and date of a year), where `time' is a continuous domain index. In this paper, we start by theoretically showing that existing invariance learning methods can fail for continuous domain problems. Specifically, the naive solution of splitting continuous domains into discrete ones ignores the underlying relationship among domains, and therefore potentially leads to suboptimal performance. To address this challenge, we then propose Continuous Invariance Learning (CIL), which extracts invariant features across continuously indexed domains. CIL is a novel adversarial procedure that measures and controls the conditional independence between the labels and continuous domain indices given the extracted features. Our theoretical analysis demonstrates the superiority of CIL over existing invariance learning methods. Empirical results on both synthetic and real-world datasets (including data collected from production systems) show that CIL consistently outperforms strong baselines among all the tasks.

Generalization to out-of-distribution (OOD) data is a critical challenge in machine learning. Ensemble-based methods, like weight space ensembles that interpolate model parameters, have been shown to achieve superior OOD performance. However, the underlying mechanism for their effectiveness remains unclear. In this study, we closely examine WiSE-FT, a popular weight space ensemble method that interpolates between a pre-trained and a fine-tuned model. We observe an unexpected ``FalseFalseTrue" phenomenon, in which WiSE-FT successfully corrects many cases where each individual model makes incorrect predictions, which contributes significantly to its OOD effectiveness. To gain further insights, we conduct theoretical analysis in a multi-class setting with a large number of spurious features. Our analysis predicts the above phenomenon and it further shows that ensemble-based models reduce prediction errors in the OOD settings by utilizing a more diverse set of spurious features. Contrary to the conventional wisdom that focuses on learning invariant features for better OOD performance, our findings suggest that incorporating a large number of diverse spurious features weakens their individual contributions, leading to improved overall OOD generalization performance. Additionally, our findings provide the first explanation for the mysterious phenomenon of weight space ensembles outperforming output space ensembles in OOD. Empirically we demonstrate the effectiveness of utilizing diverse spurious features on a MultiColorMNIST dataset, and our experimental results are consistent with the theoretical analysis. Building upon the new theoretical insights into the efficacy of ensemble methods, we further propose a novel averaging method called BAlaNced averaGing (BANG) which significantly enhances the OOD performance of WiSE-FT., Comment: ICLR 2024

It is commonplace to encounter heterogeneous data, of which some aspects of the data distribution may vary but the underlying causal mechanisms remain constant. When data are divided into distinct environments according to the heterogeneity, recent invariant learning methods have proposed to learn robust and invariant models based on this environment partition. It is hence tempting to utilize the inherent heterogeneity even when environment partition is not provided. Unfortunately, in this work, we show that learning invariant features under this circumstance is fundamentally impossible without further inductive biases or additional information. Then, we propose a framework to jointly learn environment partition and invariant representation, assisted by additional auxiliary information. We derive sufficient and necessary conditions for our framework to provably identify invariant features under a fairly general setting. Experimental results on both synthetic and real world datasets validate our analysis and demonstrate an improved performance of the proposed framework over existing methods. Finally, our results also raise the need of making the role of inductive biases more explicit in future works, when considering learning invariant models without environment partition. Codes are available at https://github.com/linyongver/ZIN_official ., Comment: Accepted by NeurIPS 2022

Three-dimensional (3D) elastic phononic topological insulator, featuring two-dimensional (2D) surface states, which support the high-efficient and robust elastic wave propagation without backscattering in all spatial dimensions, remains a challenge due to the nature of multiple polarized elastic modes and their complex hybridization in 3D. Here, a 3D elastic phononic topological insulator is designed and observed experimentally by emulating the quantum valley Hall effects. The spatial inversion of adjacent atoms gives rise to a valley topological phase to an insulating regime with complete 3D topological phononic bandgap. The 2D surface states protected by valley topology are unveiled numerically, which are confirmed experimentally to have a great robustness against the straight channel and sharp bends. Further engineering the elastic valley layer with appropriate interlayer coupling, we also demonstrate that layer pseudospin can be created in 3D elastic system which leads to 2D topological layer-dependent surface states and layer-selective transport. Our work will be a key step for the manipulation of elastic wave in 2D topological plane and the applications of 3D elastic topological-insulator-based devices with layer-selective functionality.

Preserving contour topology during image segmentation is useful in many practical scenarios. By keeping the contours isomorphic, it is possible to prevent over-segmentation and under-segmentation, as well as to adhere to given topologies. The Self-repelling Snake model (SR) is a variational model that preserves contour topology by combining a non-local repulsion term with the geodesic active contour model (GAC). The SR is traditionally solved using the additive operator splitting (AOS) scheme. In our paper, we propose an alternative solution to the SR using the Split Bregman method. Our algorithm breaks the problem down into simpler sub-problems to use lower-order evolution equations and a simple projection scheme rather than re-initialization. The sub-problems can be solved via fast Fourier transform (FFT) or an approximate soft thresholding formula which maintains stability, shortening the convergence time, and reduces the memory requirement. The Split Bregman and AOS algorithms are compared theoretically and experimentally.