Search

Multivariate time series (MTS) forecasting plays a crucial role in various real-world applications, yet simultaneously capturing both temporal and inter-variable dependencies remains a challenge. Conventional Channel-Dependent (CD) models handle these dependencies separately, limiting their ability to model complex interactions such as lead-lag dynamics. To address these limitations, we propose TiVaT (Time-Variable Transformer), a novel architecture that integrates temporal and variate dependencies through its Joint-Axis (JA) attention mechanism. TiVaT's ability to capture intricate variate-temporal dependencies, including asynchronous interactions, is further enhanced by the incorporation of Distance-aware Time-Variable (DTV) Sampling, which reduces noise and improves accuracy through a learned 2D map that focuses on key interactions. TiVaT effectively models both temporal and variate dependencies, consistently delivering strong performance across diverse datasets. Notably, it excels in capturing complex patterns within multivariate time series, enabling it to surpass or remain competitive with state-of-the-art methods. This positions TiVaT as a new benchmark in MTS forecasting, particularly in handling datasets characterized by intricate and challenging dependencies., Comment: 15pages, 5 figures

It has become common practice now to use random initialization schemes, rather than the pre-trained embeddings, when training transformer based models from scratch. Indeed, we find that pre-trained word embeddings from GloVe, and some sub-word embeddings extracted from language models such as T5 and mT5 fare much worse compared to random initialization. This is counter-intuitive given the well-known representational and transfer-learning advantages of pre-training. Interestingly, we also find that BERT and mBERT embeddings fare better than random initialization, showing the advantages of pre-trained representations. In this work, we posit two potential factors that contribute to these mixed results: the model sensitivity to parameter distribution and the embedding interactions with position encodings. We observe that pre-trained GloVe, T5, and mT5 embeddings have a wider distribution of values. As argued in the initialization studies, such large value initializations can lead to poor training because of saturated outputs. Further, the larger embedding values can, in effect, absorb the smaller position encoding values when added together, thus losing position information. Standardizing the pre-trained embeddings to a narrow range (e.g. as prescribed by Xavier) leads to substantial gains for Glove, T5, and mT5 embeddings. On the other hand, BERT pre-trained embeddings, while larger, are still relatively closer to Xavier initialization range which may allow it to effectively transfer the pre-trained knowledge.

Various modified quantum teleportation schemes are proposed to overcome experimental constraints or to meet specific application requirements for quantum communication. Hence, most schemes are developed and studied with unique methodologies, each with its inherent challenges. Our research focuses on interconnecting these schemes appearing to be unrelated to each other, based on the idea that the unique advantages of one scheme can compensate for the limitations of another. In this paper, we introduce an asymptotic teleportation scheme requiring the receiver to perform a classical selection task followed by a quantum correction. This scheme bridges standard teleportation with port-based teleportation through the transformation of joint measurements. Specifically, we categorize and analytically investigate protocols within this scheme for qubit systems. Given that linear optics teleportation protocol without ancilla qubits is contained in the two non-trivial groups, we provide a novel perspective on its expansion. Furthermore, we discuss the potential application of a protocol from one of these groups as a universal programmable processor and extend these protocols to higher-dimensional systems while maintaining the same properties and potential, providing the analytic form of the joint measurement and its performance. These results thereby propose new avenues for developing a quantum network in higher-dimensional systems., Comment: 16 pages, 5 figures

Rydberg atom arrays operated by a quantum adiabatic principle are among the most promising quantum simulating platforms due to their scalability and long coherence time. From the perspective of combinatorial optimization, they offer an efficient solution for an intrinsic maximum independent set problem because of the resemblance between the Rydberg Hamiltonian and the cost function of the maximum independent set problem. In this study, a strategy is suggested to approximate maximum independent sets by adjusting local detunings on the Rydberg Hamiltonian according to each vertex's vertex support, which is a quantity that represents connectivity between vertices. By doing so, the strategy successfully reduces the error rate three times for the checkerboard graphs with defects when the adiabaticity is sufficient. In addition, the strategy decreases the error rate for random graphs even when the adiabaticity is relatively insufficient. Moreover, it is shown that the strategy helps to prepare a quantum many-body ground state by raising the fidelity between the evolved quantum state and a 2D cat state on a square lattice. Finally, the strategy is combined with the non-abelian adiabatic mixing and this approach is highly successful in finding maximum independent sets compared to the conventional adiabatic evolution with local detunings.

The thermal Hall effect in magnetic insulators has been considered a powerful method for examining the topological nature of charge-neutral quasiparticles such as magnons. Yet, unlike the kagome system, the triangular lattice has received less attention for studying the thermal Hall effect because the scalar spin chirality cancels out between adjacent triangles. However, such cancellation cannot be perfect if the triangular lattice is distorted, which could open the possibility of a non-zero thermal Hall effect. Here, we report that the trimerized triangular lattice of multiferroic hexagonal manganite YMnO$_3$ produces a highly unusual thermal Hall effect due to topological spin fluctuations with the additional intricacy of a Dzyaloshinskii-Moriya interaction under an applied magnetic field. We conclude the thermal Hall conductivity arises from the system's topological nature of spin fluctuations. Our theoretical calculations demonstrate that the thermal Hall conductivity is also related in this material to the splitting of the otherwise degenerate two chiralities, left and right, of its 120$^{\circ}$ magnetic structure. Our result is one of the most unusual cases of topological physics due to this broken $Z_2$ symmetry of the chirality in the supposedly paramagnetic state of YMnO$_3$, with strong topological spin fluctuations. These new mechanisms in this important class of materials are crucial in exploring new thermal Hall physics and exotic excitations., Comment: 14 pages, 3 figures; accepted for publication in Nature Communications

Port-based teleportation (PBT) represents a variation of the standard quantum teleportation and is currently being employed and explored within the field of quantum information processing owing to its various applications. In this study, we focus on PBT protocol when the resource state is disrupted by local Pauli noises. Here, we fully characterise the channel of the noisy PBT protocol using Krauss representation. Especially, by exploiting the application of PBT for entanglement distribution necessary in realizing quantum networks, we investigate entanglement transmission through this protocol for each qubit considering noisy resource states, denoted as port-based entanglement teleportation (PBET). Finally, we derive upper and lower bounds for the teleported entanglement as a function of the initial entanglement and the noises. Our study demonstrates that quantum entanglement can be efficiently distributed by protocols utilizing large-sized resource states in the presence of noise and is expected to serve as a reliable guide for developing optimized PBET protocols. To obtain these results, we address that the order of entanglement of two qubit states is preserved through the local Pauli channel, and identify the boundaries of entanglement loss through this teleportation channel., Comment: 11 pages, 4 figures

Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US). Longitudinal nodule tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. This task, however, imposes a substantial cognitive load on clinicians due to the inherent challenge of maintaining a mental 3D reconstruction of the organ. We thus present a framework for automated US image slice localization within a 3D shape representation to ease how such sonographic diagnoses are carried out. Our proposed method learns a common latent embedding space between US image patches and the 3D surface of an individual's thyroid shape, or a statistical aggregation in the form of a statistical shape model (SSM), via contrastive metric learning. Using cross-modality registration and Procrustes analysis, we leverage features from our model to register US slices to a 3D mesh representation of the thyroid shape. We demonstrate that our multi-modal registration framework can localize images on the 3D surface topology of a patient-specific organ and the mean shape of an SSM. Experimental results indicate slice positions can be predicted within an average of 1.2 mm of the ground-truth slice location on the patient-specific 3D anatomy and 4.6 mm on the SSM, exemplifying its usefulness for slice localization during sonographic acquisitions. Code is publically available: \href{https://github.com/vuenc/slice-to-shape}{https://github.com/vuenc/slice-to-shape}, Comment: ShapeMI Workshop @ MICCAI 2023; 12 pages 2 figures

Traffic forecasting is a highly challenging task owing to the dynamical spatio-temporal dependencies of traffic flows. To handle this, we focus on modeling the spatio-temporal dynamics and propose a network termed Edge Squeeze Graph Convolutional Network (ESGCN) to forecast traffic flow in multiple regions. ESGCN consists of two modules: W-module and ES module. W-module is a fully node-wise convolutional network. It encodes the time-series of each traffic region separately and decomposes the time-series at various scales to capture fine and coarse features. The ES module models the spatio-temporal dynamics using Graph Convolutional Network (GCN) and generates an Adaptive Adjacency Matrix (AAM) with temporal features. To improve the accuracy of AAM, we introduce three key concepts. 1) Using edge features to directly capture the spatiotemporal flow representation among regions. 2) Applying an edge attention mechanism to GCN to extract the AAM from the edge features. Here, the attention mechanism can effectively determine important spatio-temporal adjacency relations. 3) Proposing a novel node contrastive loss to suppress obstructed connections and emphasize related connections. Experimental results show that ESGCN achieves state-of-the-art performance by a large margin on four real-world datasets (PEMS03, 04, 07, and 08) with a low computational cost., Comment: 7 Pages, 3 figures

Statistical shape models (SSMs) are an established way to represent the anatomy of a population with various clinically relevant applications. However, they typically require domain expertise, and labor-intensive landmark annotations to construct. We address these shortcomings by proposing an unsupervised method that leverages deep geometric features and functional correspondences to simultaneously learn local and global shape structures across population anatomies. Our pipeline significantly improves unsupervised correspondence estimation for SSMs compared to baseline methods, even on highly irregular surface topologies. We demonstrate this for two different anatomical structures: the thyroid and a multi-chamber heart dataset. Furthermore, our method is robust enough to learn from noisy neural network predictions, potentially enabling scaling SSMs to larger patient populations without manual segmentation annotation., Comment: Accepted at MICCAI 2023. 13 pages, 6 figures

Domain generalization (DG) is a principal task to evaluate the robustness of computer vision models. Many previous studies have used normalization for DG. In normalization, statistics and normalized features are regarded as style and content, respectively. However, it has a content variation problem when removing style because the boundary between content and style is unclear. This study addresses this problem from the frequency domain perspective, where amplitude and phase are considered as style and content, respectively. First, we verify the quantitative phase variation of normalization through the mathematical derivation of the Fourier transform formula. Then, based on this, we propose a novel normalization method, PCNorm, which eliminates style only as the preserving content through spectral decomposition. Furthermore, we propose advanced PCNorm variants, CCNorm and SCNorm, which adjust the degrees of variations in content and style, respectively. Thus, they can learn domain-agnostic representations for DG. With the normalization methods, we propose ResNet-variant models, DAC-P and DAC-SC, which are robust to the domain gap. The proposed models outperform other recent DG methods. The DAC-SC achieves an average state-of-the-art performance of 65.6% on five datasets: PACS, VLCS, Office-Home, DomainNet, and TerraIncognita., Comment: 10 pages,6 figures, Conference on Computer Vision and Pattern Recognition 2023