Search

The challenge of visual grounding and masking in multimodal machine translation (MMT) systems has encouraged varying approaches to the detection and selection of visually-grounded text tokens for masking. We introduce new methods for detection of visually and contextually relevant (concrete) tokens from source sentences, including detection with natural language processing (NLP), detection with object detection, and a joint detection-verification technique. We also introduce new methods for selection of detected tokens, including shortest $n$ tokens, longest $n$ tokens, and all detected concrete tokens. We utilize the GRAM MMT architecture to train models against synthetically collated multimodal datasets of source images with masked sentences, showing performance improvements and improved usage of visual context during translation tasks over the baseline model.

While most current work in multimodal machine translation (MMT) uses the Multi30k dataset for training and evaluation, we find that the resulting models overfit to the Multi30k dataset to an extreme degree. Consequently, these models perform very badly when evaluated against typical text-only testing sets such as the WMT newstest datasets. In order to perform well on both Multi30k and typical text-only datasets, we use a performant text-only machine translation (MT) model as the starting point of our MMT model. We add vision-text adapter layers connected via gating mechanisms to the MT model, and incrementally transform the MT model into an MMT model by 1) pre-training using vision-based masking of the source text and 2) fine-tuning on Multi30k.

A good evaluation framework should evaluate multimodal machine translation (MMT) models by measuring 1) their use of visual information to aid in the translation task and 2) their ability to translate complex sentences such as done for text-only machine translation. However, most current work in MMT is evaluated against the Multi30k testing sets, which do not measure these properties. Namely, the use of visual information by the MMT model cannot be shown directly from the Multi30k test set results and the sentences in Multi30k are are image captions, i.e., short, descriptive sentences, as opposed to complex sentences that typical text-only machine translation models are evaluated against. Therefore, we propose that MMT models be evaluated using 1) the CoMMuTE evaluation framework, which measures the use of visual information by MMT models, 2) the text-only WMT news translation task test sets, which evaluates translation performance against complex sentences, and 3) the Multi30k test sets, for measuring MMT model performance against a real MMT dataset. Finally, we evaluate recent MMT models trained solely against the Multi30k dataset against our proposed evaluation framework and demonstrate the dramatic drop performance against text-only testing sets compared to recent text-only MT models.

The primary obstacle in the field of quantum thermodynamics revolves around the development and practical implementation of quantum heat engines operating at the nanoscale. One of the key challenges associated with quantum working bodies is the occurrence of "quantum friction," which refers to irreversible wasted work resulting from quantum inter-level transitions. Consequently, the construction of a reversible quantum cycle necessitates the utilization of adiabatic shortcuts. However, the experimental realization of such shortcuts for realistic quantum substances is exceedingly complex and often unattainable. In this study, we propose a quantum heat engine that capitalizes on the plasmonic skyrmion lattice. Through rigorous analysis, we demonstrate that the quantum skyrmion substance, owing to its topological protection, exhibits zero irreversible work. Consequently, our engine operates without the need for adiabatic shortcuts. We checked by numerical calculations and observed that when the system is in the quantum skyrmion phase, the propagated states differ from the initial states only by the geometricl and dynamical phases. The adiabacit evoluation leads to the zero transition matrix elements and zero irreversible work. By employing plasmonic mods and an electric field, we drive the quantum cycle. The fundamental building blocks for constructing the quantum working body are individual skyrmions within the plasmonic lattice. As a result, one can precisely control the output power of the engine and the thermodynamic work accomplished by manipulating the number of quantum skyrmions present.

We study the topological entanglement entropy and scalar chirality of a topologically ordered skyrmion formed in a two-dimensional triangular lattice. Scalar chirality remains a smooth function of the magnetic field in both helical and quantum skyrmion phases. In contrast, topological entanglement entropy remains almost constant in the quantum skyrmion phase, whereas it experiences enhanced fluctuations in the helical phase. Therefore, topological entanglement entropy is an effective tool to distinguish between the two phases and pinpoint the quantum phase transition in the system.

Exploiting the effect of nonreciprocal magnons in a system with no inversion symmetry, we propose a concept of a quantum information diode, {\it i.e.}, a device rectifying the amount of quantum information transmitted in the opposite directions. We control the asymmetric left and right quantum information currents through an applied external electric field and quantify it through the left and right out-of-time-ordered correlation (OTOC). To enhance the efficiency of the quantum information diode, we utilize a magnonic crystal. We excite magnons of different frequencies and let them propagate in opposite directions. Nonreciprocal magnons propagating in opposite directions have different dispersion relations. Magnons propagating in one direction match resonant conditions and scatter on gate magnons. Therefore, magnon flux in one direction is damped in the magnonic crystal leading to an asymmetric transport of quantum information in the quantum information diode. A quantum information diode can be fabricated from an yttrium iron garnet (YIG) film. This is an experimentally feasible concept and implies certain conditions: low temperature and small deviation from the equilibrium to exclude effects of phonons and magnon interactions. We show that rectification of the flaw of quantum information can be controlled efficiently by an external electric field and magnetoelectric effects., Comment: Accepted for publication in Materials for Quantum Technology

Quantum skyrmionic phase is modelled in a 2D helical spin lattice. This topological skyrmionic phase retains its nature in a large parameter space before moving to a ferromagnetic phase. Next nearest-neighbour interaction improves the stability and it also causes a shift of the topological phase in the parameter space. Nonanalytic behaviour of the rate function observed, when the system which is initially in a quantum skyrmion phase is quenched to a trivial quantum ferromagnetic phase, indicates a dynamical quantum phase transition. Dynamical quantum phase transition is absent when the system initially in a skyrmion phase is quenched to a helical phase.

Biological network alignment (NA) aims to identify similar regions between molecular networks of different species. NA can be local or global. Just as the recent trend in the NA field, we also focus on global NA, which can be pairwise (PNA) and multiple (MNA). PNA produces aligned node pairs between two networks. MNA produces aligned node clusters between more than two networks. Recently, the focus has shifted from PNA to MNA, because MNA captures conserved regions between more networks than PNA (and MNA is thus considered to be more insightful), though at higher computational complexity. The issue is that, due to the different outputs of PNA and MNA, a PNA method is only compared to other PNA methods, and an MNA method is only compared to other MNA methods. Comparison of PNA against MNA must be done to evaluate whether MNA's higher complexity is justified by its higher accuracy. We introduce a framework that allows for this. We evaluate eight prominent PNA and MNA methods, on synthetic and real-world biological networks, using topological and functional alignment quality measures. We compare PNA against MNA in both a pairwise (native to PNA) and multiple (native to MNA) manner. PNA is expected to perform better under the pairwise evaluation framework. Indeed this is what we find. MNA is expected to perform better under the multiple evaluation framework. Shockingly, we find this not to always hold; PNA is often better than MNA in this framework, depending on the choice of evaluation test.

Networks can model real-world systems in a variety of domains. Network alignment (NA) aims to find a node mapping that conserves similar regions between compared networks. NA is applicable to many fields, including computational biology, where NA can guide the transfer of biological knowledge from well- to poorly-studied species across aligned network regions. Existing NA methods can only align static networks. However, most complex real-world systems evolve over time and should thus be modeled as dynamic networks. We hypothesize that aligning dynamic network representations of evolving systems will produce superior alignments compared to aligning the systems' static network representations, as is currently done. For this purpose, we introduce the first ever dynamic NA method, DynaMAGNA++. This proof-of-concept dynamic NA method is an extension of a state-of-the-art static NA method, MAGNA++. Even though both MAGNA++ and DynaMAGNA++ optimize edge as well as node conservation across the aligned networks, MAGNA++ conserves static edges and similarity between static node neighborhoods, while DynaMAGNA++ conserves dynamic edges (events) and similarity between evolving node neighborhoods. For this purpose, we introduce the first ever measure of dynamic edge conservation and rely on our recent measure of dynamic node conservation. Importantly, the two dynamic conservation measures can be optimized using any state-of-the-art NA method and not just MAGNA++. We confirm our hypothesis that dynamic NA is superior to static NA, under fair comparison conditions, on synthetic and real-world networks, in computational biology and social network domains. DynaMAGNA++ is parallelized and it includes a user-friendly graphical interface.

Network alignment (NA) aims to find a node mapping between molecular networks of different species that identifies topologically or functionally similar network regions. Analogous to genomic sequence alignment, NA can be used to transfer biological knowledge from well- to poorly-studied species between aligned network regions. Pairwise NA (PNA) finds similar regions between two networks while multiple NA (MNA) can align more than two networks. We focus on MNA. Existing MNA methods aim to maximize total similarity over all aligned nodes (node conservation). Then, they evaluate alignment quality by measuring the amount of conserved edges, but only after the alignment is constructed. Directly optimizing edge conservation during alignment construction in addition to node conservation may result in superior alignments. Thus, we present a novel MNA approach called multiMAGNA++ that can achieve this. Indeed, multiMAGNA++ generally outperforms or is on par with the existing MNA methods, while often completing faster than the existing methods. That is, multiMAGNA++ scales well to larger network data and can be parallelized effectively. During method evaluation, we also introduce new MNA quality measures to allow for more complete alignment characterization as well as more fair MNA method comparison compared to using only the existing alignment quality measures.