Search

The comic domain is rapidly advancing with the development of single- and multi-page analysis and synthesis models. Recent benchmarks and datasets have been introduced to support and assess models' capabilities in tasks such as detection (panels, characters, text), linking (character re-identification and speaker identification), and analysis of comic elements (e.g., dialog transcription). However, to provide a comprehensive understanding of the storyline, a model must not only extract elements but also understand their relationships and generate highly informative captions. In this work, we propose a pipeline that leverages Vision-Language Models (VLMs) to obtain dense, grounded captions. To construct our pipeline, we introduce an attribute-retaining metric that assesses whether all important attributes are identified in the caption. Additionally, we created a densely annotated test set to fairly evaluate open-source VLMs and select the best captioning model according to our metric. Our pipeline generates dense captions with bounding boxes that are quantitatively and qualitatively superior to those produced by specifically trained models, without requiring any additional training. Using this pipeline, we annotated over 2 million panels across 13,000 books, which will be available on the project page https://github.com/emanuelevivoli/ComiCap., Comment: Accepted at ECCV 2024 Workshop (AI for Visual Art), repo: https://github.com/emanuelevivoli/ComiCap

In the rapidly evolving field of online fashion shopping, the need for more personalized and interactive image retrieval systems has become paramount. Existing methods often struggle with precisely manipulating specific garment attributes without inadvertently affecting others. To address this challenge, we propose GAMMA (Garment Attribute Manipulation with Multi-level Attention), a novel framework that integrates attribute-disentangled representations with a multi-stage attention-based architecture. GAMMA enables targeted manipulation of fashion image attributes, allowing users to refine their searches with high accuracy. By leveraging a dual-encoder Transformer and memory block, our model achieves state-of-the-art performance on popular datasets like Shopping100k and DeepFashion., Comment: Accepted for publication at the ECCV 2024 workshop FashionAI

Vision-language models have recently evolved into versatile systems capable of high performance across a range of tasks, such as document understanding, visual question answering, and grounding, often in zero-shot settings. Comics Understanding, a complex and multifaceted field, stands to greatly benefit from these advances. Comics, as a medium, combine rich visual and textual narratives, challenging AI models with tasks that span image classification, object detection, instance segmentation, and deeper narrative comprehension through sequential panels. However, the unique structure of comics -- characterized by creative variations in style, reading order, and non-linear storytelling -- presents a set of challenges distinct from those in other visual-language domains. In this survey, we present a comprehensive review of Comics Understanding from both dataset and task perspectives. Our contributions are fivefold: (1) We analyze the structure of the comics medium, detailing its distinctive compositional elements; (2) We survey the widely used datasets and tasks in comics research, emphasizing their role in advancing the field; (3) We introduce the Layer of Comics Understanding (LoCU) framework, a novel taxonomy that redefines vision-language tasks within comics and lays the foundation for future work; (4) We provide a detailed review and categorization of existing methods following the LoCU framework; (5) Finally, we highlight current research challenges and propose directions for future exploration, particularly in the context of vision-language models applied to comics. This survey is the first to propose a task-oriented framework for comics intelligence and aims to guide future research by addressing critical gaps in data availability and task definition. A project associated with this survey is available at https://github.com/emanuelevivoli/awesome-comics-understanding., Comment: under review. project website: https://github.com/emanuelevivoli/awesome-comics-understanding

The growth of the entanglement between a disjoint subsystem and its complement after a quantum quench is regarded as a dynamical chaos indicator. Namely, it is expected to show qualitatively different behaviours depending on whether the underlying microscopic dynamics is chaotic or integrable. So far, however, this could only be verified in the context of conformal field theories. Here we present an exact confirmation of this expectation in a class of interacting microscopic Floquet systems on the lattice, i.e., dual-unitary circuits. These systems can either have zero or a super extensive number of conserved charges: the latter case is achieved via fine-tuning. We show that, for almost all dual unitary circuits the asymptotic entanglement dynamics agrees with what is expected for chaotic systems. On the other hand, if we require the systems to have conserved charges, we find that the entanglement displays the qualitatively different behaviour expected for integrable systems. Interestingly, despite having many conserved charges, charge-conserving dual-unitary circuits are in general not Yang-Baxter integrable., Comment: 6+5 pages, 4 figures

We consider how to target evolution conditioned on atypical measurement outcomes in monitored quantum circuits, i.e., the post-selection problem. We show that for a simple class of measurement schemes, post-selected light-cone dynamical correlation functions can be obtained efficiently from the averaged correlations of a different unitary circuit. This connects rare measurement outcomes in one circuit to typical outcomes in another one. We derive conditions for the existence of this rare-to-typical mapping in brickwork quantum circuits made of XYZ gates. We illustrate these general results with a model system that exhibits a dynamical crossover (a smoothed dynamical transition) in event statistics, and discuss extensions to more general dynamical correlations.

G-quadruplexes are non-canonical DNA structures rather ubiquitous in human genome, which are thought to play a crucial role in the development of 85-90 % of cancers. Here, we present a novel coarse-grained approach in modeling G-quadruplexes which accounts for their structural flexibility. We apply it to study the polymeric properties of G-quadruplex multimers, with and without crowder particles, to mimic in-vivo conditions. We find that, contrary to some suggestions found in the literature, long G-quadruplex multimers are rather flexible polymeric macromolecules, with a local persistence length comparable to monomer size, exhibiting chain stiffness variation profile consistent with a real polymer in good solvent. Moreover, in a crowded environment (up to 10% volume fraction), we report that G-quadruplex multimers exhibit an increased propensity for coiling, with a corresponding decrease in the measured chain stiffness. Accurately accounting for the polymeric properties of G4 multimers is crucial for understanding their interactions with anticancer G4-targeting drugs, thereby significantly enhancing the design and effectiveness of these drugs.

A fundamental element of scale-dependent gravity is the scale-setting procedure. We present a new covariant expression to set the scale that arises when examining the field equations. Considering the renormalization group equations and imposing energy-momentum tensor conservation, we arrive at two models of running of the gravitational and cosmological constants. In the cosmological setting, we found that in one model the Big Bang singularity is avoided, while in the other the Hubble tension can be alleviated. At the level of cosmological perturbations, we derived the basic solutions and qualitatively discussed the impacts of this scenario on structure formation., Comment: 17 pages

This paper investigates the impact of using first names in Large Language Models (LLMs) and Vision Language Models (VLMs), particularly when prompted with ethical decision-making tasks. We propose an approach that appends first names to ethically annotated text scenarios to reveal demographic biases in model outputs. Our study involves a curated list of more than 300 names representing diverse genders and ethnic backgrounds, tested across thousands of moral scenarios. Following the auditing methodologies from social sciences we propose a detailed analysis involving popular LLMs/VLMs to contribute to the field of responsible AI by emphasizing the importance of recognizing and mitigating biases in these systems. Furthermore, we introduce a novel benchmark, the Pratical Scenarios Benchmark (PSB), designed to assess the presence of biases involving gender or demographic prejudices in everyday decision-making scenarios as well as practical scenarios where an LLM might be used to make sensible decisions (e.g., granting mortgages or insurances). This benchmark allows for a comprehensive comparison of model behaviors across different demographic categories, highlighting the risks and biases that may arise in practical applications of LLMs and VLMs., Comment: Accepted at ECCV workshop FAILED

The interplay between symmetries and entanglement in out-of-equilibrium quantum systems is currently at the centre of an intense multidisciplinary research effort. Here we introduce a setting where these questions can be characterised exactly by considering dual-unitary circuits with an arbitrary number of $U(1)$ charges. After providing a complete characterisation of these systems we show that one can introduce a class of solvable states, which extends that of generic dual unitary circuits, for which the non-equilibrium dynamics can be solved exactly. In contrast to the known class of solvable states, which relax to the infinite temperature state, these states relax to a family of non-trivial generalised Gibbs ensembles. The relaxation process of these states can be simply described by a linear growth of the entanglement entropy followed by saturation to a non-maximal value but with maximal entanglement velocity. We then move on to consider the dynamics from non-solvable states, combining exact results with the entanglement membrane picture we argue that the entanglement dynamics from these states is qualitatively different from that of the solvable ones. It shows two different growth regimes characterised by two distinct slopes, both corresponding to sub-maximal entanglement velocities. Moreover, we show that non-solvable initial states can give rise to the quantum Mpemba effect, where less symmetric initial states restore the symmetry faster than more symmetric ones., Comment: 31 pages, 8 figures

Understanding thermalisation in quantum many-body systems is among the most enduring problems in modern physics. A particularly interesting question concerns the role played by quantum mechanics in this process, i.e. whether thermalisation in quantum many-body systems is fundamentally different from that in classical many-body systems and, if so, which of its features are genuinely quantum. Here we study this question in minimally structured many-body systems which are only constrained to have local interactions, i.e. local random circuits. We introduce a class of random permutation circuits (RPCs), where the gates locally permute basis states modelling generic microscopic classical dynamics, and compare them to random unitary circuits (RUCs), a standard toy model for generic quantum dynamics. We show that, like RUCs, RPCs permit the analytical computation of several key quantities such as out-of-time order correlators (OTOCs), or entanglement entropies. RPCs can be interpreted both as quantum or classical dynamics, which we use to find similarities and differences between the two. Performing the average over all random circuits, we discover a series of exact relations, connecting quantities in RUC and (quantum) RPCs. In the classical setting, we obtain similar exact results relating (quantum) purity to (classical) growth of mutual information and (quantum) OTOCs to (classical) decorrelators. Our results indicate that despite of the fundamental differences between quantum and classical systems, their dynamics exhibits qualitatively similar behaviours., Comment: 26 (15+11) pages, 2 figures

Vision-Language Models (VLMs) demonstrate remarkable zero-shot generalization to unseen tasks, but fall short of the performance of supervised methods in generalizing to downstream tasks with limited data. Prompt learning is emerging as a parameter-efficient method for adapting VLMs, but state-of-the-art approaches require annotated samples. In this paper we propose a novel approach to prompt learning based on unsupervised knowledge distillation from more powerful models. Our approach, which we call Knowledge Distillation Prompt Learning (KDPL), can be integrated into existing prompt learning techniques and eliminates the need for labeled examples during adaptation. Our experiments on more than ten standard benchmark datasets demonstrate that KDPL is very effective at improving generalization of learned prompts for zero-shot domain generalization, zero-shot cross-dataset generalization, and zero-shot base-to-novel class generalization problems. KDPL requires no ground-truth labels for adaptation, and moreover we show that even in the absence of any knowledge of training class names it can be used to effectively transfer knowledge. The code is publicly available at https://github.com/miccunifi/KDPL., Comment: Accepted for publication at ECCV24

The comic domain is rapidly advancing with the development of single-page analysis and synthesis models. However, evaluation metrics and datasets lag behind, often limited to small-scale or single-style test sets. We introduce a novel benchmark, CoMix, designed to evaluate the multi-task capabilities of models in comic analysis. Unlike existing benchmarks that focus on isolated tasks such as object detection or text recognition, CoMix addresses a broader range of tasks including object detection, speaker identification, character re-identification, reading order, and multi-modal reasoning tasks like character naming and dialogue generation. Our benchmark comprises three existing datasets with expanded annotations to support multi-task evaluation. To mitigate the over-representation of manga-style data, we have incorporated a new dataset of carefully selected American comic-style books, thereby enriching the diversity of comic styles. CoMix is designed to assess pre-trained models in zero-shot and limited fine-tuning settings, probing their transfer capabilities across different comic styles and tasks. The validation split of the benchmark is publicly available for research purposes, and an evaluation server for the held-out test split is also provided. Comparative results between human performance and state-of-the-art models reveal a significant performance gap, highlighting substantial opportunities for advancements in comic understanding. The dataset, baseline models, and code are accessible at https://github.com/emanuelevivoli/CoMix-dataset. This initiative sets a new standard for comprehensive comic analysis, providing the community with a common benchmark for evaluation on a large and varied set., Comment: Accepted at NeurIPS 2024 (D&B)

Comics, as a medium, uniquely combine text and images in styles often distinct from real-world visuals. For the past three decades, computational research on comics has evolved from basic object detection to more sophisticated tasks. However, the field faces persistent challenges such as small datasets, inconsistent annotations, inaccessible model weights, and results that cannot be directly compared due to varying train/test splits and metrics. To address these issues, we aim to standardize annotations across datasets, introduce a variety of comic styles into the datasets, and establish benchmark results with clear, replicable settings. Our proposed Comics Datasets Framework standardizes dataset annotations into a common format and addresses the overrepresentation of manga by introducing Comics100, a curated collection of 100 books from the Digital Comics Museum, annotated for detection in our uniform format. We have benchmarked a variety of detection architectures using the Comics Datasets Framework. All related code, model weights, and detailed evaluation processes are available at https://github.com/emanuelevivoli/cdf, ensuring transparency and facilitating replication. This initiative is a significant advancement towards improving object detection in comics, laying the groundwork for more complex computational tasks dependent on precise object recognition., Comment: Accepted at MANPU - COMICS workshop at ICDAR

We consider the $\phi^4_1$ measure in an interval of length $\ell$, defined by a symmetric double-well potential $W$ and inverse temperature $\beta$. Our results concern its asymptotic behavior in the joint limit $\beta, \ell \to \infty$, both in the subcritical regime $\ell \ll \rme^{\beta C_W}$ and in the supercritical regime $\ell \gg \rme^{\beta C_W}$, where $C_W$ denotes the surface tension. In the former case, in which the measure concentrates on the pure phases, we prove the corresponding large deviation principle. The associated rate function is the Modica-Mortola functional modified to take into account the entropy of the locations of the interfaces. Further, we provide the sharp asymptotics of the probability of having a given number of transitions between the two pure phases. In the supercritical regime, the measure does not longer concentrate and we show that the interfaces are asymptotically distributed according to a Poisson point process., Comment: 38 pages

Moir\'e materials represent strongly interacting electron systems bridging topological and correlated physics. Despite significant advances, decoding wavefunction properties underlying the quantum geometry remains challenging. Here, we utilize polarization-resolved photocurrent measurements to probe magic-angle twisted bilayer graphene, leveraging its sensitivity to the Berry connection that encompasses quantum "textures" of electron wavefunctions. Using terahertz light resonant with optical transitions of its flat bands, we observe bulk photocurrents driven by broken symmetries and reveal the interplay between electron interactions and quantum geometry. We observe inversion-breaking gapped states undetectable through quantum transport, sharp changes in the polarization axes caused by interaction-induced band renormalization, and recurring photocurrent patterns at integer fillings of the moir\'e unit cell that track the evolution of quantum geometry through the cascade of phase transitions. The large and tunable terahertz response intrinsic to flat-band systems offers direct insights into the quantum geometry of interacting electrons and paves the way for innovative terahertz quantum technologies.

Quantum machine learning algorithms are expected to play a pivotal role in quantum chemistry simulations in the immediate future. One such key application is the training of a quantum neural network to learn the potential energy surface and force field of molecular systems. We address this task by using the quantum extreme learning machine paradigm. This particular supervised learning routine allows for resource-efficient training, consisting of a simple linear regression performed on a classical computer. We have tested a setup that can be used to study molecules of any dimension and is optimized for immediate use on NISQ devices with a limited number of native gates. We have applied this setup to three case studies: lithium hydride, water, and formamide, carrying out both noiseless simulations and actual implementation on IBM quantum hardware. Compared to other supervised learning routines, the proposed setup requires minimal quantum resources, making it feasible for direct implementation on quantum platforms, while still achieving a high level of predictive accuracy compared to simulations. Our encouraging results pave the way towards the future application to more complex molecules, being the proposed setup scalable., Comment: 14 pages, 7 figures. Accepted on Machine Learning: Science and Technology

The finite parts of a large, locally interacting many-body system prepared out-of-equilibrium eventually equilibrate. Characterising the underlying mechanisms of this process and its timescales, however, is particularly hard as it requires to decouple universal features from observable-specific ones. Recently, new insight came by studying how certain symmetries of the dynamics that are broken by the initial state are restored at the level of the reduced state of a given subsystem. This provides a high level, observable-independent probe. Until now this idea has been applied to the restoration of internal symmetries, e.g. U(1) symmetries related to charge conservation. Here we show that that the same logic can be applied to the restoration of space-time symmetries, and hence can be used to characterise the relaxation of fully generic systems. We illustrate this idea by considering the paradigmatic example of "generic" many-body dynamics, i.e. a local random unitary circuit. We show that, surprisingly, the restoration of translation symmetry in these systems only happens on time-scales proportional to the subsystem's volume. In fact, for large enough subsystems the time of symmetry restoration becomes initial-state independent (as long as the latter breaks the symmetry at time zero) and coincides with the thermalisation time. For intermediate subsystems, however, one can observe the so-called "quantum Mpemba effect", where the state of the system restores a symmetry faster if it is initially more asymmetric., Comment: 6+4 pages, 3 figures

Given a query consisting of a reference image and a relative caption, Composed Image Retrieval (CIR) aims to retrieve target images visually similar to the reference one while incorporating the changes specified in the relative caption. The reliance of supervised methods on labor-intensive manually labeled datasets hinders their broad applicability. In this work, we introduce a new task, Zero-Shot CIR (ZS-CIR), that addresses CIR without the need for a labeled training dataset. We propose an approach named iSEARLE (improved zero-Shot composEd imAge Retrieval with textuaL invErsion) that involves mapping the visual information of the reference image into a pseudo-word token in CLIP token embedding space and combining it with the relative caption. To foster research on ZS-CIR, we present an open-domain benchmarking dataset named CIRCO (Composed Image Retrieval on Common Objects in context), the first CIR dataset where each query is labeled with multiple ground truths and a semantic categorization. The experimental results illustrate that iSEARLE obtains state-of-the-art performance on three different CIR datasets -- FashionIQ, CIRR, and the proposed CIRCO -- and two additional evaluation settings, namely domain conversion and object composition. The dataset, the code, and the model are publicly available at https://github.com/miccunifi/SEARLE., Comment: Extended version of the ICCV2023 paper arXiv:2303.15247

Large Language Models have demonstrated remarkable performance across various tasks, exhibiting the capacity to swiftly acquire new skills, such as through In-Context Learning (ICL) with minimal demonstration examples. In this work, we present a comprehensive framework for investigating Multimodal ICL (M-ICL) in the context of Large Multimodal Models. We consider the best open-source multimodal models (e.g., IDEFICS, OpenFlamingo) and a wide range of multimodal tasks. Our study unveils several noteworthy findings: (1) M-ICL primarily relies on text-driven mechanisms, showing little to no influence from the image modality. (2) When used with advanced-ICL strategy (like RICES), M-ICL is not better than a simple strategy based on majority voting over context examples. Moreover, we identify several biases and limitations of M-ICL that warrant consideration prior to deployment. Code available at https://gitlab.com/folbaeni/multimodal-icl, Comment: 20 pages, 16 figures. Accepted to CVPR 2024 Workshop on Prompting in Vision. Project page: https://folbaeni.gitlab.io/multimodal-icl

Rationally independent free fermions are those where sums of single-particle energies multiplied by arbitrary rational coefficients vanish only if the coefficients are all zero. This property guaranties that they have no degeneracies in the many-body spectrum and gives them relaxation properties more similar to those of generic systems. Using classic results from number theory we provide minimal examples of rationally independent free fermion models for every system size in one dimension. This is accomplished by considering a free fermion model with a chemical potential, and hopping terms corresponding to all the divisors of the number of sites, each one with an incommensurate complex amplitude. We further discuss the many-body spectral statistics for these models and show that local probes -- like the ratio of consecutive level spacings -- look very similar to what is expected for the Poisson statistics. We however demonstrate that free fermion models can never have Poisson statistics with an analysis of the moments of the spectral form factor., Comment: 8 pages, 4 figures, v2 presentation improved; mistake in Eq. 49 corrected

Fashion illustration is a crucial medium for designers to convey their creative vision and transform design concepts into tangible representations that showcase the interplay between clothing and the human body. In the context of fashion design, computer vision techniques have the potential to enhance and streamline the design process. Departing from prior research primarily focused on virtual try-on, this paper tackles the task of multimodal-conditioned fashion image editing. Our approach aims to generate human-centric fashion images guided by multimodal prompts, including text, human body poses, garment sketches, and fabric textures. To address this problem, we propose extending latent diffusion models to incorporate these multiple modalities and modifying the structure of the denoising network, taking multimodal prompts as input. To condition the proposed architecture on fabric textures, we employ textual inversion techniques and let diverse cross-attention layers of the denoising network attend to textual and texture information, thus incorporating different granularity conditioning details. Given the lack of datasets for the task, we extend two existing fashion datasets, Dress Code and VITON-HD, with multimodal annotations. Experimental evaluations demonstrate the effectiveness of our proposed approach in terms of realism and coherence concerning the provided multimodal inputs.

No-Reference Image Quality Assessment (NR-IQA) focuses on designing methods to measure image quality in alignment with human perception when a high-quality reference image is unavailable. The reliance on annotated Mean Opinion Scores (MOS) in the majority of state-of-the-art NR-IQA approaches limits their scalability and broader applicability to real-world scenarios. To overcome this limitation, we propose QualiCLIP (Quality-aware CLIP), a CLIP-based self-supervised opinion-unaware method that does not require labeled MOS. In particular, we introduce a quality-aware image-text alignment strategy to make CLIP generate representations that correlate with the inherent quality of the images. Starting from pristine images, we synthetically degrade them with increasing levels of intensity. Then, we train CLIP to rank these degraded images based on their similarity to quality-related antonym text prompts, while guaranteeing consistent representations for images with comparable quality. Our method achieves state-of-the-art performance on several datasets with authentic distortions. Moreover, despite not requiring MOS, QualiCLIP outperforms supervised methods when their training dataset differs from the testing one, thus proving to be more suitable for real-world scenarios. Furthermore, our approach demonstrates greater robustness and improved explainability than competing methods. The code and the model are publicly available at https://github.com/miccunifi/QualiCLIP.

Having spectral correlations that, over small enough energy scales, are described by random matrix theory is regarded as the most general defining feature of quantum chaotic systems as it applies in the many-body setting and away from any semiclassical limit. Although this property is extremely difficult to prove analytically for generic many-body systems, a rigorous proof has been achieved for dual-unitary circuits -- a special class of local quantum circuits that remain unitary upon swapping space and time. Here we consider the fate of this property when moving from dual-unitary to generic quantum circuits focussing on the \emph{spectral form factor}, i.e., the Fourier transform of the two-point correlation. We begin with a numerical survey that, in agreement with previous studies, suggests that there exists a finite region in parameter space where dual-unitary physics is stable and spectral correlations are still described by random matrix theory, although up to a maximal quasienergy scale. To explain these findings, we develop a perturbative expansion: it recovers the random matrix theory predictions, provided the terms occurring in perturbation theory obey a relatively simple set of assumptions. We then provide numerical evidence and a heuristic analytical argument supporting these assumptions., Comment: 22 pages, 12 figures

We introduce a class of quantum Markov semigroups describing the evolution of interacting quantum lattice systems, specified either as generic qudits or as fermions. The corresponding generators, which include both conservative and dissipative evolutions, are given by the superposition of local generators in the Lindblad form. Under general conditions, we show that the associated infinite volume dynamics is well defined and can be obtained as the strong limit of the finite volume dynamics. By regarding the interacting evolution as a perturbation of a non-interacting dissipative dynamics, we further obtain a quantitative criterion that yields the ergodicity of the quantum Markov semigroup together with the exponential convergence of local observables. The analysis is based on suitable a priori bounds on the resolvent equation which yield quantitive estimates on the evolution of local observables.

Terminalizations of symplectic quotients are sources of new deformation types of irreducible symplectic varieties. We classify all terminalizations of quotients of Hilbert schemes of K3 surfaces or of generalized Kummer varieties, by finite groups of symplectic automorphisms induced from the underlying K3 or abelian surface. We determine their second Betti number and the fundamental group of their regular locus. In the Kummer case, we prove that the terminalizations have quotient singularities, and determine the singularities of their universal quasi-\'etale cover. In particular, we obtain at least nine new deformation types of irreducible symplectic varieties of dimension four. Finally, we compare our deformation types with those in [FM21; Men22]. The smooth terminalizations are only three and of K$3^{[n]}$-type, and surprisingly they all appeared in different places in the literature [Fuj83; Kaw09; Flo22]., Comment: 47 pages, 11 tables, 5 pictures. Comments are welcome!

We explore the possibility of a consistent cosmology based on the gauge-fixing independent running of the gravitational and cosmological constants ($G$ and $\Lambda$) in the framework of effective quantum gravity. In particular, their running in this framework was found to satisfy $G \propto \Lambda^4$. In the cosmological setting, the covariance of the theory provides energy conservation relations, which are impossible to satisfy with the unique scale parameter. However, by introducing the second sub-dominant scale corresponding to the higher-loop corrections and higher-derivative terms, one can close the system of equations for the running of parameters and arrive at the consistent cosmological solutions. This approach yields a change in the cosmological expansion history that affects the ratio of the Hubble parameter today to the Hubble parameter at high redshift., Comment: 18 pages, 3 figures. v2: Added Cosmic Chronometers analysis. Version accepted in PDU

A new approach to linguistic watermarking of language models is presented in which information is imperceptibly inserted into the output text while preserving its readability and original meaning. A cross-attention mechanism is used to embed watermarks in the text during inference. Two methods using cross-attention are presented that minimize the effect of watermarking on the performance of a pretrained model. Exploration of different training strategies for optimizing the watermarking and of the challenges and implications of applying this approach in real-world scenarios clarified the tradeoff between watermark robustness and text quality. Watermark selection substantially affects the generated output for high entropy sentences. This proactive watermarking approach has potential application in future model development., Comment: 5 pages, 3 figures. Accepted to ICASSP 2024

We study the spreading of quantum information in a recently introduced family of brickwork quantum circuits that generalises the dual-unitary class. These circuits are unitary in time, while their spatial dynamics is unitary only in a restricted subspace. First, we show that local operators spread at the speed of light as in dual-unitary circuits, i.e., the butterfly velocity takes the maximal value allowed by the geometry of the circuit. Then, we prove that the entanglement spreading can still be characterised exactly for a family of compatible initial states (in fact, for an extension of the compatible family of dual-unitary circuits) and that the asymptotic entanglement slope is again independent on the R\'enyi index. Remarkably, however, we find that the entanglement velocity is generically smaller than one. We use these properties to find a closed-form expression for the entanglement membrane in these circuits., Comment: 7 pages in the main text, 4 pages in the SM, 2 figures in the main tex

In this paper, we present a system to restore analog videos of historical archives. These videos often contain severe visual degradation due to the deterioration of their tape supports that require costly and slow manual interventions to recover the original content. The proposed system uses a multi-frame approach and is able to deal with severe tape mistracking, which results in completely scrambled frames. Tests on real-world videos from a major historical video archive show the effectiveness of our demo system. The code and the pre-trained model are publicly available at https://github.com/miccunifi/analog-video-restoration., Comment: ACM MM 2022 (Demo)

In the latest years, videoconferencing has taken a fundamental role in interpersonal relations, both for personal and business purposes. Lossy video compression algorithms are the enabling technology for videoconferencing, as they reduce the bandwidth required for real-time video streaming. However, lossy video compression decreases the perceived visual quality. Thus, many techniques for reducing compression artifacts and improving video visual quality have been proposed in recent years. In this work, we propose a novel GAN-based method for compression artifacts reduction in videoconferencing. Given that, in this context, the speaker is typically in front of the camera and remains the same for the entire duration of the transmission, we can maintain a set of reference keyframes of the person from the higher-quality I-frames that are transmitted within the video stream and exploit them to guide the visual quality improvement; a novel aspect of this approach is the update policy that maintains and updates a compact and effective set of reference keyframes. First, we extract multi-scale features from the compressed and reference frames. Then, our architecture combines these features in a progressive manner according to facial landmarks. This allows the restoration of the high-frequency details lost after the video compression. Experiments show that the proposed approach improves visual quality and generates photo-realistic results even with high compression rates. Code and pre-trained networks are publicly available at https://github.com/LorenzoAgnolucci/Keyframes-GAN., Comment: IEEE Transactions on Multimedia 2023 (IEEE TMM 2023)

We begin a systematic investigation of quench dynamics in higher-dimensional lattice systems considering the case of non-interacting fermions with conserved particle number. We prepare the system in a translational-invariant non-equilibrium initial state -- the simplest example being a classical configuration with fermions at fixed positions on the lattice -- and let it to evolve in time. We characterise the system's dynamics by measuring the entanglement between a finite connected region and its complement. We observe the transmutation of entanglement entropy into thermodynamic entropy and investigate how this process depends on the shape and orientation of the region with respect to the underlying lattice. Interestingly, we find that irregular regions display a distinctive multi-slope entanglement growth, while the dependence on the orientation angle is generically fairly weak. This is particularly true for regions with a large (discrete) rotational symmetry group. The main tool of our analysis is the celebrated quasiparticle picture of Calabrese and Cardy, which we generalise to describe the case at hand. Specifically, we show that for generic initial configurations (even when restricting to classical ones) one has to allow for the production of multiplets involving ${n>2}$ quasiparticles and carrying non-diagonal correlations. We obtain quantitatively accurate predictions -- tested against exact numerics -- and propose an efficient Monte Carlo-based scheme to evaluate them for arbitrary connected regions of generic higher dimensional lattices., Comment: 10 pages (+5 Appendix), 8 (+2) figures

Analog magnetic tapes have been the main video data storage device for several decades. Videos stored on analog videotapes exhibit unique degradation patterns caused by tape aging and reader device malfunctioning that are different from those observed in film and digital video restoration tasks. In this work, we present a reference-based approach for the resToration of digitized Analog videotaPEs (TAPE). We leverage CLIP for zero-shot artifact detection to identify the cleanest frames of each video through textual prompts describing different artifacts. Then, we select the clean frames most similar to the input ones and employ them as references. We design a transformer-based Swin-UNet network that exploits both neighboring and reference frames via our Multi-Reference Spatial Feature Fusion (MRSFF) blocks. MRSFF blocks rely on cross-attention and attention pooling to take advantage of the most useful parts of each reference frame. To address the absence of ground truth in real-world videos, we create a synthetic dataset of videos exhibiting artifacts that closely resemble those commonly found in analog videotapes. Both quantitative and qualitative experiments show the effectiveness of our approach compared to other state-of-the-art methods. The code, the model, and the synthetic dataset are publicly available at https://github.com/miccunifi/TAPE., Comment: WACV2024

No-Reference Image Quality Assessment (NR-IQA) aims to develop methods to measure image quality in alignment with human perception without the need for a high-quality reference image. In this work, we propose a self-supervised approach named ARNIQA (leArning distoRtion maNifold for Image Quality Assessment) for modeling the image distortion manifold to obtain quality representations in an intrinsic manner. First, we introduce an image degradation model that randomly composes ordered sequences of consecutively applied distortions. In this way, we can synthetically degrade images with a large variety of degradation patterns. Second, we propose to train our model by maximizing the similarity between the representations of patches of different images distorted equally, despite varying content. Therefore, images degraded in the same manner correspond to neighboring positions within the distortion manifold. Finally, we map the image representations to the quality scores with a simple linear regressor, thus without fine-tuning the encoder weights. The experiments show that our approach achieves state-of-the-art performance on several datasets. In addition, ARNIQA demonstrates improved data efficiency, generalization capabilities, and robustness compared to competing methods. The code and the model are publicly available at https://github.com/miccunifi/ARNIQA., Comment: WACV2024

Multimodal image-text memes are prevalent on the internet, serving as a unique form of communication that combines visual and textual elements to convey humor, ideas, or emotions. However, some memes take a malicious turn, promoting hateful content and perpetuating discrimination. Detecting hateful memes within this multimodal context is a challenging task that requires understanding the intertwined meaning of text and images. In this work, we address this issue by proposing a novel approach named ISSUES for multimodal hateful meme classification. ISSUES leverages a pre-trained CLIP vision-language model and the textual inversion technique to effectively capture the multimodal semantic content of the memes. The experiments show that our method achieves state-of-the-art results on the Hateful Memes Challenge and HarMeme datasets. The code and the pre-trained models are publicly available at https://github.com/miccunifi/ISSUES., Comment: ICCV2023 CLVL Workshop

We study the nonequilibrium dynamics of the Floquet quantum East model (a Trotterized version of the kinetically constrained quantum East spin chain) at its "deterministic point", where evolution is defined in terms of CNOT permutation gates. We solve exactly the thermalization dynamics for a broad class of initial product states by means of "space evolution". We prove: (i) the entanglement of a block of spins grows at most at one-half the maximal speed allowed by locality (i.e., half the speed of dual-unitary circuits); (ii) if the block of spins is initially prepared in a classical configuration, speed of entanglement is a quarter of the maximum; (iii) thermalization to the infinite temperature state is reached exactly in a time that scales with the size of the block., Comment: 19 pages, 3 figures

The highly complicated nature of far from equilibrium systems can lead to a complete breakdown of the physical intuition developed in equilibrium. A famous example of this is the Mpemba effect, which states that non-equilibrium states may relax faster when they are further from equilibrium or, put another way, hot water can freeze faster than warm water. Despite possessing a storied history, the precise criteria and mechanisms underpinning this phenomenon are still not known. Here we study a quantum version of the Mpemba effect that takes place in closed many body systems with a U(1) conserved charge: in certain cases a more asymmetric initial configuration relaxes and restores the symmetry faster than a more symmetric one. In contrast to the classical case, we establish the criteria for this to occur in arbitrary integrable quantum systems using the recently introduced entanglement asymmetry. We describe the quantum Mpemba effect in such systems and relate properties of the initial state, specifically its charge fluctuations, to the criteria for its occurrence. These criteria are expounded using exact analytic and numerical techniques in several examples, a free fermion model, the Rule 54 cellular automaton, and the Lieb-Liniger model., Comment: 6+8 pages, 2+2 figures; v2 as appears in Phys. Rev. Lett

Given the recent advances in multimodal image pretraining where visual models trained with semantically dense textual supervision tend to have better generalization capabilities than those trained using categorical attributes or through unsupervised techniques, in this work we investigate how recent CLIP model can be applied in several tasks in artwork domain. We perform exhaustive experiments on the NoisyArt dataset which is a dataset of artwork images crawled from public resources on the web. On such dataset CLIP achieves impressive results on (zero-shot) classification and promising results in both artwork-to-artwork and description-to-artwork domain., Comment: Proc. of Florence Heri-Tech 2022: The Future of Heritage Science and Technologies: ICT and Digital Heritage, 2022

The inexorable growth of online shopping and e-commerce demands scalable and robust machine learning-based solutions to accommodate customer requirements. In the context of automatic tagging classification and multimodal retrieval, prior works either defined a low generalizable supervised learning approach or more reusable CLIP-based techniques while, however, training on closed source data. In this work, we propose OpenFashionCLIP, a vision-and-language contrastive learning method that only adopts open-source fashion data stemming from diverse domains, and characterized by varying degrees of specificity. Our approach is extensively validated across several tasks and benchmarks, and experimental results highlight a significant out-of-domain generalization capability and consistent improvements over state-of-the-art methods both in terms of accuracy and recall. Source code and trained models are publicly available at: https://github.com/aimagelab/open-fashion-clip., Comment: International Conference on Image Analysis and Processing (ICIAP) 2023

The requirements for broadband photodetection are becoming exceedingly demanding in hyperspectral imaging. Whilst intrinsic photoconductor arrays based on mercury cadmium telluride represent the most sensitive and suitable technology, their optical spectrum imposes a narrow spectral range with a sharp absorption edge that cuts their operation to < 25 um. Here, we demonstrate a giant ultra-broadband photoconductivity in twisted double bilayer graphene heterostructures spanning a spectral range of 2 - 100 um with internal quantum efficiencies ~ 40 % at speeds of 100 kHz. The giant response originates from unique properties of twist-decoupled heterostructures including pristine, crystal field induced terahertz band gaps, parallel photoactive channels, and strong photoconductivity enhancements caused by interlayer screening of electronic interactions by respective layers acting as sub-atomic spaced proximity screening gates. Our work demonstrates a rare instance of an intrinsic infrared-terahertz photoconductor that is complementary metal-oxide-semiconductor compatible and array integratable, and introduces twist-decoupled graphene heterostructures as a viable route for engineering gapped graphene photodetectors with 3D scalability.