Search

Explainable artificial intelligence (XAI) aims to make machine learning models more transparent. While many approaches focus on generating explanations post-hoc, interpretable approaches, which generate the explanations intrinsically alongside the predictions, are relatively rare. In this work, we integrate different discrete subset sampling methods into a graph-based visual question answering system to compare their effectiveness in generating interpretable explanatory subgraphs intrinsically. We evaluate the methods on the GQA dataset and show that the integrated methods effectively mitigate the performance trade-off between interpretability and answer accuracy, while also achieving strong co-occurrences between answer and question tokens. Furthermore, we conduct a human evaluation to assess the interpretability of the generated subgraphs using a comparative setting with the extended Bradley-Terry model, showing that the answer and question token co-occurrence metrics strongly correlate with human preferences. Our source code is publicly available., Comment: Accepted at COLING 2025

Dual encoder architectures like CLIP models map two types of inputs into a shared embedding space and learn similarities between them. However, it is not understood how such models compare two inputs. Here, we address this research gap with two contributions. First, we derive a method to attribute predictions of any differentiable dual encoder onto feature-pair interactions between its inputs. Second, we apply our method to CLIP-type models and show that they learn fine-grained correspondences between parts of captions and regions in images. They match objects across input modes and also account for mismatches. However, this visual-linguistic grounding ability heavily varies between object classes, depends on the training data distribution, and largely improves after in-domain training. Using our method we can identify knowledge gaps about specific object classes in individual models and can monitor their improvement upon fine-tuning.

The race towards performance increase and computing power has led to chips with heterogeneous and complex designs, integrating an ever-growing number of cores on the same monolithic chip or chiplet silicon die. Higher integration density, compounded with the slowdown of technology-driven power reduction, implies that power and thermal management become increasingly relevant. Unfortunately, existing research lacks a detailed analysis and modeling of thermal, power, and electrical coupling effects and how they have to be jointly considered to perform dynamic control of complex and heterogeneous Multi-Processor System on Chips (MPSoCs). To close the gap, in this work, we first provide a detailed thermal and power model targeting a modern High Performance Computing (HPC) MPSoC. We consider real-world coupling effects such as actuators' non-idealities and the exponential relation between the dissipated power, the temperature state, and the voltage level in a single processing element. We analyze how these factors affect the control algorithm behavior and the type of challenges that they pose. Based on the analysis, we propose a thermal capping strategy inspired by Fuzzy control theory to replace the state-of-the-art PID controller, as well as a root-finding iterative method to optimally choose the shared voltage value among cores grouped in the same voltage domain. We evaluate the proposed controller with model-in-the-loop and hardware-in-the-loop co-simulations. We show an improvement over state-of-the-art methods of up to 5x the maximum exceeded temperature while providing an average of 3.56% faster application execution runtime across all the evaluation scenarios., Comment: Paper in Review

The large success of deep learning based methods in Visual Question Answering (VQA) has concurrently increased the demand for explainable methods. Most methods in Explainable Artificial Intelligence (XAI) focus on generating post-hoc explanations rather than taking an intrinsic approach, the latter characterizing an interpretable model. In this work, we introduce an interpretable approach for graph-based VQA and demonstrate competitive performance on the GQA dataset. This approach bridges the gap between interpretability and performance. Our model is designed to intrinsically produce a subgraph during the question-answering process as its explanation, providing insight into the decision making. To evaluate the quality of these generated subgraphs, we compare them against established post-hoc explainability methods for graph neural networks, and perform a human evaluation. Moreover, we present quantitative metrics that correlate with the evaluations of human assessors, acting as automatic metrics for the generated explanatory subgraphs. Our implementation is available at https://github.com/DigitalPhonetics/Intrinsic-Subgraph-Generation-for-VQA., Comment: Accepted at LREC-COLING 2024

Research into COVID-19 has been rapidly evolving since the onset of the pandemic. This occasionally results in contradictory recommendations by credible sources of scientific opinion, public health authorities, and medical professionals. In this study, we examine whether this has resulted in a lack of trust in scientific opinion, by examining the belief patterns of social media users and their reactions to statements related to scientific facts. We devise models to mine belief and persuasion in Twitter discourse using semi-supervised approaches, and show the relationship between lack of belief and insurgence of paranoia and conspiracy theories. By investigating these belief patterns, we explore the best persuasion tactics for communicating information related to COVID-19.

Customizing voice and speaking style in a speech synthesis system with intuitive and fine-grained controls is challenging, given that little data with appropriate labels is available. Furthermore, editing an existing human's voice also comes with ethical concerns. In this paper, we propose a method to generate artificial speaker embeddings that cannot be linked to a real human while offering intuitive and fine-grained control over the voice and speaking style of the embeddings, without requiring any labels for speaker or style. The artificial and controllable embeddings can be fed to a speech synthesis system, conditioned on embeddings of real humans during training, without sacrificing privacy during inference., Comment: Published at ISCA Interspeech 2023 https://www.isca-speech.org/archive/interspeech_2023/lux23_interspeech.html

Lieb and Carlen have shown that mixed states with minimal Wehrl entropy are coherent states. We prove that mixed states with almost minimal Wehrl entropy are almost coherent states. This is proved in a quantitative sense where both the norm and the exponent are optimal and the constant is explicit. We prove a similar bound for generalized Wehrl entropies. As an application, a sharp quantitative form of the log-Sobolev inequality for functions in the Fock space is provided., Comment: 23 pages. Significantly revised to encompass density matrices as well (not only pure states)

We prove a sharp quantitative version of the Faber--Krahn inequality for the short-time Fourier transform (STFT). To do so, we consider a deficit $\delta(f;\Omega)$ which measures by how much the STFT of a function $f\in L^2(\mathbb R)$ fails to be optimally concentrated on an arbitrary set $\Omega\subset \mathbb R^2$ of positive, finite measure. We then show that an optimal power of the deficit $\delta(f;\Omega)$ controls both the $L^2$-distance of $f$ to an appropriate class of Gaussians and the distance of $\Omega$ to a ball, through the Fraenkel asymmetry of $\Omega$. Our proof is completely quantitative and hence all constants are explicit. We also establish suitable generalizations of this result in the higher-dimensional context., Comment: 46 pages

We provide a sharp monotonicity theorem about the distribution of subharmonic functions on manifolds, which can be regarded as a new, measure theoretic form of the uncertainty principle. As an illustration of the scope of this result, we deduce contractivity estimates for analytic functions on the Riemann sphere, the complex plane and the Poincar\'e disc, with a complete description of the extremal functions, hence providing a unified and illuminating perspective of a number of results and conjectures on this subject, in particular on the Wehrl entropy conjecture by Lieb and Solovej. In this connection, we completely prove that conjecture for SU(2), by showing that the corresponding extremals are only the coherent states. Also, we show that the above (global) estimates admit a local counterpart and in all cases we characterize also the extremal subsets, among those of fixed assigned measure., Comment: Fixed a few typos

In order to protect the privacy of speech data, speaker anonymization aims for hiding the identity of a speaker by changing the voice in speech recordings. This typically comes with a privacy-utility trade-off between protection of individuals and usability of the data for downstream applications. One of the challenges in this context is to create non-existent voices that sound as natural as possible. In this work, we propose to tackle this issue by generating speaker embeddings using a generative adversarial network with Wasserstein distance as cost function. By incorporating these artificial embeddings into a speech-to-text-to-speech pipeline, we outperform previous approaches in terms of privacy and utility. According to standard objective metrics and human evaluation, our approach generates intelligible and content-preserving yet privacy-protecting versions of the original recordings., Comment: IEEE Spoken Language Technology Workshop 2022

Time-frequency localization operators (with Gaussian window) $L_F:L^2(\mathbb{R}^d)\to L^2(\mathbb{R}^d)$, where $F$ is a weight in $\mathbb{R}^{2d}$, were introduced in signal processing by I. Daubechies in 1988, inaugurating a new, geometric, phase-space perspective. Sharp upper bounds for the norm (and the singular values) of such operators turn out to be a challenging issue with deep applications in signal recovery, quantum physics and the study of uncertainty principles. In this note we provide optimal upper bounds for the operator norm $\|L_F\|_{L^2\to L^2}$, assuming $F\in L^p(\mathbb{R}^{2d})$, $1

In this work, we propose a speaker anonymization pipeline that leverages high quality automatic speech recognition and synthesis systems to generate speech conditioned on phonetic transcriptions and anonymized speaker embeddings. Using phones as the intermediate representation ensures near complete elimination of speaker identity information from the input while preserving the original phonetic content as much as possible. Our experimental results on LibriSpeech and VCTK corpora reveal two key findings: 1) although automatic speech recognition produces imperfect transcriptions, our neural speech synthesis system can handle such errors, making our system feasible and robust, and 2) combining speaker embeddings from different resources is beneficial and their appropriate normalization is crucial. Overall, our final best system outperforms significantly the baselines provided in the Voice Privacy Challenge 2020 in terms of privacy robustness against a lazy-informed attacker while maintaining high intelligibility and naturalness of the anonymized speech., Comment: Accepted at Interspeech 2022

Counterfactual explanations inform ways to achieve a desired outcome from a machine learning model. However, such explanations are not robust to certain real-world changes in the underlying model (e.g., retraining the model, changing hyperparameters, etc.), questioning their reliability in several applications, e.g., credit lending. In this work, we propose a novel strategy -- that we call RobX -- to generate robust counterfactuals for tree-based ensembles, e.g., XGBoost. Tree-based ensembles pose additional challenges in robust counterfactual generation, e.g., they have a non-smooth and non-differentiable objective function, and they can change a lot in the parameter space under retraining on very similar data. We first introduce a novel metric -- that we call Counterfactual Stability -- that attempts to quantify how robust a counterfactual is going to be to model changes under retraining, and comes with desirable theoretical properties. Our proposed strategy RobX works with any counterfactual generation method (base method) and searches for robust counterfactuals by iteratively refining the counterfactual generated by the base method using our metric Counterfactual Stability. We compare the performance of RobX with popular counterfactual generation methods (for tree-based ensembles) across benchmark datasets. The results demonstrate that our strategy generates counterfactuals that are significantly more robust (nearly 100% validity after actual model changes) and also realistic (in terms of local outlier factor) over existing state-of-the-art methods., Comment: Accepted at ICML 2022

For some special window functions $\psi_{\beta} \in H^2(\mathbb{C}^+),$ we prove that, over all sets $\Delta \subset \mathbb{C}^+$ of fixed hyperbolic measure $\nu(\Delta),$ the ones over which the Wavelet transform $W_{\overline{\psi_{\beta}}}$ with window $\overline{\psi_{\beta}}$ concentrates optimally are exactly the discs with respect to the pseudohyperbolic metric of the upper half space. This answers a question raised by Abreu and D\"orfler. Our techniques make use of a framework recently developed in a previous work by F. Nicola and the second author, but in the hyperbolic context induced by the dilation symmetry of the Wavelet transform. This leads us naturally to use a hyperbolic rearrangement function, as well as the hyperbolic isoperimetric inequality, in our analysis., Comment: 16 pages

On the way towards general Visual Question Answering (VQA) systems that are able to answer arbitrary questions, the need arises for evaluation beyond single-metric leaderboards for specific datasets. To this end, we propose a browser-based benchmarking tool for researchers and challenge organizers, with an API for easy integration of new models and datasets to keep up with the fast-changing landscape of VQA. Our tool helps test generalization capabilities of models across multiple datasets, evaluating not just accuracy, but also performance in more realistic real-world scenarios such as robustness to input noise. Additionally, we include metrics that measure biases and uncertainty, to further explain model behavior. Interactive filtering facilitates discovery of problematic behavior, down to the data sample level. As proof of concept, we perform a case study on four models. We find that state-of-the-art VQA models are optimized for specific tasks or datasets, but fail to generalize even to other in-domain test sets, for example they cannot recognize text in images. Our metrics allow us to quantify which image and question embeddings provide most robustness to a model. All code is publicly available., Comment: 6.5 pages, 9 figures

We investigate the relations between normalized critical points of the nonlinear Schr\"odinger energy functional and critical points of the corresponding action functional on the associated Nehari manifold. Our first general result is that the ground state levels are strongly related by the following duality result: the (negative) energy ground state level is the Legendre-Fenchel transform of the action ground state level. Furthermore, whenever an energy ground state exists at a certain frequency, then all action ground states with that frequency have the same mass and are energy ground states too. We prove that the converse is in general false and that the action ground state level may fail to be convex. Next we analyze the differentiability of the ground state action level and we provide an explicit expression involving the mass of action ground states. Finally we show that similar results hold also for local minimizers., Comment: 25 pages, 1 figure

In this paper we solve an open problem concerning the characterization of those measurable sets $\Omega\subset \mathbb{R}^{2d}$ that, among all sets having a prescribed Lebesgue measure, can trap the largest possible energy fraction in time-frequency space, where the energy density of a generic function $f\in L^2(\mathbb{R}^d)$ is defined in terms of its Short-time Fourier transform (STFT) $\mathcal{V} f(x,\omega)$, with Gaussian window. More precisely, given a measurable set $\Omega\subset\mathbb{R}^{2d}$ having measure $s> 0$, we prove that the quantity \[ \Phi_\Omega=\max\Big\{\int_\Omega|\mathcal{V} f(x,\omega)|^2\,dxd\omega: f\in L^2(\mathbb{R}^d),\ \|f\|_{L^2}=1\Big\}, \] is largest possible if and only if $\Omega$ is equivalent, up to a negligible set, to a ball of measure $s$, and in this case we characterize all functions $f$ that achieve equality. This result leads to a sharp uncertainty principle for the "essential support" of the STFT (when $d=1$, this can be summarized by the optimal bound $\Phi_\Omega\leq 1-e^{-|\Omega|}$, with equality if and only if $\Omega$ is a ball). Our approach, using techniques from measure theory after suitably rephrasing the problem in the Fock space, also leads to a local version of Lieb's uncertainty inequality for the STFT in $L^p$ when $p\in [2,\infty)$, as well as to $L^p$-concentration estimates when $p\in [1,\infty)$, thus proving a related conjecture. In all cases we identify the corresponding extremals., Comment: 23 pages

This work presents a control-oriented structural design approach for a 2-DOF underactuated mechanical system, with the purpose of generating an optimal oscillatory behavior of the end-effector. To achieve the desired periodic motion, we propose to adjust the dynamic response of the mechanism by selecting its mass distribution and the characteristic of a nonlinear spring. In particular, we introduce a two-step optimization strategy to shape the system's zero dynamics, obtained via input-output linearization. The first part of the procedure aims to minimize the root-mean-square value of the input torque by optimizing the mechanism's mass distribution. In this context, we show that a perfect matching with the desired trajectory can be reached by assuming the ability to design an arbitrary shape of the system's elastic properties. Then, in order to favor a simpler physical implementation of the structure, we dedicate the second optimization step to the piecewise linear approximation of the previously defined stiffness characteristic. The proposed procedure is finally tested in detailed numerical simulations, confirming its effectiveness in generating a complex and efficient periodic motion., Comment: accepted for presentation at the 2021 IEEE/ASME International Conference on Advanced Intelligent Mechatronics

Fractional graph isomorphism is the linear relaxation of an integer programming formulation of graph isomorphism. It preserves some invariants of graphs, like degree sequences and equitable partitions, but it does not preserve others like connectivity, clique and independence numbers, chromatic number, vertex and edge cover numbers, matching number, domination and total domination numbers. In this work, we extend the concept of fractional graph isomorphism to hypergraphs, and give an alternative characterization, analogous to one of those that are known for graphs. With this new concept we prove that the fractional packing, covering, matching and transversal numbers on hypergraphs are invariant under fractional hypergraph isomorphism. As a consequence, fractional matching, vertex and edge cover, independence, domination and total domination numbers are invariant under fractional graph isomorphism. This is not the case of fractional chromatic, clique, and clique cover numbers. In this way, most of the classical fractional parameters are classified with respect to their invariance under fractional graph isomorphism.

In this work, we present the integrated structure-control design of a 2-DOF underactuated mechanical system, aiming to achieve a periodic motion of the end-effector. The desired behavior is generated via input-output linearization, followed by structural optimization of the zero dynamics. Inspired by recent works on the control-oriented design of multibody systems, we define an optimization problem based on the simulation of the system's response. In particular, relevant model parameters are used to match the reference with a specific orbit of the zero dynamics, while also penalizing the input energy. For the considered application, the selected parameters are related to the mechanism's elasticities and mass distribution. Notably, we show that it is possible to reach a desirable trade-off between mass reduction and periodic motion accuracy. With an optimal zero dynamics response available, the control scheme can be completed with established orbital stabilization techniques, ensuring a robust oscillating behavior., Comment: 12 pages, 7 figures, 1 table

We study the existence of segregated solutions to a class of reaction-diffusion systems with strong interactions, arising in many physical applications. These special solutions are obtained as weak limits of minimizers of a family of perturbed functionals. We prove some a priori estimates through a minimization procedure which is nonstandard in the parabolic theory: our approach is purely variational and all the information is encoded in the family of functionals we consider., Comment: 22 pages

We consider the problem of uniqueness of ground states of prescribed mass for the Nonlinear Schr\"odinger Energy with power nonlinearity on noncompact metric graphs. We first establish that the Lagrange multiplier appearing in the NLS equation is constant on the set of ground states of mass $\mu$, up to an at most countable set of masses. Then we apply this result to obtain uniqueness of ground states on two specific noncompact graphs. Finally we construct a graph that admits at least two ground states with the same mass having different Lagrange multipliers. Our proofs are based on careful variational arguments and rearrangement techniques, and hold both for the subcritical range $p\in(2,6)$ and in the critical case $p = 6$., Comment: 30 pages, 2 figures

In this paper, we present a hybrid sensorless observer for Permanent Magnets Synchronous Machines, with no a priori knowledge of the mechanical dynamics and without the typical assumption of constant or slowly-varying speed. Instead, we impose the rotor speed to have a constant (unknown) sign and a non-zero magnitude at all times. For the design of the proposed scheme, meaningful Lie group formalism is adopted to describe the rotor position as an element of the unit circle. This choice, however, leads to a non-contractible state space, and therefore it introduces topological constraints that complicate the achievement of global/semi-global and robust results. In this respect it is shown that the proposed observer, which employs a clock to periodically reset the estimates, is semi-globally practically asymptotically stable, and thus it improves a continuous-time version designed under the same assumptions. As highlighted in the simulation results, the novel hybrid strategy leads to enhanced transient performance, notably without any modification of the gains employed in the continuous-time solution. These features motivate to augment the observer with a discrete-time identifier, leading to significantly faster rotor flux reconstruction., Comment: submitted to the 21st IFAC World Congress; bibliography corrected

In this work, we present a mixed sensorless strategy for Permanent Magnets Synchronous Machines, combining a torque/current controller and an observer for position, speed, flux and stator resistance. The proposed co-design is motivated by the need of an appropriate signal injection technique, in order to guarantee full state observability. Neither the typical constant or slowly-varying speed assumptions, nor a priori mechanical model information are used in the observer design. Instead, the rotor speed is modeled as an unknown input disturbance with constant (unknown) sign and uniformly non-zero magnitude. With the proposed architecture, it is shown that the torque tracking and signal injection tasks can be achieved and asymptotically decoupled. Because of these features, we refer to this strategy as a sensorless controller-observer with no mechanical model. Employing a gradient descent resistance/back-EMF estimation, combined with the unit circle formalism to describe the rotor position, we rigorously prove regional practical asymptotic stability of the overall structure, with a domain of attraction that can be made arbitrarily large, not including a lower dimensional manifold. The effectiveness of this design is further validated with numerical simulations, related to a challenging application of UAV propellers control., Comment: submitted to the 21st IFAC World Congress

We study the optimal partitioning of a (possibly unbounded) interval of the real line into $n$ subintervals in order to minimize the maximum of certain set-functions, under rather general assumptions such as continuity, monotonicity, and a Radon-Nikodym property. We prove existence and uniqueness of a solution to this minimax partition problem, showing that the values of the set-functions on the intervals of any optimal partition must coincide. We also investigate the asymptotic distribution of the optimal partitions as $n$ tends to infinity. Several examples of set-functions fit in this framework, including measures, weighted distances and eigenvalues. We recover, in particular, some classical results of Sturm-Liouville theory: the asymptotic distribution of the zeros of the eigenfunctions, the asymptotics of the eigenvalues, and the celebrated Weyl law on the asymptotics of the counting function.

We consider the minimization of the NLS energy on a metric tree, either rooted or unrooted, subject to a mass constraint. With respect to the same problem on other types of metric graphs, several new features appear, such as the existence of minimizers with positive energy, and the emergence of unexpected threshold phenomena. We also study the problem with a radial symmetry constraint that is in principle different from the free problem due to the failure of the P\'olya-Szeg\H{o} inequality for radial rearrangements. A key role is played by a new Poincar\'e inequality with remainder., Comment: 21 pages, 3 figures

We look for best partitions of the unit interval that minimize certain functionals defined in terms of the eigenvalues of Sturm-Liouville problems. Via \Gamma-convergence theory, we study the asymptotic distribution of the minimizers as the number of intervals of the partition tends to infinity. Then we discuss several examples that fit in our framework, such as the sum of (positive and negative) powers of the eigenvalues and an approximation of the trace of the heat Sturm-Liouville operator.

We investigate the existence of ground states for the focusing nonlinear Schroedinger equation on a prototypical doubly periodic metric graph. When the nonlinearity power is below 4, ground states exist for every value of the mass, while, for every nonlinearity power between 4 (included) and 6 (excluded), a mark of $L^2$-criticality arises, as ground states exist if and only if the mass exceeds a threshold value that depends on the power. This phenomenon can be interpreted as a continuous transition from a two-dimensional regime, for which the only critical power is 4, to a one-dimensional behavior, in which criticality corresponds to the power 6. We show that such a dimensional crossover is rooted in the coexistence of one-dimensional and two-dimensional Sobolev inequalities, leading to a new family of Gagliardo-Nirenberg inequalities that account for this continuum of critical exponents., Comment: 17 pages, 2 figures

We discuss a purely variational approach to the study of a wide class of second order nonhomogeneous dissipative hyperbolic PDEs. Precisely, we focus on the wave-like equations that present also a nonzero source term and a first-order-in-time linear term. The paper carries on the research program initiated in (Serra&Tilli'12), and developed in (Serra&Tilli'16), (Tentarelli&Tilli '18), on the De Giorgi approach to hyperbolic equations., Comment: 21 pages; keywords: minimization, nonhomogeneous hyperbolic equations, dissipation, De Giorgi conjecture

We consider the problem of maximizing the first eigenvalue of the $p$-laplacian (possibly with non-constant coefficients) over a fixed domain $\Omega$, with Dirichlet conditions along $\partial\Omega$ and along a supplementary set $\Sigma$, which is the unknown of the optimization problem. The set $\Sigma$, that plays the role of a supplementary stiffening rib for a membrane $\Omega$, is a compact connected set (e.g. a curve or a connected system of curves) that can be placed anywhere in $\overline{\Omega}$, and is subject to the constraint of an upper bound $L$ to its total length (one-dimensional Hausdorff measure). This upper bound prevents $\Sigma$ from spreading throughout $\Omega$ and makes the problem well-posed. We investigate the behavior of optimal sets $\Sigma_L$ as $L\to\infty$ via $\Gamma$-convergence, and we explicitly construct certain asymptotically optimal configurations. We also study the behavior as $p\to\infty$ with $L$ fixed, finding connections with maximum-distance problems related to the principal frequency of the $\infty$-laplacian.