Search

Clinical machine learning research and AI driven clinical decision support models rely on clinically accurate labels. Manually extracting these labels with the help of clinical specialists is often time-consuming and expensive. This study tests the feasibility of automatic span- and document-level diagnosis extraction from unstructured Dutch echocardiogram reports. We included 115,692 unstructured echocardiogram reports from the UMCU a large university hospital in the Netherlands. A randomly selected subset was manually annotated for the occurrence and severity of eleven commonly described cardiac characteristics. We developed and tested several automatic labelling techniques at both span and document levels, using weighted and macro F1-score, precision, and recall for performance evaluation. We compared the performance of span labelling against document labelling methods, which included both direct document classifiers and indirect document classifiers that rely on span classification results. The SpanCategorizer and MedRoBERTa$.$nl models outperformed all other span and document classifiers, respectively. The weighted F1-score varied between characteristics, ranging from 0.60 to 0.93 in SpanCategorizer and 0.96 to 0.98 in MedRoBERTa$.$nl. Direct document classification was superior to indirect document classification using span classifiers. SetFit achieved competitive document classification performance using only 10% of the training data. Utilizing a reduced label set yielded near-perfect document classification results. We recommend using our published SpanCategorizer and MedRoBERTa$.$nl models for span- and document-level diagnosis extraction from Dutch echocardiography reports. For settings with limited training data, SetFit may be a promising alternative for document classification., Comment: 28 pages, 5 figures

Emotions play important epistemological and cognitive roles in our lives, revealing our values and guiding our actions. Previous work has shown that LLMs display biases in emotion attribution along gender lines. However, unlike gender, which says little about our values, religion, as a socio-cultural system, prescribes a set of beliefs and values for its followers. Religions, therefore, cultivate certain emotions. Moreover, these rules are explicitly laid out and interpreted by religious leaders. Using emotion attribution, we explore how different religions are represented in LLMs. We find that: Major religions in the US and European countries are represented with more nuance, displaying a more shaded model of their beliefs. Eastern religions like Hinduism and Buddhism are strongly stereotyped. Judaism and Islam are stigmatized -- the models' refusal skyrocket. We ascribe these to cultural bias in LLMs and the scarcity of NLP literature on religion. In the rare instances where religion is discussed, it is often in the context of toxic language, perpetuating the perception of these religions as inherently toxic. This finding underscores the urgent need to address and rectify these biases. Our research underscores the crucial role emotions play in our lives and how our values influence them.

The rapid advancement of Large Language Models (LLMs) necessitates robust and challenging benchmarks. Leaderboards like Chatbot Arena rank LLMs based on how well their responses align with human preferences. However, many tasks such as those related to emotional intelligence, creative writing, or persuasiveness, are highly subjective and often lack majoritarian human agreement. Judges may have irreconcilable disagreements about what constitutes a better response. To address the challenge of ranking LLMs on highly subjective tasks, we propose a novel benchmarking framework, the Language Model Council (LMC). The LMC operates through a democratic process to: 1) formulate a test set through equal participation, 2) administer the test among council members, and 3) evaluate responses as a collective jury. We deploy a council of 20 newest LLMs on an open-ended emotional intelligence task: responding to interpersonal dilemmas. Our results show that the LMC produces rankings that are more separable, robust, and less biased than those from any individual LLM judge, and is more consistent with a human-established leaderboard compared to other benchmarks.

Large language models (LLMs) reflect societal norms and biases, especially about gender. While societal biases and stereotypes have been extensively researched in various NLP applications, there is a surprising gap for emotion analysis. However, emotion and gender are closely linked in societal discourse. E.g., women are often thought of as more empathetic, while men's anger is more socially accepted. To fill this gap, we present the first comprehensive study of gendered emotion attribution in five state-of-the-art LLMs (open- and closed-source). We investigate whether emotions are gendered, and whether these variations are based on societal stereotypes. We prompt the models to adopt a gendered persona and attribute emotions to an event like 'When I had a serious argument with a dear person'. We then analyze the emotions generated by the models in relation to the gender-event pairs. We find that all models consistently exhibit gendered emotions, influenced by gender stereotypes. These findings are in line with established research in psychology and gender studies. Our study sheds light on the complex societal interplay between language, gender, and emotion. The reproduction of emotion stereotypes in LLMs allows us to use those models to study the topic in detail, but raises questions about the predictive use of those same LLMs for emotion applications., Comment: Accepted to ACL 2024

Emotions are a central aspect of communication. Consequently, emotion analysis (EA) is a rapidly growing field in natural language processing (NLP). However, there is no consensus on scope, direction, or methods. In this paper, we conduct a thorough review of 154 relevant NLP publications from the last decade. Based on this review, we address four different questions: (1) How are EA tasks defined in NLP? (2) What are the most prominent emotion frameworks and which emotions are modeled? (3) Is the subjectivity of emotions considered in terms of demographics and cultural factors? and (4) What are the primary NLP applications for EA? We take stock of trends in EA and tasks, emotion frameworks used, existing datasets, methods, and applications. We then discuss four lacunae: (1) the absence of demographic and cultural aspects does not account for the variation in how emotions are perceived, but instead assumes they are universally experienced in the same manner; (2) the poor fit of emotion categories from the two main emotion theories to the task; (3) the lack of standardized EA terminology hinders gap identification, comparison, and future goals; and (4) the absence of interdisciplinary research isolates EA from insights in other fields. Our work will enable more focused research into EA and a more holistic approach to modeling emotions in NLP., Comment: Accepted to LREC-COLING 2024

Background: Proximal Junctional Failure (PJF) is a post-operative complication in adult spine surgery, often requiring reoperation. Osteotomy is often used in revision surgeries, leading to 34.8% complications. Hence, suboptimal decisions might be extending hardware without osteotomy, which yields to severe Global Alignment and Proportion (GAP) scores. High GAPs increase PJF risk, but Hardware Density Reduction (HDR) might limit it. Methods: Two clinical cases were evaluated: 1) Initially operated with hardware extended to T10, GAP 10; 2) PJF at T11 and hardware extended to T3, GAP 11. Two patient-personalized spine FE models were constructed through Statistical Shape Modelling (SSM) and mesh morphing. Intervertebral Disk (IVD) fiber strain, screw pull-out force, and rod stress were evaluated for the cases 1) and 2), also for 91 virtual HDR scenarios with different GAP scores, using Finite Element (FE) simulations. Different rod and bone material properties were also assessed. Results: HDR could decrease IVD fiber strain (-70% at most) and increase screw pull-out forces (+142% at most) for cases with Ti rod and normal bone. Cr-Co rod and osteopenia, and osteoporotic bones had high PJF risk. Trade-off analyses could determine the best configurations avoiding PJF. Virtual cohort study showed that GAP 12 and 13 could not avoid PJF in any HDR scenarios either with Ti or Cr-Co rods. HDR in a UIV T10 virtual patient with GAP 11 could not de-risk in case of Cr-Co rods. UIV T3 with GAP 13 could not benefit any HDR strategy, independently of rod properties. In contrast, Ti rods might allow HDR to de-risk GAP 12 patients with UIV T3. Conclusions: HDR could avoid PJF in the patients with medium high GAP scores, depending on the screw reduction pattern, and bone and rod material properties. Remarkably, HDR technique might avoid excessive spine surgeries and minimize the surgery cost.

Adult spine deformity (ASD) is prevalent and leads to a sagittal misalignment in the vertebral column. Computational methods, including Finite Element (FE) Models, have emerged as valuable tools for investigating the causes and treatment of ASD through biomechanical simulations. However, the process of generating personalized FE models is often complex and time-consuming. To address this challenge, we present a repository of FE models with diverse spine morphologies that statistically represent real geometries from a cohort of patients. These models are generated using EOS images, which are utilized to reconstruct 3D surface spine models. Subsequently, a Statistical Shape Model (SSM) is constructed, enabling the adaptation of a FE hexahedral mesh template for both the bone and soft tissues of the spine through mesh morphing. The SSM deformation fields facilitate the personalization of the mean hexahedral FE model based on sagittal balance measurements. Ultimately, this new hexahedral SSM tool offers a means to generate a virtual cohort of 16807 thoracolumbar FE spine models, which are openly shared in a public repository.

Recent instruction fine-tuned models can solve multiple NLP tasks when prompted to do so, with machine translation (MT) being a prominent use case. However, current research often focuses on standard performance benchmarks, leaving compelling fairness and ethical considerations behind. In MT, this might lead to misgendered translations, resulting, among other harms, in the perpetuation of stereotypes and prejudices. In this work, we address this gap by investigating whether and to what extent such models exhibit gender bias in machine translation and how we can mitigate it. Concretely, we compute established gender bias metrics on the WinoMT corpus from English to German and Spanish. We discover that IFT models default to male-inflected translations, even disregarding female occupational stereotypes. Next, using interpretability methods, we unveil that models systematically overlook the pronoun indicating the gender of a target occupation in misgendered translations. Finally, based on this finding, we propose an easy-to-implement and effective bias mitigation solution based on few-shot learning that leads to significantly fairer translations., Comment: Accepted at EMNLP 2023. Code and data at https://github.com/MilaNLProc/interpretability-mt-gender-bias

In this work we explore the low energy effects induced from the integration of the heavy Higgs boson modes, $H$, $A$ and $H^\pm$, within the Two Higgs Doublet Model (2HDM) by assuming that the lightest Higgs boson $h$ is the one observed experimentally at $m_h \sim 125$ GeV. We work within the context of Effective Field Theories, focusing on the Higgs Effective Field Theory (HEFT), although some comparisons with the Standard Model Effective Field Theory (SMEFT) case are also discussed through this work. Our main focus is placed in the computation of the non-decoupling effects from the heavy Higgs bosons and the capture of such effects by means of the HEFT coefficients which are expressed in terms of the input parameters of the 2HDM. Our approach to solve this issue is by matching the amplitudes of the 2HDM and the HEFT for physical processes involving the light Higgs boson $h$ in the external legs, instead of the most frequently used matching procedure at the Lagrangian level. More concretely, we perform the matching at the amplitudes level for the following physical processes, including scattering and decays: $h\to WW^*\to Wf\bar{f'}$, $h\to ZZ^*\to Zf\bar{f}$, $WW \to hh$, $ZZ \to hh$, $hh \to hh$, $h \to \gamma \gamma$ and $h \to \gamma Z$. One important point of this work is that the matching is required to happen at low energies compared to the heavy Higgs boson masses, and these are heavier than the other particle masses. The proper expansion for this heavy mass limit is also defined here, which provides the results for the non-decoupling effects presented in this work. We finally discuss the implications of the resulting effective coefficients, and remark on the interesting correlations detected among them., Comment: 35 pages, 15 figures

Large Language Models (LLMs) exhibit remarkable text classification capabilities, excelling in zero- and few-shot learning (ZSL and FSL) scenarios. However, since they are trained on different datasets, performance varies widely across tasks between those models. Recent studies emphasize the importance of considering human label variation in data annotation. However, how this human label variation also applies to LLMs remains unexplored. Given this likely model specialization, we ask: Do aggregate LLM labels improve over individual models (as for human annotators)? We evaluate four recent instruction-tuned LLMs as annotators on five subjective tasks across four languages. We use ZSL and FSL setups and label aggregation from human annotation. Aggregations are indeed substantially better than any individual model, benefiting from specialization in diverse tasks or languages. Surprisingly, FSL does not surpass ZSL, as it depends on the quality of the selected examples. However, there seems to be no good information-theoretical strategy to select those. We find that no LLM method rivals even simple supervised models. We also discuss the tradeoffs in accuracy, cost, and moral/ethical considerations between LLM and human annotation., Comment: Accepted to the 3rd Workshop on Perspectivist Approaches to NLP at LREC-COLING 2024

Within the Two Higgs Doublet Model (2HDM) with $\mathcal{CP}$ conservation and a softly broken $Z_2$ symmetry, we analyze the flavor changing Higgs decays $h\to bs$ ($bs$ refers jointly to the two decay channels $b\bar s$ and $\bar b s$), where $h$ is identified with the SM-like Higgs boson discovered at the LHC. We provide a comprehensive study of the decay width $\Gamma (h \to bs)$ with particular focus on the most relevant effects from the triple Higgs coupling $\lambda_{hH^+H^-}$. Furthermore, we consider all the relevant theoretical and experimental constraints to determine which predictions for the $\mathrm{BR}\left(h\to bs\right)$ are still allowed by the current data. We find that the predictions for $\mathrm{BR}\left(h\to bs\right)$ in types II and III can be several orders of magnitude smaller compared to the SM value. In contrast, in type I and IV we find that the predicted enhancements in the decay rates with respect to the SM of up to about 70% and 50%, respectively, are still allowed. We discuss how these deviations from the SM are caused by interference effects controlled by the coupling $\lambda_{hH^+H^-}$ which can be large for very heavy $H^\pm$. To better understand the role of $\lambda_{hH^+H^-}$ in the $h \to bs$ decay we derive and analyze here the analytical results for the $hbs$ one-loop effective vertex that is generated by integrating out the heavy $H^\pm$., Comment: 33 pages, 5 figures, version published in Phys. Rev. D

Low-field (<1T) magnetic resonance imaging (MRI) scanners remain in widespread use in low- and middle-income countries (LMICs) and are commonly used for some applications in higher income countries e.g. for small child patients with obesity, claustrophobia, implants, or tattoos. However, low-field MR images commonly have lower resolution and poorer contrast than images from high field (1.5T, 3T, and above). Here, we present Image Quality Transfer (IQT) to enhance low-field structural MRI by estimating from a low-field image the image we would have obtained from the same subject at high field. Our approach uses (i) a stochastic low-field image simulator as the forward model to capture uncertainty and variation in the contrast of low-field images corresponding to a particular high-field image, and (ii) an anisotropic U-Net variant specifically designed for the IQT inverse problem. We evaluate the proposed algorithm both in simulation and using multi-contrast (T1-weighted, T2-weighted, and fluid attenuated inversion recovery (FLAIR)) clinical low-field MRI data from an LMIC hospital. We show the efficacy of IQT in improving contrast and resolution of low-field MR images. We demonstrate that IQT-enhanced images have potential for enhancing visualisation of anatomical structures and pathological lesions of clinical relevance from the perspective of radiologists. IQT is proved to have capability of boosting the diagnostic value of low-field MRI, especially in low-resource settings., Comment: Accepted in Medical Image Analysis

In recent years, transmission system operators have started requesting converter-interfaced generators (CIGs) to participate in grid services such as power oscillation damping (POD). As power systems are prone to topology changes because of connection and disconnection of generators and electrical lines, one of the most important requirements in the POD controller design is to account for these changes and to deal with them by using either adaptive or robust approaches. The robust approach is usually preferred by system operators because of the fixed structure of the controller. In this paper, a procedure to design POD controllers for CIG-based power plants that takes into consideration all possible network configurations is presented. This procedure is based on frequency-response techniques, so it is suitable for the commissioning in newly installed power plants, even in those cases when a detailed small-signal model of the system is not available. This procedure can be used to damp critical system modes by using active power, reactive power, or both power components simultaneously. The proposed procedure is applied to the design of the POD controller for a CIG-based power plant connected to the IEEE 39 Bus system. Simulations performed in Matlab and SimPowerSystems are used to validate the proposed design procedure.