Search

The widespread use of machine learning in credit scoring has brought significant advancements in risk assessment and decision-making. However, it has also raised concerns about potential biases, discrimination, and lack of transparency in these automated systems. This tutorial paper performed a non-systematic literature review to guide best practices for developing responsible machine learning models in credit scoring, focusing on fairness, reject inference, and explainability. We discuss definitions, metrics, and techniques for mitigating biases and ensuring equitable outcomes across different groups. Additionally, we address the issue of limited data representativeness by exploring reject inference methods that incorporate information from rejected loan applications. Finally, we emphasize the importance of transparency and explainability in credit models, discussing techniques that provide insights into the decision-making process and enable individuals to understand and potentially improve their creditworthiness. By adopting these best practices, financial institutions can harness the power of machine learning while upholding ethical and responsible lending practices.

The tilt-to-length coupling during the LISA Pathfinder mission has been numerically and analytically modeled for particular timespans. In this work, we investigate the long-term stability of the coupling coefficients of this noise. We show that they drifted slowly (by 1\,$\mu$m/rad and 6$\times10^{-6}$ in 100 days) and strongly correlated to temperature changes within the satellite (8\,$\mu$m/rad/K and 30$\times10^{-6}$/K). Based on analytical TTL coupling models, we attribute the temperature-driven coupling changes to rotations of the test masses and small distortions in the optical setup. Particularly, we show that LISA Pathfinder's optical baseplate was bent during the cooldown experiment, which started in late 2016 and lasted several months.

A precise characterization of the magnetic properties of LISA Pathfinder free falling test-masses is of special interest for future gravitational wave observatory in space. Magnetic forces have an important impact on the instrument sensitivity in the low frequency regime below the millihertz. In this paper we report on the magnetic injection experiments performed throughout LISA Pathfinder operations. We show how these experiments allowed a high precision estimate of the instrument magnetic parameters. The remanent magnetic moment was found to have a modulus of $(0.245\pm0.081)\,\rm{nAm}^2$, the x-component of the background magnetic field within the test masses position was measured to be $(414 \pm 74)$ nT and its gradient had a value of $(-7.4\pm 2.1)\,\mu$T/m. Finally, we also measured the test mass magnetic susceptibility to be $(-3.35\pm0.15)\times$10$^{-5}$ in the low frequency regime. All results are in agreement with on-ground estimates.

LISA Pathfinder was a mission designed to test key technologies required for gravitational wave detection in space. Magnetically driven forces play a key role in the instrument sensitivity in the low-frequency regime, which corresponds to the measurement band of interest for future space-borne gravitational wave observatories. Magnetic-induced forces couple to the test mass motion, introducing a contribution to the relative acceleration noise between the free falling test masses. In this Letter we present the first complete estimate of this term of the instrument performance model. Our results set the magnetic-induced acceleration noise during the February 2017 noise run of $\rm 0.25_{-0.08}^{+0.15}\,fm\,s^{-2}/\sqrt{Hz}$ at 1 mHz and $\rm 1.01_{-0.24}^{+0.73}\, fm\,s^{-2}/\sqrt{Hz}$ at 0.1 mHz. We also discuss how the non-stationarities of the interplanetary magnetic field can affect these values during extreme space weather conditions.

Hallucinations -- the generation of untrue claims -- pose a challenge to the application of large language models (LLMs) [1] thereby motivating the development of metrics to evaluate factual precision. We observe that popular metrics using the Decompose-Then-Verify framework, such as FActScore [2], can be manipulated by adding obvious or repetitive claims to artificially inflate scores. We expand the FActScore dataset to design and analyze factual precision metrics, demonstrating that models can be trained to achieve high scores under existing metrics through exploiting the issues we identify. This motivates our new customizable plug-and-play subclaim selection component called Core, which filters down individual subclaims according to their uniqueness and informativeness. Metrics augmented by Core are substantially more robust as shown in head-to-head comparisons. We release an evaluation framework supporting the modular use of Core (https://github.com/zipJiang/Core) and various decomposition strategies, and we suggest its adoption by the LLM community. [1] Hong et al., "The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models", arXiv:2404.05904v2 [cs.CL]. [2] Min et al., "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation", arXiv:2305.14251v2 [cs.CL].

A fundamental problem in quantum many-body physics is that of finding ground states of local Hamiltonians. A number of recent works gave provably efficient machine learning (ML) algorithms for learning ground states. Specifically, [Huang et al. Science 2022], introduced an approach for learning properties of the ground state of an $n$-qubit gapped local Hamiltonian $H$ from only $n^{\mathcal{O}(1)}$ data points sampled from Hamiltonians in the same phase of matter. This was subsequently improved by [Lewis et al. Nature Communications 2024], to $\mathcal{O}(\log n)$ samples when the geometry of the $n$-qubit system is known. In this work, we introduce two approaches that achieve a constant sample complexity, independent of system size $n$, for learning ground state properties. Our first algorithm consists of a simple modification of the ML model used by Lewis et al. and applies to a property of interest known beforehand. Our second algorithm, which applies even if a description of the property is not known, is a deep neural network model. While empirical results showing the performance of neural networks have been demonstrated, to our knowledge, this is the first rigorous sample complexity bound on a neural network model for predicting ground state properties. We also perform numerical experiments that confirm the improved scaling of our approach compared to earlier results., Comment: 11 pages, 7 figures + 36-page appendix

We present an in-depth analysis of the LISA Pathfinder differential acceleration performance over the entire course of its science operations, spanning approximately 500 days. We find that: 1) the evolution of the Brownian noise that dominates the acceleration amplitude spectral density (ASD), for frequencies $f\gtrsim 1\,\text{mHz}$, is consistent with the decaying pressure due to the outgassing of a single gaseous species. 2) between $f=36\,\mu\text{Hz}$ and $1\,\text{mHz}$, the acceleration ASD shows a $1/f$ tail in excess of the Brownian noise of almost constant amplitude, with $\simeq 20\%$ fluctuations over a period of a few days, with no particular time pattern over the course of the mission; 3) at the lowest considered frequency of $f=18\,\mu\text{Hz}$, the ASD significantly deviates from the $1/f$ behavior, because of temperature fluctuations that appear to modulate a quasi-static pressure gradient, sustained by the asymmetries of the outgassing pattern. We also present the results of a projection of the observed acceleration noise on the potential sources for which we had either a direct correlation measurement, or a quantitative estimate from dedicated experiments. These sources account for approximately $40\%$ of the noise power in the $1/f$ tail. Finally, we analyze the possible sources of the remaining unexplained fraction, and identify the possible measures that may be taken to keep those under control in LISA.

As generated text becomes more commonplace, it is increasingly important to evaluate how well-supported such text is by external knowledge sources. Many approaches for evaluating textual support rely on some method for decomposing text into its individual subclaims which are scored against a trusted reference. We investigate how various methods of claim decomposition -- especially LLM-based methods -- affect the result of an evaluation approach such as the recently proposed FActScore, finding that it is sensitive to the decomposition method used. This sensitivity arises because such metrics attribute overall textual support to the model that generated the text even though error can also come from the metric's decomposition step. To measure decomposition quality, we introduce an adaptation of FActScore, which we call DecompScore. We then propose an LLM-based approach to generating decompositions inspired by Bertrand Russell's theory of logical atomism and neo-Davidsonian semantics and demonstrate its improved decomposition quality over previous methods.

We present first-order models for tilt-to-length (TTL) coupling in LISA, both for the individual interferometers as well as in the time-delay interferometry (TDI) Michelson observables. These models include the noise contributions from angular and lateral jitter coupling of the six test masses, six movable optical subassemblies (MOSAs), and three spacecraft. We briefly discuss which terms are considered to be dominant and reduce the TTL model for the second-generation TDI Michelson X observable to these primary noise contributions to estimate the resulting noise level. We show that the expected TTL noise will initially violate the entire mission displacement noise budget, resulting in the known necessity to fit and subtract TTL noise in data post-processing. By comparing the noise levels for different assumptions prior to subtraction, we show why noise mitigation by realignment prior to subtraction is favorable. We then discuss that the TTL coupling in the individual interferometers will have noise contributions that will not be present in the TDI observables. Models for TTL coupling noise in TDI and in the individual interferometers are therefore different, and commonly made assumptions are valid as such only for TDI but not for the individual interferometers. Finally, we analyze what implications can be drawn from the presented models for the subsequent fit-and-subtraction in post-processing. We show that noise contributions from the test mass and inter-satellite interferometers are indistinguishable, such that only the combined coefficients can be fit and used for subtraction. However, a distinction is considered not necessary. Additionally, we show a correlation between coefficients for transmitter and receiver jitter couplings in each individual TDI Michelson observable. This full correlation can be resolved by using all three Michelson observables for fitting the TTL coefficients.

Online hate detection suffers from biases incurred in data sampling, annotation, and model pre-training. Therefore, measuring the averaged performance over all examples in held-out test data is inadequate. Instead, we must identify specific model weaknesses and be informed when it is more likely to fail. A recent proposal in this direction is HateCheck, a suite for testing fine-grained model functionalities on synthesized data generated using templates of the kind "You are just a [slur] to me." However, despite enabling more detailed diagnostic insights, the HateCheck test cases are often generic and have simplistic sentence structures that do not match the real-world data. To address this limitation, we propose GPT-HateCheck, a framework to generate more diverse and realistic functional tests from scratch by instructing large language models (LLMs). We employ an additional natural language inference (NLI) model to verify the generations. Crowd-sourced annotation demonstrates that the generated test cases are of high quality. Using the new functional tests, we can uncover model weaknesses that would be overlooked using the original HateCheck dataset., Comment: LREC-COLING 2024. Content Warning: This paper contains model outputs that are offensive in nature

Wavefront errors are a common artifact in laser light generation and imaging. They can be described as an aberration from the spherical wavefront of an ideal Gaussian beam by combinations of higher-order Hermite- or Laguerre-Gaussian terms. Here, we present an algorithm called Beamfit to estimate the mode composition from a series of CCD images taken over the Rayleigh range of a laser beam. The algorithm uses a user-defined set of Hermite- or Laguerre-Gaussian modes as the basis of its theoretical model. A novel method reduces the number of calculations needed to compute the model's intensity profiles. For a given model containing $N$ modes, the number of Hermite-Gaussian complex amplitudes needed to calculate are reduced from orders of $\mathcal{O}(N^2)$ to $\mathcal{O}(N)$ and replaced by simple multiplications. Additionally, non-beam parameters are pre-calculated to further reduce the search space dimension and its resulting calculation time. It is planned to release the Beamfit software to the public under an open-source license., Comment: 13 pages, 5 figures

Electrostatic force actuation is a key component of the system of geodesic reference test masses (TM) for the LISA orbiting gravitational wave observatory and in particular for performance at low frequencies, below 1 mHz, where the observatory sensitivity is limited by stray force noise. The system needs to apply forces of order 10$^{-9}$ N while limiting fluctuations in the measurement band to levels approaching 10$^{-15}$ N/Hz$^{1/2}$. We present here the LISA actuation system design, based on audio-frequency voltage carrier signals, and results of its in-flight performance test with the LISA Pathfinder test mission. In LISA, TM force actuation is used to align the otherwise free-falling TM to the spacecraft-mounted optical metrology system, without any forcing along the critical gravitational wave-sensitive interferometry axes. In LISA Pathfinder, on the other hand, the actuation was used also to stabilize the TM along the critical $x$ axis joining the two TM, with the commanded actuation force entering directly into the mission's main differential acceleration science observable. The mission allowed demonstration of the full compatibility of the electrostatic actuation system with the LISA observatory requirements, including dedicated measurement campaigns to amplify, isolate, and quantify the two main force noise contributions from the actuation system, from actuator gain noise and from low frequency ``in band'' voltage fluctuations. These campaigns have shown actuation force noise to be a relevant, but not dominant, noise source in LISA Pathfinder and have allowed performance projections for the conditions expected in the LISA mission.

We develop Conley's theory for multivalued maps on finite topological spaces. More precisely, for discrete-time dynamical systems generated by the iteration of a multivalued map which satisfies appropriate regularity conditions, we establish the notions of isolated invariant sets and index pairs, and use them to introduce a well-defined Conley index. In addition, we verify some of its fundamental properties such as the Wazewski property and continuation.

Social media users often hold several accounts in their effort to multiply the spread of their thoughts, ideas, and viewpoints. In the particular case of objectionable content, users tend to create multiple accounts to bypass the combating measures enforced by social media platforms and thus retain their online identity even if some of their accounts are suspended. User identity linkage aims to reveal social media accounts likely to belong to the same natural person so as to prevent the spread of abusive/illegal activities. To this end, this work proposes a machine learning-based detection model, which uses multiple attributes of users' online activity in order to identify whether two or more virtual identities belong to the same real natural person. The models efficacy is demonstrated on two cases on abusive and terrorism-related Twitter content.

We present a study of the tilt-to-length coupling noise during the LISA Pathfinder mission and how it depended on the system's alignment. Tilt-to-length coupling noise is the unwanted coupling of angular and lateral spacecraft or test mass motion into the primary interferometric displacement readout. It was one of the major noise sources in the LISA Pathfinder mission and is likewise expected to be a primary noise source in LISA. We demonstrate here that a recently derived and published analytical model describes the dependency of the LISA Pathfinder tilt-to-length coupling noise on the alignment of the two freely falling test masses. This was verified with the data taken before and after the realignments performed in March (engineering days) and June 2016, and during a two-day experiment in February 2017 (long cross-talk experiment). The latter was performed with the explicit goal of testing the tilt-to-length coupling noise dependency on the test mass alignment. Using the analytical model, we show that all realignments performed during the mission were only partially successful and explain the reasons why. In addition to the analytical model, we computed another physical tilt-to-length coupling model via a minimising routine making use of the long cross-talk experiment data. A similar approach could prove useful for the LISA mission.

The Sparse Identification of Nonlinear Dynamics (SINDy) algorithm can be applied to stochastic differential equations to estimate the drift and the diffusion function using data from a realization of the SDE. The SINDy algorithm requires sample data from each of these functions, which is typically estimated numerically from the data of the state. We analyze the performance of the previously proposed estimates for the drift and diffusion function to give bounds on the error for finite data. However, since this algorithm only converges as both the sampling frequency and the length of trajectory go to infinity, obtaining approximations within a certain tolerance may be infeasible. To combat this, we develop estimates with higher orders of accuracy for use in the SINDy framework. For a given sampling frequency, these estimates give more accurate approximations of the drift and diffusion functions, making SINDy a far more feasible system identification method., Comment: 25 pages, 6 figures and 1 table

High-contrast long-slit spectrographs can be used to characterize exoplanets. High-contrast long-slit spectroscopic data are however corrupted by stellar leakages which largely dominate other signals and make the process of extracting the companion spectrum very challenging. This paper presents a complete method to calibrate the spectrograph and extract the signal of interest. The proposed method is based on a flexible direct model of the high-contrast long-slit spectroscopic data. This model explicitly accounts for the instrumental response and for the contributions of both the star and the companion. The contributions of these two components and the calibration parameters are jointly estimated by solving a regularized inverse problem. This problem having no closed-form solution, we propose an alternating minimization strategy to effectively find the solution. We have tested our method on empirical long-slit spectroscopic data and by injecting synthetic companion signals in these data. The proposed initialization and the alternating strategy effectively avoid the self-subtraction bias, even for companions observed very close to the coronagraphic mask. Careful modeling and calibration of the angular and spectral dispersion laws of the instrument clearly reduce the contamination by the stellar leakages. In practice, the outputs of the method are mostly driven by a single hyper-parameter which tunes the level of regularization of the companion SED., Comment: Paper under review by Astronomy & Astrophysics

Tilt-to-length coupling was the limiting noise source in LISA Pathfinder between 20 and 200 mHz before subtraction in post-processing. To prevent the adding of sensing noise to the data by the subtraction process, the success of this strategy depended on a previous direct noise reduction by test mass alignment. The exact dependency of the level of tilt-to-length coupling on the set-points of LISA Pathfinder's test masses was not understood until the end of the mission. Here, we present, for the first time, an analytical tilt-to-length coupling model that describes the coupling noise changes due to the realignments. We report on the different mechanisms, namely the lever arm and piston effect as well as the coupling due to transmissive components, and how they contribute to the full coupling. Further, we show that a pure geometric model would not have been sufficient to describe the coupling in LISA Pathfinder. Therefore, we model also the non-geometric tilt-to-length noise contributions. For the resulting coupling coefficients of the full model, we compute the expected error bars based on the known individual error sources. Also, we validated the analytical model against numerical simulations. A detailed study and thorough understanding of this noise are the basis for a successful analysis of the LISA Pathfinder data with respect to tilt-to-length coupling.

As pointed out by several scholars, current research on hate speech (HS) recognition is characterized by unsystematic data creation strategies and diverging annotation schemata. Subsequently, supervised-learning models tend to generalize poorly to datasets they were not trained on, and the performance of the models trained on datasets labeled using different HS taxonomies cannot be compared. To ease this problem, we propose applying extremely weak supervision that only relies on the class name rather than on class samples from the annotated data. We demonstrate the effectiveness of a state-of-the-art weakly-supervised text classification model in various in-dataset and cross-dataset settings. Furthermore, we conduct an in-depth quantitative and qualitative analysis of the source of poor generalizability of HS classification models., Comment: WOAH 7@ACL 2023

We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible. We present our results and findings, which include that just 13\% of papers had (i) sufficiently low barriers to reproduction, and (ii) enough obtainable information, to be considered for reproduction, and that all but one of the experiments we selected for reproduction was discovered to have flaws that made the meaningfulness of conducting a reproduction questionable. As a result, we had to change our coordinated study design from a reproduce approach to a standardise-then-reproduce-twice approach. Our overall (negative) finding that the great majority of human evaluations in NLP is not repeatable and/or not reproducible and/or too flawed to justify reproduction, paints a dire picture, but presents an opportunity for a rethink about how to design and report human evaluations in NLP., Comment: 5 pages plus appendix, 4 tables, 1 figure. To appear at "Workshop on Insights from Negative Results in NLP" (co-located with EACL2023). Updated author list and acknowledgements