Search

Sentence encoder encode the semantics of their input, enabling key downstream applications such as classification, clustering, or retrieval. In this paper, we present Serafim PT*, a family of open-source sentence encoders for Portuguese with various sizes, suited to different hardware/compute budgets. Each model exhibits state-of-the-art performance and is made openly available under a permissive license, allowing its use for both commercial and research purposes. Besides the sentence encoders, this paper contributes a systematic study and lessons learned concerning the selection criteria of learning objectives and parameters that support top-performing encoders.

Retrieval-augmented generation resorts to content retrieved from external sources in order to leverage the performance of large language models in downstream tasks. The excessive volume of retrieved content, the possible dispersion of its parts, or their out of focus range may happen nevertheless to eventually have a detrimental rather than an incremental effect. To mitigate this issue and improve retrieval-augmented generation, we propose a method to refine the retrieved content before it is included in the prompt by resorting to meta-prompting optimization. Put to empirical test with the demanding multi-hop question answering task from the StrategyQA dataset, the evaluation results indicate that this method outperforms a similar retrieval-augmented system but without this method by over 30%.

Leveraging research on the neural modelling of Portuguese, we contribute a collection of datasets for an array of language processing tasks and a corresponding collection of fine-tuned neural language models on these downstream tasks. To align with mainstream benchmarks in the literature, originally developed in English, and to kick start their Portuguese counterparts, the datasets were machine-translated from English with a state-of-the-art translation engine. The resulting PORTULAN ExtraGLUE benchmark is a basis for research on Portuguese whose improvement can be pursued in future work. Similarly, the respective fine-tuned neural language models, developed with a low-rank adaptation approach, are made available as baselines that can stimulate future work on the neural processing of Portuguese. All datasets and models have been developed and are made available for two variants of Portuguese: European and Brazilian., Comment: Preprint - Paper accepted for BUCC 2024

We present a Wikidata-based framework, called KIF, for virtually integrating heterogeneous knowledge sources. KIF is written in Python and is released as open-source. It leverages Wikidata's data model and vocabulary plus user-defined mappings to construct a unified view of the underlying sources while keeping track of the context and provenance of their statements. The underlying sources can be triplestores, relational databases, CSV files, etc., which may or may not use the vocabulary and RDF encoding of Wikidata. The end result is a virtual knowledge base which behaves like an "extended Wikidata" and which can be queried using a simple but expressive pattern language, defined in terms of Wikidata's data model. In this paper, we present the design and implementation of KIF, discuss how we have used it to solve a real integration problem in the domain of chemistry (involving Wikidata, PubChem, and IBM CIRCA), and present experimental results on the performance and overhead of KIF

To foster the neural encoding of Portuguese, this paper contributes foundation encoder models that represent an expansion of the still very scarce ecosystem of large language models specifically developed for this language that are fully open, in the sense that they are open source and openly distributed for free under an open license for any purpose, thus including research and commercial usages. Like most languages other than English, Portuguese is low-resourced in terms of these foundational language resources, there being the inaugural 900 million parameter Albertina and 335 million Bertimbau. Taking this couple of models as an inaugural set, we present the extension of the ecosystem of state-of-the-art open encoders for Portuguese with a larger, top performance-driven model with 1.5 billion parameters, and a smaller, efficiency-driven model with 100 million parameters. While achieving this primary goal, further results that are relevant for this ecosystem were obtained as well, namely new datasets for Portuguese based on the SuperGLUE benchmark, which we also distribute openly.

To advance the neural decoding of Portuguese, in this paper we present a fully open Transformer-based, instruction-tuned decoder model that sets a new state of the art in this respect. To develop this decoder, which we named Gerv\'asio PT*, a strong LLaMA~2 7B model was used as a starting point, and its further improvement through additional training was done over language resources that include new instruction data sets of Portuguese prepared for this purpose, which are also contributed in this paper. All versions of Gerv\'asio are open source and distributed for free under an open license, including for either research or commercial usage, and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.

After a brief introduction to quasinormal modes in dissipative systems, we review the WKB formalism in the context of the analytical calculation of quasinormal frequencies. We apply these results to the calculation of quasinormal frequencies associated with gravitational perturbations of d-dimensional spherically symmetric black holes with string corrections. We do this for two distinct limits: the eikonal limit and the asymptotic limit., Comment: 25 pages, 5 figures. Lecture presented at 11th Aegean Summer School

Machine learning plays an important and growing role in molecular simulation. The newest version of the OpenMM molecular dynamics toolkit introduces new features to support the use of machine learning potentials. Arbitrary PyTorch models can be added to a simulation and used to compute forces and energy. A higher-level interface allows users to easily model their molecules of interest with general purpose, pretrained potential functions. A collection of optimized CUDA kernels and custom PyTorch operations greatly improves the speed of simulations. We demonstrate these features on simulations of cyclin-dependent kinase 8 (CDK8) and the green fluorescent protein (GFP) chromophore in water. Taken together, these features make it practical to use machine learning to improve the accuracy of simulations at only a modest increase in cost., Comment: 15 pages, 4 figures

With the availability of extensive databases of inorganic materials, data-driven approaches leveraging machine learning have gained prominence in materials science research. In this study, we propose an innovative adaptation of data-driven concepts to the mapping and exploration of chemical compound space. Recommender systems, widely utilized for suggesting items to users, employ techniques such as collaborative filtering, which rely on bipartite graphs composed of users, items, and their interactions. Building upon the Open Quantum Materials Database (OQMD), we constructed a bipartite graph where elements from the periodic table and sites within crystal structures are treated as separate entities. The relationships between them, defined by the presence of ions at specific sites and weighted according to the thermodynamic stability of the respective compounds, allowed us to generate an embedding space that contains vector representations for each ion and each site. Through the correlation of ion-site occupancy with their respective distances within the embedding space, we explored new ion-site occupancies, facilitating the discovery of novel stable compounds. Moreover, the graph's embedding space enabled a comprehensive examination of chemical similarities among elements, and a detailed analysis of local geometries of sites. To demonstrate the effectiveness and robustness of our method, we conducted a historical evaluation using different versions of the OQMD and recommended new compounds with Kagome lattices, showcasing the applicability of our approach to practical materials design.

We study the large N dynamics of two massless Yang-Mills coupled matrix quantum mechanics, by minimization of a loop truncated Jevicki-Sakita effective collective field Hamiltonian. The loop space constraints are handled by the use of master variables. The method is successfully applied directly in the massless limit for a range of values of the Yang-Mills coupling constant, and the scaling behaviour of different physical quantities derived from their dimensions are obtained with a high level of precision. We consider both planar properties of the theory, such as the large N ground state energy and multi-matrix correlator expectation values, and also the spectrum of the theory. For the spectrum, we establish that the U(N) traced fundamental constituents remain massless and decoupled from other states, and that bound states develop well defined mass gaps, with the mass of the two degenerate lowest lying bound states being determined with a particularly high degree of accuracy. In order to confirm, numerically, the physical interpretation of the spectrum properties of the U(N) traced constituents, we add masses to the system and show that, indeed, the U(N) traced fundamental constituents retain their "bare masses". For this system, we draw comparisons with planar results available in the literature., Comment: 35 pages; discussions of charges associated with the global SO(2)~U(1) symmetry added; typos corrected

To advance the neural encoding of Portuguese (PT), and a fortiori the technological preparation of this language for the digital age, we developed a Transformer-based foundation model that sets a new state of the art in this respect for two of its variants, namely European Portuguese from Portugal (PT-PT) and American Portuguese from Brazil (PT-BR). To develop this encoder, which we named Albertina PT-*, a strong model was used as a starting point, DeBERTa, and its pre-training was done over data sets of Portuguese, namely over data sets we gathered for PT-PT and PT-BR, and over the brWaC corpus for PT-BR. The performance of Albertina and competing models was assessed by evaluating them on prominent downstream language processing tasks adapted for Portuguese. Both Albertina PT-PT and PT-BR versions are distributed free of charge and under the most permissive license possible and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.

In 1944 Zariski discovered that Bertini's theorem on variable singular points is no longer true when we pass from a field of characteristic zero to a field of positive characteristic. In other words, he found fibrations by singular curves, which only exist in positive characteristic. Such fibrations are connected with many interesting phenomena. For instance, the extension of Enrique's classification of surfaces to positive characteristic (Bombieri and Mumford in 1976), the counterexamples of Kodaira vanishing theorem (Mukai in 2013 and Zheng in 2016) and the isolated singularities with infinity Milnor number (Hefez, Rodrigues and Salom\~ao in 2019). In this work we are going to show that the smoothing process introduced by Shimada in 1991 can be used to classify the set of fibrations by genus two singular curves, up to isomorphism among their generic fibers, such that their smoothing are elliptic fibrations on rational surfaces. Moreover we will also describe the vector fields that can be used to recover such fibrations by singular curves via quotient of rational elliptic surfaces., Comment: We have been informed by an anonymous referee that Theorems 4.7 and 4.8, which are the main results appearing in this preprint, are direct consequences of W. Lang's work on the classification of extremal rational elliptic surfaces

Creating visual 3D sensing characters that interact with AI peers and virtual environments can be a difficult task for those with less experience in using learning algorithms or creating visual environments to execute an agent-based simulation. In this paper, the use of game engines as a tool to create and execute graphic simulations with 3D sensing characters is being explored with plugins such as ML-Agents for the Unity3D game engine. This allows the simulation of agents using off-the-shelf algorithms and using the game engine's motor for the visualizations of these agents. We explore the use of these tools to create visual bots for games, and teach them how to play the game until they reach a level where they can serve as adversaries for real-life players in interactive games., Comment: 4 pages, 3 figures, short paper accepted and presented at ICGI 2022, https://gpcg.pt/icgi2022/program/#session4

We describe a new open-source Python-based package for high accuracy correlated electron calculations using quantum Monte Carlo (QMC) in real space: PyQMC. PyQMC implements modern versions of QMC algorithms in an accessible format, enabling algorithmic development and easy implementation of complex workflows. Tight integration with the PySCF environment allows for simple comparison between QMC calculations and other many-body wave function techniques, as well as access to high accuracy trial wave functions.

In this paper we present a constructive method to characterize ideals of the local ring $\mathscr{O}_{\mathbb{C}^n,0}$ of germs of holomorphic functions at $0\in\mathbb{C}^n$ which arise as the moduli ideal $\langle f,\mathfrak{m}\, j(f)\rangle$, for some $f\in\mathfrak{m}\subset\mathscr{O}_{\mathbb{C}^n,0}$. A consequence of our characterization is an effective solution to a problem dating back to the 1980's, called the Reconstruction Problem of the hypersurface singularity from its moduli algebra. Our results work regardless of whether the hypersurface singularity is isolated or not., Comment: 11 pages now. Minor improvements in the Introduction. Better illustrative examples, I believe

Argument mining tasks require an informed range of low to high complexity linguistic phenomena and commonsense knowledge. Previous work has shown that pre-trained language models are highly effective at encoding syntactic and semantic linguistic phenomena when applied with transfer learning techniques and built on different pre-training objectives. It remains an issue of how much the existing pre-trained language models encompass the complexity of argument mining tasks. We rely on experimentation to shed light on how language models obtained from different lexical semantic families leverage the performance of the identification of argumentative discourse units task. Experimental results show that transfer learning techniques are beneficial to the task and that current methods may be insufficient to leverage commonsense knowledge from different lexical semantic families.

We compute the quasinormal frequencies of d-dimensional large spherically symmetric black holes with Gauss-Bonnet corrections in the highly damped regime. We solve perturbatively the master differential equation and we compute the monodromies of the master perturbation variable (analytically continued to the complex plane) in different contours, in order to obtain the quasinormal mode spectra. We consider tensorial, vectorial and scalar gravitational perturbations, obtaining the same frequencies for the three cases like in Einstein gravity. We also separately perform the same calculation for test scalar fields., Comment: 26 pages, 3 figures. v2: clarifications and references added. Published version

The Tjurina ideal of a germ of an holomorphic function $f$ is the ideal of $\mathscr{O}_{\mathbbm{C}^n,0}$ - the ring of those germs at $0\in\mathbbm{C}^n$ - generated by $f$ itself and by its partial derivatives. Here it is denoted by $T(f)$. The ideal $T(f)$ gives the structure of closed subscheme of $(\mathbbm{C}^n,0)$ to the hypersurface singularity defined by $f$, being an object of central interest in Singularity Theory. In this note we introduce \emph{$T$-fullness} and \emph{$T$-dependence}, two easily verifiable properties for arbitrary ideals of germs of holomorphic functions. These two properties allow us to give necessary and sufficient conditions on an ideal $I\subset \mathscr{O}_{\mathbbm{C}^n,0}$, for the equation $I=T(f)$ to admit a solution $f$., Comment: 18 pages. Included information about Reconstruction Problem and Reconstruction problem as well as linked the interest of the results obtained to the solution of both problems. In this process, different references were included as well. Comments are still welcome

In the context of the $AdS_{4}/CFT_{3}$ correspondence between higher spin fields and vector theories, we use the constructive bilocal fields based approach to this correspondence, to demonstrate, at the $IR$ critical point of the interacting vector theory and directly in the bulk, the removal of the $\Delta=1$ ($s=0$) state from the higher spins field spectrum, and to exhibit simple Klein-Gordon higher spin Hamiltonians. The bulk variables and higher spin fields are obtained in a simple manner from boundary bilocals, by the change of variables previously derived for the $UV$ critical point (in momentum space), together with a field redefinition., Comment: Derivation of the \Delta=2 Hamiltonian added to Section 3.3; 19 pages including references

In this work we revisit the problem of solving multi-matrix systems through numerical large $N$ methods. The framework is a collective, loop space representation which provides a constrained optimization problem, addressed through master-field minimization. This scheme applies both to multi-matrix integrals ($c=0$ systems) and multi-matrix quantum mechanics ($c=1$). The complete fluctuation spectrum is also computable in the above scheme, and is of immediate physical relevance in the later case. The complexity (and the growth of degrees of freedom) at large $N$ have stymied earlier attempts and in the present work we present significant improvements in this regard. The (constrained) minimization and spectrum calculations are easily achieved with close to $10^4$ variables, giving solution to Migdal-Makeenko, and collective field equations. Considering the large number of dynamical (loop) variables and the extreme nonlinearity of the problem, high precision is obtained when confronted with solvable cases. Through numerical results presented, we prove that our scheme solves, by numerical loop space methods, the general two matrix model problem., Comment: 38 pages. v2: minor typos corrected and refs added. v3: Published version

Paper holds the promise to replace silicon substrates in applications like internet of things or disposable electronics that require ultra-low-cost electronic components and an environmentally friendly electronic waste management. In the last years, spurred by the abovementioned properties of paper as a substrate and the exceptional electronic, mechanical and optical properties of van der Waals (vdW) materials, many research groups have worked towards the integration of vdW materials-based devices on paper. Recently, a method to deposit a continuous film of densely packed interconnects of vdW materials on paper by simply rubbing the vdW crystals against the rough surface of paper has been presented. This method utilizes the weak interlayer vdW interactions and allows cleaving of the crystals into micro platelets through the abrasion against the paper. Here, we aim to illustrate the general character and the potential of this technique by fabricating films of 39 different vdW materials (including superconductors, semi-metals, semiconductors, and insulators) on standard copier paper. We have thoroughly characterized their optical properties showing their high optical quality: one can easily resolve the absorption band edge of semiconducting vdW materials and even the excitonic features present in some vdW materials with high exciton binding energy. We also measured the electrical resistivity for several vdW materials films on paper finding exceptionally low values, which are in some cases, orders of magnitude lower than those reported for analogous films produced by inkjet printing. We finally demonstrate the fabrication of field-effect devices with vdW materials on paper using the paper substrate as an ionic gate., Comment: 4 figures in main text, 21 figures in Supp. Info

We compute the quasinormal frequencies of $d$-dimensional spherically symmetric black holes with leading string $\alpha'$ corrections for tensorial gravitational perturbations in the highly damped regime. We solve perturbatively the master differential equation and we compute the monodromies of the master perturbation variable (analytically continued to the complex plane) in different contours, in order to obtain the quasinormal mode spectra. We proceed analogously for the quasinormal modes of test scalar fields. Differently than in Einstein gravity, we obtain distinct results for the two cases., Comment: 31 pages, 4 figures. v2: references added; published version