Search

Significant advances have been made in natural language processing in recent years. However, our current deep learning approach to language modeling requires substantial resources in terms of data and computation. One of the side effects of this data-hungry paradigm is the current schism between languages, separating those considered high-resource, where most of the development happens and resources are available, and the low-resource ones, which struggle to attain the same level of performance and autonomy. This study aims to introduce a new set of resources to stimulate the future development of neural text generation in Portuguese. In this work, we document the development of GigaVerbo, a concatenation of deduplicated Portuguese text corpora amounting to 200 billion tokens. Via this corpus, we trained a series of decoder-transformers named Tucano. Our models perform equal or superior to other Portuguese and multilingual language models of similar size in several Portuguese benchmarks. The evaluation of our models also reveals that model performance on many currently available benchmarks used by the Portuguese NLP community has little to no correlation with the scaling of token ingestion during training, highlighting the limitations of such evaluations when it comes to the assessment of Portuguese generative language models. All derivatives of our study are openly released on GitHub and Hugging Face. See https://nkluge-correa.github.io/Tucano/

Large language models (LLMs) have significantly advanced natural language processing, but their progress has yet to be equal across languages. While most LLMs are trained in high-resource languages like English, multilingual models generally underperform monolingual ones. Additionally, aspects of their multilingual foundation sometimes restrict the byproducts they produce, like computational demands and licensing regimes. In this study, we document the development of open-foundation models tailored for use in low-resource settings, their limitations, and their benefits. This is the TeenyTinyLlama pair: two compact models for Brazilian Portuguese text generation. We release them under the permissive Apache 2.0 license on GitHub and Hugging Face for community use and further development. See https://github.com/Nkluge-correa/TeenyTinyLlama, Comment: 21 pages, 5 figures

The long-range, parity-odd nucleon interaction generated by single pion exchange is captured in the parity-odd pion-nucleon coupling $h^1_\pi$. Its calculation in lattice QCD requires the evaluation of 4-quark operator nucleon 3-point functions. We investigate a new numerical approach to compute $h^1_\pi$ based on nucleon matrix elements of parity-even 4-quark operators and related to the parity-violating electro-weak theory by PCAC and chiral perturbation theory. This study is performed with 2+1+1 dynamical flavors of twisted mass fermions at pion mass $m_{\pi} \approx 260\,\text{MeV}$ in a lattice box of $L \approx 3 \,\text{fm} $ and with a lattice spacing of $a \approx 0.091 \,\text{fm}$. From a calculation excluding fermion loop diagrams we find a bare coupling of $h^1_\pi = 8.08 \,(98) \cdot 10^{-7}$.

Staple-shaped Wilson line operators are necessary for the study of transverse momentum-dependent parton distribution functions (TMDPDFs) in lattice QCD and beyond. In this work, we study the renormalization of such operators in the general case of an asymmetric staple. We analyze the mixing pattern of these operators using their symmetry properties, where we find that the possible mixing is restricted within groups of four operators. We then present numerical results using the regularization independent momentum subtraction (RI/MOM) scheme to study the importance of mixing using one operator in particular, the $\gamma_0$ operator. Based on these results, we consider the short distance ratio (SDR) scheme, which is desirable in the absence of mixing. Finally, we investigate a variant of the RI/MOM scheme, where the renormalization factors are computed at short distances., Comment: 20 pages, 14 figures

We present results for an exploratory lattice calculation of the leading parity-violating pion-nucleon coupling $h_\pi^1$. Based on the PCAC relation we use a parity-conserving Lagrangian and focus on the techniques to determine the nucleon matrix elements of the effective 4-quark operators. For our study we employ an ensemble of twisted mass fermions with $260$ MeV pion mass. Barring mixing with lower-dimensional operators and renormalization at this stage, we discuss our estimate for $h_\pi^1$., Comment: LATTICE2022 proceedings, revised coupling $h_\pi^1$ in Tabs. 3 and 4 and in corresponding Fig. 5

We present an exploratory investigation of the parity odd $\Delta I = 1$ pion-nucleon coupling $h_\pi^1$ from lattice QCD. Based on the PCAC relation, we study the parity-conserving effective Hamiltonian and extract the coupling by determining the nucleon mass splitting arising from the effective 4-quark interactions using the Feynman-Hellmann theorem. We present preliminary results of the mass shift for a $32^3 \times 64$ ensemble of $N_f = 2 + 1 + 1$ twisted mass fermions at pion mass $260$ MeV and lattice spacing $a = 0.097$ fm., Comment: LATTICE2021 proceedings