Search

Despite different applications, transformer-based language models, like BERT and GPT, learn about language by predicting missing parts of text. BERT is pretrained in Masked Language Modelling and GPT generates text from a given sequence. We explore such models for answering cloze questions in Portuguese, following different approaches. When options are not considered, the largest BERT model, trained exclusively for Portuguese, is the most accurate. But when selecting the best option, top performance is achieved by computing the most probable sentence, and GPT-2 fine-tuned for Portuguese beats BERT.

Word2vec-like word embeddings are known for keeping linguistic regularities and thus good for solving analogies. Following this, we explore such embeddings for Portuguese in the discovery of lexical-semantic relations, which can be used for augmenting lexical-semantic knowledge bases. In this exploratory approach, we tested different methods for discovering relations of different types and confirm that word embeddings can be used, at least, for suggesting new candidate relations.

We describe an IR-based dialogue system that, in order to match user interactions with FAQs on a list, leverages on a model for computing the semantic similarity between two fragments of Portuguese text. It was mainly used for answering questions about the economic activity in Portugal and, when no FAQ has a higher score than a threshold, it may search for similar interactions in a corpus of movie subtitles and still tries to give a suitable response. Besides describing the underlying model and its integration, we assess it when answering variations of FAQs and report on an experiment to set the aforementioned threshold.

This paper offers a brief overview on the ASSIN 2, an evaluation shared task collocated with STIL 2019. ASSIN 2 covered two different but related tasks: Recognizing Textual Entailment (RTE), also known as Natural Language Inference (NLI), and Semantic Textual Similarity (STS). The ASSIN 2 collection was made of pairs of sentences annotated with human judgments for NLI and STS. Participating teams could take part in any of the tasks or both: nine teams participated in the STS task and eight in the NLI task.

New forms of communication, like emojis, are frequent today in social media. Having in mind their strong connection with expressed emotions, we exploit emojis towards the creation of a model for emotion recognition in Portuguese. We gather short texts from Twitter and follow a traditional text classification task, where emojis are used as labels. After the process of feature engineering, two types of Naive Bayes and SVM classifiers are trained: one for classifying emotion, based on related emojis; another for predicting emojis. Interesting but debatable results were obtained on the former task, while the latter revealed to be more challenging, mainly due to emoji similarity. Yet, this also suggests that we can rely on them as an alternative to manually labelling emotions.

Within the domain of Artificial Intelligence, humor has been a research topic for some time, but the automatic recognition of its verbal expression has never been tackled for Portuguese. This work aims to change this scenario. We describe a set of experiments towards the development of computational models that recognize humor written in Portuguese, based on content and humor-specific features extracted. Interesting results, with F1-scores up to 0.93, are achieved when classifiers for this purpose are trained and tested on texts with a similar style (question-answers or news headlines). Yet, when the testing examples are of a different style, results are poor, which suggests that much more has to be done towards effective humor recognition.

Automatic recognition of named entities from clinical text lightens the work of health professionals by helping in the interpretation and easing tasks such as the population of databases with patient health information. In this study, we evaluated the performance of Conditional Random Fields, a sequence labelling model, for extracting entities from neurology clinical texts written in Portuguese. More than achieving F1-scores of about 73% or 80%, respectively for a relaxed or strict evaluation, the more discriminant features in this task were also analyzed.

This paper describes the creation of PT-LKB, new Portuguese word embeddings learned from a large lexical-semantic knowledge base (LKB), using the node2vec method. Resulting embeddings combine the strengths of word vector representations and, even with lower dimensions, achieve high scores in genuine similarity, which so far were obtained by exploiting the graph structure of LKBs.

This paper presents several approaches for computing word similarity in Portuguese and is motivated by the recent availability of state-of-the-art distributional models of Portuguese words, which add to several lexical knowledge bases (LKBs) for this language, available for a longer time. The previous resources were exploited to answer word similarity tests, also recently available for Portuguese. We conclude that there are several valid approaches for this task, but not one that outperforms all the others in every single test. For instance, distributional models seem to capture relatedness better, but LKBs are better suited for computing genuine similarity.

There is much research on Semantic Textual Similarity (STS) in English, specially since its inclusion in the SemEval evaluations. For other languages, it is not as common, mostly due to the unavailability of benchmarks. Recently, the ASSIN shared task targeted STS in Portuguese and released training and test collections. This paper describes an incremental approach to ASSIN, where the computed similarity is gradually improved by exploiting different features (e.g., token overlap, semantic relations, chunks, and negation) and approaches. The best reported results, obtained with a supervised approach, would get second place overall in ASSIN.

This paper presents MemeGera, a prototype tool that generates image-based memes from Portuguese news headlines. All is done automatically, with the help of computational linguistic resources, uncovered here with the rules for selecting images and adapting the text.

This paper revisits PoeTryMe, a poetry generation platform, and presents its most recent instantiation for producing poetry inspired by trends in the Twitter social network. The presented system searches for tweets that mention a given topic, extracts the most frequent words in those tweets, and uses them as seeds for the generation of new poems. The set of seeds might still be expanded with semantically-relevant words. Generation is performed by the classic PoeTryMe system, based on a semantic network and a grammar, with a previously used generate&test strategy. Illustrative results are presented using different seed expansion settings. They show that the produced poems use semantically-coherent lines with words that, at the time of generation, were associated with the topic. Resulting poems are not really about the topic, but they are a way of expressing, poetically, what the system knows about the semantic domain set by the topic.

There are several lexical resources available for the computational processing of Portuguese, organised differently and created by different people with different approaches and limitations. This paper presents the first experiments towards the exploitation of seven of those resources in the automatic creation of a large wordnet, where numerical scores are assigned to the inclusion of words in synsets and to the connection of synsets by semantic relations. Experiments confirm that a large wordnet can indeed be created and, to some extent, computed scores can be used as a confidence measure, which will enable the users to select only a portion of the resource, depending on the needs of their application on quantity and quality of lexical-semantic knowledge.

Although typically used in classic topic models, surface words cannot represent meaning on their own. Consequently, redundancy is common in those topics, which may, for instance, include synonyms. To face this problem, we present SemLDA, an extended topic model that incorporates semantics from an external lexical-semantic knowledge base. SemLDA is introduced and explained in detail, pointing out where semantics is included both in the pre-pocessing and generative phase of topic distributions. As a result, instead of topics as distributions over words, we obtain distributions over concepts, each represented by a set of synonymous words. In order to evaluate SemLDA, we applied preliminary qualitative tests automatically against a state-of-the-art classical topic model. The results were promising and confirm our intuition towards the benefits of incorporating general semantics in a topic model.

This paper discusses the utility of term-based lexical networks, with focus on Portuguese. Although less complex and often undervalued towards wordnets, for Portuguese, these resources have been used in varied natural language processing tasks. We enumerate those, and then introduce a larger resource of this kind together with two additional tasks where it was useful.

A wordnet is an important tool for developing natural language processing applications for a language, but the manual creation of such a resource limits its development. This dissertation studied the automatic construction of Onto.PT, a large Portuguese wordnet, aiming to minimise the main limitations of existing Portuguese wordnets. On this context, we propose ECO, an approach for creating wordnets automatically from text – relation instances are extracted, synonymy clusters (synsets) are discovered, and the remaining relations are then attached to suitable synsets. This document also reports on the contents of Onto.PT, its comparison to other wordnets, and its evaluation.