Search

When creating a story, there is a valuable world-building process to be engaged with. This project presents the development of a Procedural Content Generator to aid in this process. This is done with the use of quantized LLM model, hosted in a local computer (with limited capacity), as an inference engine of our structure, based on Knowledge-Based Systems (KBS). We also make use of an ontology to represent the various elements of a story, and act as the permanence and knowledge of our KBS. The Procedural Content Generator obtained starts from an initial ontology, which represents a story in some point of it's course. From there it starts a looping structure in which the program has the ability to choose a sub-pipeline, which are the means through which the program expands on the story. It keeps adding content onto the ontology until the program considers that it sufficiently expanded the original ontology, after which it stops the execution. To achieve results gotten, we first made a series of experiments with the LLM to see the capacities and limitations of the model. This experiments include tasks of categorization in personalized simple categories and contextual content generation in relation to one or two pre-existing elements of a story, in between others. This document presents both the experiments done with the LLM to achieve useful results and the finally designed pipeline that would be developed., A l'hora de crear una història, hi ha un valuós procés de construcció del món en el què un es pot involucrar. Aquest projecte presenta el desenvolupament d'un Generador de Contingut Procedimental per ajudar en aquest procés. Això es fa amb l'ús d'un model LLM quantitzat, utilitzat en un ordinador local (amb capacitat limitada), com a motor d'inferència de la nostra estructura, que està basada en Sistemes Basats en el Coneixement (SBC). També fem servir una ontologia per representar els diversos elements d'una història i actuar com la permanència i coneixement del nostre SBC. El Generador de Contingut Procedimental obtingut parteix d'una ontologia inicial, que representa una història en algun moment del seu recorregut. Des d'aquí comença una estructura amb bucle en què el programa té la capacitat de triar una sub-linea, que són els mitjans a través dels quals el programa amplia la història. Aquesta continua afegint contingut a l'ontologia fins que el programa considera que ha expandit prou l'ontologia original, moment en què atura l'execució. Per obtenir els resultats, primer vam realitzar una sèrie d'experiments amb el LLM per veure les capacitats i limitacions del model. Aquests experiments inclouen tasques de categorització en categories simples personalitzades i generació de contingut contextual en relació amb un o dos elements preexistents d'una història, entre d'altres. Aquest document presenta tant els experiments realitzats amb el LLM per obtenir resultats com el disseny resultant del programa que es desenvolupa.

The adoption of algorithms based on Artificial Intelligence (AI) has been rapidly increasing during the last few years. However, some aspects of AI techniques are under heavy scrutiny. For instance, in many use cases, it is not clear whether the decisions of an algorithm are well informed and conforming to human understanding. Having ways to address these concerns is crucial in many domains, especially whenever humans and intelligent (physical or virtual) agents must cooperate in a shared environment. In this paper, we apply an explainability method based on the creation of a Policy Graph (PG) based on discrete predicates that represent and explain a trained agent’s behaviour in a multi-agent cooperative environment. We show that from these policy graphs, policies for surrogate interpretable agents can be automatically generated. These policies can be used to measure the reliability of the explanations enabled by the PGs through a fair behavioural comparison between the original opaque agent and the surrogate one. The contributions of this paper represent the first use case of policy graphs in the context of explaining agent behaviour in cooperative multi-agent scenarios and present experimental results that sets this kind of scenario apart from previous implementations in single-agent scenarios: when requiring cooperative behaviour, predicates that allow representing observations about the other agents are crucial to replicate the opaque agent’s behaviour and increase the reliability of explanations., This work has been partially supported by the H2020 knowlEdge European project (Grant agreement ID: 957331)., Peer Reviewed, Postprint (published version)

When integrating artificial agents into physical or digital environments that are shared with humans, agents are often equipped with opaque Machine Learning methods to enable adapting their behaviour to dynamic human needs and environment. This brings about agents that are also opaque and therefore hard to explain. In previous work, we show that we can reduce an opaque agent into an explainable Policy Graph (PG) which works accurately in multi-agent environments. Policy Graphs are based on a discretisation of the world into propositional logic to identify states, and the choice of which discretiser to apply is key to the performance of the reduced agent. In this work, we explore this further by 1) reducing a single agent into an explainable PG, and 2) enforcing collaboration between this agent and an agent trained from human behaviour. The human agent is computed by using GAIL from a series of human-played episodes, and kept unchanged. We show that an opaque agent created and trained to collaborate with the human agent can be reduced to an explainable, non-opaque PG, so long as predicates regarding collaboration are included in the state representation, by showing the difference in reward between the agent and its PG. Code is available at https://github.com/HPAI-BSC/explainable-agents-with-humans, This work has been partially supported by EU Horizon 2020 Project StairwAI (grant agreement No. 101017142)., Peer Reviewed, Postprint (published version)

Agent-Based Simulation is a suitable approach used now-a-days to simulate and analyze complex societal environments and scenarios. Current Agent-Based Simulation frameworks either scale quite well in computation but implement very simple reasoning mechanisms, or employ complex reasoning systems at the expense of scalability. In this paper we present our work to extend an agent-based HPC platform, enabling goal-driven agents with HTN planning capabilities to scale and run parallelly. Our extension includes preferences over their objectives, preferences over their plans, actions, and moral values. We show the expresiveness of the extended platform with a sample scenario., This work has been partially supported by EU Horizon 2020 Project StairwAI (grant agreement No.101017142)., Peer Reviewed, Postprint (published version)

The adoption of algorithms based on Artificial Intelligence (AI) has been rapidly increasing during the last years. However, some aspects of AI techniques are under heavy scrutiny. For instance, in many cases, it is not clear whether the decisions of an algorithm are well-informed and reliable. Having an answer to these concerns is crucial in many domains, such as those in were humans and intelligent agents must cooperate in a shared environment. In this paper, we introduce an application of an explainability method based on the creation of a Policy Graph (PG) based on discrete predicates that represent and explain a trained agent’s behaviour in a multi-agent cooperative environment. We also present a method to measure the similarity between the explanations obtained and the agent’s behaviour, by building an agent with a policy based on the PG and comparing the behaviour of the two agents., This work has been partially supported by the H2020 knowlEdge European project (Grant agreement ID: 957331)., Peer Reviewed, Postprint (published version)

Studying misinformation and how to deal with unhealthy behaviours within online discussions has recently become an important field of research within social studies. With the rapid development of social media, and the increasing amount of available information and sources, rigorous manual analysis of such discourses has become unfeasible. Many approaches tackle the issue by studying the semantic and syntactic properties of discussions following a supervised approach, for example using natural language processing on a dataset labeled for abusive, fake or bot-generated content. Solutions based on the existence of a ground truth are limited to those domains which may have ground truth. However, within the context of misinformation, it may be difficult or even impossible to assign labels to instances. In this context, we consider the use of temporal dynamic patterns as an indicator of discussion health. Working in a domain for which ground truth was unavailable at the time (early COVID-19 pandemic discussions) we explore the characterization of discussions based on the the volume and time of contributions. First we explore the types of discussions in an unsupervised manner, and then characterize these types using the concept of ephemerality, which we formalize. In the end, we discuss the potential use of our ephemerality definition for labeling online discourses based on how desirable, healthy and constructive they are., This work is supported by the scheme ‘INFRAIA-01-2018-2019: Research and Innovation action’, Grant Agreement n. 871042 ‘SoBigData++: European Integrated Infrastructure for Social Mining and Big Data Analytics’, Preprint

El modelado basado en agentes es una de las formas más apropiadas para simular y analizar problemas y simulaciones complejas, como la simulación de entornos y escenarios sociales. El tipo de plataforma que más se utiliza en estas tareas es la de un sistema multiagente. Los sistemas multiagente se componen de varios actores (agentes) en un entorno de simulación concreto, y cada uno de ellos posee un conocimiento y un comportamiento individual. Estos sistemas pueden utilizarse para analizar el comportamiento emergente colectivo en contextos como la sociología, la economía, la elaboración de políticas sociales y económicas, etc. Las plataformas multiagente actuales o bien escalan bastante bien en computación pero implementan mecanismos de razonamiento muy simples, o bien emplean sistemas de razonamiento complejos a costa de escalabilidad. En un trabajo reciente realizado en la UPC, se ha propuesto, teorizado e implementado una plataforma que permite escalar y ejecutar paralelamente agentes complejos con planificación HTN. Este proyecto amplía dicha plataforma para permitir un mejor análisis de las relaciones sociales entre los agentes mediante las preferencias sobre sus objetivos, las preferencias sobre sus planes, sus acciones y valores morales, a la vez que nos aseguramos de que nuestras adiciones sean escalables, para mantener el espíritu y el propósito de la plataforma. En este trabajo, partimos del trabajo previo realizado por Dmitry Gnatyshak sobre la implementación de dicha plataforma, y lo ampliamos, tanto formalmente como a nivel de implementación. Formalizamos las ampliaciones del modelo del sistema, así como sus modificaciones, y hacemos lo mismo con la implementación. Al final, proporcionamos un complejo escenario de ejemplo para mostrar todas las ampliaciones que hemos creado y/o añadido., Agent-based modeling is one of the most suitable ways to simulate and analyze complex problems and simulations, such as the simulation of societal environments and scenarios. The kind of platform most commonly used in these endeavors is that of a multi-agent system. Multi-agent systems are comprised of various actors (agents) in a concrete simulation environment, each of them possessing an individual knowledge and an individual behavior. These systems can be used to analyze collective emergent behavior in contexts such as sociology, economics, policy making, etc. Current Multi-agent platforms either scale in computation quite well but implement very simple reasoning mechanisms, or employ complex reasoning systems at the expense of scalability. In recent work done at UPC, a platform enabling complex agents with HTN planning to scale and run parallelly was proposed, theorized, and implemented. This project extends said platform to enable a better analysis of the social relationships between agents by means of preferences over their objectives, preferences over their plans, actions, and moral values, while making sure our additions are scalable, to maintain the spirit and purpose of the platform. In this work, we start from the previous work done by Dmitry Gnatyshak on implementing said platform, and we expand it, both formally and imple- mentation-wise. We formalize the additions to the model of the system, as well as its modifications, and we do the same for the implementation. In the end, we provide a complex example scenario to showcase all the additions we have created.

L'adopció d'algorismes basats en Intel·ligència Artificial (IA) ha augmentat ràpida\-ment durant els darrers anys. Tanmateix, alguns aspectes d'aquestes tècniques estan sota un escrutini intens. Per exemple, en molts casos, no és clar si les seves decisions estan ben fonamentades ni tampoc si hi podem confiar. Tenir una resposta a aquestes preocupacions és crucial en molts dominis, com aquells en què els humans i els agents intel·ligents han de cooperar en un entorn compartit. En aquest article us presentem una aplicació d'un mètode d'explicabilitat basat en la creació d'un Policy Graph. Aquest graf es construeix fent servir predicats discrets que representen i expliquen el comportament d'un agent entrenat en un entorn cooperatiu amb múltiples agents. També presentem un mètode per mesurar la semblança entre les explicacions obtingudes i el comportament de l'agent. Aquest mètode consisteix a construir un nou agent amb una política basada en el PG, per després poder comparar el seu comportament amb el de l'agent original., The adoption of algorithms based on Artificial Intelligence (AI) has been rapidly increasing during the last years. However, some aspects of AI techniques are under heavy scrutiny. For instance, in many cases, it is not clear whether their decisions are well-informed or whether we can trust them. Having an answer to these concerns is crucial in many domains, such as those in were humans and intelligent agents must cooperate in a shared environment. In this paper, we introduce an application of an explainability method based on the creation of a Policy Graph (PG). This graph is built using discrete predicates that represent and explain the behaviour of a trained agent in a cooperative environment with multiple agents. We also present a method to measure the similarity between the explanations obtained and the behaviour of the agent. This method consists of building a new agent with a policy based on the PG, in order to compare its behaviour with that of the original agent.

En este trabajo se realiza una búsqueda del estado del arte en entornos multiagente para la técnica de aprendizaje Reinforcement Learning. Más adelante, se filtran los entornos encontrados y se escoge uno para su adaptación, incorporando elementos sociales al nuevo entorno. Se explica detalladamente el proceso de adaptación, sus etapas y las diversas decisiones tomadas en su desarrollo. Para comprobar que el entorno es funcional, se realiza el entrenamiento de varios agentes en el entorno. Finalmente se analizan los resultados obtenidos y se plantea trabajo futuro para este entorno.

Rocket League es un popular juego en el que los jugadores pueden competir entre sí o contra bots. Sin embargo, desde un punto de vista competitivo, todavía hay espacio para mejorar la calidad de las capacidades de los bots. Es posible desarrollar, utilizando el aprendizaje por refuerzo, un bot competitivo de Rocket League capaz de competir contra jugadores humanos y otros bots tradicionales. El objetivo de este proyecto es tanto hacer un estudio teórico de los fundamentos del aprendizaje por refuerzo como una solución práctica para un problema real: la necesidad de mejores bots de Rocket League. La parte teórica recorrerá problemas de toma de decisiones, para luego explicar los principales métodos de aprendizaje por refuerzo diseñados para resolverlos. En primer lugar, se mostrarán métodos sencillos, capaces de resolver pequeños problemas. Estos problemas simples apenas existen en la vida real, por lo que estos métodos sirven como base teórica y como fundamento para métodos más complejos necesarios para resolver problemas reales. Finalmente, estos métodos más complejos serán presentados y analizados. Para la parte práctica, el objetivo es desarrollar un bot para Rocket League mediante aprendizaje por refuerzo, capaz de competir contra jugadores humanos con experiencia en el juego y mejorar el rendimiento de los bots que vienen con el juego. En primer lugar, se mostrará una breve descripción de los conceptos a tener en cuenta a la hora de desarrollar una solución de aprendizaje por refuerzo. Luego, se presentará la implementación de la solución, brindando a los lectores una guía sobre cómo desarrollar sus propios proyectos de aprendizaje por refuerzo. Además, también se describe la planificación del proyecto, mostrando tanto los recursos humanos como materiales, y cómo se distribuyen en el tiempo. La planificación incluye un presupuesto basado en esos recursos. Este texto tiene como objetivo ayudar a los lectores a comprender mejor los conceptos básicos del aprendiza, Rocket League is a popular game where players can compete against each other or against bots. However, from a competitive point of view, there is still room for improvement on the quality of the bots' capabilities. It is possible to develop, using reinforcement learning, a competitive Rocket League bot able to compete against human players and other traditional bots. The goal of this project is to both make a theoretical study of the foundations of reinforcement learning and a practical solution for a real problem: the need for better Rocket League bots. The theoretical part will go through decision-making problems, to then explain the main reinforcement learning methods designed to solve them. First, simple methods will be shown, capable of solving small problems. These simple problems hardly exist in real life, so these methods serve as a theoretical background and as a foundation for more complex methods needed to solve real problems. Finally, these more complex methods will be presented and analyzed. For the practical part, the goal is to develop a bot for Rocket League, using reinforcement learning, capable of competing against human players that are experienced in the game and improving the performance of the game stock bots. First, a brief description will be shown of the concepts to take into account when developing a reinforcement learning solution. Then, the implementation of the solution will be presented, giving readers a guideline of how to develop their own reinforcement learning projects. Additionally, the project planning is also described, showing both the human and material resources, and how they are distributed over time. The planning includes a budget based on those resources. This text aims to help people better understand the basics of reinforcement learning, theoretically and practically, so they get the courage to start their own projects.

La inteligencia artificial es una de las áreas de la computación que se encuentra actualmente en auge, este trabajo se encuentra bajo el marco del aprendizaje por refuerzo. El aprendizaje por refuerzo se centra en la resolución de problemas conocidos como escenarios mediante la creación de modelos de inteligencia artificial llamados agentes. Los objetivos de esto es maximizar el valor que nos retorna una función que una vez el agente ha realizado una acción en el escenario. Generalmente todos estos agentes han sido creados mediante el uso de redes neuronales. Al tener entrenada una red neuronal, desde fuera puede parecer como una caja negra ya que no sabemos porqué o cómo está solucionando el problema debido a su complejidad. La explicabilidad son un conjunto técnicas que se encargan de arrojar luz a esta caja intentando así dar explicaciones del porqué de su funcionamiento. En este trabajo se aplican este tipo de técnicas en el videojuego ViZDoom, en el cual se encuentran escenarios complejos de resolver., Since the beginning of artificial intelligence as a field it has shown humanity how useful it has been to solve complex problems which there were no feasible algorithms for. One of the main problems is that once we have a system trained is like a black box. Explainability tries to cast light on those black boxes and explain how they are working or the case of Reinforcement Learning explains how they behave. In this project we will use some explainability techniques in the ViZDoom game which contains complex environments.

The advancement on explainability techniques is quite relevant in the field of Reinforcement Learning (RL) and its applications can be beneficial for the development of intelligent agents that are understandable by humans and are able cooperate with them. When dealing with Deep RL some approaches already exist in the literature, but a common problem is that it can be tricky to define whether the explanations generated for an agent really reflect the behaviour of the trained agent. In this work we will apply an approach for explainability based on the creation of a Policy Graph (PG) that represents the agent’s behaviour. Our main contribution is a way to measure the similarity between the explanations and the agent’s behaviour, by building another agent that follows a policy based on the explainability method and comparing the behaviour of both agents., Peer Reviewed, Postprint (published version)

La popularidad de los métodos explicativos está aumentando en el contexto de la Inteligencia Artificial y consiste en dar explicaciones inteligibles a modelos complejos. Recientemente, en el contexto del Aprendizaje Reforzado (RL) han aparecido muchos trabajos teóricos que intentan explicar el comportamiento de agentes entrenados, con predicados y usando representaciones del comportamiento observado en forma de grafo MDP. Nosotros aplicaremos este método a un entorno de RL y crearemos una política que intentará demostrar su correcto funcionamiento., The popularity of explainability methods is increasing in the Artificial Intelligence context and consists of giving intelligible explanations to complex models. Recently, in the Reinforcement Learning (RL) context several theoretical works have appeared that try to explain the behavior of trained agents, with predicates and using representations of the observed behavior in a graph form (MDP). We will apply this method to an RL environment and create a policy that will try to prove its correct functioning.

Currently, agent-based simulation frameworks force the user to choose between simulations involving a large number of agents (at the expense of limited agent reasoning capability) or simulations including agents with increased reasoning capabilities (at the expense of a limited number of agents per simulation). This paper describes a first attempt at putting goal-oriented agents into large agentbased (micro-)simulations. We discuss a model for goal-oriented agents in HighPerformance Computing (HPC) and then briefly discuss its implementation in PyCOMPSs (a library that eases the parallelisation of tasks) to build such a platform that benefits from a large number of agents with the capacity to execute complex cognitive agents., Peer Reviewed, Postprint (author's final draft)