Search

Agent-Based Simulation is a suitable approach used now-a-days to simulate and analyze complex societal environments and scenarios. Current Agent-Based Simulation frameworks either scale quite well in computation but implement very simple reasoning mechanisms, or employ complex reasoning systems at the expense of scalability. In this paper we present our work to extend an agent-based HPC platform, enabling goal-driven agents with HTN planning capabilities to scale and run parallelly. Our extension includes preferences over their objectives, preferences over their plans, actions, and moral values. We show the expresiveness of the extended platform with a sample scenario. This work has been partially supported by EU Horizon 2020 Project StairwAI (grant agreement No.101017142).

When integrating artificial agents into physical or digital environments that are shared with humans, agents are often equipped with opaque Machine Learning methods to enable adapting their behaviour to dynamic human needs and environment. This brings about agents that are also opaque and therefore hard to explain. In previous work, we show that we can reduce an opaque agent into an explainable Policy Graph (PG) which works accurately in multi-agent environments. Policy Graphs are based on a discretisation of the world into propositional logic to identify states, and the choice of which discretiser to apply is key to the performance of the reduced agent. In this work, we explore this further by 1) reducing a single agent into an explainable PG, and 2) enforcing collaboration between this agent and an agent trained from human behaviour. The human agent is computed by using GAIL from a series of human-played episodes, and kept unchanged. We show that an opaque agent created and trained to collaborate with the human agent can be reduced to an explainable, non-opaque PG, so long as predicates regarding collaboration are included in the state representation, by showing the difference in reward between the agent and its PG. Code is available at https://github.com/HPAI-BSC/explainable-agents-with-humans This work has been partially supported by EU Horizon 2020 Project StairwAI (grant agreement No. 101017142).

Studying misinformation and how to deal with unhealthy behaviours within online discussions has recently become an important field of research within social studies. With the rapid development of social media, and the increasing amount of available information and sources, rigorous manual analysis of such discourses has become unfeasible. Many approaches tackle the issue by studying the semantic and syntactic properties of discussions following a supervised approach, for example using natural language processing on a dataset labeled for abusive, fake or bot-generated content. Solutions based on the existence of a ground truth are limited to those domains which may have ground truth. However, within the context of misinformation, it may be difficult or even impossible to assign labels to instances. In this context, we consider the use of temporal dynamic patterns as an indicator of discussion health. Working in a domain for which ground truth was unavailable at the time (early COVID-19 pandemic discussions) we explore the characterization of discussions based on the the volume and time of contributions. First we explore the types of discussions in an unsupervised manner, and then characterize these types using the concept of ephemerality, which we formalize. In the end, we discuss the potential use of our ephemerality definition for labeling online discourses based on how desirable, healthy and constructive they are. This work is supported by the scheme ‘INFRAIA-01-2018-2019: Research and Innovation action’, Grant Agreement n. 871042 ‘SoBigData++: European Integrated Infrastructure for Social Mining and Big Data Analytics’

Studying misinformation and how to deal with unhealthy behaviours within online discussions has recently become an important field of research within social studies. With the rapid development of social media, and the increasing amount of available information and sources, rigorous manual analysis of such discourses has become unfeasible. Many approaches tackle the issue by studying the semantic and syntactic properties of discussions following a supervised approach, for example using natural language processing on a dataset labeled for abusive, fake or bot-generated content. Solutions based on the existence of a ground truth are limited to those domains which may have ground truth. However, within the context of misinformation, it may be difficult or even impossible to assign labels to instances. In this context, we consider the use of temporal dynamic patterns as an indicator of discussion health. Working in a domain for which ground truth was unavailable at the time (early COVID-19 pandemic discussions) we explore the characterization of discussions based on the the volume and time of contributions. First we explore the types of discussions in an unsupervised manner, and then characterize these types using the concept of ephemerality, which we formalize. In the end, we discuss the potential use of our ephemerality definition for labeling online discourses based on how desirable, healthy and constructive they are., Comment: 15 pages. Related to the SoBigData++ project: https://plusplus.sobigdata.eu/

Currently, agent-based simulation frameworks force the user to choose between simulations involving a large number of agents (at the expense of limited agent reasoning capability) or simulations including agents with increased reasoning capabilities (at the expense of a limited number of agents per simulation). This paper describes a first attempt at putting goal-oriented agents into large agentbased (micro-)simulations. We discuss a model for goal-oriented agents in HighPerformance Computing (HPC) and then briefly discuss its implementation in PyCOMPSs (a library that eases the parallelisation of tasks) to build such a platform that benefits from a large number of agents with the capacity to execute complex cognitive agents.

Multimodal clustering is an unsupervised technique for mining interesting patterns in $n$-adic binary relations or $n$-mode networks. Among different types of such generalized patterns one can find biclusters and formal concepts (maximal bicliques) for 2-mode case, triclusters and triconcepts for 3-mode case, closed $n$-sets for $n$-mode case, etc. Object-attribute biclustering (OA-biclustering) for mining large binary datatables (formal contexts or 2-mode networks) arose by the end of the last decade due to intractability of computation problems related to formal concepts; this type of patterns was proposed as a meaningful and scalable approximation of formal concepts. In this paper, our aim is to present recent advance in OA-biclustering and its extensions to mining multi-mode communities in SNA setting. We also discuss connection between clustering coefficients known in SNA community for 1-mode and 2-mode networks and OA-bicluster density, the main quality measure of an OA-bicluster. Our experiments with 2-, 3-, and 4-mode large real-world networks show that this type of patterns is suitable for community detection in multi-mode cases within reasonable time even though the number of corresponding $n$-cliques is still unknown due to computation difficulties. An interpretation of OA-biclusters for 1-mode networks is provided as well.

El modelado basado en agentes es una de las formas más apropiadas para simular y analizar problemas y simulaciones complejas, como la simulación de entornos y escenarios sociales. El tipo de plataforma que más se utiliza en estas tareas es la de un sistema multiagente. Los sistemas multiagente se componen de varios actores (agentes) en un entorno de simulación concreto, y cada uno de ellos posee un conocimiento y un comportamiento individual. Estos sistemas pueden utilizarse para analizar el comportamiento emergente colectivo en contextos como la sociología, la economía, la elaboración de políticas sociales y económicas, etc. Las plataformas multiagente actuales o bien escalan bastante bien en computación pero implementan mecanismos de razonamiento muy simples, o bien emplean sistemas de razonamiento complejos a costa de escalabilidad. En un trabajo reciente realizado en la UPC, se ha propuesto, teorizado e implementado una plataforma que permite escalar y ejecutar paralelamente agentes complejos con planificación HTN. Este proyecto amplía dicha plataforma para permitir un mejor análisis de las relaciones sociales entre los agentes mediante las preferencias sobre sus objetivos, las preferencias sobre sus planes, sus acciones y valores morales, a la vez que nos aseguramos de que nuestras adiciones sean escalables, para mantener el espíritu y el propósito de la plataforma. En este trabajo, partimos del trabajo previo realizado por Dmitry Gnatyshak sobre la implementación de dicha plataforma, y lo ampliamos, tanto formalmente como a nivel de implementación. Formalizamos las ampliaciones del modelo del sistema, así como sus modificaciones, y hacemos lo mismo con la implementación. Al final, proporcionamos un complejo escenario de ejemplo para mostrar todas las ampliaciones que hemos creado y/o añadido. Agent-based modeling is one of the most suitable ways to simulate and analyze complex problems and simulations, such as the simulation of societal environments and scenarios. The kind of platform most commonly used in these endeavors is that of a multi-agent system. Multi-agent systems are comprised of various actors (agents) in a concrete simulation environment, each of them possessing an individual knowledge and an individual behavior. These systems can be used to analyze collective emergent behavior in contexts such as sociology, economics, policy making, etc. Current Multi-agent platforms either scale in computation quite well but implement very simple reasoning mechanisms, or employ complex reasoning systems at the expense of scalability. In recent work done at UPC, a platform enabling complex agents with HTN planning to scale and run parallelly was proposed, theorized, and implemented. This project extends said platform to enable a better analysis of the social relationships between agents by means of preferences over their objectives, preferences over their plans, actions, and moral values, while making sure our additions are scalable, to maintain the spirit and purpose of the platform. In this work, we start from the previous work done by Dmitry Gnatyshak on implementing said platform, and we expand it, both formally and imple- mentation-wise. We formalize the additions to the model of the system, as well as its modifications, and we do the same for the implementation. In the end, we provide a complex example scenario to showcase all the additions we have created.

En este trabajo se realiza una búsqueda del estado del arte en entornos multiagente para la técnica de aprendizaje Reinforcement Learning. Más adelante, se filtran los entornos encontrados y se escoge uno para su adaptación, incorporando elementos sociales al nuevo entorno. Se explica detalladamente el proceso de adaptación, sus etapas y las diversas decisiones tomadas en su desarrollo. Para comprobar que el entorno es funcional, se realiza el entrenamiento de varios agentes en el entorno. Finalmente se analizan los resultados obtenidos y se plantea trabajo futuro para este entorno.

L'adopció d'algorismes basats en Intel·ligència Artificial (IA) ha augmentat ràpida\-ment durant els darrers anys. Tanmateix, alguns aspectes d'aquestes tècniques estan sota un escrutini intens. Per exemple, en molts casos, no és clar si les seves decisions estan ben fonamentades ni tampoc si hi podem confiar. Tenir una resposta a aquestes preocupacions és crucial en molts dominis, com aquells en què els humans i els agents intel·ligents han de cooperar en un entorn compartit. En aquest article us presentem una aplicació d'un mètode d'explicabilitat basat en la creació d'un Policy Graph. Aquest graf es construeix fent servir predicats discrets que representen i expliquen el comportament d'un agent entrenat en un entorn cooperatiu amb múltiples agents. També presentem un mètode per mesurar la semblança entre les explicacions obtingudes i el comportament de l'agent. Aquest mètode consisteix a construir un nou agent amb una política basada en el PG, per després poder comparar el seu comportament amb el de l'agent original. The adoption of algorithms based on Artificial Intelligence (AI) has been rapidly increasing during the last years. However, some aspects of AI techniques are under heavy scrutiny. For instance, in many cases, it is not clear whether their decisions are well-informed or whether we can trust them. Having an answer to these concerns is crucial in many domains, such as those in were humans and intelligent agents must cooperate in a shared environment. In this paper, we introduce an application of an explainability method based on the creation of a Policy Graph (PG). This graph is built using discrete predicates that represent and explain the behaviour of a trained agent in a cooperative environment with multiple agents. We also present a method to measure the similarity between the explanations obtained and the behaviour of the agent. This method consists of building a new agent with a policy based on the PG, in order to compare its behaviour with that of the original agent.

Rocket League es un popular juego en el que los jugadores pueden competir entre sí o contra bots. Sin embargo, desde un punto de vista competitivo, todavía hay espacio para mejorar la calidad de las capacidades de los bots. Es posible desarrollar, utilizando el aprendizaje por refuerzo, un bot competitivo de Rocket League capaz de competir contra jugadores humanos y otros bots tradicionales. El objetivo de este proyecto es tanto hacer un estudio teórico de los fundamentos del aprendizaje por refuerzo como una solución práctica para un problema real: la necesidad de mejores bots de Rocket League. La parte teórica recorrerá problemas de toma de decisiones, para luego explicar los principales métodos de aprendizaje por refuerzo diseñados para resolverlos. En primer lugar, se mostrarán métodos sencillos, capaces de resolver pequeños problemas. Estos problemas simples apenas existen en la vida real, por lo que estos métodos sirven como base teórica y como fundamento para métodos más complejos necesarios para resolver problemas reales. Finalmente, estos métodos más complejos serán presentados y analizados. Para la parte práctica, el objetivo es desarrollar un bot para Rocket League mediante aprendizaje por refuerzo, capaz de competir contra jugadores humanos con experiencia en el juego y mejorar el rendimiento de los bots que vienen con el juego. En primer lugar, se mostrará una breve descripción de los conceptos a tener en cuenta a la hora de desarrollar una solución de aprendizaje por refuerzo. Luego, se presentará la implementación de la solución, brindando a los lectores una guía sobre cómo desarrollar sus propios proyectos de aprendizaje por refuerzo. Además, también se describe la planificación del proyecto, mostrando tanto los recursos humanos como materiales, y cómo se distribuyen en el tiempo. La planificación incluye un presupuesto basado en esos recursos. Este texto tiene como objetivo ayudar a los lectores a comprender mejor los conceptos básicos del aprendizaje por refuerzo, tanto en la teoría como en la práctica, para que se animen a comenzar sus propios proyectos. Rocket League is a popular game where players can compete against each other or against bots. However, from a competitive point of view, there is still room for improvement on the quality of the bots' capabilities. It is possible to develop, using reinforcement learning, a competitive Rocket League bot able to compete against human players and other traditional bots. The goal of this project is to both make a theoretical study of the foundations of reinforcement learning and a practical solution for a real problem: the need for better Rocket League bots. The theoretical part will go through decision-making problems, to then explain the main reinforcement learning methods designed to solve them. First, simple methods will be shown, capable of solving small problems. These simple problems hardly exist in real life, so these methods serve as a theoretical background and as a foundation for more complex methods needed to solve real problems. Finally, these more complex methods will be presented and analyzed. For the practical part, the goal is to develop a bot for Rocket League, using reinforcement learning, capable of competing against human players that are experienced in the game and improving the performance of the game stock bots. First, a brief description will be shown of the concepts to take into account when developing a reinforcement learning solution. Then, the implementation of the solution will be presented, giving readers a guideline of how to develop their own reinforcement learning projects. Additionally, the project planning is also described, showing both the human and material resources, and how they are distributed over time. The planning includes a budget based on those resources. This text aims to help people better understand the basics of reinforcement learning, theoretically and practically, so they get the courage to start their own projects.

La inteligencia artificial es una de las áreas de la computación que se encuentra actualmente en auge, este trabajo se encuentra bajo el marco del aprendizaje por refuerzo. El aprendizaje por refuerzo se centra en la resolución de problemas conocidos como escenarios mediante la creación de modelos de inteligencia artificial llamados agentes. Los objetivos de esto es maximizar el valor que nos retorna una función que una vez el agente ha realizado una acción en el escenario. Generalmente todos estos agentes han sido creados mediante el uso de redes neuronales. Al tener entrenada una red neuronal, desde fuera puede parecer como una caja negra ya que no sabemos porqué o cómo está solucionando el problema debido a su complejidad. La explicabilidad son un conjunto técnicas que se encargan de arrojar luz a esta caja intentando así dar explicaciones del porqué de su funcionamiento. En este trabajo se aplican este tipo de técnicas en el videojuego ViZDoom, en el cual se encuentran escenarios complejos de resolver. Since the beginning of artificial intelligence as a field it has shown humanity how useful it has been to solve complex problems which there were no feasible algorithms for. One of the main problems is that once we have a system trained is like a black box. Explainability tries to cast light on those black boxes and explain how they are working or the case of Reinforcement Learning explains how they behave. In this project we will use some explainability techniques in the ViZDoom game which contains complex environments.

La popularidad de los métodos explicativos está aumentando en el contexto de la Inteligencia Artificial y consiste en dar explicaciones inteligibles a modelos complejos. Recientemente, en el contexto del Aprendizaje Reforzado (RL) han aparecido muchos trabajos teóricos que intentan explicar el comportamiento de agentes entrenados, con predicados y usando representaciones del comportamiento observado en forma de grafo MDP. Nosotros aplicaremos este método a un entorno de RL y crearemos una política que intentará demostrar su correcto funcionamiento. The popularity of explainability methods is increasing in the Artificial Intelligence context and consists of giving intelligible explanations to complex models. Recently, in the Reinforcement Learning (RL) context several theoretical works have appeared that try to explain the behavior of trained agents, with predicates and using representations of the observed behavior in a graph form (MDP). We will apply this method to an RL environment and create a policy that will try to prove its correct functioning.