Search

Recently, there have been several high-profile achievements of agents learning to play games against humans and beat them. In this article, we study the problem of training intelligent agents in service of game development. Unlike the agents built to “beat the game,” our agents aim to produce human-like behavior to help with game evaluation and balancing. We discuss two fundamental metrics based on which we measure the human-likeness of agents, namely skill and style, which are multifaceted concepts with practical implications outlined in this article. We report four case studies in which the style and skill requirements inform the choice of algorithms and metrics used to train agents; ranging from A* search to state-of-the-art deep reinforcement learning (RL). Furthermore, we, show that the learning potential of state-of-the-art deep RL models does not seamlessly transfer from the benchmark environments to target ones without heavily tuning their hyperparameters, leading to linear scaling of the engineering efforts, and computational cost with the number of target domains.

Achieving highly detailed terrain models spanning vast areas is crucial to modern computer graphics. The pipeline for obtaining such terrains is via amplification of a low-resolution terrain to refine the details given a desired theme, which is a time-consuming and labor-intensive process. Recently, data-driven methods, such as the sparse construction tree, have provided a promising direction to equip the artist with better control over the theme. These methods learn to amplify terrain details by using an exemplar of high-resolution detailed terrains to transfer the theme. In this paper, we propose Generative Adversarial Terrain Amplification (GATA) that achieves better local/global coherence compared to the existing data-driven methods while providing even more ways to control the theme. GATA is comprised of two key ingredients. Thefi rst one is a novel embedding of themes into vectors of real numbers to achieve a single tool for multi-theme amplification. The theme component can leverage existing LIDAR data to generate similar terrain features. It can also generate newfi ctional themes by tuning the embedding vector or even encoding a new example terrain into an embedding. The second one is an adversarially trained model that, conditioned on an embedding and a low-resolution terrain, generates a high-resolution terrain adhering to the desired theme. The proposed integral approach reduces the need for unnecessary manual adjustments, can speed up the development, and brings the model quality to a new level. Our implementation of the proposed method has proved successful in large-scale terrain authoring for an open-world game.

Generating realistic urban environments by scattering or placing buildings on maps is a challenging problem. Unlike the existing procedural methods, we employ a data-driven approach to this problem. We combine two recent advances in machine learning techniques, Generative Latent optimization (GLO) together with adversarial training, to learn a model that can easily generate and place buildings on a given map. Such a model enables its users, particularly artists, to easily generate areas with specific styles, e.g. residential or commercial, just by providing examples. In contrast, traditional procedural methods require lengthy manual tuning of hyper-parameters. Using a more flexible method like ours allows artists to iterate over their designs of urban layouts much faster. Finally, our experiments on real-world data show that our method outperforms state-of-the-art methods in visual quality and can better match the underlying distribution of the building placements.

Training a bipedal character to play basketball and interact with objects, or a quadruped character to move in various locomotion modes, are difficult tasks due to the fast and complex contacts happening during the motion. In this paper, we propose a novel framework to learn fast and dynamic character interactions that involve multiple contacts between the body and an object, another character and the environment, from a rich, unstructured motion capture database. We use one-on-one basketball play and character interactions with the environment as examples. To achieve this task, we propose a novel feature called local motion phase, that can help neural networks to learn asynchronous movements of each bone and its interaction with external objects such as a ball or an environment. We also propose a novel generative scheme to reproduce a wide variation of movements from abstract control signals given by a gamepad, which can be useful for changing the style of the motion under the same context. Our scheme is useful for animating contact-rich, complex interactions for real-time applications such as computer games.

Competition is a primary driver of player satisfaction and engagement in multiplayer online games. Traditional matchmaking systems aim at creating matches involving teams of similar aggregated individual skill levels, such as Elo score or TrueSkill. However, team dynamics cannot be solely captured using such linear predictors. Recently, it has been shown that nonlinear predictors that target to learn probability of winning as a function of player and team features significantly outperforms these linear skill-based methods. In this paper, we show that using final score difference provides yet a better prediction metric for competitive balance. We also show that a linear model trained on a carefully selected set of team and individual features achieves almost the performance of the more powerful neural network model while offering two orders of magnitude inference speed improvement. This shows significant promise for implementation in online matchmaking systems., Comment: 2020 IEEE Conference in Games (COG 2020), 8 pages

The digital game industry has recently adopted recommender systems to deliver the most relevant content and suggest the most suitable activities to players. Because of diverse game designs and dynamic experiences, recommender systems typically operate in highly heterogeneous and time-sensitive environments. In this paper, we describe a recommender system at a digital game company which aims to provide recommendations for a large variety of use-cases while being easy to integrate and operate. The system leverages a unified data platform, standardized context and tracking data pipelines, robust naive linear contextual multi-armed bandit algorithms, and experimentation platform for extensibility as well as flexibility. Several games and applications have successfully launched with the recommender system and have achieved significant improvements.

The digital game industry has recently adopted recommendation systems to provide suitable game and content choices to players. Recommendations in digital games have several unique applications and challenges compared to other well known recommendation system such as those for movies and books. Designers must adopt different architectures and algorithms to overcome these challenges. In this talk, we describe the game recommendation system at Electronic Arts. It leverages heterogeneous player data across many games to provide intelligent recommendations. We discuss three example applications: recommending games for purchase, suitable game map, and game difficulty.Like the movie and book recommendation problem, one application is to recommend the next favorite games for a player. Digital games fall into a wide range of genres such as first player shooting (FPS), sports, and role-playing games (RPG). Games within the same genre however tend to be unique and creative. While the recommendation item space is smaller, the recommendation system should also manage different types of contents such as games and extra downloadable contents to play, editorial videos and tutorials to watch.The second application provides the game mode and map recommendations within a game to improve player experience. Many online digital games, especially FPS and sports games, contain different maps and game modes to provide diverse gameplay experience. Different maps and game modes often require different skill levels, strategies, or cooperation from players, and the maps and game modes are often played repeatedly. Therefore, recommending the most suitable map and game mode is important from smooth onboarding experience to retain players who are likely to churn. In the map and game mode recommendation application, the algorithms need to evaluate both the short-term actions as well as long term effects of playing different maps and game modes to optimize player's engagement.In addition, we also use the same recommendation system to adjust in-game configurations such as difficulty. Players have a wide variety of experiences, skills, learning rates, and playing styles, and will react differently to the same difficulty setting. Second, even for an individual player, one's difficulty preference may also change over time. For example, in a level progression game, a player who loses the first several attempts to one level might feel much less frustrated compared to losing after tens of unsuccessful trials. The difficulty recommendation provides suggestions and adjustments on game configuration based on the player's previous gameplay experience to maximize the engagement. For online multiplayer games, recommending partners and opponents in matchmaking is also an effective way to improve player experience.We developed one flexible recommendation system to satisfy the need of different applications and that executes data-driven algorithms such collaborative filtering and multi-armed bandit. The centralized system leverages entire player and game data for all recommendation applications in digital games, supports unified roll-out and update, and at the same time measures the performance together via A/B testing experiments. Moreover, the one system strategy is easy to generate consistent recommendations across multiple games and platforms. We tested these recommendation applications in EA website and games, an observed significant improvements in click-through-rate and engagement.

Video game crash events are characterized primarily by large media payloads and by highly bursty traffic patterns, with hundreds of thousands or millions of reports being issued in only a few minutes. These events are invaluable in quickly responding to game breaking issues that directly impact user experience. Even the slightest delay in capturing, processing and reporting these events can lead to user abandonment and significant financial cost.A traditional standalone RESTful service, backed by a vertically scaled SQL database is neither a reliable nor cost-effective solution to this problem. An architecture that decouples capture and persistence and uses a horizontally scalable NoSQL database is not only easier to provision, but also uses fewer cpu and memory resources to provide the same end to end latency and throughput.By replacing our RESTful implementation with one that takes advantage both of the aforementioned design and multi-tenant provisioning, we have reduced our dedicated cpu footprint by 63% and memory footprint by 59%. Additionally, we have decreased our data loss during spikes to essentially 0, maintained sub-second persistence latency and improved query latency in the average case by 54% with only a 3% sacrifice for worst case queries.

Matchmaking connects multiple players to participate in online player-versus-player games. Current matchmaking systems depend on a single core strategy: create fair games at all times. These systems pair similarly skilled players on the assumption that a fair game is best player experience. We will demonstrate, however, that this intuitive assumption sometimes fails and that matchmaking based on fairness is not optimal for engagement. In this paper, we propose an Engagement Optimized Matchmaking (EOMM) framework that maximizes overall player engagement. We prove that equal-skill based matchmaking is a special case of EOMM on a highly simplified assumption that rarely holds in reality. Our simulation on real data from a popular game made by Electronic Arts, Inc. (EA) supports our theoretical results, showing significant improvement in enhancing player engagement compared to existing matchmaking methods., WWW2017

Dynamic difficulty adjustment (DDA) is a technique for adaptively changing a game to make it easier or harder. A common paradigm to achieve DDA is through heuristic prediction and intervention, adjusting game difficulty once undesirable player states (e.g., boredom or frustration) are observed. Without quantitative objectives, it is impossible to optimize the strength of intervention and achieve the best effectiveness. In this paper, we propose a DDA framework with a global optimization objective of maximizing a player's engagement throughout the entire game. Using level-based games as our example, we model a player's progression as a probabilistic graph. Dynamic difficulty reduces to optimizing transition probabilities to maximize a player's stay time in the progression graph. We have successfully developed a system that applies this technique in multiple games by Electronic Arts, Inc., and have observed up to 9% improvement in player engagement with a neutral impact on monetization.

Physical data layout is a crucial factor in the performance of queries and updates in large data warehouses. Data layout enhances and complements other performance features such as materialized views and dynamic caching of aggregated results. Prior work has identified that the multidimensional nature of large data warehouses imposes natural restrictions on the query workload. A method based on a “uniform” query class approach has been proposed for data clustering and shown to be optimal. However, we believe that realistic query workloads will exhibit data access skew. For instance, if time is a dimension in the data model, then more queries are likely to focus on the most recent time interval. The query class approach does not adequately model the possibility of multidimensional data access skew. We propose the affinity graph model for capturing workload characteristics in the presence of access skew and describe an efficient algorithm for physical data layout. Our proposed algorithm considers declustering and load balancing issues which are inherent to the multidisk data layout problem. We demonstrate the validity of this approach experimentally.

Large organizations have a multitude of data sources across the enterprise and want to obtain business value from all of them. While the majority of these data sources may be consolidated in an enterprise data warehouse, many business units have their own data marts where analysis is carried out against data stored in multidimensional data structures. It is often critical to pose queries which span both these sources. This is a challenge since these sources have differing models and query languages (SQL vs MDX). The Siebel Analytics Server enables this requirement to be fulfilled. In this paper, we describe how the multidimensional metadata is modeled relationally within Siebel Analytics, efficient SQL to MDX translation algorithms and the conversion protocols required to convert a multidimensional result into a relational rowset.

In this paper we describe an ongoing research project called the Columbia Digital News System. The goal of this project is to develop a suite of effective interoperable tools with which people can find relevant information (text, images, video, and structured documents) from distributed sources and track it over a period of time. Our initial focus is on the development of a system with which researchers, journalists, and students can keep track of current news events in specific areas.

Datacubes compute aggregates of a set of database records at a variety of different granularities. For large datasets with many dimensions, the complete datacube may be very large. In order to support on-line access to datacube results, one would like to perform some precomputation to enhance query performance. Existing schemes materialize selected datacube tuples on disk, choosing the most beneficial cuboids (i.e., combinations of dimensions) to materialize given a space limit. However, in the context of a data-warehouse receiving frequent "append" updates to the database, the cost of keeping these disk-resident cuboids up-to-date can be high. In this paper, we propose a main memory based framework which provides rapid response to queries and requires considerably less maintenance cost than a disk based scheme in an append-only environment. We materialize in main memory (a) selected coarse-granularity tuples of the datacube, and (b) all tuples at the finest level of granularity of the cube. Our approach is limited to applications in which the finest granularity tuples of the datacube fit in main memory. We argue that there are important applications that meet this requirement. Further, as main memory sizes grow over the coming years, more and more applications will meet this requirement. For a given datacube query, we first look among our coarse-granularity tuples to see if we have a direct answer for the query. If so, that answer is returned directly. If not, we use a hash based scheme reminiscent of partial match retrieval to rapidly compute the answer to the query from the finest-level data without having to scan all of the base tuples. Our in-memory data structures allow for rapid updates in response to the appearance of new base data. We show how to choose which coarse-level tuples to precompute, and how to select partial-match keys to minimize the effects of skew. We present analytical and experimental results demonstrating the benefits of our approach.

Datacube queries compute aggregates over database relations at a variety of granularities. Often one wants only datacube output tuples whose aggregate value satisfies a certain condition, such as exceeding a given threshold. We develop algorithms for processing a datacube query using the selection condition internally during the computation. Thus, we can safely prune parts of the computation and end up with a more efficient computation of the answer Our first technique, called "specialization", uses the fact that a tuple in the datacube does not meet the given threshold to infer that all finer level aggregates cannot meet the threshold. Our second technique is called "generalization", and applies in the case where the actual value of the aggregate is not needed in the output, but used just to compare with the threshold. We demonstrate the efficiency of these techniques by implementing them within the sparse datacube algorithm of Ross and Srivastava. We present a performance study using synthetic and real-world data sets. Our results indicate substantial performance improvements for queries with selective conditions.

This chapter describes the progress of the Digital Government Research Center in tackling the challenges of integrating and accessing the massive amount of statistical and text data available from government agencies. In particular, we address the issues of database heterogeneity, size, distribution, and control of terminology. In this chapter we provide an overview of our results in addressing problems such as (1) ontological mappings for terminology standardization, (2) data integration across data bases with high speed query processing, and (3) interfaces for query input and presentation of results. The DGRC is a collaboration between researchers from Columbia University and the Information Sciences Institute of the University of Southern California employing technology developed at both locations, in particular the SENSUS ontology, the SIMS multi-database access planner, the LEXING automated dictionary and terminology analysis system, the main-memory query processing component and others. The pilot application targets gasoline data from the Bureau of Labor Statistics, the Energy Information Administration of the Department of Energy, the Census Bureau, and other government agencies.

In this paper we describe an ongoing research project called the Columbia Digital News Project. The goal of this project is to develop a suite of effective interoperable tools with which people can find relevant information (text, images, video, and structured documents) from distributed sources and track it over a period of time. Our initial focus is on the development of a system with which researchers, journalists, and students can keep track of current news events in specific areas.