Search

The interaction of multiple autonomous agents gives rise to highly dynamic and nondeterministic environments, contributing to the complexity in applications such as automated financial markets, smart grids, or robotics. Due to the sheer number of situations that may arise, it is not possible to foresee and program the optimal behaviour for all agents beforehand. Consequently, it becomes essential for the success of the system that the agents can learn their optimal behaviour and adapt to new situations or circumstances. The past two decades have seen the emergence of reinforcement learning, both in single and multi-agent settings, as a strong, robust and adaptive learning paradigm. Progress has been substantial, and a wide range of algorithms are now available. An important challenge in the domain of multi-agent learning is to gain qualitative insights into the resulting system dynamics. In the past decade, tools and methods from evolutionary game theory have been successfully employed to study multi-agent learning dynamics formally in strategic interactions. This article surveys the dynamical models that have been derived for various multi-agent reinforcement learning algorithms, making it possible to study and compare them qualitatively. Furthermore, new learning algorithms that have been introduced using these evolutionary game theoretic tools are reviewed. The evolutionary models can be used to study complex strategic interactions. Examples of such analysis are given for the domains of automated trading in stock markets and collision avoidance in multi-robot systems. The paper provides a roadmap on the progress that has been achieved in analysing the evolutionary dynamics of multi-agent learning by highlighting the main results and accomplishments.

This article tackles the Distribution Network Expansion Planning (DNEP) problem that has to be solved by distribution network operators to decide which, where, and/or when enhancements to electricity networks should be introduced to satisfy the future power demands. Because of many real-world details involved, the structure of the problem is not exploited easily using mathematical programming techniques, for which reason we consider solving this problem with evolutionary algorithms (EAs). We compare three types of EAs for optimizing expansion plans: the classic genetic algorithm (GA), the estimation-of-distribution algorithm (EDA), and the Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA). Not fully knowing the structure of the problem, we study the effect of linkage learning through the use of three linkage models: univariate, marginal product, and linkage tree. We furthermore experiment with the impact of incorporating different levels of problem-specific knowledge in the variation operators. Experiments show that the use of problem-specific variation operators is far more important for the classic GA to find high-quality solutions. In all EAs, the marginal product model and its linkage learning procedure have difficulty in capturing and exploiting the DNEP problem structure. GOMEA, especially when combined with the linkage tree structure, is found to have the most robust performance by far, even when an out-of-the-box variant is used that does not exploit problem-specific knowledge. Based on experiments, we suggest that when selecting optimization algorithms for power system expansion planning problems, EAs that have the ability to effectively model and efficiently exploit problem structures, such as GOMEA, should be given priority, especially in the case of black-box or grey-box optimization.

Various smart grid control systems have been developed with different architectures. Comparison helps developers identify their strong and weak points. A three-step analysis method is proposed to facilitate the comparison of independently developed control systems. In the first step, a microgrid model is created describing demand and supply patterns of controllable and non-controllable devices (Flex Street). In the second step, a version of Flex Street is used to design a case, with a given control objective and key performance indicators. In the last step, simulations of different control systems are performed and their results are analysed and compared. The Flex Street model describes a diverse set of households based on realistic data. Furthermore, its bottom-up modelling approach makes it a flexible tool for designing cases. Currently, three cases with peak-shaving objectives are developed based on scenarios of the Dutch residential sector, specifying various penetration rates of renewable and controllable devices. The proposed method is demonstrated by comparing IntelliGator and TRIANA, two independently developed control systems, on peak reduction, energy efficiency, savings and abated emissions. Results show that IntelliGator---a real-time approach---is proficient in reducing peak demand, while TRIANA---a planning approach---also levels intermediate demand. Both systems yield benefits (\geneuro5--54 per household per year) through reduced transport losses and network investments in the distribution network.

We present a new method to register a pair of images captured in different image modalities. Unlike most of existing systems that register images by aligning single type of visual features, e.g., interest point or contour, we try to align hybrid visual features, including straight lines and interest points. The entire algorithm is carried out in two stages: line-based global transform approximation and point-based local transform adaptation. In the first stage, straight lines derived from edge pixels are employed to find correspondences between two images in order to estimate a global perspective transformation. In the second stage, we divide the entire image into non-overlapping cells with fixed size. The point having the strongest corner response within each cell is selected as the interest point. These points are transformed to other image based on the global transform, and then used to bootstrap a local correspondence search. Experimental evidence shows this method achieves better accuracy for registering visible and long wavelength infrared images/videos as compared to state-of-the-art approaches.

Display Omitted Predicting stress levels from real world data collected from smartphones of 30 employees.Addressing scarcity of labelled data through ensemble methods, semisupervised learning and transfer learning.Improving classification accuracy of stress levels by 1072% through the use of transfer learning. ObjectiveStress at work is a significant occupational health concern. Recent studies have used various sensing modalities to model stress behaviour based on non-obtrusive data obtained from smartphones. However, when the data for a subject is scarce it becomes a challenge to obtain a good model. MethodsWe propose an approach based on a combination of techniques: semi-supervised learning, ensemble methods and transfer learning to build a model of a subject with scarce data. Our approach is based on the comparison of decision trees to select the closest subject for knowledge transfer. ResultsWe present a real-life, unconstrained study carried out with 30 employees within two organisations. The results show that using information (instances or model) from similar subjects can improve the accuracy of the subjects with scarce data. However, using transfer learning from dissimilar subjects can have a detrimental effect on the accuracy. Our proposed ensemble approach increased the accuracy by 10% to 71.58% compared to not using any transfer learning technique. ConclusionsIn contrast to high precision but highly obtrusive sensors, using smartphone sensors for measuring daily behaviours allowed us to quantify behaviour changes, relevant to occupational stress. Furthermore, we have shown that use of transfer learning to select data from close models is a useful approach to improve accuracy in presence of scarce data.

Many different value-based or policy-search reinforcement learning algorithms have been applied to multi-agent settings. Value-based learners estimate the expected return (value) for each state-action combination and then derive a policy from these expectations. Policy-search learners optimize the agent's policy directly by using a parameterized representation of the policy and then optimizing the parameter values to maximize the expected return. While the two classes of algorithms have been considered as contrasting one another, we note that several policy-search algorithms (e.g., Weighted Policy Learner and Infinitesimal Gradient Ascent) need a method for estimating the expected returns. In practice, these policy-search algorithms internally use an update equation for incrementally improving value estimates. In this paper we present the first detailed study of the effect of using different value-based learning algorithms as components of policy-search learners. Our results show that the particular choice can significantly affect performance.

We present a dynamic and distributed approach to the hospital patient scheduling problem, in which patients can have multiple appointments that have to be scheduled to different resources. To efficiently solve this problem we develop a multi-agent Pareto-improvement appointment exchanging algorithm: MPAEX. It respects the decentralization of scheduling authorities and continuously improves patient schedules in response to the dynamic environment. We present models of the hospital patient scheduling problem in terms of the health care cycle where a doctor repeatedly orders sets of activities to diagnose and/or treat a patient. We introduce the Theil index to the health care domain to characterize different hospital patient scheduling problems in terms of the degree of relative workload inequality between required resources. In experiments that simulate a broad range of hospital patient scheduling problems, we extensively compare the performance of MPAEX to a set of scheduling benchmarks. The distributed and dynamic MPAEX performs almost as good as the best centralized and static scheduling heuristic, and is robust for variations in the model settings.

In this paper, we propose a multi-scale model of energy demand that is consistent with observations at a macro scale, in our use-case standard load profiles for (residential) electric loads. We employ the model to study incentives to assume the risk of volatile market prices for intelligent energy cooperatives at different aggregation scales of energy consumption. Next to scale, we investigate the benefits of demand response, more precisely intelligent scheduling of time-shiftable electric processes, and virtual storage intraday and between days. Results show that the increasing electrification and introduction of flexibilities (electric vehicles, thermal applications, storage, etc.) is going to make market participation viable for smaller groups of consumers. Retailers may thus introduce innovative tariffs for intelligent energy cooperatives to share the risk of volatility in wholesale markets for electricity.

Smart energy systems integrate renewables and demand response. Most European electricity markets coordinate the resulting time-varying flexibility in demand and supply by organising day-ahead trade with Walrasian mechanisms, using simultaneous call auctions and sealed bids. These mechanisms give bidders no information on each other's values and flexibilities until after clearing. In this paper we simulate two alternative day-ahead market mechanisms which share information, such that bidders obtain a better position before entering the intraday market. One mechanism uses an ascending shared market price signal rather than sealed bids. The other auctions off future timeslots consecutively rather than simultaneously. We perform a case study on 400 households with electric vehicles, either with or without volatile wind generation. Results show that a price-taking flexible consumer can obtain higher utility in the market with simultaneous ascending-price auctions, because online price information reduces uncertainty over available energy and prices.

The use of soft robots in future space exploration is still a far-fetched idea, but an attractive one. Soft robots are inherently compliant mechanisms that are well suited for locomotion on rough terrain as often faced in extra-planetary environments. Depending on the particular application and requirements, the best shape (or body morphology) and locomotion strategy for such robots will vary substantially. Recent developments in soft robotics and evolutionary optimization showed the possibility to simultaneously evolve the morphology and locomotion strategy in simulated trials. The use of techniques such as generative encoding and neural evolution were key to these findings. In this paper, we improve further on this methodology by introducing the use of a novelty measure during the evolution process. We compare fitness search and novelty search in different gravity levels and we consistently find novelty-based search to perform as good as or better than a fitness--based search, while also delivering a greater variety of designs. We propose a combination of the two techniques using fitness-elitism in novelty search to obtain a further improvement. We then use our methodology to evolve the gait and morphology of soft robots at different gravity levels, finding a taxonomy of possible locomotion strategies that are analyzed in the context of space-exploration.

We propose a novel clustering-based model-building evolutionary algorithm to tackle optimization problems that have both binary and real-valued variables. The search space is clustered every generation using a distance metric that considers binary and real-valued variables jointly in order to capture and exploit dependencies between variables of different types. After clustering, linkage learning takes place within each cluster to capture and exploit dependencies between variables of the same type. We compare this with a model-building approach that only considers dependencies between variables of the same type. Additionally, since many real-world problems have constraints, we examine the use of different well-known approaches to handling constraints: constraint domination, dynamic penalty and global competitive ranking. We experimentally analyze the performance of the proposed algorithms on various unconstrained problems as well as a selection of well-known MINLP benchmark problems that all have constraints, and compare our results with the Mixed-Integer Evolution Strategy (MIES). We find that our approach to clustering that is aimed at the processing of dependencies between binary and real-valued variables can significantly improve performance in terms of required population size and function evaluations when solving problems that exhibit properties such as multiple optima, strong mixed dependencies and constraints.

Linkage-learning Evolutionary Algorithms (EAs) use link- age learning to construct a linkage model, which is exploited to solve problems efficiently by taking into account important linkages, i.e. dependencies between problem variables, during variation. It has been shown that when this linkage model is aligned correctly with the structure of the problem, these EAs are capable of solving problems efficiently by per- forming variation based on this linkage model [2]. The Link- age Tree Genetic Algorithm (LTGA) uses a Linkage Tree (LT) as a linkage model to identify the problem's structure hierarchically, enabling it to solve various problems very efficiently. Understanding the reasons for LTGA's excellent performance is highly valuable as LTGA is also able to efficiently solve problems for which a tree-like linkage model seems inappropriate. This brings us to ask what in fact makes a linkage model ideal for LTGA to be used.

In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.

Agent-based computational economics (ACE) combines elements from economics and computer science. In this paper, we focus on the relation between the evolutionary technique that is used and the economic problem that is modeled. In the field of ACE, economic simulations often derive parameter settings for the evolutionary algorithm directly from the values of the economic model parameters. In this paper, we compare two important approaches that are dominating ACE research and show that the above practice may hinder the performance of the evolutionary algorithm and thereby hinder agent learning. More specifically, we show that economic model parameters and evolutionary algorithm parameters should be treated separately by comparing the two widely used approaches to social learning with respect to their convergence properties and robustness. This leads to new considerations for the methodological aspects of evolutionary algorithm design within the field of ACE.

In future energy systems, peaks in the daily electricity generation and consumption are expected to increase. The "smart grid" concept aims to maintain high levels of efficiency in the energy system by establishing distributed intelligence. Software agents (operating on devices with unknown computational capabilities) can implement dynamic and autonomous decision making about energy usage and generation, e.g. in domestic households, farms or offices. To reach satisfactory levels of efficiency and reliability, it is crucial to include planning-ahead of the energy-involving activities. Market mechanisms are a promising approach for large-scale coordination problems about energy supply and demand, but existing electricity markets either do not involve planning-ahead sufficiently or require a high level of sophistication and computing power from participants, which is not suitable for smart grid settings. This paper proposes a new market mechanism for smart grids, ABEM (Ahead- and Balancing Energy Market). ABEM performs an ahead market and a last-minute balancing market, where planning-ahead in the ahead market supports both binding ahead-commitments and reserve capacities in bids (which can be submitted as price functions). These features of planning-ahead reflect the features in modern wholesale electricity markets. However, constructing bids in ABEM is straightforward and fast. We also provide a model of a market with the features mentioned above, which a strategic agent can use to construct a bid (e.g. in ABEM), using a decision-theoretic approach. We evaluate ABEM experimentally in various stochastic scenarios and show favourable outcomes in comparison with a benchmark mechanism.

Database needs are changing, driven by the Internet and increasing amounts of scientific and sensor data. In this article, the authors propose research into several important new directions for database management systems.

The evolution of cooperation in a system of agents playing the iterated prisoner''s dilemma (IPD) is investigated. We present results for the standard two-person IPD as well as the more general N-person IPD (NIPD) game. In our computational model, agents can recognize each other and decide whether to interact or not, based upon ``tags'''' (labels). We consider the evolutionary stability of the evolving populations. Previous work is extended by introducing sexual reproduction (recombination) of agents and by analyzing its influence on the evolving populations. We observed the occasional formation of very stable cooperative societies, as opposed to previous results without sexual reproduction. These cooperative societies are able to resist invasions of ``mimics'''' (defecting agents with the tag of a cooperating agent).

Low voltage cables in distribution grids will operate under harmful conditions more often in the near future, due to the integration of renewable energy sources and the addition of powerful consumption appliances. It is crucial to evaluate smart solutions that can prolong cable lifetime. Because most knowledge about thermal behaviour of cables has been collected in the medium and high voltage grids, it is also important to study, in real-world circumstances, the precise thermal reaction of low voltage cables to novel types of usage spikes and to protection mechanisms. We emulated a low voltage cable of a Dutch neighbourhood in a laboratory and conducted several overheating experiments for multiple hours. One scenario involved the usage of a smart battery with a control algorithm for the protection of the cable from overheating. We report on results of systematically varying currents and cable insulation. We demonstrate the positive effect of regular interruptions of high currents on cable temperature and that even a small battery (e.g. a used EV battery) can make a significant contribution to the reduction of overheating.

The recently introduced Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA), with a lean, but sufficient, linkage model and an efficient variation operator, has been shown to be a robust and efficient methodology for solving single objective (SO) optimization problems with superior performance compared to classic genetic algorithms (GAs) and estimation-of-distribution algorithms (EDAs). In this paper, we bring the strengths of GOMEAs to the multi-objective (MO) optimization realm. To this end, we modify the linkage learning procedure and the variation operator of GOMEAs to better suit the need of finding the whole Pareto-optimal front rather than a single best solution. Based on state-of-the-art studies on MOEAs, we further pinpoint and incorporate two other essential components for a scalable MO optimizer. First, the use of an elitist archive is beneficial for keeping track of non-dominated solutions when the main population size is limited. Second, clustering can be crucial if different parts of the Pareto-optimal front need to be handled differently. By combining these elements, we construct a multi-objective GOMEA (MO-GOMEA). Experimental results on various MO optimization problems confirm the capability and scalability of our MO-GOMEA that compare favorably with those of the well-known GA NSGA-II and the more recently introduced EDA mohBOA.

The Power TAC simulates a smart grid energy market. In this simulation, broker agents compete for customers on a tariff market and trade energy on a wholesale market. It provides a platform for testing strategies of broker agents against other strategies. In this paper we describe the strategies of our broker agent. Amongst others, due to a beneficial trading technique related to equilibria in continuous auctions on the wholesale market and a strategy inspired by Tit-for-Tat in the Iterated Prisoner’s Dilemma game on the tariff market, our broker ended second in the 2013 Power TAC.

The EU-driven integration of European energy systems and the development of a Smart Energy System involves many key players. The success of a European Smart Energy Systems relies heavily on the development of well-designed ICT solutions in all related sectors. Because such ICT solutions should be well aligned, ICT innovation goals are needed that have the support of key European players and consortia. To this end, Round Tables have been organised by EIT ICT Labs from 2013 onwards, in which key players in the European energy sector establish a common vision on and align forces in the development of a European Smart Energy System. We reflect and build upon these discussions to formulate a joint input to the European goals on ICT and Smart Energy Systems, and for the innovation activities of EIT ICT Labs.

Offshore wind farms with high installed capacities and located further from the shore are starting to be built by northern European countries. Furthermore, it is expected that by 2020, several dozens of large offshore wind farms will be built in the Baltic, Irish and North seas. These wind farms will be constituted of a considerable number of turbines packed together. Due to shadowing effects between turbines, the power production is reduced, resulting in a decreased wind farm efficiency. Moreover, the collection system length and complexity rapidly increase with the number of wind turbines installed, resulting in higher investment costs and power losses. Hence, when large wind farms are considered, wake losses and collection system total length are important aspects that need to be optimized. Nonetheless, designing a large offshore wind project is a very complex task. In the development phase, an enormous amount of time is used to generate feasible wind farm designs. For example, the manual layout optimization and cable routing are highly time consuming. Amongst others, this drawback leads to offshore wind projects with very high investments costs. In fact, electricity generated offshore is, up to today, substantially more expensive when compared to onshore wind or combined cycle gas turbines. Wind farm layout optimization is a possible solution to further increase the efficiency of wind projects leading to lower prices of energy generated offshore. Moreover, investment costs would be reduced if the collection system could be automatically designed via optimization tools. The aim of this work is to obtain optimized offshore wind farm layouts with enhanced energy production while considering the seabed constraints, e.g. maximum sea depth, inappropriate soil types and large seabed gradients. Through the use of optimization tools it is also desired to alleviate the designer teams of mundane manual layout optimization and cable routing. The optimized wind farm layout obtained presents higher energy yield. Although, the collection system total length is higher than for the standard case, the overall system efficiency is improved. Moreover, installation costs of the collection system were reduced through the avoidance of challenging areas of the wind farm. The wind turbine support structures costs were also reduced since deep areas of the seabed were avoided. In this work it is presented a novel optimization tool to obtain optimized offshore wind farm layouts while taking into consideration the seabed profile. Due to the increased offshore activity in recent years, as well as the expected installation in the coming years, this tool becomes very important and relevant for the offshore field.

A key characteristic of Mixed-Integer (MI) problems is the presence of both continuous and discrete problem variables. These variables can interact in various ways, resulting in challenging optimization problems. In this paper, we study the design of an algorithm that combines the strengths of LTGA and iAMaLGaM: state-of-the-art model-building EAs designed for discrete and continuous search spaces, respectively. We examine and discuss issues which emerge when trying to integrate those two algorithms into the MI setting. Our considerations lead to a design of a new algorithm for solving MI problems, which we motivate and compare with alternative approaches.

Distribution network operators require more advanced planning tools to deal with the challenges of future network planning. An appropriate planning and optimization tool can identify which option for network extension should be selected from available alternatives. However, many optimization approaches described in the literature are quite theoretical and do not yield results that are practically relevant and feasible. In this paper, a distribution network planning approach is proposed which meets requirements originating from network planning practice to guarantee realistic outcomes. This approach uses a state-of-the-art evolutionary algorithm: Gene-pool Optimal Mixing Evolutionary Algorithm. The performance of this algorithm, as well as the proposed model, is demonstrated using a real-world case study.

Market Garden is a scalable research environment and demonstration tool, in which market mechanisms for smart energy systems and the interaction between end users, traders and system operators can be simulated. Users can create scenarios in a user-friendly editor in which a hierarchical market architecture and a model of the physical grid can be defined. The system sets up an interactive and competitive environment of modular market mechanisms. A visualiser in which the results can be explored in a graphical and intuitive way is included, making Market Garden very suitable for user experience and demonstration purposes, next to its application for research.

Currently, two major challenges dominate the field of deformable image registration. The first challenge is related to the tuning of the developed methods to specific problems (i.e. how to best combine different objectives such as similarity measure and transformation effort). This is one of the reasons why, despite significant progress, clinical implementation of such techniques has proven to be difficult. The second challenge is to account for large anatomical differences (e.g. large deformations, (dis)appearing structures) that occurred between image acquisitions. In this paper, we study a framework based on multi-objective optimization to improve registration robustness and to simplify tuning for specific applications. Within this framework we specifically consider the use of an advanced model-based evolutionary algorithm for optimization and a dual-dynamic transformation model (i.e. two "non-fixed" grids: one for the source- and one for the target image) to accommodate for large anatomical differences. The framework computes and presents multiple outcomes that represent efficient trade-offs between the different objectives (a so-called Pareto front). In image processing it is common practice, for reasons of robustness and accuracy, to use a multi-resolution strategy. This is, however, only well-established for single-objective registration methods. Here we describe how such a strategy can be realized for our multi-objective approach and compare its results with a single-resolution strategy. For this study we selected the case of prone-supine breast MRI registration. Results show that the well-known advantages of a multi-resolution strategy are successfully transferred to our multi-objective approach, resulting in superior (i.e. Pareto-dominating) outcomes.

The Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) is a well-known, state-of-the-art optimization algorithm for single-objective real-valued problems, especially in black-box settings. Although several extensions of CMA-ES to multi-objective (MO) optimization exist, no extension incorporates a key component of the most robust and general CMA-ES variant: the association of a population with each Gaussian distribution that drives optimization. To achieve this, we use a recently introduced framework for extending population-based algorithms from single- to multi-objective optimization. We compare, using six well-known benchmark problems, the performance of the newly constructed MO-CMA-ES with existing variants and with the estimation of distribution algorithm (EDA) known as iMAMaLGaM, that is also an instance of the framework, extending the single-objective EDA iAMaLGaM to MO. Results underline the advantages of being able to use populations. Because many real-world problems have constraints, we also study the use of four constraint-handling techniques. We find that CMA-ES is typically less robust to these techniques than iAMaLGaM. Moreover, whereas we could verify that a penalty method that was previously used in literature leads to fast convergence, we also find that it has a high risk of finding only nearly, but not entirely, feasible solutions. We therefore propose that other constraint-handling techniques should be preferred in general.

Distributed computing is the theoretical foundation for applications and technologies like interactive architecture, wearable computing, and smart materials. It evolves continuously, following needs rising from scientific developments, novel uses of technology, or simply the curiosity to better understand the world around us. The pace of evolution is fast: in a short span of time distributed computing helped the Internet develop into how we know it today and overcame the dynamics of the large spread of mobile wireless communication. Soon, we will see the effects of distributed computing in the rise of the Internet of Things. Interactive architecture brings a specific set of requirements to distributed computing (sheer number of devices, specific dynamics, resources, control constraints, etc.). The mapping between the requirements of interactive installation and distributed systems theory is well understood. Architects are already acquainted with concepts such as emergent behavior and swarm behavior and use them in their concepts. Nevertheless, despite the availability of all the constituent ingredients, the number of large-scale installations employing distributed behavior in hardware and software is very small. The vast majority of systems are still controlled from a central computer that creates the desired behavior and that faces obvious issues of scale. This is the main research question that we would like to tackle in this essay. Before going into details, let us contemplate for a moment the larger picture, making abstraction of the technological barriers. The dream of large-scale interactive systems responding and adapting instantly to the user’s desires will come one step closer to actualization. For example, the myriad inputs triggered by the crowds inhabiting a city will be translated automatically through extensively equipped intelligent surroundings. Aspects of control, safety, and privacy will be automatically dealt with in real time. Humans will have a new understanding of the world around them, which will simply adapt to suit their needs. Smart cities and the Internet of Things will finally become a reality.

In this position paper, we address the applicability of spatial computing in the field of interactive architecture. The process of designing large-scale interactive systems is cumbersome, due in fact to single design cycles (transforming ideas into prototypes) taking a period of time usually measured in months, most of it dedicated to writing the software controlling the system. As most interactive architecture projects pass through several design cycles interleaved with user studies, speeding up the generation of the needed software becomes of crucial importance. The global-to-local programming approach is in fact a perfect tool for this task. Describing complex behaviors with simple rules is rarely seen in the existing installations, the large majority employing a central computer for the control of the system. Building up on our previous experience in this area, we created HiveKit, a proof of concept allowing bridging between the two fields, giving non-specialists easy access to distributed algorithms. HiveKit is a software package which allows designers to specify the desired behavior and automatically generate and deploy the needed code on networks of embedded devices. We introduce several projects where HiveKit is employed and create an argument, based on user studies, favoring the need for large-scale adoption of such tools.

Mixing of partial solutions is a key mechanism used for creating new solutions in many Genetic Algorithms (GAs). However, this mixing can be disruptive and generate improved solutions inefficiently. Exploring a problem’s structure can help in establishing less disruptive operators, leading to more efficient mixing. One way of using a problem’s structure is to consider variable linkage information. This paper focuses on exploring different methods of building family of subsets (FOS) linkage models, which are then used with the Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA) to solve MAX-SAT problems. The GOMEA framework provides an efficient mechanism for mixing partial solutions and generating new candidate solutions, given a FOS linkage model. We wish to examine if learning variable linkage information, and representing this linkage within a FOS model can be beneficial in creating highly fit solutions more efficiently.

Demand response is a crucial mechanism for flattening of peak loads. For its implementation, we not only require consumers who react to price changes, but also intelligent strategies to select prices. We propose a parametrised meta-strategy for dynamic pricing and identify suitable strategies for given scenarios through offline optimisation using a population model. We also model an important and novel constraint: a price cap (a maximal unit price) for consumer protection. We show in computational simulations that the maximal unit price influences the peak reduction potential of dynamic pricing. We compare our dynamic pricing approach with a constant pricing approach and show that our approach, used by a profit-optimising seller, is both peak-reducing and equally profitable.

The linkage tree genetic algorithm (LTGA) learns, each generation, a linkage model by building a hierarchical cluster tree. The LTGA is an instance of the more general gene-pool optimal mixing evolutionary algorithm (GOMEA) that uses a family of subsets (FOS) linkage model. We compare the performance of the linkage model learning LTGA with several predetermined FOS linkage models applied by GOMEA. The predetermined models are matched to the underlying problem structure of four benchmark functions: onemax, deceptive trap functions, maximal overlapping nearest-neighbor NK-landscapes, and weighted MAXCUT problems. Although the a priori fixed models are specifically designed to capture the interactions between the problem variables, experimental results show that - for problems with intricate interaction structure - they are actually less efficient than LTGA that dynamically learns a hierarchical tree model. Some of these observations were unexpected and raise the question of what exactly is the optimal linkage structure for a given problem as used by GOMEA. In the case of the NK-problem a linkage model that is an accurate description of the underlying additively decomposable fitness structure is clearly not an optimal linkage model. Being able to rebuild the linkage model each generation has clear benefits above using fixed, predetermined linkage models.

This paper proposes a saliency-aware fusion algorithm for integrating infrared (IR) and visible light (ViS) images (or videos) with the aim to enhance the visualization of the latter. Our algorithm involves saliency detection followed by a biased fusion. The goal of the saliency detection is to generate a saliency map for the IR image, highlighting the co-occurrence of high brightness values ("hot spots") and motion. Markov Random Fields (MRFs) are used to combine these two sources of information. The subsequent fusion step is employed to bias the end result in favor of the ViS image, except when a region shows clear IR saliency, in which case the IR image gains (local) dominance. By doing so, the fused image succeeds in depicting both the salient foreground object (gleaned from the IR image), against as an easily recognizable background as supplied by the ViS image. An evaluation of the proposed saliency detection method indicates improvements in detection accuracy when compared to state-of-the-art alternatives. Moreover, both objective and subjective assessments reveal the effectiveness of the proposed fusion algorithm in terms of visual context enhancement.

Low-voltage radial electricity cables will have more and more difficulties to carry the increasing load of novel consumption devices (e.g. electric vehicles) and the expected generated input of decentrally-generated power (e.g. from photovoltaic cells). One solution to avoid replacement is to install a battery at the end of a cable which is expected to be overloaded frequently. The intelligent operation of this battery needs to combine the protection of the cable with optimizing its revenue, in order to be economically viable. This paper formulates the offline optimization problem and proposes two robust heuristic online strategies. We show in computer simulations that these heuristics, which make fast just-in-time responses, reliably deliver good results. Our second heuristic, H2, reaches up to 83% of the approximated theoretical optimum.

Exploiting a problem’s structure to arrive at the most efficient optimization algorithm is key in many optimization disciplines. In evolutionary computation, especially for solving discrete optimization problems from a black-box optimization (BBO) perspective, linkage learning is an important research line because if important linkages are disrupted during variation, optimization will not proceed efficiently [4]. Estimation-of-distribution algorithms (EDAs) are well-k nown for building and using models to exploit problem structure [2, 3]. Models in EDAs represent probability distributions and linkage information is processed via probabilistic dependency relations within these distributions. Although EDAs can be very powerful, estimating complete distributions might be more than what is required to respect important linkage relations. Here, we therefore consider the class of Genepool Optimal Mixing Evolutionary Algorithms (GOMEAs) as they exploit linkage information by integrating greedy local search, genetic recombination and fitness-based selection [1] based on linkage models, which can typically be learned more efficiently. Recent results indicate that the use of hierarchical linkage models in GOMEAs leads to the best performance [1]. There are, arguably, however still potential ine fficiencies. In this paper, we consider ways to filter these out. We further consider a way to combine the strengths of different linkage models.

The community of agent researchers and engineers has produced a number of interesting and mature results. However, agent technology is still not widely adopted by industrial software developers or software companies - possibly because existing frameworks are infused with academic premises that rarely apply to industrial settings. In this paper, we analyse the requirements of current industry-driven software projects and show how we are able to cope with these requirements in the Java Intelligent Agent Componentware agent framework, JIAC V. We argue that the lack of industry-grade requirements and features in other agent frameworks is one of the reasons for the slow acceptance of agent technology in the software industry. The JIAC V framework tries to bridge that gap - not as a final solution, but as a stepping stone towards industrial acceptance.

Q-learning is a very popular reinforcement learning algorithm being proven to converge to optimal policies in Markov decision processes. However, Q-learning shows artifacts in non-stationary environments, e.g., the probability of playing the optimal action may decrease if Q-values deviate significantly from the true values, a situation that may arise in the initial phase as well as after changes in the environment.These artifacts were resolved in literature by the variant Frequency Adjusted Q-learning (FAQL). However, FAQL also suffered from practical concerns that limited the policy subspace for which the behavior was improved. Here, we introduce the Repeated Update Q-learning (RUQL), a variant of Q-learning that resolves the undesirable artifacts of Q-learning without the practical concerns of FAQL.We show (both theoretically and experimentally) the similarities and differences between RUQL and FAQL (the closest state-of-the-art). Experimental results verify the theoretical insights and show how RUQL outperforms FAQL and QL in non-stationary environments.

Satisfiability in propositional logic is well researched and many approaches to checking and solving exist. In infinite-valued or fuzzy logics, however, there have only recently been attempts at developing methods for solving satisfiability. In this paper, we propose new benchmark problems and analyse the function landscape of different problem classes, focussing our analysis on plateaus. Based on this study, we develop Mixing CMA-ES (M-CMA-ES), an extension to CMA-ES that is well suited to solving problems with many large plateaus. We empirically show the relation between certain function landscape properties and M-CMA-ES performance.

We consider the problem of stacking containers in a given set of stacks of fixed maximum capacity when the pick-up times are stochastic with unknown probability distributions. The goal is to minimize the expected number of times a container is picked up while it is not at the top of its stack. We formulate several algorithms under varying assumptions about the available knowledge about the pick-up-time distributions. We distinguish four qualitatively different settings: 1) we know nothing about the actual distributions, 2) we have point estimates of the means, 3) we have point estimates of means and variances, or 4) we have historical samples of actual pick-up times. For each assumption we propose one or more algorithms. We test the algorithms empirically in many different scenarios, considering both sequential and concurrent arrivals. Additionally, we consider the computational complexity and ease of use of each algorithm.

Given a domain of interest, an ontology provides a formal specification of the concepts and relationships in this domain. An align- ment identifies a semantic connection between the concepts and relation- ships of ontologies purporting to describe the same knowledge. The ma- jority of approaches to ontology alignment take a general view and rely on terminological and structural techniques to handle the widest possible class of ontologies. Due to the fuzzy nature of concepts and the varia- tions in their representations, this matching process is often complicated. One area that should provide a more solid foundation for comparison is mathematical concepts. Mathematical concepts are covered by numerous ontologies, making them a potentially important basis for comparison. One part of this doctoral work is to investigate the potential of using mathematical concepts to improve ontology alignment.

How will we trade energy in the future? We can expect vast technological changes in our energy systems, leading to high heterogeneity in both supply and demand. Knowing the true value of energy at a given time and location will be crucial. Market mechanisms are methods of determining prices in complex, multi-actor settings. In the Intelligent Systems group at CWI, we have developed a research environment in which different market mechanisms for electricity can be studied and evaluated - in interaction, remotely and in a scalable manner.

Satisfiability in propositional logic is well researched and many approaches to checking and solving exist. In infinite-valued or fuzzy logics, however, there have only recently been attempts at developing methods for solving satisfiability. In this paper, we analyse the function landscape of different problem classes, focussing our analysis on plateaus. Based on this study, we develop Mixing CMA-ES (M-CMA-ES), an extension to CMA-ES that is well suited to solving problems with many large plateaus. We empirically show the relation between certain function landscape properties and M-CMA-ES performance.

Hierarchical problems represent an important class of nearly decomposable problems. The concept of near decomposability is central to the study of complex systems. When little a priori information is available, a black box problem solver is needed to optimize these hierarchical problems. The solver should be able to learn linkage information, and to preserve and test partial solutions at different levels in the hierarchy. Two well known benchmark functions - shuffled Hierarchical If-And-Only-If (HIFF) and shuffled Hierarchical Trap (HTRAP) functions - exemplify the challenges posed by hierarchical problems. Standard genetic algorithms are unable to solve these problems, and specific methods, like SEAM and hBOA, have been designed to address them. In this paper, we investigate how the recently developed Linkage Tree Genetic Algorithm (LTGA) performs on these hierarchical problems. We compare LTGA with SEAM and hBOA on HIFF and HTRAP functions. Results show that, although LTGA is a simple algorithm compared to SEAM and hBOA, it nevertheless is a very efficient, reliable, and scalable algorithm for solving the randomly shuffled versions of HIFF and HTRAP, two hard, hierarchical problems.

Satisfiability solving in fuzzy logics is a subject that has not been researched much, certainly compared to satisfiability in propositional logics. Yet, fuzzy logics are a powerful tool for modelling complex problems. Recently, we proposed an optimization approach to solving satisfiability in fuzzy logics and compared the standard Covariance Matrix Adaptation Evolution Strategy algorithm (CMA-ES) with an analytical solver on a set of benchmark problems. Especially on more finegrained problems did CMA-ES compare favourably to the analytical approach. In this paper, we evaluate two types of hillclimber in addition to CMA-ES, as well as restart strategies for these algorithms. Our results show that a population-based hillclimber outperforms CMA-ES on the harder problem class.