Search

The interaction of multiple autonomous agents gives rise to highly dynamic and nondeterministic environments, contributing to the complexity in applications such as automated financial markets, smart grids, or robotics. Due to the sheer number of situations that may arise, it is not possible to foresee and program the optimal behaviour for all agents beforehand. Consequently, it becomes essential for the success of the system that the agents can learn their optimal behaviour and adapt to new situations or circumstances. The past two decades have seen the emergence of reinforcement learning, both in single and multi-agent settings, as a strong, robust and adaptive learning paradigm. Progress has been substantial, and a wide range of algorithms are now available. An important challenge in the domain of multi-agent learning is to gain qualitative insights into the resulting system dynamics. In the past decade, tools and methods from evolutionary game theory have been successfully employed to study multi-agent learning dynamics formally in strategic interactions. This article surveys the dynamical models that have been derived for various multi-agent reinforcement learning algorithms, making it possible to study and compare them qualitatively. Furthermore, new learning algorithms that have been introduced using these evolutionary game theoretic tools are reviewed. The evolutionary models can be used to study complex strategic interactions. Examples of such analysis are given for the domains of automated trading in stock markets and collision avoidance in multi-robot systems. The paper provides a roadmap on the progress that has been achieved in analysing the evolutionary dynamics of multi-agent learning by highlighting the main results and accomplishments.

This article tackles the Distribution Network Expansion Planning (DNEP) problem that has to be solved by distribution network operators to decide which, where, and/or when enhancements to electricity networks should be introduced to satisfy the future power demands. Because of many real-world details involved, the structure of the problem is not exploited easily using mathematical programming techniques, for which reason we consider solving this problem with evolutionary algorithms (EAs). We compare three types of EAs for optimizing expansion plans: the classic genetic algorithm (GA), the estimation-of-distribution algorithm (EDA), and the Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA). Not fully knowing the structure of the problem, we study the effect of linkage learning through the use of three linkage models: univariate, marginal product, and linkage tree. We furthermore experiment with the impact of incorporating different levels of problem-specific knowledge in the variation operators. Experiments show that the use of problem-specific variation operators is far more important for the classic GA to find high-quality solutions. In all EAs, the marginal product model and its linkage learning procedure have difficulty in capturing and exploiting the DNEP problem structure. GOMEA, especially when combined with the linkage tree structure, is found to have the most robust performance by far, even when an out-of-the-box variant is used that does not exploit problem-specific knowledge. Based on experiments, we suggest that when selecting optimization algorithms for power system expansion planning problems, EAs that have the ability to effectively model and efficiently exploit problem structures, such as GOMEA, should be given priority, especially in the case of black-box or grey-box optimization.

Various smart grid control systems have been developed with different architectures. Comparison helps developers identify their strong and weak points. A three-step analysis method is proposed to facilitate the comparison of independently developed control systems. In the first step, a microgrid model is created describing demand and supply patterns of controllable and non-controllable devices (Flex Street). In the second step, a version of Flex Street is used to design a case, with a given control objective and key performance indicators. In the last step, simulations of different control systems are performed and their results are analysed and compared. The Flex Street model describes a diverse set of households based on realistic data. Furthermore, its bottom-up modelling approach makes it a flexible tool for designing cases. Currently, three cases with peak-shaving objectives are developed based on scenarios of the Dutch residential sector, specifying various penetration rates of renewable and controllable devices. The proposed method is demonstrated by comparing IntelliGator and TRIANA, two independently developed control systems, on peak reduction, energy efficiency, savings and abated emissions. Results show that IntelliGator---a real-time approach---is proficient in reducing peak demand, while TRIANA---a planning approach---also levels intermediate demand. Both systems yield benefits (\geneuro5--54 per household per year) through reduced transport losses and network investments in the distribution network.

We present a new method to register a pair of images captured in different image modalities. Unlike most of existing systems that register images by aligning single type of visual features, e.g., interest point or contour, we try to align hybrid visual features, including straight lines and interest points. The entire algorithm is carried out in two stages: line-based global transform approximation and point-based local transform adaptation. In the first stage, straight lines derived from edge pixels are employed to find correspondences between two images in order to estimate a global perspective transformation. In the second stage, we divide the entire image into non-overlapping cells with fixed size. The point having the strongest corner response within each cell is selected as the interest point. These points are transformed to other image based on the global transform, and then used to bootstrap a local correspondence search. Experimental evidence shows this method achieves better accuracy for registering visible and long wavelength infrared images/videos as compared to state-of-the-art approaches.

Display Omitted Predicting stress levels from real world data collected from smartphones of 30 employees.Addressing scarcity of labelled data through ensemble methods, semisupervised learning and transfer learning.Improving classification accuracy of stress levels by 1072% through the use of transfer learning. ObjectiveStress at work is a significant occupational health concern. Recent studies have used various sensing modalities to model stress behaviour based on non-obtrusive data obtained from smartphones. However, when the data for a subject is scarce it becomes a challenge to obtain a good model. MethodsWe propose an approach based on a combination of techniques: semi-supervised learning, ensemble methods and transfer learning to build a model of a subject with scarce data. Our approach is based on the comparison of decision trees to select the closest subject for knowledge transfer. ResultsWe present a real-life, unconstrained study carried out with 30 employees within two organisations. The results show that using information (instances or model) from similar subjects can improve the accuracy of the subjects with scarce data. However, using transfer learning from dissimilar subjects can have a detrimental effect on the accuracy. Our proposed ensemble approach increased the accuracy by 10% to 71.58% compared to not using any transfer learning technique. ConclusionsIn contrast to high precision but highly obtrusive sensors, using smartphone sensors for measuring daily behaviours allowed us to quantify behaviour changes, relevant to occupational stress. Furthermore, we have shown that use of transfer learning to select data from close models is a useful approach to improve accuracy in presence of scarce data.

Many different value-based or policy-search reinforcement learning algorithms have been applied to multi-agent settings. Value-based learners estimate the expected return (value) for each state-action combination and then derive a policy from these expectations. Policy-search learners optimize the agent's policy directly by using a parameterized representation of the policy and then optimizing the parameter values to maximize the expected return. While the two classes of algorithms have been considered as contrasting one another, we note that several policy-search algorithms (e.g., Weighted Policy Learner and Infinitesimal Gradient Ascent) need a method for estimating the expected returns. In practice, these policy-search algorithms internally use an update equation for incrementally improving value estimates. In this paper we present the first detailed study of the effect of using different value-based learning algorithms as components of policy-search learners. Our results show that the particular choice can significantly affect performance.

We present a dynamic and distributed approach to the hospital patient scheduling problem, in which patients can have multiple appointments that have to be scheduled to different resources. To efficiently solve this problem we develop a multi-agent Pareto-improvement appointment exchanging algorithm: MPAEX. It respects the decentralization of scheduling authorities and continuously improves patient schedules in response to the dynamic environment. We present models of the hospital patient scheduling problem in terms of the health care cycle where a doctor repeatedly orders sets of activities to diagnose and/or treat a patient. We introduce the Theil index to the health care domain to characterize different hospital patient scheduling problems in terms of the degree of relative workload inequality between required resources. In experiments that simulate a broad range of hospital patient scheduling problems, we extensively compare the performance of MPAEX to a set of scheduling benchmarks. The distributed and dynamic MPAEX performs almost as good as the best centralized and static scheduling heuristic, and is robust for variations in the model settings.

In this paper, we propose a multi-scale model of energy demand that is consistent with observations at a macro scale, in our use-case standard load profiles for (residential) electric loads. We employ the model to study incentives to assume the risk of volatile market prices for intelligent energy cooperatives at different aggregation scales of energy consumption. Next to scale, we investigate the benefits of demand response, more precisely intelligent scheduling of time-shiftable electric processes, and virtual storage intraday and between days. Results show that the increasing electrification and introduction of flexibilities (electric vehicles, thermal applications, storage, etc.) is going to make market participation viable for smaller groups of consumers. Retailers may thus introduce innovative tariffs for intelligent energy cooperatives to share the risk of volatility in wholesale markets for electricity.

Smart energy systems integrate renewables and demand response. Most European electricity markets coordinate the resulting time-varying flexibility in demand and supply by organising day-ahead trade with Walrasian mechanisms, using simultaneous call auctions and sealed bids. These mechanisms give bidders no information on each other's values and flexibilities until after clearing. In this paper we simulate two alternative day-ahead market mechanisms which share information, such that bidders obtain a better position before entering the intraday market. One mechanism uses an ascending shared market price signal rather than sealed bids. The other auctions off future timeslots consecutively rather than simultaneously. We perform a case study on 400 households with electric vehicles, either with or without volatile wind generation. Results show that a price-taking flexible consumer can obtain higher utility in the market with simultaneous ascending-price auctions, because online price information reduces uncertainty over available energy and prices.

The use of soft robots in future space exploration is still a far-fetched idea, but an attractive one. Soft robots are inherently compliant mechanisms that are well suited for locomotion on rough terrain as often faced in extra-planetary environments. Depending on the particular application and requirements, the best shape (or body morphology) and locomotion strategy for such robots will vary substantially. Recent developments in soft robotics and evolutionary optimization showed the possibility to simultaneously evolve the morphology and locomotion strategy in simulated trials. The use of techniques such as generative encoding and neural evolution were key to these findings. In this paper, we improve further on this methodology by introducing the use of a novelty measure during the evolution process. We compare fitness search and novelty search in different gravity levels and we consistently find novelty-based search to perform as good as or better than a fitness--based search, while also delivering a greater variety of designs. We propose a combination of the two techniques using fitness-elitism in novelty search to obtain a further improvement. We then use our methodology to evolve the gait and morphology of soft robots at different gravity levels, finding a taxonomy of possible locomotion strategies that are analyzed in the context of space-exploration.

We propose a novel clustering-based model-building evolutionary algorithm to tackle optimization problems that have both binary and real-valued variables. The search space is clustered every generation using a distance metric that considers binary and real-valued variables jointly in order to capture and exploit dependencies between variables of different types. After clustering, linkage learning takes place within each cluster to capture and exploit dependencies between variables of the same type. We compare this with a model-building approach that only considers dependencies between variables of the same type. Additionally, since many real-world problems have constraints, we examine the use of different well-known approaches to handling constraints: constraint domination, dynamic penalty and global competitive ranking. We experimentally analyze the performance of the proposed algorithms on various unconstrained problems as well as a selection of well-known MINLP benchmark problems that all have constraints, and compare our results with the Mixed-Integer Evolution Strategy (MIES). We find that our approach to clustering that is aimed at the processing of dependencies between binary and real-valued variables can significantly improve performance in terms of required population size and function evaluations when solving problems that exhibit properties such as multiple optima, strong mixed dependencies and constraints.

Linkage-learning Evolutionary Algorithms (EAs) use link- age learning to construct a linkage model, which is exploited to solve problems efficiently by taking into account important linkages, i.e. dependencies between problem variables, during variation. It has been shown that when this linkage model is aligned correctly with the structure of the problem, these EAs are capable of solving problems efficiently by per- forming variation based on this linkage model [2]. The Link- age Tree Genetic Algorithm (LTGA) uses a Linkage Tree (LT) as a linkage model to identify the problem's structure hierarchically, enabling it to solve various problems very efficiently. Understanding the reasons for LTGA's excellent performance is highly valuable as LTGA is also able to efficiently solve problems for which a tree-like linkage model seems inappropriate. This brings us to ask what in fact makes a linkage model ideal for LTGA to be used.

In this paper, we investigate Reinforcement learning (RL) in multi-agent systems (MAS) from an evolutionary dynamical perspective. Typical for a MAS is that the environment is not stationary and the Markov property is not valid. This requires agents to be adaptive. RL is a natural approach to model the learning of individual agents. These Learning algorithms are however known to be sensitive to the correct choice of parameter settings for single agent systems. This issue is more prevalent in the MAS case due to the changing interactions amongst the agents. It is largely an open question for a developer of MAS of how to design the individual agents such that, through learning, the agents as a collective arrive at good solutions. We will show that modeling RL in MAS, by taking an evolutionary game theoretic point of view, is a new and potentially successful way to guide learning agents to the most suitable solution for their task at hand. We show how evolutionary dynamics (ED) from Evolutionary Game Theory can help the developer of a MAS in good choices of parameter settings of the used RL algorithms. The ED essentially predict the equilibriums outcomes of the MAS where the agents use individual RL algorithms. More specifically, we show how the ED predict the learning trajectories of Q-Learners for iterated games. Moreover, we apply our results to (an extension of) the COllective INtelligence framework (COIN). COIN is a proved engineering approach for learning of cooperative tasks in MASs. The utilities of the agents are re-engineered to contribute to the global utility. We show how the improved results for MAS RL in COIN, and a developed extension, are predicted by the ED.

Agent-based computational economics (ACE) combines elements from economics and computer science. In this paper, we focus on the relation between the evolutionary technique that is used and the economic problem that is modeled. In the field of ACE, economic simulations often derive parameter settings for the evolutionary algorithm directly from the values of the economic model parameters. In this paper, we compare two important approaches that are dominating ACE research and show that the above practice may hinder the performance of the evolutionary algorithm and thereby hinder agent learning. More specifically, we show that economic model parameters and evolutionary algorithm parameters should be treated separately by comparing the two widely used approaches to social learning with respect to their convergence properties and robustness. This leads to new considerations for the methodological aspects of evolutionary algorithm design within the field of ACE.

In future energy systems, peaks in the daily electricity generation and consumption are expected to increase. The "smart grid" concept aims to maintain high levels of efficiency in the energy system by establishing distributed intelligence. Software agents (operating on devices with unknown computational capabilities) can implement dynamic and autonomous decision making about energy usage and generation, e.g. in domestic households, farms or offices. To reach satisfactory levels of efficiency and reliability, it is crucial to include planning-ahead of the energy-involving activities. Market mechanisms are a promising approach for large-scale coordination problems about energy supply and demand, but existing electricity markets either do not involve planning-ahead sufficiently or require a high level of sophistication and computing power from participants, which is not suitable for smart grid settings. This paper proposes a new market mechanism for smart grids, ABEM (Ahead- and Balancing Energy Market). ABEM performs an ahead market and a last-minute balancing market, where planning-ahead in the ahead market supports both binding ahead-commitments and reserve capacities in bids (which can be submitted as price functions). These features of planning-ahead reflect the features in modern wholesale electricity markets. However, constructing bids in ABEM is straightforward and fast. We also provide a model of a market with the features mentioned above, which a strategic agent can use to construct a bid (e.g. in ABEM), using a decision-theoretic approach. We evaluate ABEM experimentally in various stochastic scenarios and show favourable outcomes in comparison with a benchmark mechanism.

Database needs are changing, driven by the Internet and increasing amounts of scientific and sensor data. In this article, the authors propose research into several important new directions for database management systems.

The evolution of cooperation in a system of agents playing the iterated prisoner''s dilemma (IPD) is investigated. We present results for the standard two-person IPD as well as the more general N-person IPD (NIPD) game. In our computational model, agents can recognize each other and decide whether to interact or not, based upon ``tags'''' (labels). We consider the evolutionary stability of the evolving populations. Previous work is extended by introducing sexual reproduction (recombination) of agents and by analyzing its influence on the evolving populations. We observed the occasional formation of very stable cooperative societies, as opposed to previous results without sexual reproduction. These cooperative societies are able to resist invasions of ``mimics'''' (defecting agents with the tag of a cooperating agent).

Low voltage cables in distribution grids will operate under harmful conditions more often in the near future, due to the integration of renewable energy sources and the addition of powerful consumption appliances. It is crucial to evaluate smart solutions that can prolong cable lifetime. Because most knowledge about thermal behaviour of cables has been collected in the medium and high voltage grids, it is also important to study, in real-world circumstances, the precise thermal reaction of low voltage cables to novel types of usage spikes and to protection mechanisms. We emulated a low voltage cable of a Dutch neighbourhood in a laboratory and conducted several overheating experiments for multiple hours. One scenario involved the usage of a smart battery with a control algorithm for the protection of the cable from overheating. We report on results of systematically varying currents and cable insulation. We demonstrate the positive effect of regular interruptions of high currents on cable temperature and that even a small battery (e.g. a used EV battery) can make a significant contribution to the reduction of overheating.

The recently introduced Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA), with a lean, but sufficient, linkage model and an efficient variation operator, has been shown to be a robust and efficient methodology for solving single objective (SO) optimization problems with superior performance compared to classic genetic algorithms (GAs) and estimation-of-distribution algorithms (EDAs). In this paper, we bring the strengths of GOMEAs to the multi-objective (MO) optimization realm. To this end, we modify the linkage learning procedure and the variation operator of GOMEAs to better suit the need of finding the whole Pareto-optimal front rather than a single best solution. Based on state-of-the-art studies on MOEAs, we further pinpoint and incorporate two other essential components for a scalable MO optimizer. First, the use of an elitist archive is beneficial for keeping track of non-dominated solutions when the main population size is limited. Second, clustering can be crucial if different parts of the Pareto-optimal front need to be handled differently. By combining these elements, we construct a multi-objective GOMEA (MO-GOMEA). Experimental results on various MO optimization problems confirm the capability and scalability of our MO-GOMEA that compare favorably with those of the well-known GA NSGA-II and the more recently introduced EDA mohBOA.

The Power TAC simulates a smart grid energy market. In this simulation, broker agents compete for customers on a tariff market and trade energy on a wholesale market. It provides a platform for testing strategies of broker agents against other strategies. In this paper we describe the strategies of our broker agent. Amongst others, due to a beneficial trading technique related to equilibria in continuous auctions on the wholesale market and a strategy inspired by Tit-for-Tat in the Iterated Prisoner’s Dilemma game on the tariff market, our broker ended second in the 2013 Power TAC.

The EU-driven integration of European energy systems and the development of a Smart Energy System involves many key players. The success of a European Smart Energy Systems relies heavily on the development of well-designed ICT solutions in all related sectors. Because such ICT solutions should be well aligned, ICT innovation goals are needed that have the support of key European players and consortia. To this end, Round Tables have been organised by EIT ICT Labs from 2013 onwards, in which key players in the European energy sector establish a common vision on and align forces in the development of a European Smart Energy System. We reflect and build upon these discussions to formulate a joint input to the European goals on ICT and Smart Energy Systems, and for the innovation activities of EIT ICT Labs.

Offshore wind farms with high installed capacities and located further from the shore are starting to be built by northern European countries. Furthermore, it is expected that by 2020, several dozens of large offshore wind farms will be built in the Baltic, Irish and North seas. These wind farms will be constituted of a considerable number of turbines packed together. Due to shadowing effects between turbines, the power production is reduced, resulting in a decreased wind farm efficiency. Moreover, the collection system length and complexity rapidly increase with the number of wind turbines installed, resulting in higher investment costs and power losses. Hence, when large wind farms are considered, wake losses and collection system total length are important aspects that need to be optimized. Nonetheless, designing a large offshore wind project is a very complex task. In the development phase, an enormous amount of time is used to generate feasible wind farm designs. For example, the manual layout optimization and cable routing are highly time consuming. Amongst others, this drawback leads to offshore wind projects with very high investments costs. In fact, electricity generated offshore is, up to today, substantially more expensive when compared to onshore wind or combined cycle gas turbines. Wind farm layout optimization is a possible solution to further increase the efficiency of wind projects leading to lower prices of energy generated offshore. Moreover, investment costs would be reduced if the collection system could be automatically designed via optimization tools. The aim of this work is to obtain optimized offshore wind farm layouts with enhanced energy production while considering the seabed constraints, e.g. maximum sea depth, inappropriate soil types and large seabed gradients. Through the use of optimization tools it is also desired to alleviate the designer teams of mundane manual layout optimization and cable routing. The optimized wind farm layout obtained presents higher energy yield. Although, the collection system total length is higher than for the standard case, the overall system efficiency is improved. Moreover, installation costs of the collection system were reduced through the avoidance of challenging areas of the wind farm. The wind turbine support structures costs were also reduced since deep areas of the seabed were avoided. In this work it is presented a novel optimization tool to obtain optimized offshore wind farm layouts while taking into consideration the seabed profile. Due to the increased offshore activity in recent years, as well as the expected installation in the coming years, this tool becomes very important and relevant for the offshore field.

A key characteristic of Mixed-Integer (MI) problems is the presence of both continuous and discrete problem variables. These variables can interact in various ways, resulting in challenging optimization problems. In this paper, we study the design of an algorithm that combines the strengths of LTGA and iAMaLGaM: state-of-the-art model-building EAs designed for discrete and continuous search spaces, respectively. We examine and discuss issues which emerge when trying to integrate those two algorithms into the MI setting. Our considerations lead to a design of a new algorithm for solving MI problems, which we motivate and compare with alternative approaches.

Distribution network operators require more advanced planning tools to deal with the challenges of future network planning. An appropriate planning and optimization tool can identify which option for network extension should be selected from available alternatives. However, many optimization approaches described in the literature are quite theoretical and do not yield results that are practically relevant and feasible. In this paper, a distribution network planning approach is proposed which meets requirements originating from network planning practice to guarantee realistic outcomes. This approach uses a state-of-the-art evolutionary algorithm: Gene-pool Optimal Mixing Evolutionary Algorithm. The performance of this algorithm, as well as the proposed model, is demonstrated using a real-world case study.

Market Garden is a scalable research environment and demonstration tool, in which market mechanisms for smart energy systems and the interaction between end users, traders and system operators can be simulated. Users can create scenarios in a user-friendly editor in which a hierarchical market architecture and a model of the physical grid can be defined. The system sets up an interactive and competitive environment of modular market mechanisms. A visualiser in which the results can be explored in a graphical and intuitive way is included, making Market Garden very suitable for user experience and demonstration purposes, next to its application for research.

Currently, two major challenges dominate the field of deformable image registration. The first challenge is related to the tuning of the developed methods to specific problems (i.e. how to best combine different objectives such as similarity measure and transformation effort). This is one of the reasons why, despite significant progress, clinical implementation of such techniques has proven to be difficult. The second challenge is to account for large anatomical differences (e.g. large deformations, (dis)appearing structures) that occurred between image acquisitions. In this paper, we study a framework based on multi-objective optimization to improve registration robustness and to simplify tuning for specific applications. Within this framework we specifically consider the use of an advanced model-based evolutionary algorithm for optimization and a dual-dynamic transformation model (i.e. two "non-fixed" grids: one for the source- and one for the target image) to accommodate for large anatomical differences. The framework computes and presents multiple outcomes that represent efficient trade-offs between the different objectives (a so-called Pareto front). In image processing it is common practice, for reasons of robustness and accuracy, to use a multi-resolution strategy. This is, however, only well-established for single-objective registration methods. Here we describe how such a strategy can be realized for our multi-objective approach and compare its results with a single-resolution strategy. For this study we selected the case of prone-supine breast MRI registration. Results show that the well-known advantages of a multi-resolution strategy are successfully transferred to our multi-objective approach, resulting in superior (i.e. Pareto-dominating) outcomes.

The Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) is a well-known, state-of-the-art optimization algorithm for single-objective real-valued problems, especially in black-box settings. Although several extensions of CMA-ES to multi-objective (MO) optimization exist, no extension incorporates a key component of the most robust and general CMA-ES variant: the association of a population with each Gaussian distribution that drives optimization. To achieve this, we use a recently introduced framework for extending population-based algorithms from single- to multi-objective optimization. We compare, using six well-known benchmark problems, the performance of the newly constructed MO-CMA-ES with existing variants and with the estimation of distribution algorithm (EDA) known as iMAMaLGaM, that is also an instance of the framework, extending the single-objective EDA iAMaLGaM to MO. Results underline the advantages of being able to use populations. Because many real-world problems have constraints, we also study the use of four constraint-handling techniques. We find that CMA-ES is typically less robust to these techniques than iAMaLGaM. Moreover, whereas we could verify that a penalty method that was previously used in literature leads to fast convergence, we also find that it has a high risk of finding only nearly, but not entirely, feasible solutions. We therefore propose that other constraint-handling techniques should be preferred in general.

Distributed computing is the theoretical foundation for applications and technologies like interactive architecture, wearable computing, and smart materials. It evolves continuously, following needs rising from scientific developments, novel uses of technology, or simply the curiosity to better understand the world around us. The pace of evolution is fast: in a short span of time distributed computing helped the Internet develop into how we know it today and overcame the dynamics of the large spread of mobile wireless communication. Soon, we will see the effects of distributed computing in the rise of the Internet of Things. Interactive architecture brings a specific set of requirements to distributed computing (sheer number of devices, specific dynamics, resources, control constraints, etc.). The mapping between the requirements of interactive installation and distributed systems theory is well understood. Architects are already acquainted with concepts such as emergent behavior and swarm behavior and use them in their concepts. Nevertheless, despite the availability of all the constituent ingredients, the number of large-scale installations employing distributed behavior in hardware and software is very small. The vast majority of systems are still controlled from a central computer that creates the desired behavior and that faces obvious issues of scale. This is the main research question that we would like to tackle in this essay. Before going into details, let us contemplate for a moment the larger picture, making abstraction of the technological barriers. The dream of large-scale interactive systems responding and adapting instantly to the user’s desires will come one step closer to actualization. For example, the myriad inputs triggered by the crowds inhabiting a city will be translated automatically through extensively equipped intelligent surroundings. Aspects of control, safety, and privacy will be automatically dealt with in real time. Humans will have a new understanding of the world around them, which will simply adapt to suit their needs. Smart cities and the Internet of Things will finally become a reality.

In this position paper, we address the applicability of spatial computing in the field of interactive architecture. The process of designing large-scale interactive systems is cumbersome, due in fact to single design cycles (transforming ideas into prototypes) taking a period of time usually measured in months, most of it dedicated to writing the software controlling the system. As most interactive architecture projects pass through several design cycles interleaved with user studies, speeding up the generation of the needed software becomes of crucial importance. The global-to-local programming approach is in fact a perfect tool for this task. Describing complex behaviors with simple rules is rarely seen in the existing installations, the large majority employing a central computer for the control of the system. Building up on our previous experience in this area, we created HiveKit, a proof of concept allowing bridging between the two fields, giving non-specialists easy access to distributed algorithms. HiveKit is a software package which allows designers to specify the desired behavior and automatically generate and deploy the needed code on networks of embedded devices. We introduce several projects where HiveKit is employed and create an argument, based on user studies, favoring the need for large-scale adoption of such tools.

Demand response is a crucial mechanism for flattening of peak loads. For its implementation, we not only require consumers who react to price changes, but also intelligent strategies to select prices. We propose a parametrised meta-strategy for dynamic pricing and identify suitable strategies for given scenarios through offline optimisation using a population model. We also model an important and novel constraint: a price cap (a maximal unit price) for consumer protection. We show in computational simulations that the maximal unit price influences the peak reduction potential of dynamic pricing. We compare our dynamic pricing approach with a constant pricing approach and show that our approach, used by a profit-optimising seller, is both peak-reducing and equally profitable.

The linkage tree genetic algorithm (LTGA) learns, each generation, a linkage model by building a hierarchical cluster tree. The LTGA is an instance of the more general gene-pool optimal mixing evolutionary algorithm (GOMEA) that uses a family of subsets (FOS) linkage model. We compare the performance of the linkage model learning LTGA with several predetermined FOS linkage models applied by GOMEA. The predetermined models are matched to the underlying problem structure of four benchmark functions: onemax, deceptive trap functions, maximal overlapping nearest-neighbor NK-landscapes, and weighted MAXCUT problems. Although the a priori fixed models are specifically designed to capture the interactions between the problem variables, experimental results show that - for problems with intricate interaction structure - they are actually less efficient than LTGA that dynamically learns a hierarchical tree model. Some of these observations were unexpected and raise the question of what exactly is the optimal linkage structure for a given problem as used by GOMEA. In the case of the NK-problem a linkage model that is an accurate description of the underlying additively decomposable fitness structure is clearly not an optimal linkage model. Being able to rebuild the linkage model each generation has clear benefits above using fixed, predetermined linkage models.

This paper proposes a saliency-aware fusion algorithm for integrating infrared (IR) and visible light (ViS) images (or videos) with the aim to enhance the visualization of the latter. Our algorithm involves saliency detection followed by a biased fusion. The goal of the saliency detection is to generate a saliency map for the IR image, highlighting the co-occurrence of high brightness values ("hot spots") and motion. Markov Random Fields (MRFs) are used to combine these two sources of information. The subsequent fusion step is employed to bias the end result in favor of the ViS image, except when a region shows clear IR saliency, in which case the IR image gains (local) dominance. By doing so, the fused image succeeds in depicting both the salient foreground object (gleaned from the IR image), against as an easily recognizable background as supplied by the ViS image. An evaluation of the proposed saliency detection method indicates improvements in detection accuracy when compared to state-of-the-art alternatives. Moreover, both objective and subjective assessments reveal the effectiveness of the proposed fusion algorithm in terms of visual context enhancement.

Low-voltage radial electricity cables will have more and more difficulties to carry the increasing load of novel consumption devices (e.g. electric vehicles) and the expected generated input of decentrally-generated power (e.g. from photovoltaic cells). One solution to avoid replacement is to install a battery at the end of a cable which is expected to be overloaded frequently. The intelligent operation of this battery needs to combine the protection of the cable with optimizing its revenue, in order to be economically viable. This paper formulates the offline optimization problem and proposes two robust heuristic online strategies. We show in computer simulations that these heuristics, which make fast just-in-time responses, reliably deliver good results. Our second heuristic, H2, reaches up to 83% of the approximated theoretical optimum.

How will we trade energy in the future? We can expect vast technological changes in our energy systems, leading to high heterogeneity in both supply and demand. Knowing the true value of energy at a given time and location will be crucial. Market mechanisms are methods of determining prices in complex, multi-actor settings. In the Intelligent Systems group at CWI, we have developed a research environment in which different market mechanisms for electricity can be studied and evaluated - in interaction, remotely and in a scalable manner.

Satisfiability solving in fuzzy logics is a subject that has not been researched much, certainly compared to satisfiability in propositional logics. Yet, fuzzy logics are a powerful tool for modelling complex problems. Recently, we proposed an optimization approach to solving satisfiability in fuzzy logics and compared the standard Covariance Matrix Adaptation Evolution Strategy algorithm (CMA-ES) with an analytical solver on a set of benchmark problems. Especially on more finegrained problems did CMA-ES compare favourably to the analytical approach. In this paper, we evaluate two types of hillclimber in addition to CMA-ES, as well as restart strategies for these algorithms. Our results show that a population-based hillclimber outperforms CMA-ES on the harder problem class.

Ontologies provide a formal description of concepts and their relationships in a knowledge domain. The goal of ontology alignment is to identify semantically matching concepts and relationships across independently developed ontologies that purport to describe the same knowledge. In order to handle the widest possible class of ontologies, many alignment algorithms rely on terminological and structural methods, but the often fuzzy nature of concepts complicates the matching process. However, one area that should provide clear matching solutions due to its mathematical nature, is units of measurement. Several ontologies for units of measurement are available, but there has been no attempt to align them, notwithstanding the obvious importance for technical interoperability. We propose a general strategy to map these (and similar) ontologies by introducing MathML to accurately capture the semantic description of concepts specified therein. We provide mapping results for three ontologies, and show that our approach improves on lexical comparisons.

Modern electricity markets conduct a two-settlement procedure. Ahead of time, they allocate definite supply as well as reserves. Close to the time of consumption, they balance supply and demand. Bidding in these two auctions poses a challenge for automated bidding by agents, which will be more common in future electricity markets and so-called “smart grids”. In a decision-theoretic model, we implement the current bidding practice that uses two independent bids and a novel, unified format that simplifies computation. We show through Monte-Carlo simulations in one-shot settings that the unified format restricts market power of suppliers in exploitable settings, and is also less vulnerable to uncertainty of bidders about market outcomes.

Objective-space discretization is a popular method to control the elitist archive size for evolutionary multi-objective optimization and avoid problems with convergence. By setting the level of discretization, the proximity and diversity of the Pareto approximation set can be controlled. This paper proposes an adaptive archiving strategy which is developed from a rigid-grid discretization mechanism. The main advantage of this strategy is that the practitioner just decides the desirable target size for the elitist archive while all the maintenance details are automatically handled. We compare the adaptive and rigid archiving strategies on the basis of a performance indicator that measures front quality, success rate, and running time. Experimental results confirm the competitiveness of the adaptive method while showing its advantages in terms of transparency and ease of use.

Previous work on demand response in smart grids considers dynamic real-time prices, but has so far neglected to consider how consumers can also be involved in planning ahead, both for scheduling of consumption and reserving their ability to regulate downward during balancing. This work models a flexible consumer in a novel two-settlement electricity auction. The consumer buys electricity on an ahead market and offers downward regulation on the balancing market. Bidding in two- settlement procedures is a hard problem and the use case of a flexible consumer is a novel problem setting. This poses a challenge for the design of smart ICT that automates bidding in smart grid operations. In a decision-theoretic model that takes the viewpoint of a flexible consumer, we implement a novel, unified format that simplifies computation of bids and can make the guarantee that offering downward regulation increases overall utility. Simulations show that the unified format, when compared to a benchmark format with two independent bids, attains the same utility under a wide range of market conditions despite its simplicity of use. Furthermore, it ensures that the consumer’s offer for downward regulation is indeed executed with the UNI format, which is often not the case with the benchmark format. Index Terms—markets, balancing, flexibility, demand side

Efficient operation and planning of power systems is important for a reliable and sustainable electricity supply. Therefore, optimization techniques have been applied to several optimization problems in power systems in order to achieve technical and economic efficiency. This paper presents an overview of existing optimization techniques and applications in power systems, with a special focus on multi-objective optimization in power system planning. Power system planning is by its nature a very complex multi-objective optimization problem involving perspectives of different stakeholders. Besides, a single stakeholder can also have various objectives that need to be optimized at the same time. This paper provides a review of the state-of-the-art in multi-objective evolutionary algorithms applied to power systems planning problems.

The term smart environment refers to physical spaces equipped with sensors feeding into adaptive algorithms that enable the environment to become sensitive and responsive to the presence and needs of its occupants. People with special needs, such as the elderly or disabled people, stand to benefit most from such environments as they offer sophisticated assistive functionalities supporting independent living and improved safety. In a smart environment, the key issue is to sense the location and identity of its users. In this paper, we intend to tackle the problems of detecting and tracking humans in a realistic home environment by exploiting the complementary nature of (synchronized) color and depth images produced by a low-cost consumer-level RGB-D camera. Our system selectively feeds the complementary data emanating from the two vision sensors to different algorithmic modules which together implement three sequential components: (1) object labeling based on depth data clustering, (2) human re-entry identification based on comparing visual signatures extracted from the color (RGB) information, and (3) human tracking based on the fusion of both depth and RGB data. Experimental results show that this division of labor improves the system’s efficiency and classification performance.

Many traditional reinforcement-learning algorithms have been designed for problems with small finite state and action spaces. Learning in such discrete problems can been difficult, due to noise and delayed reinforcements. However, many real-world problems have continuous state or action spaces, which can make learning a good decision policy even more involved. In this chapter we discuss how to automatically find good decision policies in continuous domains. Because analytically computing a good policy from a continuous model can be infeasible, in this chapter we mainly focus on methods that explicitly update a representation of a value function, a policy or both. We discuss considerations in choosing an appropriate representation for these functions and discuss gradient-based and gradient-free ways to update the parameters. We show how to apply these methods to reinforcement-learning problems and discuss many specific algorithms. Amongst others, we cover gradient-based temporal-difference learning, evolutionary strategies, policy-gradient algorithms and actor-critic methods. We discuss the advantages of different approaches and compare the performance of a state-of-the-art actor-critic method and a state-of-the-art evolutionary strategy empirically.

Standard multiresolution image fusion of multimodal images may yield an output image with artifacts due to the occurrence of opposite contrast in the input images. Equal but opposite contrast leads to noisy patches, instable with respect to slight changes in the input images. Unequal and opposite contrast leads to uncertainty of how to interpret the modality of the result. In this paper a biased fusion is proposed to remedy this, where the bias is towards one image, the so-called iconic image, in a preferred spectrum. A nonlinear fusion rule is proposed to prevent that the fused image reverses the local contrasts as seen in the iconic image. The rule involves saliency and a local match measure. The method is demonstrated by artificial and real-life examples.

We present a new method to register a pair of visible (ViS) and infrared (IR) images. Unlike most of existing systems that align interest points of two images, we align lines derived from edge pixels, because the interest points extracted from both images are not always identical, but most major edges detected from one image do appear in another image. To solve feature matching problem, we emphasize the geometric structure alignment of features (lines), instead of descriptor-based individual feature matching. This is due to the fact that image properties and patch statistics of corresponding features might be quite different, especially when one compares ViS image with long wave IR images (thermal information). However, the spatial layout of features for both images always preserves consistency. The last step of our algorithm is to compute the image transform matrix, given minimum 4 pairs of line correspondence. The comparative evaluation for algorithms demonstrates higher accuracy attained by our method when compared to the state-of-the-art approaches.

Dynamic Pricing (DyP) is a form of Revenue Management in which the price of a (usually) perishable good is changed over time to increase revenue. It is an effective method that has become even more relevant and useful with the emergence of Internet firms and the possibility of readily and frequently updating prices. In this paper a new approach to DyP is presented. We design adaptive dynamic pricing strategies and optimize their parameters with an Evolutionary Algorithm (EA) offline while the strategies can deal with stochastic market dynamics quickly online. We design two adaptive heuristic dynamic pricing strategies in a duopoly where each firm has a finite inventory of a single type of good. We consider two cases, one in which the average of a customer population’s stochastic valuation for each of the goods is constant throughout the selling horizon and one in which the average customer valuation for each good is changed according to a random Brownian motion. We also design an agent-based software framework for simulating various dynamic pricing strategies in agent-based marketplaces with multiple firms in a bounded time horizon. We use an EA to optimize the parameters for each of the pricing strategies in each of the settings and compare the strategies with other strategies from the literature. We also perform sensitivity a analysis and show that the optimized strategies work well even when used in settings with varied demand functions.

As power markets become liberalised and include more intermittent generation, the trade of reserve energy will become more important. We propose a novel bidding mechanism to integrate power and reserve markets. It facilitates planning for bidding in both markets and adds expressivity to reserve bids\footnote{This work is a part of the IDeaNeD project and sponsored by Agentschap NL, a research funding agency of the dutch ministry of economic affairs, in the IOP-EMVT program. It has also been presented at the AAMAS 2011 conference.