Search

Classical reinforcement learning (RL) techniques are generally concerned with the design of decision-making policies driven by the maximisation of the expected outcome. Nevertheless, this approach does not take into consideration the potential risk associated with the actions taken, which may be critical in certain applications. To address that issue, the present research work introduces a novel methodology based on distributional RL to derive sequential decision-making policies that are sensitive to the risk, the latter being modelled by the tail of the return probability distribution. The core idea is to replace the Q function generally standing at the core of learning schemes in RL by another function, taking into account both the expected return and the risk. Named the risk-based utility function U, it can be extracted from the random return distribution Z naturally learnt by any distributional RL algorithm. This enables the spanning of the complete potential trade-off between risk minimisation and expected return maximisation, in contrast to fully risk-averse methodologies. Fundamentally, this research yields a truly practical and accessible solution for learning risk-sensitive policies with minimal modification to the distributional RL algorithm, with an emphasis on the interpretability of the resulting decision-making process.

Estimating the distance to objects is crucial for autonomous vehicles, but cost, weight or power constraints sometimes prevent the use of dedicated depth sensors. In this case, the distance has to be estimated from on-board mounted RGB cameras, which is a complex task especially for environments such as natural outdoor landscapes. In this paper, we present a new depth estimation method suitable for use in such landscapes. First, we establish a bijective relationship between depth and the visual parallax of two consecutive frames and show how to exploit it to perform motion-invariant pixel-wise depth estimation. Then, we detail our architecture which is based on a pyramidal convolutional neural network where each level refines an input parallax map estimate by using two customized cost volumes. We use these cost volumes to leverage the visual spatio-temporal constraints imposed by motion and make the network robust for varied scenes. We benchmarked our approach both in test and generalization modes on public datasets featuring synthetic camera trajectories recorded in a wide variety of outdoor scenes. Results show that our network outperforms the state of the art on these datasets, while also performing well on a standard depth estimation benchmark.

For transient stability analysis of a multi-machine power system, the Extended Equal Area Criterion (EEAC) method applies the classic Equal Area Criterion (EAC) concept to an approximate One Machine Infinite Bus (OMIB) equivalent of the system to find the critical clearing angle. The system-critical clearing time can then be obtained by numerical integration of OMIB equations. The EEAC method was proposed in the 1980s and 1990s as a substitute for time-domain simulation for Transmission System Operators (TSOs) to provide fast, transient stability analysis with the limited computational power available those days. To ensure the secure operation of the power system, TSOs have to identify and prevent potential critical scenarios through offline analyses of a few dangerous ones. These days, due to increased uncertainties in electrical power systems, the number of these critical scenarios is increasing, substantially, calling for fast, transient stability analysis techniques once more. Among them, the EEAC is a unique approach that provides not only valuable information, but also a graphical representation of system dynamics. This paper revisits the EEAC but from a modern, functional point of view. First, the definition of the OMIB model of a multi-machine power system is redrawn in its general form. To achieve fast, transient stability analysis, EEAC relies on approximate models of the true OMIB model. These approximations are clarified, and the EAC concept is redefined with a general definition for instability, and its conditions. Based on the defined conditions and definitions, functions are developed for each EEAC building block, which are later put out together to provide a full-resolution, functional scheme. This functional scheme not only covers the previous literature on the subject, but also allows to introduce several possible new EEAC approaches and provides a detailed description of their implementation procedure. A number of approaches are applied to the French EHV network, and the approximations are examined.

Retailers and major consumers of electricity generally purchase an important percentage of their estimated electricity needs years ahead in the forward market. This long-term electricity procurement task consists of determining when to buy electricity so that the resulting energy cost is minimised, and the forecast consumption is covered. In this scientific article, the focus is set on a yearly base load product from the Belgian forward market, named calendar (CAL), which is tradable up to three years ahead of the delivery period. This research paper introduces a novel algorithm providing recommendations to either buy electricity now or wait for a future opportunity based on the history of CAL prices. This algorithm relies on deep learning forecasting techniques and on an indicator quantifying the deviation from a perfectly uniform reference procurement policy. On average, the proposed approach surpasses the benchmark procurement policies considered and achieves a reduction in costs of 1.65% with respect to the perfectly uniform reference procurement policy achieving the mean electricity price. Moreover, in addition to automating the complex electricity procurement task, this algorithm demonstrates more consistent results throughout the years. Eventually, the generality of the solution presented makes it well suited for solving other commodity procurement problems.

In the context of increasing decentralised electricity generation, this paper evaluates the effect of different regulatory frameworks on the evolution of distribution networks. This problem is addressed by means of agent based modelling in which the interactions between the agents of a distribution network and an environment are described. The consumers and the distribution system operator are the agents, which act in an environment that is composed by a set of rules. For a given environment, we can simulate the evolution of the distribution network by computing the actions of the agents at every time step of a discrete time dynamical system. We assume the electricity consumers are rational agents that may deploy distributed energy installations. The deployment of such installations may alter the remuneration mechanism of the distribution system operator. By modelling this mechanism, we may compute the evolution of the electricity distribution tariff in response to the deployment of distributed generation.