Search

This paper studies novel subgoaling relaxations for automated planning with propositional and numeric state variables. Subgoaling relaxations address one source of complexity of the planning problem: the requirement to satisfy conditions simultaneously. The core idea is to relax this requirement by recursively decomposing conditions into atomic subgoals that are considered in isolation. Such relaxations are typically used for pruning, or as the basis for computing admissible or inadmissible heuristic estimates to guide optimal or satisficing heuristic search planners. In the last decade or so, the subgoaling principle has underpinned the design of an abundance of relaxation-based heuristics whose formulations have greatly extended the reach of classical planning. This paper extends subgoaling relaxations to support numeric state variables and numeric conditions. We provide both theoretical and practical results, with the aim of reaching a good trade-off between accuracy and computation costs within a heuristic state-space search planner. Our experimental results validate the theoretical assumptions, and indicate that subgoaling substantially improves on the state of the art in optimal and satisficing numeric planning via forward state-space search. [ABSTRACT FROM AUTHOR]

In this paper, we discuss the learning of generalised policies for probabilistic and classical planning problems using Action Schema Networks (ASNets). The ASNet is a neural network architecture that exploits the relational structure of (P)PDDL planning problems to learn a common set of weights that can be applied to any problem in a domain. By mimicking the actions chosen by a traditional, non-learning planner on a handful of small problems in a domain, ASNets are able to learn a generalised reactive policy that can quickly solve much larger instances from the domain. This work extends the ASNet architecture to make it more expressive, while still remaining invariant to a range of symmetries that exist in PPDDL problems. We also present a thorough experimental evaluation of ASNets, including a comparison with heuristic search planners on seven probabilistic and deterministic domains, an extended evaluation on over 18,000 Blocksworld instances, and an ablation study. Finally, we show that sparsity-inducing regularisation can produce ASNets that are compact enough for humans to understand, yielding insights into how the structure of ASNets allows them to generalise across a domain. [ABSTRACT FROM AUTHOR]

A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decision-theoretic planning, where many desirable behaviours are more naturally expressed as properties of execution sequences rather than as properties of states, NMRDPs form a more natural model than the commonly adopted fully Markovian decision process (MDP) model. While the more tractable solution methods developed for MDPs do not directly apply in the presence of non-Markovian rewards, a number of solution methods for NMRDPs have been proposed in the literature. These all exploit a compact specification of the non-Markovian reward function in temporal logic, to automatically translate the NMRDP into an equivalent MDP which is solved using efficient MDP solution methods. This paper presents NMRDPP (Non-Markovian Reward Decision Process Planner), a software platform for the development and experimentation of methods for decision-theoretic planning with non-Markovian rewards. The current version of NMRDPP implements, under a single interface, a family of methods based on existing as well as new approaches which we describe in detail. These include dynamic programming, heuristic search, and structured methods. Using NMRDPP, we compare the methods and identify certain problem features that affect their performance. NMRDPP's treatment of non-Markovian rewards is inspired by the treatment of domain-specific search control knowledge in the TLPlan planner, which it incorporates as a special case. In the First International Probabilistic Planning Competition, NMRDPP was able to compete and perform well in both the domain-independent and hand-coded tracks, using search control knowledge in the latter. [ABSTRACT FROM AUTHOR]

Building planning systems that operate in real domains requires coping with both uncertainty and time pressure. This article describes a model of reaction plans, which are generated using a formalization of actions and of state descriptions in probabilistic logic, as a basis for anytime planning under uncertainty. The model has the following main features. At the action level, we handle incomplete and ambiguous domain information, and reason about alternative action effects whose probabilities are given. On this basis, we generate reaction plans that specify different courses of action, reflecting the domain uncertainty and alternative action effects; if generation time was insufficient, these plans may be left unfinished, but they can be reused, incrementally improved, and finished later. At the planning level, we develop a framework for measuring the quality of plans that takes domain uncertainty and probabilistic information into account using Markov chain theory; based on this framework, one can design anytime algorithms focusing on those parts of an unfinished plan first, whose completion promises the most "gain". Finally, the plan quality can be updated during execution, according to additional information acquired, and can therefore be used for on-line planning. [ABSTRACT FROM AUTHOR]

• Proposing a co-optimisation approach for consumers to participate in the energy and frequency control ancillary service (FCAS) markets. • Guaranteeing network feasibility by including three AC-OPFs for each operating condition, i.e., one for energy market, one for FCAS raise and another for FCAS lower market. • Developing a distributed algorithm based on the alternating direction method of multipliers (ADMM) which enables the consumers and the network to solve their problem separately and in parallel. This on one hand increases privacy and on the other hand, reduces the computation time of the problem. • Implementing the proposed approach on a variable-time discretisised receding horizon framework to avoid shortsighted decisions while reducing the problem size. With the increasing uptake of distributed energy resources (DER), such as rooftop PV and batteries, a significant amount of electricity is generated at a smaller scale. Consequently, the large-scale generators, responsible for frequency response, are having less share in our electricity markets. This makes it challenging to provide the grid with the required frequency response. To overcome this challenge, we introduce a network-aware co-optimisation approach that enables consumers to co-optimise their DER in energy and frequency markets. Our approach is based on the alternating direction method of multipliers where consumers and the grid negotiate on a receding horizon framework to obtain consensus solutions which satisfy the grid constraints under all operating conditions. Our experiments on 69 and 141 bus networks show a significant improvement over a case where decisions are not co-optimised. We also show how, compared to the majority of the literature, we can avoid network violations. [ABSTRACT FROM AUTHOR]