Search

Continual model learning for nonlinear dynamic systems, such as autonomous robots, presents several challenges. First, it tends to be computationally expensive as the amount of data collected by the robot quickly grows in time. Second, the model accuracy is impaired when data from repetitive motions prevail in the training set and outweigh scarcer samples that also capture interesting properties of the system. It is not known in advance which samples will be useful for model learning. Therefore, effective methods need to be employed to select informative training samples from the continuous data stream collected by the robot. Existing literature does not give any guidelines as to which of the available sample-selection methods are suitable for such a task. In this paper, we compare five sample-selection methods, including a novel method using the model prediction error. We integrate these methods into a model learning framework based on symbolic regression, which allows for learning accurate models in the form of analytic equations. Unlike the currently popular data-hungry deep learning methods, symbolic regression is able to build models even from very small training data sets. We demonstrate the approach on two real robots: the TurtleBot mobile robot and the Parrot Bebop drone. The results show that an accurate model can be constructed even from training sets as small as 24 samples. Informed sample-selection techniques based on prediction error and model variance clearly outperform uninformed methods, such as sequential or random selection.

State-of-the-art critic-only reinforcement learning methods can deal with a small discrete action space. The most common approach to real-world problems with continuous actions is to discretize the action space. In this paper a method is proposed to derive a continuous-action policy based on a value function that has been computed for discrete actions by using any known algorithm such as value iteration. Several variants of the policy-derivation algorithm are introduced and compared on two continuous state-action benchmarks: double pendulum swing-up and 3D mountain car.

It is well known that reinforcement learning (RL) can benefit from the use of a dynamic prediction model which is learned on data samples collected online from the process to be controlled. Most RL algorithms are formulated in the state-space domain and use state-space models. However, learning state-space models is difficult, mainly because in the vast majority of problems the full state cannot be measured on the system or reconstructed from the measurements. To circumvent this limitation, we propose to use input-output models of the NARX (nonlinear autoregressive with exogenous input) type. Symbolic regression is employed to construct parsimonious models and the corresponding value functions. Thanks to this approach, we can learn accurate models and compute optimal policies even from small amounts of training data. We demonstrate the approach on two simulated examples, a hopping robot and a 1-DOF robot arm, and on a real inverted pendulum system. Results show that our proposed method can reliably determine a good control policy based on a symbolic input-output process model and value function.

Reinforcement learning (RL) is a suitable approach for controlling systems with unknown or time-varying dynamics. RL in principle does not require a model of the system, but before it learns an acceptable policy, it needs many unsuccessful trials, which real robots usually cannot withstand. It is well known that RL can be sped up and made safer by using models learned online. In this paper, we propose to use symbolic regression to construct compact, parsimonious models described by analytic equations, which are suitable for realtime robot control. Single node genetic programming (SNGP) is employed as a tool to automatically search for equations fitting the available data. We demonstrate the approach on two benchmark examples: a simulated mobile robot and the pendulum swing-up problem; the latter both in simulations and real-time experiments. The results show that through this approach we can find accurate models even for small batches of training data. Based on the symbolic model found, RL can control the system well.

This paper presents an analysis of selected social networking portals for science, research and innovation as the first step in development of a portal for social network of IT professionals in the Czech Republic. First, actual portal implementations of four portals dedicated to scientific communities of researchers are analyzed, compared from the perspective of provided functions, layout and their interdependence and the main features are summarized. Then, the most important recommendations for the portal being developed are presented. The last part of the paper introduces two concepts of structured profiles - knowledge profile and scientific profile - that are proposed in order to precisely and explicitly express knowledge and scientific alignment of an IT professional. These two profiles will make it possible to search and compare IT specialists, their groups and projects or pair the users to the demands on the market. The paper concludes with a list of typical use cases where the profiles will be utilized.

This paper presents a first step of our research on designing an effective and efficient GP-based method for solving the symbolic regression. We have proposed three extensions of the standard Single Node GP, namely (1) a selection strategy for choosing nodes to be mutated based on the depth of the nodes, (2) operators for placing a compact version of the best tree to the beginning and to the end of the population, and (3) a local search strategy with multiple mutations applied in each iteration. All the proposed modifications have been experimentally evaluated on three symbolic regression problems and compared with standard GP and SNGP. The achieved results are promising showing the potential of the proposed modifications to significantly improve the performance of the SNGP algorithm.

This paper presents a novel evolutionary algorithm for solving routing and sequencing problems. It builds directly on the observation that the optimal solution is composed mostly of short and low-cost links. It uses an indirect representation and an extended nearest neighbor constructive procedure. The representation scheme is redundant making it possible to have multiple attractors in the search space representing the same optimal solution. The order-based crossover is used that allows for combining partial solutions contained in parental individuals. Further, an adaptive iterated scheme is used to allow for escaping from a region of attraction of a local optimum. Proof-of-concept experiments have been carried out on the classical TSP with data sets ranging from 100 to 2392 nodes. The proposed algorithm was compared with two other relevant EAs. The achieved results clearly show the proposed method is competitive to and better than the compared EAs.

This paper describes the concept of a social network of the ICT specialists in the regions of the Czech Republic. In particular, we focus on the web portal under development, i.e. a software tool serving for the network implementation. Associated activities concerning collecting and analyzing ICT requirements from companies and educational ICT knowledge of university graduates are presented. The novelty of our approach is in focusing on specificities of the social network (entities and relationships specific for ICT) and in the concept of professional profile (PP). Professional profiles enable to express the ICT focus of a given entity (e.g., person, research team or project) in a structured way. Based of the PP concept, we introduce specific portal functionalities which are not offered by existing portals.

This paper proposes a novel grammar-based framework of concept representation for randomized search in Relational Learning (RL), namely for Inductive Logic Programming. The utilization of grammars guarantees that the search operations produce syntactically correct concepts and that the background knowledge encoded in the grammar can be used both for directing the search and for restricting the space of possible concepts to relevant candidate concepts (semantically valid concepts). Not only that it enables handling and incorporating the domain knowledge in a declarative fashion, but grammars also make the new approach transparent, flexible, less problem-specific and allow it to be easily used by almost any randomized algorithm within RL. Initial test results suggest that the grammar-based algorithm has strong potential for RL tasks.

Recently, a new iterative optimization framework utilizing an evolutionary algorithm called "Prototype Optimization with Evolved iMprovement Steps" (POEMS) was introduced, which showed good performance on hard optimization problems - large instances of TSP and real-valued optimization problems. Especially, on discrete optimization problems such as the TSP the algorithm exhibited much better search capabilities than the standard evolutionary approaches. In many real-world optimization problems a solution is sought for multiple (conflicting) optimization criteria. This paper proposes a multiobjective version of the POEMS algorithm (mPOEMS), which was experimentally evaluated on the multiobjective 0/1 knapsack problem with alternative multiobjective evolutionary algorithms. Major result of the experiments was that the proposed algorithm performed comparable to or better than the alternative algorithms.

Evolutionary algorithms have already been more or less successfully applied to a wide range of optimisation problems. Typically, they are used to evolve a population of complete candidate solutions to a given problem, which can be further refined by some problem-specific heuristic algorithm. In this paper, we introduce a new framework called Iterative Prototype Optimisation with Evolved Improvement Steps. This is a general optimisation framework, where an initial prototype solution is being improved iteration by iteration. In each iteration, a sequence of actions/operations, which improves the current prototype the most, is found by an evolutionary algorithm. The proposed algorithm has been tested on problems from two different optimisation problem domains – binary string optimisation and the traveling salesman problem. Results show that the concept can be used to solve hard problems of big size reliably achieving comparably good or better results than classical evolutionary algorithms and other selected methods.