Search

• Multi-agent framework for optimization using metaheuristics. • Agents modify their actions using concepts of Reinforcement Learning. • Learning ability of the agents directly influences the quality of solutions. • Framework validated using Vehicle Routing Problem with Time-Windows (VRPTW) and Unrelated Parallel Machine Scheduling Problem with Sequence-Dependent Setup Times (UPMSP-ST). This article presents a multi-agent framework for optimization using metaheuristics, called AMAM. In this proposal, each agent acts independently in the search space of a combinatorial optimization problem. Agents share information and collaborate with each other through the environment. The goal is to enable the agent to modify their actions based on experiences gained in interacting with the other agents and the environment using the concepts of Reinforcement Learning. For better introduction and validation of the AMAM framework, this article uses the instantiation of the Vehicle Routing Problem with Time Windows (VRPTW) and the Unrelated Parallel Machine Scheduling Problem with Sequence-Dependent Setup Times (UPMSP-ST), i.e., two classic combinatorial optimization problems. The main objective of the experiments is to evaluate the performance of the proposed adaptive agents. The experiments confirm that the ability to learn attributed to the agent directly influences the quality of solutions, both from the individual point of view and from the point of view of teamwork. In this way, the framework presented here is a step forward in relation to the other frameworks of the literature regarding to the adaptation to the particular aspects of the problems. Additionally, the cooperation between agents and their ability to influence the quality of the solutions of the agents involved in the search of the solution is confirmed. The results also strengthen the issue of the scalability of the framework, since, with the addition of new agents, there is an improvement of the solutions obtained. [ABSTRACT FROM AUTHOR]

Graphical abstract Highlights • A review concerning frameworks for solving Optimization problems using metaheuristics. • A comparative analysis between frameworks for solving Optimization problems using metaheuristics. • Identify gaps in the characteristics necessary for the development of frameworks for optimization using metaheuristics. • A special focus on the use of multi-agent structures in the development of hybrid metaheuristics. • Multi-agents structures as the central role of this revision, perspective of analysis not yet addressed in the literature. Abstract This article presents a review and a comparative analysis between frameworks for solving optimization problems using metaheuristics. The aim is to identify both the desirable characteristics as the existing gaps in the current state of the art, with a special focus on the use of multi-agent structures in the development of hybrid metaheuristics. A literature review of existing frameworks is introduced, with emphasis on their characteristics of hybridization, cooperation, and parallelism, particularly focusing on issues related to the use of multi-agents. For the comparative analysis, a set of twenty-two characteristics was listed, according to four categories: basics, advanced, multi-agent approach and support to the optimization process. Strategies used in hybridization, such as parallelism, cooperation, decomposition of the search space, hyper-heuristic and multi-agent systems are assessed in respect to their use in the various analyzed frameworks. Specific features of multi-agent systems, such as learning and interaction between agents, are also analyzed. The comparative analysis shows that the hybridization is not a strong feature in existing frameworks. On the other hand, proposals using multi-agent systems stand out in the implementation of hybrid methods, as they allow the interaction between metaheuristics. It also notes that the concept of hyper-heuristic is little explored by the analyzed frameworks, as well as there is a lack of tools that offer support to the optimization process, such as statistical analysis, self-tuning of parameters and graphical interfaces. Based on the presented analysis, it can be said that there are important gaps to be filled in the development of Frameworks for Optimization using metaheuristics, which open important possibilities for future works, particularly by implementing the approach of multi-agent systems. [ABSTRACT FROM AUTHOR]