Software is becoming the core aspect in robotics development and it is growing in terms of complexity and size. However, roboticists and researchers are struggling in ensuring and even measuring the quality of their software with respect to run-time properties such as energy efficiency and performance. This paper presents Robot Runner, a tool for streamlining the execution of measurement-based experiments involving robotics software. The tool is able to automatically setup, start, resume, and fully replicate user-defined experiments. Thanks to its plugin-based architecture, the tool is fully independent of the number, type, and complexity of the used robots (both real and simulated). GitHub repository - https://github.com/S2-group/robot-runner Youtube video - https://youtu.be/le-SAXI2k1.