A meta-evaluation of climate policy evaluations : findings from the freight transport sector

Knowledge about how implemented policy instruments have performed is important for designing effective and efficient policy instruments that contribute to reductions of greenhouse gas emissions. This paper carries out a meta-evaluation of ex-post evaluations of climate policy instruments in the freight transport sector. By analysing the outcomes and quality of evaluations, the aim is to identify whether estimated effects of policy instruments can be compared between evaluations and if the results are appropriate to use for evidence-based decision making. To analyse these aspects, commonly applied evaluation criteria are assessed and classified according to an assessment scale. We confirm that few ex-post evaluations are carried out and that there is a gap between evaluation theory and how ex-post policy evaluations are performed in practice, where evaluation criteria recommended in policy evaluation guidelines are found to often be neglected in evaluations. The result is a lack of systematic climate policy evaluation which hinders reliable conclusions about the effect of policy instruments. There is a need for more systematic monitoring and evaluation of implemented policy instruments and we suggest that evidence-based decision making can be improved by adjusting current policy evaluation guidelines and by introducing an evaluation obligation.
Research funding was provided by the Swedish Transport Administration via Lindholmen Science Park AB for the project Triple F: Policy Instruments for Fossil Free Freight (PIFF) (VTI project number 204918 and Triple F project number 2020.3.2.14).