Is the methodology used in reviews of restoration outcomes reliable? A systematic map protocol

Abstract 1. Over the past decades, evidence‐based research has become increasingly important in restoration ecology. Evidence synthesis can be a powerful tool to identify the most effective strategies to conserve and restore ecosystems. However, reviews in the environmental sector have been described as non‐systematic and exhibit a diverse range of approaches. While it is known that environmental syntheses can substantially improve in quality, the reliability of restoration reviews (i.e. the level of confidence an end‐user may place in their methodology) remains poorly known. 2. Given the importance of literature reviews of restoration practice and outcomes for informing management and policy, as well as research, this systematic map protocol aims to scrutinize the peer‐reviewed literature for an assessment of the methodological reliability and reproducibility of restoration reviews. We will use bibliographic databases and search engines to collect studies published in peer‐reviewed journals dealing with the ecological restoration of terrestrial ecosystems. 3. Through a scoping exercise, a search string was developed which was based on a previously prepared test list. The search string was then tested for validity with one independent reference list. After searching, the screening process will be done on the title, abstract and full‐text level and consistency checking will be done on a random subsample by a second assessor, with decisions being compared using the kappa test of agreement. After retrieving studies and checking for relevance to the synthesis, we will appraise the methodological reliability of restoration reviews by applying the Collaboration for Environmental Evidence Assessment Tool – CEESAT. Lastly, we will collect bibliometric information to qualitatively describe the retrieved body of literature, and then key trends in data will be synthesized according to a range of generic questions. 4. To conduct the resulting review, we will follow the procedures specified in this protocol, considering guidelines from the Collaboration for Environmental Evidence and ROSES form. The resulting review will yield a useful overview of applying systematic reviews principles for various end users. At the same time, it will help restoration practitioners to identify critical points where restoration evidence syntheses must be improved to move forward.