Back to Search Start Over

Publisher preferences for a journal transparency tool: A modified three-round Delphi study [version 2; peer review: 2 approved with reservations]

Authors :
Jeremy Y. Ng
Henry Liu
Mehvish Masood
Rubaina Farin
Mireille Messih
Amaya Perez
IJsbrand Jan Aalbersberg
Juan Alperin
Gregory L. Bryson
Qiuxia Chen
Alan Ehrlich
Alfonso Iorio
Wim J. N. Meester
John Willinsky
Agnes Grudniewicz
Erik Cobo
Imogen Cranston
Phaedra Eve Cress
Julia Gunn
R. Brian Haynes
Bibi Sumera Keenoo
Ana Marušić
Eleanor-Rose Papas
Alan Purvis
João de Deus Barreto Segundo
Pathiyil Ravi Shankar
Pavel Stoev
Josephine Weisflog
Margaret Winker
Kelly D. Cobey
David Moher
Author Affiliations :
<relatesTo>1</relatesTo>Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada<br /><relatesTo>2</relatesTo>Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada<br /><relatesTo>3</relatesTo>Elsevier, Amsterdam, The Netherlands<br /><relatesTo>4</relatesTo>School of Publishing, Simon Fraser University, Vancouver, British Columbia, Canada<br /><relatesTo>5</relatesTo>Department of Anesthesiology and Pain Medicine, University of Ottawa, Ottawa, Ontario, Canada<br /><relatesTo>6</relatesTo>Department of Family Medicine and Community Health, UMass Chan Medical School, Worcester, Massachusetts, USA<br /><relatesTo>7</relatesTo>EBSCO Information Services, Ipswich, Massachusetts, USA<br /><relatesTo>8</relatesTo>Department of Medicine, McMaster University, Hamilton, Ontario, Canada<br /><relatesTo>9</relatesTo>Graduate School of Education, Stanford University, Stanford, California, USA<br /><relatesTo>10</relatesTo>Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada<br /><relatesTo>11</relatesTo>Department of Statistics and Operations Research, Barcelona-Tech, UPC, Barcelona, Spain<br /><relatesTo>12</relatesTo>F1000, Taylor & Francis, London, UK<br /><relatesTo>13</relatesTo>The Aesthetic Society, Garden Grove, California, USA<br /><relatesTo>14</relatesTo>Taylor & Francis, London, UK<br /><relatesTo>15</relatesTo>Faculty of Medicine and Health Sciences, University of Mauritius, Réduit, Moka, Mauritius<br /><relatesTo>16</relatesTo>Department of Research in Biomedicine and Health, Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia<br /><relatesTo>17</relatesTo>Independent Researcher, Croydon, London, UK<br /><relatesTo>18</relatesTo>Hong Kong Academy of Medicine Press, Hong Kong, Hong Kong<br /><relatesTo>19</relatesTo>Escola Bahiana de Medicina e Saude Publica, Salvador, State of Bahia, Brazil<br /><relatesTo>20</relatesTo>IMU Centre for Education, IMU University, Kuala Lumpur, Malaysia<br /><relatesTo>21</relatesTo>Pensoft Publishers, Sofia, Bulgaria<br /><relatesTo>22</relatesTo>National Museum of Natural History at the Bulgarian Academy of Sciences, Sofia, Bulgaria<br /><relatesTo>23</relatesTo>BMJ Publishing Group, London, UK<br /><relatesTo>24</relatesTo>Program Director and Trustee, World Association of Medical Editors, Bellagio, Italy<br /><relatesTo>25</relatesTo>University of Ottawa Heart Institute, Ottawa, Ontario, Canada<br /><relatesTo>26</relatesTo>School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
Source :
F1000Research. 13:915
Publication Year :
2025
Publisher :
London, UK: F1000 Research Limited, 2025.

Abstract

Background We propose the creation of a journal transparency tool (JTT), which will allow users to obtain information about a given scholarly journal’s operations and policies. We are obtaining preferences from different stakeholders to inform the development of this tool. This study aimed to identify the publishing community’s preferences for the JTT. Methods We conducted a modified three-round Delphi survey. Representatives from publishing houses and journal publishers were recruited through purposeful and snowball sampling. The first two Delphi rounds involved an online survey with items about JTT metrics and user features. During the third round, participants discussed and voted on JTT metric items that did not reach consensus after round 2 within a virtual consensus meeting. We defined consensus as 80% agreement to include or exclude an item in the JTT. Results Eighty-six participants completed the round 1 survey, and 43 participants (50% of round 1) completed the round 2 survey. In both rounds, respondents voted on JTT user feature and JTT metric item preferences and answered open-ended survey questions regarding the JTT. In round 3, a total of 21 participants discussed and voted on JTT metric items that did not reach consensus after round 2 during an online consensus group meeting. Fifteen out of 30 JTT metric items and none of the four JTT user feature items reached the 80% consensus threshold after all rounds of voting. Analysis of the round 3 online consensus group transcript resulted in two themes: ‘factors impacting support for JTT metrics’ and ‘suggestions for user clarity.’ Conclusions Participants suggested that the publishing community’s primary concerns for a JTT are to ensure that the tool is relevant, user-friendly, accessible, and equitable. The outcomes of this research will contribute to developing and refining the tool in accordance with publishing preferences.

Subjects

Subjects :
Research Article
Articles
health literacy
scholarly publishing
periodicals as topic
publications
ethics in publishing
journal transparency tool

Details

ISSN :
20461402
Volume :
13
Database :
F1000Research
Journal :
F1000Research
Notes :
Revised Amendments from Version 1 We thank both reviewers for their feedback. In response to comments provided by Andreas Nishikawa-Pacher, machine-readable survey results have been uploaded to Open Science Framework (OSF), and references to similar initiatives (e.g., the Journal Observatory) were added to the manuscript. Further, concerns raised by this reviewer regarding defining 'transparency' and difficulties with feasibly enacting all participant preferences are acknowledged and appreciated. In response, we note that the Journal Transparency Tool (JTT) is currently within preliminary stages, and defining such items will require the collation of data from multiple stakeholders, leading to the limited information being provided regarding such elements within the present report. Further, we appreciate the comment that the abstract’s results may be more interesting if it focused on the contents of the survey outcome rather than numerical data. However, we believe that the present focus aptly represents this report’s findings due to the ‘numeric’ nature of the data provided by the present study design (i.e., a modified Delphi). Alternatively, we appreciate the concerns raised by Bahar Memarian regarding the tool's contribution and design, along with the purpose of the present research study. In response, we clarify that the JTT aims to help users identify journals that adhere to transparent research practices and highlight how this addresses a known gap within the literature. Further, the goal of this research study was to use a modified Delphi approach to capture publisher preferences for the JTT. The sections of the manuscript highlighting such information have been provided within our response for further clarity., , [version 2; peer review: 2 approved with reservations]
Publication Type :
Academic Journal
Accession number :
edsfor.10.12688.f1000research.154408.2
Document Type :
research-article
Full Text :
https://doi.org/10.12688/f1000research.154408.2
Full Text Access
View/download PDF

Tools

Authors Abstract Subjects Details