Search

Background: In the evaluation of research proposals, reviewers are often required to provide their opinions using various forms of quantitative and qualitative criteria. In 2020, the European Commission removed, for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) funding scheme, the numerical scores from the individual evaluations but retained them in the consensus report. This study aimed to assess whether there were any differences in reviewer comments’ linguistic characteristics after the numerical scoring was removed, compared to comments from 2019 when numerical scoring was still present. Methods: This was an observational study and the data were collected for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) evaluation reports from the calls of 2019 and 2020, for both individual and consensus comments and numerical scores about the quality of the research proposal on three evaluation criteria: Excellence, Impact and Implementation. All comments were analyzed using the Linguistic Inquiry and Word Count (LIWC) program. Results: For both years, the comments for proposal’s strengths were written in a style that reflects objectivity, clout, and positive affect, while in weaknesses cold and objective style dominated, and that pattern remained stable across proposal status and research domains. Linguistic variables explained a very small proportion of the variance of the differences between 2019 and 2020 (McFadden R2=0.03). Conclusions: Removing the numerical scores was not associated with the differences in linguistic characteristics of the reviewer comments. Future studies should adopt a qualitative approach to assess whether there are conceptual changes in the content of the comments.

Abstract This cross-sectional study compared plain language summaries (PLSs) from medical and non-medical organizations regarding conclusiveness, readability and textual characteristics. All Cochrane (medical PLSs, n = 8638) and Campbell Collaboration and International Initiative for Impact Evaluation (non-medical PLSs, n = 163) PLSs of latest versions of systematic reviews published until 10 November 2022 were analysed. PLSs were classified into three conclusiveness categories (conclusive, inconclusive and unclear) using a machine learning tool for medical PLSs and by two experts for non-medical PLSs. A higher proportion of non-medical PLSs were conclusive (17.79% vs 8.40%, P

Abstract Background Cochrane plain language summaries (PLSs) are an important format to present high‐quality healthcare evidence to patients with cancer and their families. They should be written in a way everyone can understand, since they serve as a tool in decision‐making and present a bridge to overcome the gap between the healthcare users and professionals. Objective The aim of the study was to assess the language characteristics of PLSs of Cochrane systematic reviews of oncology interventions in comparison with corresponding Cochrane scientific abstracts (SAs). Methods In this cross‐sectional study, we included all Cochrane PLSs and SAs of systematic reviews of oncology interventions available in the Cochrane Database of Systematic Reviews. We assessed text readability, measured using the Simple Measure of Gobbledygook (SMOG) index, and the prevalence of words related to different language tones (clout, authenticity, emotions and analytical tones). Two independent assessors categorized the conclusiveness of the efficacy of interventions into nine categories. Results The overall median SMOG index for 275 PLSs was 13.0 (95% confidence interval [CI] 12.8–13.3). Readability scores did not differ across Cochrane Review Groups. SAs had a higher readability index than the corresponding PLSs (median = 16.6, 95% CI = 16.4–16.8). Regarding linguistic characteristics, PLSs were shorter than SAs, with less use of analytical tone, but more use of a positive emotional tone and authenticity. Overall, the ‘Unclear’ category of conclusiveness was the most common among all PLSs. Also, PLSs with ‘No evidence’ conclusions were the shortest and had the lowest SMOG index. Conclusion PLSs of Cochrane systematic reviews of oncological interventions have low readability and most give unclear conclusions about the efficacy of interventions. PLSs should be simplified so that patients and their families can benefit from appropriate health information on evidence synthesis. Further research is needed into reasons for unclear language to describe evidence from oncology trials.

Background and Objectives: This study demonstrates the factor structure and reliability of the Croatian version of the TOPICOP (Topical Corticosteroid Phobia) questionnaire, thereby contributing to further validation and standardization of the measurement of topical corticophobia with dermatological patients or their parents, which directly affects patient or parent compliance, as well as the final therapeutic effect. Materials and Methods: The cross-sectional, observational study was conducted at the University Hospital Centre Split, Department of Dermatovenerology. The research involved inviting 120 participants (age 12–68) who attended the University Hospital Centre Split’s Atopy School, patients examined in the Dermatology Outpatient Clinic of the University Hospital Centre Split and diagnosed with atopic dermatitis (AD) or allergic contact dermatitis (ACD), and parents or legal representatives of patients younger than 12 years old. The TOPICOP questionnaire consists of 12 items assessing the three different components of topical corticophobia (worries, beliefs, and behaviour). Respondents evaluated their perception of the correctness of each statement within 10 min of filling out the questionnaire on a four-point Likert scale. Results: The response rate was 94%, resulting in a sample of 113 respondents (109 adults and 4 children). Factor analysis yielded one common factor of relatively high reliability (Cronbach α = 0.85, 95% CI 0.81 to 0.89). No differences were found in questionnaire scores between male and female participants, nor between the parents/legal representatives of dermatological patients and other patients. Conclusions: This research contributes to further development of the appropriate measuring instrument, its practical application, and thus, the better recognition, resolution, and prevention of topical corticophobia as part of the better overall healthcare and treatment of chronic dermatological patients.

ObjectiveTo evaluate the impact of research design on the perceived medical treatment effectiveness among researchers, healthcare workers (HCWs) and consumers in Croatia.MethodsA cross-sectional study was conducted from November 2021 to February 2022 using an online survey. The participants were researchers, HCWs and consumers from Croatia. The survey had six scenarios about the same medical treatment presented within different study designs and in random order. Participants were asked to assess on a scale from 1 to 10 if the descriptions presented a sufficient level of evidence to conclude that the treatment was effective.ResultsFor researchers (n = 97), as the number of participants and degree of variable control in the study design increased, the perceived level of sufficient evidence also increased significantly. Among consumers (n = 286) and HCWs (n = 201), no significant differences in scores were observed between the cross-sectional study, cohort study, RCT, and systematic review.ConclusionThere is a need to implement educational courses on basic research methodology in lower levels of education and as part of Continuing Medical Education for all stakeholders in the healthcare system. Trial registration: this study has been registered on the Open Science Framework prior to study commencement (https://osf.io/t7xmf).

Abstract Background Health information and patient education on lifestyle changes may have a positive effect on the prevention of many chronic conditions, especially cardiovascular diseases (CVDs). We performed a parallel, three-arm randomized controlled trial (RCT) of 6-month educational intervention in a form of letters containing a reminder of the participant’s CVD risk with or without Cochrane blogshots to reduce CVD risk among women aged 45–65 with one or more known CVD risk factors. Methods The control group received a letter about their CVD risk at the beginning of the trial. The intervention groups received the initial letter about their CVD risk and remainder letters about their CVD risk every 2 months, with or without Cochrane blogshots: (1) effect of calcium in the prevention of high blood pressure, (2) effect of reducing saturated fat acids in eating habits, and (3) effects of green and black tea in CVD prevention. The primary outcome was CVD risk reduction calculated as the difference between the baseline and 6-month score for a 10-year risk of fatal CVD according to the ACC/AHA guidelines. Results After both interventions, CVD risk reduction was significantly higher compared to the control group (P < 0.001, Kruskal-Wallis H test). The number of participants who decreased their CV risk was 29% (20/70) in the control group, 69% (48/70) in the group receiving the reminder letters, and 70% (49/70) in the group receiving the reminder letters and blogshots. The number needed to treat to achieve risk reduction was 2.41 (95% CI = 1.77 to 3.78) for letters with a CVD risk reminder and 2.50 (1.81 to 4.03) for letters with a reminder and a blogshot. The group receiving reminder letters with Cochrane blogshots had a significant change in the category of CVD risk, mainly from high to moderate and from moderate to low CVD risk category. Conclusions A simple and inexpensive intervention method in a form of letters reminding women about their CVD risk with or without providing additional health information in the form of Cochrane blogshots about interventions for important CVD risk factors may be effective in CVD management and could be considered by primary care providers. Trial registration ClinicalTrials.gov, NCT04601558. Retrospectively registered on October 19, 2020

Abstract Background Lack of knowledge about systematic reviews (SRs) could prevent individual healthcare workers (HCWs) from using SRs as a source of information in their clinical practice or discourage them from participating in such research. In this study, we aimed to explore in-depth the opinion of a sample of HCWs about the newly created online educational intervention designed to improve knowledge about SRs. Methods We created a brief online educational intervention on SRs, consisting of 11 textual modules. We evaluated it among practicing HCWs who graduated from a university-level health sciences program using a mixed-methods pilot study that consisted of pre- and post-intervention questionnaires and qualitative evaluation via semi-structured interviews. We assessed participants’ knowledge about SR methodology before and after the intervention, and compared the responses. We sought their opinions about the characteristics of SRs. Also, the participants were presented with four scientific abstracts, where they were asked to distinguish whether those abstracts presented summaries of a systematic or a non-systematic review. Results Twelve participants took part in the study. In the pilot study, the participants’ knowledge about SRs was improved after the intervention compared to the baseline. Participants provided positive feedback regarding the educational intervention. Suggestions to improve the educational intervention were to provide more details about the forest plot, add more digital content or images, provide more details about the methodological steps of an SR, add descriptions about practical applications of SRs and provide links to additional educational materials. The participants suggested that HCWs could be motivated to take part in such an education if it is offered as continuing medical education (CME) course or credit for academic/career advancement. Conclusion HCWs provided positive feedback about the newly designed online educational intervention on SRs; they considered it an appropriate tool for learning about SRs and resulted in increased knowledge about SRs. In addition, participants gave suggestions for improving education, which can be used to tailor the education for end-users. In future studies, it would be useful to examine the effectiveness of the modified educational intervention on increasing knowledge in a larger sample and in the form of a randomized controlled trial.

Abstract Although there is research interest to assess attitudes on psychedelics, no validated instrument exists for this purpose. We aimed to develop and examine the psychometric properties of the Attitudes on Psychedelics Questionnaire (APQ) in a sample of the Croatian general population. A cross-sectional, web-based survey among the general population was conducted on 1153 participants (62.1% female, 77.7% with a graduate or high school degree, 15.1% health care workers). We assessed participants’ ability to recognize psychedelic substances using a short knowledge test. The APQ consists of 20 items with four sub-scales: Legal Use of Psychedelics, Effects of Psychedelics, Risk Assessment of Psychedelics, and Openness to Psychedelics. This model demonstrated best fit in a confirmatory factor analysis. Total scale reliability was excellent (McDonald’s ω = 0.949, 95% CI = 0.944–0.953). A strong correlation with a similar unvalidated measure (r = 0.885, P

Background: In the evaluation of research proposals, reviewers are often required to provide their opinions using various forms of quantitative and qualitative criteria. In 2020, the European Commission removed, for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) funding scheme, the numerical scores from the individual evaluations but retained them in the consensus report. This study aimed to assess whether there were any differences in reviewer comments’ linguistic characteristics after the numerical scoring was removed, compared to comments from 2019 when numerical scoring was still present. Methods: This was an observational study and the data were collected for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) evaluation reports from the calls of 2019 and 2020, for both individual and consensus comments and numerical scores about the quality of the proposal on three evaluation criteria: Excellence, Impact and Implementation. All comments were analyzed using the Linguistic Inquiry and Word Count (LIWC) program. Results: For both years, the comments for proposal's strengths were written in a style that reflects objectivity, clout, and positive affect, while in weaknesses cold and objective style dominated, and that pattern remained stable across proposal status and research domains. Linguistic variables explained a very small proportion of the variance of the differences between 2019 and 2020 (McFadden R2=0.03). Conclusions: Removing the numerical scores was not associated with the differences in linguistic characteristics of the reviewer comments. Future studies should adopt a qualitative approach to assess whether there are conceptual changes in the content of the comments.

Abstract Background One of the most important formats to disseminate the evidence in health to different populations are Cochrane Plain Language Summaries (PLSs). PLSs should be written in a simplified language, easily understandable and providing clear message for the consumer. The aim of this study was to examine the extent to which PLSs are customized for lay persons, specifically by providing conclusive, comprehensible, and readable messages. Methods The study analyzed Cochrane PLSs of interventional studies (N = 4360) in the English language published from 1995 to 2019. We categorized the conclusiveness into one of the following categories: “positive”, “positive inconclusive”, “no evidence”, “no opinion”, “negative”, “negative inconclusive”, “unclear”, “equal”, “equal inconclusive”. Language characteristics were analyzed using Linguistic Inquiry and Word Count (LIWC) software. The level of readability was measured by SMOG (Simple Measure of Gobbledygook) index, indicating the number of years of education required to read the text. For each PLS, we also collected the following data: Cochrane Review Network, year of publication and number of authors. Results Most of the PLSs (80%) did not have a conclusive message. In 53% PLSs there was no concluding opinion about the studied intervention or the conclusion was unclear. The most frequent conclusiveness category was “no opinion” (30%), and its frequency increased over time. The conclusiveness categories were similarly dispersed across Cochrane Networks. PLSs were written in an objective style, with high levels of analytical tone and clout above neutral, but a lower relation to authenticity and tone. The median number of years of non-specific education needed to read the PLSs was 14.9 (IQR 13.8 to 16.1), indicating that the person needs almost 15 years of general education to read the content with ease. Conclusion Most of the Cochrane PLSs provided no concluding opinion or unclear conclusion regarding the effects of analyzed intervention. Analysis of readability indicated that they may be difficult to read for the lay population without medical education. Our results indicate that PLSs may not be so plain, and that the writing of Cochrane PLSs requires more effort. Tools used in this study could improve PLSs and make them better suited for lay audiences.

Abstract Background Medical schools should also evaluate applicants’ non-academic characteristics in the search for successful students and future physicians, but ideal non-academic criteria have not yet been found. We followed two successive generations of medical students at the University of Split School of Medicine (USSM) to assess both academic and non-academic constructs as predictors of academic performance, defined as medical school grade point average (GPA). We also interviewed some of the participants to gain additional insight for future studies. Methods We measured study GPA in first and last year, as well as attitudes towards science, motivation, emotional intelligence, self-esteem, and perceived personal incompetence in first year. We also obtained their scores on existing medical school enrollment criteria, the State Graduation Exam (SGE) and high-school GPA. Regression models were constructed for predictors of GPA in the last year of medical school. Four structured pilot interviews were conducted to explore participants’ perceptions of necessary traits for medical school and later practice. Results Regression analysis showed that only SGE predicted final academic performance in medical school (β=0.35, 95% confidence interval (CI)=0.06-0.64), while none of the non-academic constructs we assessed predicted this outcome of education. The two generations did not significantly differ in any variable except that intrinsic motivation was higher in the generation that enrolled in 2011 (OR=1.47, 95%CI=1.12-1.93, P=0.005). Discussion None of the non-academic constructs predicted academic performance in medical school. Their use as selection criteria may not be warranted as they could impact the academic quality of enrolling medical students.

Background: Reports of questionable or detrimental research practices (QRPs) call into question the reliability of scientific evidence and the trustworthiness of research. A critical component of the research ecosystem is the organization within which research takes place. We conducted a survey to explore the attitudes and beliefs of European and American researchers about the organisations in which they work, their own research practices and their attitudes towards research integrity and research integrity policies. Methods: We administered an online survey (International Research Integrity Survey (IRIS)) to 2,300 active researchers based in the US and 45,000 in Europe (including UK, Norway, Iceland and Switzerland). We employed a stratified probability sample of the authors of research articles published between 2016 and 2020 included in Clarivate’s Web of Science citation database. Coverage includes researchers in the humanities, social sciences, natural sciences and medical sciences, who hold at least a master’s level degree. Results: In comparison to researchers in the US, European researchers admit to more QRPs and are less confident in maintaining high research integrity (RI) standards. In the US and Europe, many researchers judge their organization to fall short of best RI practice. All researchers recognize the benefits of RI, reliable knowledge and the trust of colleagues and the public, and there is support for RI training particularly among Europeans. Conclusion: To create and maintain a culture of integrity in scientific research, a collective commitment from researchers, their institutions and funders is needed. Researchers rely on many channels of communication about research integrity and thus the involvement of many different participants in the research system is required to make improvements. Policies must be developed to reinforce best practice rather than being seen as an irrelevance to the real business of research.

Responsible research and innovation, or RRI, is a concept that aims to bring together society and science for a better future. There are six key elements of RRI: public engagement, gender equality, science education, open access, ethics and governance. Higher Education Institutions and Responsible Research and Innovation (HEIRRI) project aimed to bring the concept of RRI into the educational system. Using state-of-the-art review of good practices, HEIRRI team developed 10 training programs on RRI for different higher education institution educational levels, including a summer school and a massive open online course (MOOC). We conducted pilot of the trainings and evaluated participants’ experiences. Satisfaction with HEIRRI training programs on responsible research and innovation was high, both for participants and for the trainers, and trainings raised awareness of RRI. Participants’ feedback was used to identify areas that need improvement and provided for recommendations for final versions of the HEIRRI training programs. In order to equip researchers with skills to recognize and apply RRI values, RRI should be included in their education. HEIRRI training is suitable for a range of different disciplines, including forensic science, and is free to use and adjust for specific contexts (available from: https://rri-tools.eu/heirri-training-programmes). Supplemental data for this article is available online at https://doi.org/10.1080/20961790.2021.1970319 .

BackgroundLack of knowledge of systematic reviews (SRs) could prevent individual health care professionals from using SRs as a source of information in their clinical practice or discourage them from participating in such research. ObjectiveIn this randomized controlled trial, we evaluated the effect of a short web-based educational intervention on short-term knowledge of SRs. MethodsEligible participants were 871 Master’s students of university health sciences studies in Croatia; 589 (67.6%) students who agreed to participate in the trial were randomized using a computer program into 2 groups. Intervention group A (294/589, 49.9%) received a short web-based educational intervention about SR methodology, and intervention group B (295/589, 50.1%) was presented with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. The participants’ knowledge of SRs was assessed before and after the intervention. The participants could not be blinded because of the nature of the intervention. The primary outcome was the difference in the percentage of correct answers about SR methodology per participant between the groups after the intervention, expressed as relative risk and 95% CI. ResultsResults from 162 and 165 participants in the educational intervention and PRISMA checklist groups, respectively, were available for analysis. Most of them (educational intervention group: 130/162, 80.2%; PRISMA checklist group: 131/165, 79.4%) were employed as health care professionals in addition to being health sciences students. After the intervention, the educational intervention group had 23% (relative risk percentage) more correct answers in the postintervention questionnaire than the PRISMA checklist group (relative risk=1.23, 95% CI 1.17-1.29). ConclusionsA short web-based educational intervention about SRs is an effective tool for short-term improvement of knowledge of SRs among health care studies students, most of whom were also employed as health care professionals. Further studies are needed to explore the long-term effects of the tested education. Trial RegistrationOSF Registries 10.17605/OSF.IO/RYMVC; https://osf.io/rymvc

Abstract Background Knowledge assessment in evidence-based medicine (EBM) is usually performed by the measurement of memorised facts, understanding of EBM concepts and application of learned knowledge in familiar situations, all of which are considered lower-level educational objectives. The aim of this study was to assess EBM knowledge both on higher and lower cognitive levels across EBM topics. Methods In order to assess knowledge on different EBM topics across learning levels, we created a knowledge test (Six Progressive Levels in Testing – SPLIT instrument), which consists of 36 multiple choice items and measures knowledge in EBM at six cognitive levels (Remembering, Understanding, Applying, Analysing, Evaluating and Creating) and addresses six EBM topics (Evidence-based practice, Internal validity, Clinical importance, Study design, Sources of evidence, Diagnostic studies). Three independent assessors defined the minimum passing score (MPS) for the overall test, based on the first-year course content and educational objectives. The instrument was assessed in a sample of first- (n = 119) and third-year medical students (n = 70) and EBM experts (n = 14). Results The MPS was 16 correct answers out of total 36 questions, and was achieved by 21 out of 119 first-year students, 14 out of 70 third-year students and 9 out of 14 EBM experts (χ2 = 13.3; P

Abstract Background Cochrane, an organization dedicated to the production and dissemination of high-quality evidence on health, endeavors to reach consumers by developing appropriate summary formats of its systematic reviews. However, the optimal type of presentation of evidence to consumers is still unknown. Objective The aim of this study was to investigate consumer preferences for different summary formats of Cochrane systematic reviews (CSRs), using both qualitative and quantitative approaches. Methods Initially, we conducted three focus groups with medical students (n = 7), doctors (n = 4), and patients (n = 9) in 2017 to explore their health information search habits and preferences for CSR summary formats. Based on those findings, we conducted a randomized trial with medical students at the University of Split School of Medicine, Croatia, and with patients from three Dalmatian family practices to determine whether they prefer CSR blogshots (n = 115) or CSR plain language summaries (PLSs; n = 123). Results Participants in the focus groups favored brief and explicit CSR summary formats with fewer numbers. Although we found no difference in participants’ preferences for a specific summary format in the overall sample, subgroup analysis showed that patients preferred blogshots over PLSs in comparison to medical students (P = 0.003, eta squared effect size η2 = 0.04). Conclusion CSR summaries should be produced in a format that meets the expectations and needs of consumers. Use of blogshots as a summary format could enhance the dissemination of CSRs among patients. Trial registration The trial was registered in ClinicalTrials.gov , NCT03542201 . Registered on May 31st 2018.

Abstract Background Cochrane systematic review Plain language Summaries (CSR PLSs should serve as a tool for the evidence translation to non-medical population. However, the evidence of optimal type of numerical presentation in CSR PLSs is still scarce. The aim of this study was to investigate readers’ comprehension and preferences for different presentation of findings, including framing and numerical data, in Cochrane systematic review Plain Language Summaries (CSR PLSs). Methods We conducted a parallel randomized trial and a crossover randomized trial at the School of Medicine and family practice offices in Split, Croatia. The participants were students and consumers. We assessed possible differences in comprehension, measured by four questions on PLS content, of CSR PLSs depending on the positive or negative framing of results (n = 91) (Trial 1) or using percentages or frequencies for the presentation of results (n = 245) (Trial 2). The outcome measures were comprehension of PLS content, perceived effectiveness of the treatment and readiness to use the treatment (all on 1–10 scales). Results In Trial 1 we found no difference in readers’ perception of the effectiveness of the described treatment, desire that the treatment be offered by their family doctor, readiness to use the treatment, or comprehension when CSR PLS results were presented positively or negatively. In Trial 2 we found no difference in CSR PLS comprehension when results were presented as natural frequencies or percentages (BF 10 = 0.62, Bayesian t-test for independent samples). Conclusions Numerical presentation and framing direction of results appear to have no significant impact on understanding of messages in CSR PLSs. Trial registration The trials were registered in ClinicalTrials.gov. Protocol registration numbers: Trial 1: NCT03442387 ; Trial 2: NCT03554252 .

Hypovitaminosis D has been recognized as a worldwide pandemic, but there are disagreements regarding its diagnosis and treatment. This study aimed to evaluate the knowledge and practice of European pediatricians concerning vitamin D (VD) and hypovitaminosis D and their adherence to relevant guidelines. A cross-sectional study was conducted through an anonymous survey via SurveyMonkey on 304 European pediatricians. Most of the participants were general pediatricians, followed by endocrinologists and neonatologists. ESPGHAN’s and the national guidelines were the most frequently used. VD testing was mostly performed in patients with skeletal, chronic kidney, and autoimmune diseases. Participants predominantly answered the questions regarding the definition of hypovitaminosis D and VD supplementation correctly, especially in children younger than 12 months. They showed the least knowledge regarding the cut off-point for the initiation of the therapy and therapeutic doses used to treat confirmed hypovitaminosis D. Participants’ conflicting answers could be related to the differences between the guidelines. We consider that experts in this field should create uniform guidelines, and that those guidelines should also be promoted by national or local pediatric associations.

Background: We assessed the ethics review of proposals selected for funding under the Marie Skłodowska-Curie Actions (MSCA) and the European Research Council (ERC) in Horizon 2020, EU’s framework programme for research and innovation, 2014-2020. Methods: We analysed anonymized datasets for 3,054 MSCA individual fellowships (IF), 417 MSCA Innovative Training Networks (ITN), and 1,465 ERC main-listed proposals with ethics conditional clearance, over four years (2016 to 2019). The datasets included the information on ethics issues identified by applicants in their proposal and ethics issues and requirements identified by ethics experts during the ethics review. Results: 42% of proposals received ethical clearance. For proposals with conditional ethics clearance (n=3546), most of the identified ethics issues by both applicants and ethics experts were in the ethics categories related to humans; protection of personal data; environment, health and safety; and non-EU countries. Ethics experts identified twice as many ethics issues compared to applicants across funding schemes, years, and from high- and low-research performing countries. ERC grants had the highest number of ethics requirements per proposal (median (Md)=8, interquartile range (IQR=4-14), compared to ITN (Md=6, IQR=3-13) and IF grants (Md=3, IQR=2-6). The majority of requirements had to be fulfilled after grant agreement: 99.4% for IF, 99.5% for ITN, and 26.0% for ERC. For 9% of the proposals, the requirements included the appointment of an independent ethics advisor and 1% of the proposals had to appoint an ethics advisory board. Conclusions: Many applicants for highly competitive H2020 funding schemes lack awareness of ethics issues raised by their proposed research. There is a need for better training of researchers at all career stages about ethics issues in research, more support to researchers from research organizations to follow the funding agencies requirements, as well as further development and harmonization of the ethics appraisal process during grant assessment.

Abstract Background Although numeracy, defined as understanding and handling numbers, is an important skill for the medical profession, it is not clear whether it changes during graduate medical education and whether it can be improved by specific interventions. The objective of this study was to assess objective and subjective numeracy levels at different stages of medical education and explore whether a research methodology/statistics course improves numeracy levels in a longer period. Methods We performed cross-sectional and controlled before-and-after studies. First-year sociology students and first- to sixth-year medical students from the in the cross sectional study and two groups of first-year medical students in a controlled before-and-after study. The intervention was a course on biostatistics and research methodology using blended approach. Numeracy was measured using Subjective Numeracy Scale (Cronbach α = 0.70) and Numeracy Understanding in Medicine instrument (Cronbach α = 0.75). Results Whereas first-year medical students did not differ from first-year sociology students in objective numeracy, medicine students had higher results on subjective numeracy. Students from higher years of medical school had generally higher subjective and objective numeracy scores. In the controlled before-and-after study, the intervention group improved more in subjective numeracy (median difference on a 0–8 scale = 0.5, 95% CI 0.3 to 0.7 vs − 0.4, 95% CI − 0.4 to − 0.1, P

Abstract Background Although subjective expressions and linguistic fluency have been shown as important factors in processing and interpreting textual facts, analyses of these traits in textual health information for different audiences are lacking. We analyzed the readability and linguistic psychological and emotional characteristics of different textual summary formats of Cochrane systematic reviews. Methods We performed a multitrait-multimethod cross-sectional study of Press releases available at Cochrane web site (n = 162) and corresponding Scientific abstracts (n = 158), Cochrane Clinical Answers (n = 35) and Plain language summaries in English (n = 156), French (n = 101), German (n = 41) and Croatian (n = 156). We used SMOG index to assess text readability of all text formats, and natural language processing tools (IBM Watson Tone Analyzer, Stanford NLP Sentiment Analysis and Linguistic Inquiry and Word Count) to examine the affective states and subjective information in texts of Scientific abstracts, Plain language summaries and Press releases. Results All text formats had low readability, with SMOG index ranging from a median of 15.6 (95% confidence interval (CI) 15.3–15.9) for Scientific abstracts to 14.7 (95% CI 14.4–15.0) for Plain language summaries. In all text formats, “Sadness” was the most dominantly perceived emotional tone and the style of writing was perceived as “Analytical” and “Tentative”. At the psychological level, all text formats exhibited the predominant “Openness” tone, and Press releases scored higher on the scales of “Conscientiousness”, “Agreeableness” and “Emotional range”. Press releases had significantly higher scores than Scientific abstracts and Plain language summaries on the dimensions of “Clout”, and “Emotional tone”. Conclusions Although the readability of Plain language summaries was higher than that of text formats targeting more expert audiences, the required literacy was much higher than the recommended US 6th grade level. The language of Press releases was generally more engaging than that of Scientific abstracts and Plain language summaries, which are written by the authors of systematic reviews. Preparation of textual summaries about health evidence for different audiences should take into account readers’ subjective experiences to encourage cognitive processing and reaction to the provided information.

Abstract Background Uncontrolled blood pressure remains an urgent issue in clinical practice worldwide. This study aimed to compare the characteristics and effectiveness of hypertension control in family medicine pratice in the first treatment year, in relation to the geographical position, socio-economic standard, and access to medical services and public pharmacies in urban, rural and island environments (city of Split vs. Dalmatian Hinterland vs. islands in Southern Croatia). Methods A historical cohort study included 213 patients diagnosed from 2008 to 2014 with essential arterial hypertension (AH) and without related complications or diabetes mellitus. Each patient was followed up for 365 days from the visit when the diagnosis of hypertension was ascertained. Normotension was defined as arterial pressure

Abstract Background Optimising breastfeeding rates is a public health priority. Studies have shown that all forms of extra breastfeeding support increase breastfeeding rates, including support provided by trained health professionals. International Board Certified Lactation Consultants (IBCLCs) are trained healthcare professionals in the clinical management of breastfeeding and human lactation. The IBCLC certification is a sought-after credential and can only be obtained after passing the exam administered by the International Board of Lactation Consultant Examiners (IBLCE). In Slovenia and Croatia, the IBLCE exam has been offered since 2006 and 2009, respectively. In this study, our aim was to 1) determine which candidate characteristics are associated with a passing grade on the IBLCE exam; and 2) analyse differences between candidates from Slovenia and Croatia, given Slovenians’ higher achievements in the past. Methods In February, 2017, a 4-page, 36-question survey was sent via Survey Monkey to the available email addresses of all past IBLCE exam candidates in Croatia and Slovenia. Questions covered sociodemographic data, breastfeeding education, exam preparation, motivation and experience taking the IBLCE exam. Results Ninety-two participants completed the online survey: 36 from Croatia and 55 from Slovenia, giving a response of 47 and 52%, respectively. No significant difference was found in pass rates between the two countries, despite Slovenians being younger and spending more time observing normal breastfeeding dyads. Variables found to be significantly more common among respondents who passed the IBLCE exam included: attending breastfeeding conferences/symposiums, using a breastfeeding atlas and studying with others. Statistical predictors of IBLCE exam success were: number of hours of bedside teaching, perceived clarity of photographs and breastfeeding conference/symposium attendance. Respondents who reported that they had attended a breastfeeding conference/symposium, had less hours of bedside teaching and perceived exam photographs as completely clear, were 7.49 (95% CI 2.26, 24.84), 0.48 (95% CI 0.28, 0.82), and 3.49 (95% CI 1.17, 10.41) times more likely to pass the exam, respectively. Conclusion Breastfeeding conference attendance, less bedside teaching and perceived clarity of exam photographs may be predictors of IBLCE exam success. Further studies on larger samples of exam candidates are required to confirm our findings and determine other factors associated with passing the IBLCE exam.

Objectives To evaluate the impact of the first COVID-19 lockdown in 2020 on the burnout and study satisfaction of medical students.Design A cross-sectional study with a presurvey and postsurvey.Setting University of Split School of Medicine (USSM), Split, Croatia. The lockdown in the COVID-19 pandemic lasted from late March to mid-May 2020. There was a full switch to e-learning at the USSM during this period, and all clinical teaching was stopped.Participants Students enrolled in the 2019/2020 academic year. Data were collected before lockdown in December 2019 and January 2020 and again after the end of lockdown in June 2020.Primary and secondary outcome measures Study satisfaction was assessed using the study satisfaction survey. Burnout was assessed using two instruments: Oldenburg Burnout Inventory and Copenhagen Burnout Inventory. We used Bayesian statistics to compare before-and-after differences.Results 437 independent responses (77.2% response rate) were collected before and 235 after lockdown (41.5% response rate). 160 participant responses were eligible for pairing. There was no significant difference for both paired and unpaired participants in study satisfaction before (3.38 on a 1–5 scale; 95% credible interval (95% CrI) 3.32 to 3.44) and after (3.49, 95% CrI 3.41 to 3.57) lockdown. We found no evidence (Bayes factor (BF10) >3.00 as a cut-off value) for an increase in the level of burnout before and after lockdown, both in independent and paired samples.Conclusions It seems that the first pandemic-related lockdown and a switch to e-learning did not affect burnout levels among medical students or their perception of their study programme. More insight is needed on the short-term and long-term effects of the COVID-19 pandemic on medical students and their education. Well-structured longitudinal studies on medical student burnout during the COVID-19 pandemic are needed.

Background: We assessed the ethics review of proposals selected for funding under the Marie Skłodowska-Curie Actions (MSCA) and the European Research Council (ERC) in Horizon 2020, EU’s framework programme for research and innovation, 2014-2020. Methods: We analysed anonymized datasets for 3,054 MSCA individual fellowships (IF), 417 MSCA Innovative Training Networks (ITN), and 1,465 ERC main-listed proposals with ethics conditional clearance, over four years (2016 to 2019). The datasets included the information on ethics issues identified by applicants in their proposal and ethics issues and requirements identified by ethics experts during the ethics review. Results: 42% of proposals received ethical clearance. For proposals with conditional ethics clearance (n=3546), most of the identified ethics issues by both applicants and ethics experts were in the ethics categories related to humans; protection of personal data; environment, health and safety; and non-EU countries. Ethics experts identified twice as many ethics issues compared to applicants across funding schemes, years, and from high- and low-research performing countries. ERC grants had the highest number of ethics requirements per proposal (median (Md)=8, interquartile range (IQR=4-14), compared to ITN (Md=6, IQR=3-13) and IF grants (Md=3, IQR=2-6). The majority of requirements had to be fulfilled after grant agreement: 99.4% for IF, 99.5% for ITN, and 26.0% for ERC. For 9% of the proposals, the requirements included the appointment of an independent ethics advisor and 1% of the proposals had to appoint an ethics advisory board. Conclusions: Many applicants for highly competitive H2020 funding schemes lack awareness of ethics issues raised by their proposed research. There is a need for better training of researchers at all career stages about ethics issues in research, more support to researchers from research organizations to follow the funding agencies requirements, as well as further development and harmonization of the ethics appraisal process during grant assessment.

Most funding agencies rely on peer review to evaluate grant applications and proposals, but research into the use of this process by funding agencies has been limited. Here we explore if two changes to the organization of peer review for proposals submitted to various funding actions by the European Union has an influence on the outcome of the peer review process. Based on an analysis of more than 75,000 applications to three actions of the Marie Curie programme over a period of 12 years, we find that the changes – a reduction in the number of evaluation criteria used by reviewers and a move from in-person to virtual meetings – had little impact on the outcome of the peer review process. Our results indicate that other factors, such as the type of grant or area of research, have a larger impact on the outcome.

Peer review is often criticized for being flawed, subjective and biased, but research into peer review has been hindered by a lack of access to peer review reports. Here we report the results of a study in which text-analysis software was used to determine the linguistic characteristics of 472,449 peer review reports. A range of characteristics (including analytical tone, authenticity, clout, three measures of sentiment, and morality) were studied as a function of reviewer recommendation, area of research, type of peer review and reviewer gender. We found that reviewer recommendation had the biggest impact on the linguistic characteristics of reports, and that area of research, type of peer review and reviewer gender had little or no impact. The lack of influence of research area, type of review or reviewer gender on the linguistic characteristics is a sign of the robustness of peer review.

Abstract Background There are a few studies of alignment between different knowledge-indices for evidence-based medicine (EBM). The aim of this study was to investigate whether the type of test used to assess knowledge of EBM affects the estimation of this knowledge in medical students. Methods Medical students enrolled in 1-week EBM course were tested with the Fresno, Berlin, and ACE tests at the beginning and the end of the course. We evaluated the ability of these tests to detect a change in the acquired level of EBM knowledge and compared the estimates of change with those of the Control group that was tested with the ACE and Berlin tests before and after an unrelated non-EBM course. The distributions of test scores and average item difficulty indices were compared among the tests and the groups. Results Test scores improved on all three tests when compared with their pre-test results and the control. Students had on average a “good” performance on the ACE test, “sufficient” performance on the Berlin test, and “insufficient” performance or have “not passed” on the Fresno test. The post-test improvements in performance on the Fresno test (median 31% increase in percent scores, 95% confidence interval (CI) 25–42%) outperformed those on the ACE (13, 95% CI 13–20%) and Berlin tests (13, 95% CI 7–20%). Post-test score distributions demonstrated that the ACE test had less potential to discriminate between levels of EBM knowledge than other tests. Conclusion The use of different EBM tests resulted in different assessment of general EBM knowledge in a sample of graduate medical students, with lowest results on the Fresno and highest on the ACE test. In the light of these findings, EBM knowledge assessment should be based on the course’s content and learning objectives.

Abstract Background Health practice guidelines (HPGs) are important tools for the translation of evidence into practice. Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist provides guidance on reporting health practice guidelines (HPGs). We assessed the reporting completeness and quality of a set of national (Croatian) and relevant transnational (European) HPGs. Methods The national sample included all HPGs published in the official journal of the Croatian Medical Association in 2014–2016. We searched PubMed to identify relevant European guidelines (n = 24). Two independent reviewers assessed the adherence with the items on the RIGHT checklist. Kappa score was used to measure the level of agreement. Frequentist and Bayes statistics Bayes factor (BF10) was used to evaluate the differences between the national and transnational HPGs. Results Overall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0–15.0) RIGHT reporting items, and European counterparts reported a median of 16.0 (95% CI 14.0–17.2) out of the total of 35 checklist items (Mann Whitney U test, P = 0.048; BF10 = 1.543). European HPGs were better than Croatian HPGs in reporting stakeholder involvement and values and preferences (BF10 = 80.63), as well as describing the implications of costs and resources (BF10 = 55.15). Croatian HPGs better reported HPGs specified aims (BF10 = 16.90), primary intended users (BF10 = 8.70), and sources of funding (BF10 = 122.90). Most insufficiently reported items for both HPG sets were defining the guideline questions and clear outcomes, quality assurance, management of funding and conflicts of interest, and guideline limitations. Conclusions Important methodological details are missing from most published HPGs at national and transnational levels. To ensure better quality and adequate use of HPGs, reporting guidelines should be endorsed and used by developers and users alike.

Assessing the success and performance of researchers is a difficult task, as their grant output is influenced by a series of factors, including seniority, gender and geographical location of their host institution. In order to assess the effects of these factors, we analysed the publication and citation outputs, using Scopus and Web of Science, and the collaboration networks of European Research Council (ERC) starting (junior) and advanced (senior) grantees. For this study, we used a cohort of 355 grantees from the Life Sciences domain of years 2007-09. While senior grantees had overall greater publication output, junior grantees had a significantly greater pre-post grant award increase in their overall number of publications and in those on which they had last authorship. The collaboration networks size and the number of sub-communities increased for all grantees, although more pronounced for juniors, as they departed from smaller and more compact pre-award co-authorship networks. Both junior and senior grantees increased the size of the community within which they were collaborating in the post-award period. Pre-post grant award performance of grantees was not related to gender, although male junior grantees had more publications than female grantees before and after the grant award. Junior grantees located in lower research-performing countries published less and had less diverse collaboration networks than their peers located in higher research-performing countries. Our study suggests that research environment has greater influence on post-grant award publications than gender especially for junior grantees. Also, collaboration networks may be a useful complement to publication and citation outputs for assessing post-grant research performance, especially for grantees who already have a high publication output and who get highly competitive grants such as those from ERC.

The project named Higher Education Reform was established in 2004 as one of the implementation elements of the European higher education policy. The core activities of the project were executed throughout the formation of National Teams of Bologna Experts who had the task and duty to contribute to the general and real awareness-rising on the topic of the higher education reform among different stakeholders in participating countries of the Bologna Process. The Croatian National Team of Bologna Expert (Hrvatska stručna skupina za Bolonjski proces) was established in 2011. Among the diversity of activities executed by the student representatives in the Team, the important place is reserved for the deliberative workshops held during 2013. The target groups of the deliberative workshops were students and other stakeholders in the higher education. This paper presents the process of including the stakeholders in educational reform through the deliberative workshops. During the workshops, the organizers found out how the stakeholders are rethinking on some of the aspects of the Bologna Process, how they perceive and value the work of student representative and volunteering activities, and what they think about the extra-curricular activities of the student and how they value such activities. The form of deliberative workshops encouraged participants to freely and in constructive way express their thoughts and expectances in order to exchange ideas and knowledge about the matter, and to make a decision on common suggestions to solve a particular problem in the context of higher education (introduction of a new practice, modifications and alternations of existing practices, etc). This type of the application of deliberation method is extremely useful, which is the reason for the suggestion of the method’s use in preparing, implementing and evaluating the educational reforms.

Background: In the evaluation of research proposals, reviewers are often required to provide their opinions using various forms of quantitative and qualitative criteria. In 2020, the European Commission removed, for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) funding scheme, the numerical scores from the individual evaluations but retained them in the consensus report. This study aimed to assess whether there were any differences in reviewer comments’ linguistic characteristics after the numerical scoring was removed, compared to comments from 2019 when numerical scoring was still present. Methods: This was an observational study and the data were collected for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) evaluation reports from the calls of 2019 and 2020, for both individual and consensus comments and numerical scores about the quality of the research proposal on three evaluation criteria: Excellence, Impact and Implementation. All comments were analyzed using the Linguistic Inquiry and Word Count (LIWC) program. Results: For both years, the comments for proposal’s strengths were written in a style that reflects objectivity, clout, and positive affect, while in weaknesses cold and objective style dominated, and that pattern remained stable across proposal status and research domains. Linguistic variables explained a very small proportion of the variance of the differences between 2019 and 2020 (McFadden R 2=0.03). Conclusions: Removing the numerical scores was not associated with the differences in linguistic characteristics of the reviewer comments. Future studies should adopt a qualitative approach to assess whether there are conceptual changes in the content of the comments.

Background: In the evaluation of research proposals, reviewers are often required to provide their opinions using various forms of quantitative and qualitative criteria. In 2020, the European Commission removed, for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) funding scheme, the numerical scores from the individual evaluations but retained them in the consensus report. This study aimed to assess whether there were any differences in reviewer comments’ linguistic characteristics after the numerical scoring was removed, compared to comments from 2019 when numerical scoring was still present. Methods: This was an observational study and the data were collected for the Marie Skłodowska-Curie Actions (MSCA) Innovative Training Networks (ITN) evaluation reports from the calls of 2019 and 2020, for both individual and consensus comments and numerical scores about the quality of the research proposal on three evaluation criteria: Excellence, Impact and Implementation. All comments were analyzed using the Linguistic Inquiry and Word Count (LIWC) program. Results: For both years, the comments for proposal’s strengths were written in a style that reflects objectivity, clout, and positive affect, while in weaknesses cold and objective style dominated, and that pattern remained stable across proposal status and research domains. Linguistic variables explained a very small proportion of the variance of the differences between 2019 and 2020 (McFadden R 2=0.03). Conclusions: Removing the numerical scores was not associated with the differences in linguistic characteristics of the reviewer comments. Future studies should adopt a qualitative approach to assess whether there are conceptual changes in the content of the comments.

Background: Reports of questionable or detrimental research practices (QRPs) call into question the reliability of scientific evidence and the trustworthiness of research. A critical component of the research ecosystem is the organization within which research takes place. We conducted a survey to explore the attitudes and beliefs of European and American researchers about the organisations in which they work, their own research practices and their attitudes towards research integrity and research integrity policies. Methods: We administered an online survey (International Research Integrity Survey (IRIS)) to 2,300 active researchers based in the US and 45,000 in Europe (including UK, Norway, Iceland and Switzerland). We employed a stratified probability sample of the authors of research articles published between 2016 and 2020 included in Clarivate’s Web of Science citation database. Coverage includes researchers in the humanities, social sciences, natural sciences and medical sciences, who hold at least a master’s level degree. Results: In comparison to researchers in the US, European researchers admit to more QRPs and are less confident in maintaining high research integrity (RI) standards. In the US and Europe, many researchers judge their organization to fall short of best RI practice. All researchers recognize the benefits of RI, reliable knowledge and the trust of colleagues and the public, and there is support for RI training particularly among Europeans. Conclusion: To create and maintain a culture of integrity in scientific research, a collective commitment from researchers, their institutions and funders is needed. Researchers rely on many channels of communication about research integrity and thus the involvement of many different participants in the research system is required to make improvements. Policies must be developed to reinforce best practice rather than being seen as an irrelevance to the real business of research.