Search

Abstract Simulation studies are essential tools to assess statistical methods. Functioning as controlled experiments, simulations generate data from known underlying processes. However, unclear or incomplete reporting of simulation studies can impact their interpretability and reproducibility, potentially leading to the misuse of statistical methods. While Morris et al. (2019, Stat Med, 38, p. 2074) recently provided guidance on the planning and conduct of simulation studies for statistical method evaluation, there is currently no comprehensive set of reporting guidelines in ecology and evolutionary biology. Here, we propose 11 reporting items for statistical simulation studies extending on Morris and colleagues' guidance. These items span across three stages: planning, coding and analysis. We also clarify the terminology related to statistical components and the broad purposes of statistical simulation studies. To highlight our proposed reporting items with current practices, we surveyed 100 articles in ecology and evolution journals that included a simulation study evaluating a statistical method. Our survey found room for improvement in more transparent reporting to ensure clear evaluation of statistical methods. Most notably, only a small proportion of articles reported a Monte Carlo uncertainty (17%; 17 out of 98), and 32% (32 out of 100) articles did not provide code. Beyond the proposed reporting items, we discuss the benefits of open science tools to enhance the reproducibility of simulation studies. Specifically, we propose the registration of statistical simulation studies to enhance planning, reporting and collaboration. We aim to instigate discussions to improve the reporting of simulation studies for statistical method research. The reporting items we propose, along with open science tools, serve as a template for developing standards and guidelines for simulation studies evaluating statistical methods. These reporting guidelines will help enhance reproducibility and indirectly encourage more consideration in the design and conduct of these simulation studies.

Abstract Background Spinal cord injury presents a significant burden globally, with traumatic spinal cord injury being the predominant cause historically. However, nontraumatic spinal cord injury (NTSCI) is emerging as a significant contributor, particularly in developed nations, yet it remains poorly understood due to underreporting and misclassification. NTSCI, spanning various etiologies such as bony growths, vascular conditions, infections, neoplastic conditions, and immune disorders, poses unique challenges in diagnosis and treatment, often resulting in lifelong morbidity. This study addresses the lack of suitable animal models for NTSCI research, especially in neonatal animals. Methods Utilizing a solid spacer approach, we developed a compression NTSCI model applicable to both neonatal and adult Sprague–Dawley rats. Results Through anatomical measurements and in vivo experiments, we confirmed the feasibility and safety of the spacer insertion procedure and observed no acute off‐target effects. Conclusion The versatility of this model lies in its adaptability to different ages of rats, offering a cost‐effective and reproducible means to induce graded injuries. Moreover, behavioral assessments demonstrated observable hindlimb function, validating the model's utility for studying functional outcomes. Although challenges persist, particularly in accounting for spinal column growth in neonatal animals, this model fills a crucial gap in pediatric NTSCI research. By providing a platform to investigate pathophysiological mechanisms and test potential treatments, it offers promising avenues for advancing our understanding and management of pediatric NTSCI.

ABSTRACT The marine aquaculture industry and regulators are in the process of implementing environmental DNA (eDNA) metabarcoding of microbial communities for compliance monitoring. This requires standardization of sampling, laboratory, and data analysis protocols. Towards this goal, we in this study completed two further milestones using samples collected from two Scottish salmon farms: (i) We tested the effect of using two different PCR protocols (i.e., different DNA polymerases, master mixes, and annealing temperatures), which are frequently being used in eDNA biomonitoring of aquaculture installations, for the amplification of the taxonomic marker gene (V3‐V4 hypervariable region of the bacterial 16S rRNA gene). (ii) We quantified sampling background noise obtained from eDNA samples and statistically compared results with the sampling bias observed in macrofaunal samples from the same source sediments. We detected differences in bacterial community structures resulting from the performance of different PCR protocols, profoundly influencing the interpretation of biomonitoring results. Furthermore, we found that sampling‐induced errors for eDNA samples were similar to errors for macrofaunal samples collected according to compliance monitoring protocol (~25% variability in both cases). Finally, we showed that within‐grab variances of microbial community structures were in the same order of magnitude (less than 10× difference in all cases) as the one obtained from replicate grabs collected from the same locale (impact category). Based on our findings, we suggest using a consistent PCR protocol for biomonitoring efforts to improve the comparability of results, especially when different service providers are conducting the biomonitoring. We propose a sampling scheme to be considered in eDNA biomonitoring that includes taking three replicate grabs at each locale, with one replicate sample from each grab. This minimizes sampling‐induced errors and makes upcoming eDNA‐based monitoring results comparable with previous compliance monitoring results obtained from macrofaunal data.

Abstract Heavy training has been reported to be immunosuppressive in athletes and lead to blunted cortisol responses to exercise. Cortisol elevates the number of dendritic cells (DCs), key antigen‐presenting cells that interact with T cells to initiate an immune response. Reproducible cortisol responses to a 30‐min cycle test have been identified but were based on percentage of work rate maximum. To ensure physiological consistency, submaximal anchors, that is, ventilatory threshold (VT1) should prescribe intensity. This study aims to assess the reproducibility of the DC and T cell responses to an adapted stress test to assess its usefulness in assessing DC dysfunction with intensified training. Twelve males cycled for 1 min at 20% below VT1 and 4 min at 50% between VT1 and V̇O2max, for 30 min (20/50), with blood samples pre‐, post‐ and 30 min post‐exercise. This was repeated twice, 2–7 days apart. Flow cytometry assessed total DCs, plasmacytoid DCs, myeloid DCs, total T cells, T helper cells and T cytotoxic cells. No significant trial or interaction effects were found for any variable. A significant main effect of time for all variables was found; immune cells increased from pre‐ to post‐exercise and decreased to baseline 30 min post‐exercise, apart from plasmacytoid DCs, which remained elevated 30 min post‐exercise. Intraclass correlation coefficients showed overall good‐to‐excellent reliability for all immune cells, with smallest real difference and Bland–Altman analysis verifying high reproducibility between trials. These results suggest that the 20/50 exercise test induces reproducible DC and T cell count changes, which, implemented before and after a period of intensified training, may highlight the negative states of overtraining.

Abstract Biologists increasingly rely on computer code to collect and analyze their data, reinforcing the importance of published code for transparency, reproducibility, training, and a basis for further work. Here, we conduct a literature review estimating temporal trends in code sharing in ecology and evolution publications since 2010, and test for an influence of code sharing on citation rate. We find that code is rarely published (only 6% of papers), with little improvement over time. We also found there may be incentives to publish code: Publications that share code have tended to be low‐impact initially, but accumulate citations faster, compensating for this deficit. Studies that additionally meet other Open Science criteria, open‐access publication, or data sharing, have still higher citation rates, with publications meeting all three criteria (code sharing, data sharing, and open access publication) tending to have the most citations and highest rate of citation accumulation.

Abstract Habitat suitability models infer the geographical distribution of species using occurrence data and environmental variables. While data on species presence are increasingly accessible, the difficulty of confirming real absences in the field often forces researchers to generate them in silico. To this aim, pseudo‐absences are commonly sampled randomly across the study area (i.e. the geographical space). However, this introduces sample location bias (i.e. the sampling is unbalanced towards the most frequent habitats occurring within the geographical space) and favours class overlap (i.e. overlap between environmental conditions associated with species presences and pseudo‐absences) in the training dataset. To mitigate this, we propose an alternative methodology (i.e. the uniform approach) that systematically samples pseudo‐absences within a portion of the environmental space delimited by a kernel‐based filter, which seeks to minimise the number of false absences included in the training dataset. We simulated 50 virtual species and modelled their distribution using training datasets assembled with the presence points of the virtual species and pseudo‐absences collected using the uniform approach and other approaches that randomly sample pseudo‐absences within the geographical space. We compared the predictive performance of habitat suitability models and evaluated the extent of sample location bias and class overlap associated with the different sampling strategies. Results indicated that the uniform approach: (i) effectively reduces sample location bias and class overlap; (ii) provides comparable predictive performance to sampling strategies carried out in the geographical space; and (iii) ensures gathering pseudo‐absences adequately representing the environmental conditions available across the study area. We developed a set of R functions in an accompanying R package called USE to disseminate the uniform approach.

Abstract Stage‐based demographic methods, such as matrix population models (MPMs), are powerful tools used to address a broad range of fundamental questions in ecology, evolutionary biology and conservation science. Accordingly, MPMs now exist for over 3000 species worldwide. These data are being digitised as an ongoing process and periodically released into two large open‐access online repositories: the COMPADRE Plant Matrix Database and the COMADRE Animal Matrix Database. During the last decade, data archiving and curation of COMPADRE and COMADRE, and subsequent comparative research, have revealed pronounced variation in how MPMs are parameterized and reported. Here, we summarise current issues related to the parameterisation and reporting of MPMs that arise most frequently and outline how they affect MPM construction, analysis, and interpretation. To quantify variation in how MPMs are reported, we present results from a survey identifying key aspects of MPMs that are frequently unreported in manuscripts. We then screen COMPADRE and COMADRE to quantify how often key pieces of information are omitted from manuscripts using MPMs. Over 80% of surveyed researchers (n = 60) state a clear benefit to adopting more standardised methodologies for reporting MPMs. Furthermore, over 85% of the 300 MPMs assessed from COMPADRE and COMADRE omitted one or more elements that are key to their accurate interpretation. Based on these insights, we identify fundamental issues that can arise from MPM construction and communication and provide suggestions to improve clarity, reproducibility and future research utilising MPMs and their required metadata. To fortify reproducibility and empower researchers to take full advantage of their demographic data, we introduce a standardised protocol to present MPMs in publications. This standard is linked to www.compadre‐db.org, so that authors wishing to archive their MPMs can do so prior to submission of publications, following examples from other open‐access repositories such as DRYAD, Figshare and Zenodo. Combining and standardising MPMs parameterized from populations around the globe and across the tree of life opens up powerful research opportunities in evolutionary biology, ecology and conservation research. However, this potential can only be fully realised by adopting standardised methods to ensure reproducibility.

Abstract Pressing environmental research questions demand the integration of increasingly diverse and large‐scale ecological datasets as well as complex analytical methods, which require specialized tools and resources. Computational training for ecological and evolutionary sciences has become more abundant and accessible over the past decade, but tool development has outpaced the availability of specialized training. Most training for scripted analyses focuses on individual analysis steps in one script rather than creating a scripted pipeline, where modular functions comprise an ecosystem of interdependent steps. Although current computational training creates an excellent starting place, linear styles of scripting can risk becoming labor‐ and time‐intensive and less reproducible by often requiring manual execution. Pipelines, however, can be easily automated or tracked by software to increase efficiency and reduce potential errors. Ecology and evolution would benefit from techniques that reduce these risks by managing analytical pipelines in a modular, readily parallelizable format with clear documentation of dependencies. Workflow management software (WMS) can aid in the reproducibility, intelligibility and computational efficiency of complex pipelines. To date, WMS adoption in ecology and evolutionary research has been slow. We discuss the benefits and challenges of implementing WMS and illustrate its use through a case study with the targets r package to further highlight WMS benefits through workflow automation, dependency tracking and improved clarity for reviewers. Although WMS requires familiarity with function‐oriented programming and careful planning for more advanced applications and pipeline sharing, investment in training will enable access to the benefits of WMS and impart transferable computing skills that can facilitate ecological and evolutionary data science at large scales.

Abstract Objectives Amino acid variations across more than 30 immunoglobulin (Ig) allotypes may introduce structural changes that influence recognition by anti‐Ig detection reagents, consequently confounding interpretation of antibody responses, particularly in genetically diverse cohorts. Here, we assessed a panel of commercial monoclonal anti‐IgG1 clones for capacity to universally recognise two dominant IgG1 haplotypes (G1m‐1,3 and G1m1,17). Methods Four commercial monoclonal anti‐human IgG1 clones were assessed via ELISAs and multiplex bead‐based assays for their ability to bind G1m‐1,3 and G1m1,17 IgG1 variants. Detection antibodies were validated against monoclonal IgG1 allotype standards and tested for capacity to recognise antigen‐specific plasma IgG1 from G1m‐1,3 and G1m1,17 homozygous and heterozygous SARS‐CoV‐2 BNT162b2 vaccinated (n = 28) and COVID‐19 convalescent (n = 44) individuals. An Fc‐specific pan‐IgG detection antibody corroborated differences between hinge‐ and Fc‐specific anti‐IgG1 responses. Results Hinge‐specific anti‐IgG1 clone 4E3 preferentially bound G1m1,17 compared to G1m‐1,3 IgG1. Consequently, SARS‐CoV‐2 Spike‐specific IgG1 levels detected in G1m1,17/G1m1,17 BNT162b2 vaccinees appeared 9‐ to 17‐fold higher than in G1m‐1,3/G1m‐1,3 vaccinees. Fc‐specific IgG1 and pan‐IgG detection antibodies equivalently bound G1m‐1,3 and G1m1,17 IgG1 variants, and detected comparable Spike‐specific IgG1 levels between haplotypes. IgG1 responses against other human coronaviruses and influenza were similarly poorly detected by 4E3 anti‐IgG1 in G1m‐1,3/G1m‐1,3 subjects. Conclusion Anti‐IgG1 clone 4E3 confounds assessment of antibody responses in clinical cohorts owing to bias towards detection of G1m1,17 IgG1 variants. Validation of anti‐Ig clones should include evaluation of binding to relevant antibody variants, particularly as the role of immunogenetics upon humoral immunity is increasingly explored in diverse populations.

Abstract Blood is the most commonly used body fluid for extracellular vesicle (EV) research. The composition of a blood sample and its derivatives (i.e., plasma and serum) are not only donor‐dependent but also influenced by collection and preparation protocols. Since there are hundreds of pre‐analytical protocols and over forty variables, the development of standard operating procedures for EV research is very challenging. To improve the reproducibility of blood EV research, the International Society for Extracellular Vesicles (ISEV) Blood EV Task Force proposes standardized reporting of (i) the applied blood collection and preparation protocol and (ii) the quality of the prepared plasma and serum samples. Gathering detailed information will provide insight into the performance of the protocols and more effectively identify potential confounders in the prepared plasma and serum samples. To collect this information, the ISEV Blood EV Task Force created the Minimal Information for Blood EV research (MIBlood‐EV), a tool to record and report information about pre‐analytical protocols used for plasma and serum preparation as well as assays used to assess the quality of these preparations. This tool does not require modifications of established local pre‐analytical protocols and can be easily implemented to enhance existing databases thereby enabling evidence‐based optimization of pre‐analytical protocols through meta‐analysis. Taken together, insight into the quality of prepared plasma and serum samples will (i) improve the quality of biobanks for EV research, (ii) guide the exchange of plasma and serum samples between biobanks and laboratories, (iii) facilitate inter‐laboratory comparative EV studies, and (iv) improve the peer review process.

Abstract Animal models of cardiovascular disease are often evaluated by invasive instrumentation for phenotyping. As no consensus exists, both open‐ and closed‐chest approaches are used, which might compromise rigour and reproducibility in preclinical research. We aimed to quantify the cardiopulmonary changes induced by sternotomy and pericardiotomy in a large animal model. Seven pigs were anaesthetized, mechanically ventilated and evaluated by right heart catheterization and bi‐ventricular pressure–volume loop recordings at baseline and after sternotomy and pericardiotomy. Data were compared by ANOVA or the Friedmann test where appropriate, with post‐hoc analyses to control for multiple comparisons. Sternotomy and pericardiotomy caused reductions in mean systemic (−12 ± 11 mmHg, P = 0.027) and pulmonary pressures (−4 ± 3 mmHg, P = 0.006) and airway pressures. Cardiac output decreased non‐significantly (−1329 ± 1762 ml/min, P = 0.052). Left ventricular afterload decreased, with an increase in ejection fraction (+9 ± 7%, P = 0.027) and coupling. No changes were observed in right ventricular systolic function or arterial blood gases. In conclusion, open‐ versus closed‐chest approaches to invasive cardiovascular phenotyping cause a systematic difference in key haemodynamic variables. Researchers should adopt the most appropriate approach to ensure rigour and reproducibility in preclinical cardiovascular research.

Abstract Introduction The use of neoadjuvant therapy (NAT) has been showing an incraesing tendency in the treatment of locally advanced breast cancer. The evaluation of residual cancer could be performed by Residual Cancer Burden (RCB) calculator. The prognostic system takes the two largest diameters of the tumor, the cellularity, the amount of in situ carcinoma, the number of metastatic lymph nodes, and the size of the largest metastatic deposit into account. The aim of our study was to examine the reproducibility of RCB in NAT treated patients. Methods Patients who were treated with NAT and had resection specimens between 2018 and 2021 were selected. Histological examination was performed by five pathologists. After assessment of the examined variables, RCB points and RCB classes were defined. For statistical analysis, interclass correlation was used (SPSS Statistics V.22.0 software). Results Altogether 100 patients were included in our retrospective, cohort study (average age: 57 years). In two‐thirds of the cases, third generation chemotherapy was used, and mastectomy was performed. Significant concordance was found in the two largest diameters of the tumor (coefficients, 0.984 and 0.973), the cellularity (coefficient, 0.970), and the largest metastatic deposit (coefficient, 0.998). Although the amount of in situ carcinoma proved to be the least reproducible factor, it resulted in almost 90% of agreement (coefficient, 0.873). Regarding RCB points and classes, similar results were observed (coefficients, 0.989 and 0.960). Conclusions Significant agreement was observed between examiners based on almost all RCB parameters, points, and classes, reflecting the optimal reproducibility of RCB. Therefore, we recommend the use of the calculator in routine histopathological reports in NAT cases.

Abstract Receiving the same results from repeated analysis of the same sample is a basic principle in science. The inability to reproduce previously published results has led to discussions of a reproducibility crisis within science. For studies of microbial communities, the problem of reproducibility is more pronounced and has, in some fields, led to a discussion on the very existence of a constantly present microbiota. In this study, DNA from 44 bovine milk samples were extracted twice and the V3–V4 region of the 16S rRNA gene was sequenced in two separate runs. The FASTQ files from the two data sets were run through the same bioinformatics pipeline using the same settings and results from the two data sets were compared. Milk samples collected maximally 2 h apart were used as replicates and permitted comparisons to be made within the same run. Results show a significant difference in species richness between the two sequencing runs although Shannon and Simpson's diversity was the same. Multivariate analyses of all samples demonstrate that the sequencing run was a driver for variation. Direct comparison of similarity between samples and sequencing run showed an average similarity of 42%–45% depending on whether binary or abundance‐based similarity indices were used. Within‐run comparisons of milk samples collected maximally 2 h apart showed an average similarity of 39%–47% depending on the similarity index used and that similarity differed significantly between runs. We conclude that repeated DNA extraction and sequencing significantly can affect the results of a low microbial biomass microbiota study.

Abstract Unprecedented technological advances in digitization and the steadily expanding open‐access digital repositories are yielding new opportunities to quickly and efficiently measure morphological traits without transportation and advanced/expensive microscope machinery. A prime example is the AntWeb.org database, which allows researchers from all over the world to study taxonomic, ecological, or evolutionary questions on the same ant specimens with ease. However, the reproducibility and reliability of morphometric data deduced from AntWeb compared to traditional microscope measurements has not yet been tested. Here, we compared 12 morphological traits of 46 Temnothorax ant specimens measured either directly by stereomicroscope on physical specimens or via the widely used open‐access software tpsDig utilizing AntWeb digital images. We employed a complex statistical framework to test several aspects of reproducibility and reliability between the methods. We estimated (i) the agreement between the measurement methods and (ii) the trait value dependence of the agreement, then (iii) compared the coefficients of variation produced by the different methods, and finally, (iv) tested for systematic bias between the methods in a mixed modeling‐based statistical framework. The stereomicroscope measurements were extremely precise. Our comparisons showed that agreement between the two methods was exceptionally high, without trait value dependence. Furthermore, the coefficients of variation did not differ between the methods. However, we found systematic bias in eight traits: apart from one trait where software measurements overestimated the microscopic measurements, the former underestimated the latter. Our results shed light on the fact that relying solely on the level of agreement between methods can be highly misleading. In our case, even though the software measurements predicted microscope measurements very well, replacing traditional microscope measurements with software measurements, and especially mixing data collected by the different methods, might result in erroneous conclusions. We provide guidance on the best way to utilize virtual specimens (2D z‐stacked images) as a source of morphometric data, emphasizing the method's limitations in certain fields and applications.

Abstract Urban ecological studies have the potential to expand our understanding of socioecological systems beyond that of an individual city or region. Cross‐comparative empirical work and synthesis are imperative to develop a general urban ecological theory. This can be achieved only if studies are replicable and generalizable. Transparency in methods reporting facilitates generalizability and replicability by documenting the decisions scientists make during the various steps of research design; this is particularly true for sampling design and selection because of their impact on both internal and external validity and the potential to unintentionally introduce bias. Three interdependent aspects of sample design are study sample selection (e.g., specific organisms, soils, or water), sample specification (measurement of specific variable of interest), and site selection (locations sampled). Of these, documentation of site selection—the where component of sample design—is underrepresented in the urban ecology literature. Using a stratified random sample of 158 papers from 12 major urban ecology journals, we investigated how researchers selected study sites in urban ecosystems and evaluated whether their site selection methods were transparent. We extracted data from these papers using a 50‐question, theory‐based questionnaire and a multiple‐reviewer approach. Our sample represented almost 45 years of urban ecology research across 40 different countries. We found that more than 80% of the papers we read were not transparent in their site selection methodology. We do not believe site selection methods are replicable for 70% of the papers read. Key weaknesses include incomplete descriptions of populations and sampling frames, urban gradients, sample selection methods, and property access. Low transparency in reporting the where methodology limits urban ecologists' ability to assess the internal and external validity of studies' findings and to replicate published studies; it also limits the generalizability of existing studies. The challenges of low transparency are particularly relevant in urban ecology, a field where standard protocols for site selection and delineation are still being developed. These limitations interfere with the fields' ability to build theory and inform policy. We conclude by offering a set of recommendations to increase transparency, replicability, and generalizability.

Abstract Purpose To assess the reproducibility of a new 2‐dimensional computed tomography (CT) method of assessing graft positioning in arthroscopic bone block procedure. Methods This is a prospective observational study. Twenty‐seven patients, (all men, mean [Standard deviation] age at surgery 30.9 [8.49] years) were included. Vertical graft position was assessed on the sagittal view by measuring the amount of glenoid bone defect covered by the graft. The length of the bone defect and the amount of graft covering the defect were measured. Positioning of the graft on the sagittal plane was classified as accurate if the graft covered at least 90% of the defect. Intraobserver and interobserver reproducibility was analyzed using intraclass correlation coefficients (ICC) and Kappa coefficient with 95% confidence. Results Excellent intraobserver reproducibility was found, with an ICC of 0.94 (CI 95%, 0.86‐0.97). Interobserver reproducibility was good, with the ICC value of 0.71, ranging from 0.45 to 0.86 (CI 95%). Conclusion This new method of assessing graft positioning in arthroscopic bone block procedures on 2‐dimensional computed tomography scans is reliable, with an excellent intraobserver and good interobserver reproducibility. Level of evidence III

Abstract Purpose Robotic‐assisted surgery has been recently introduced to improve biomechanical restoration, and thus better clinical and functional outcomes, after knee joint arthroplasty operations. Robotic‐assisted uni‐compartmental knee arthroplasty (UKA) aims indeed to improve surgical bone resection and alignment accuracy, optimized component positioning and knee balancing, relying on a series of calibration measurements performed during the surgery. These advantages focus therefore on improving the reproducibility of UKA surgeries, reducing (if not eliminating) eventual differences among high‐ and low‐volume surgeons. The purpose of this study is to investigate and quantify the reproducibility of in‐vivo measurements performed with a robotic system: the intra‐ and inter‐observer variability of a series of measurements was therefore analyzed and compared among differently experienced operators. Methods Five patients were analyzed and underwent robotic‐assisted UKA using a semi‐active robotic system. Three different observers with different experience levels were involved to independently perform the measurements of two parameters of the preoperative knee (Hip‐Knee‐Ankle angle [HKAa], Internal‐External Rotation) at different degrees of knee flexion. Inter‐observer and intra‐observer comparisons were performed. Results The average variability in the measurements obtained from the intra‐observer and inter‐observer comparisons were always 0.75) for most cases and good agreement (> 0.60) in the remaining ones. Conclusion This study demonstrated high reproducibility of the measurements obtainable in clinical environment with the robotic system. The inter‐observer results furthermore showed that the level of confidence with the robotic system is not significantly influencing the measurement.

Abstract Background and Aims Single‐use electrocardiography (ECG) leads have been developed to reduce healthcare‐associated infection. This study compared the validity and reliability of short‐term heart rate variability (HRV) obtained from single‐use disposable ECG leads. Methods Thirty healthy subjects (33 ± 10 years; 9 females) underwent 5‐min resting HRV assessments using disposable (single use) ECG cable and wire system (Kendall DL™ Cardinal Health) and a standard, reusable ECG leads (CardioExpress, Spacelabs Healthcare). Results Intraclass correlation coefficient (ICC) with 95% confidence interval (CI) between disposable and reusable ECG leads was for the time domain [R‐R interval (ms); 0.99 (0.91, 1.00)], the root mean square of successive normal R‐R interval differences (RMSSD) (ms); 0.91 (0.76, 0.96), the SD of normal‐to‐normal R‐R intervals (SDNN) (ms); 0.91 (0.68, 0.97) and frequency domain [low‐frequency (LF) normalized units (nu); 0.90 (0.79, 0.95), high frequency (HF) nu; 0.91 (0.80, 0.96), LF power (ms2); 0.89 (0.62, 0.96), HF power (ms2); 0.90 (0.72, 0.96)] variables. The mean difference and upper and lower limits of agreement between disposable and reusable leads for time‐ and frequency‐domain variables were acceptable. Analysis of repeated measures using disposable leads demonstrated excellent reproducibility (ICC 95% CI) for R‐R interval (ms); 0.93 (0.85, 0.97), RMSSD (ms); 0.93 (0.85, 0.97), SDNN (ms); 0.88 (0.75, 0.95), LF power (ms2); 0.87 (0.72, 0.94), and HF power (ms2); 0.88 (0.73, 0.94) with coefficient of variation ranging from 2.2% to 5% (p > 0.37 for all variables). Conclusion Single‐use Kendall DL™ ECG leads demonstrate a valid and reproducible tool for the assessment of HRV.

Abstract Perovskite is an attractive building block for future optoelectronic applications. However, the strict fabrication conditions of perovskite devices impede the transformation of lab techniques into commercial applications. Here, a facile annealing‐free posttreatment is proposed to reconstruct the perovskite film to obtain high‐performance photodetectors with an optimized production rate. With posttreatment by methylamine thiocyanate, the prefabricated formamidinium‐lead triiodide (FAPbI3) film will undergo a recrystallization process consisting of a repeating phase‐transition‐cycle (PTC) between the black and yellow phases of FAPbI3, which improves the crystal quality and eliminates defects. As a result, some casually prepared or even decomposed perovskite films can be reconstructed, and the dispersion degree of the device performance based on the posttreatment method decreases by ≈21% compared to the traditional antisolvent method. This facile and annealing‐free posttreatment will be an attractive method for the future industrial production of perovskite devices.

Purpose: Cardiac-induced liver motion can bias liver ADC measurements and compromise reproducibility. The purpose of this work was to enable motion-robust DWI on multiple MR scanners and assess reproducibility of the resulting liver ADC measurements., Methods: First moment-optimized diffusion imaging (MODI) was implemented on three MR scanners with various gradient performances and field strengths. MODI-DWI and conventional Stejskal-Tanner monopolar (MONO) DWI were acquired in eight (N = 8) healthy volunteers on each scanner, and DWI repetitions were combined using three different averaging methods. For each combination of scanner, acquisition, and averaging method, ADC measurements from each liver segment were collected. Systematic differences in ADC values between scanners and methods were assessed with linear mixed effects modeling, and reproducibility was quantified via reproducibility coefficients., Results: MODI reduced left-right liver lobe ADC bias from 0.43 × 10 -3  mm 2 /s (MONO) to 0.19 × 10 -3  mm 2 /s (MODI) when simple (unweighted) repetition averaging was used. The bias was reduced from 0.23 × 10 -3  mm 2 /s to 0.06 × 10 -3  mm 2 /s using weighted averaging, and 0.14 × 10 -3  mm 2 /s to 0.01 × 10 -3  mm 2 /s using squared weighted averaging. There was no significant difference in ADC measurements between field strengths or scanner gradient performance. MODI improved reproducibility coefficients compared to MONO: 0.84 × 10 -3  mm 2 /s vs. 0.63 × 10 -3  mm 2 /s (MODI vs. MONO) for simple averaging, 0.66 × 10 -3  mm 2 /s vs. 0.50 × 10 -3  mm 2 /s for weighted averaging, and 0.61 × 10 -3  mm 2 /s vs. 0.47 × 10 -3  mm 2 /s for squared weighted averaging., Conclusion: The feasibility of motion-robust liver DWI using MODI was demonstrated on multiple MR scanners. MODI improved interlobar agreement and reproducibility of ADC measurements in a healthy cohort., (© 2024 The Author(s). Magnetic Resonance in Medicine published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.)

Background and Purpose: Irwin tests are key preclinical study elements for characterising drug-induced neurological side effects. This multicentre study aimed to assess the robustness of Irwin tests across multinational sites during three stages of protocol harmonisation. The projects were part of the Enhanced Quality in Preclinical Data framework, aiming to increase success rates in transition from preclinical testing to clinical application., Experimental Approach: Female and male NMRI mice were assigned to one of three groups (vehicle, MK-801 0.1 and 0.3 mg kg -1 ). Irwin scores were assessed at baseline and multiple times following intraperitoneal injection of MK-801 using local protocols (Stage 1), shared protocols with harmonised environmental design (Stage 2) and fully harmonised Irwin scoring protocols (Stage 3)., Key Results: The analysis based on the four functional domains (motor, autonomic, sedation and excitation) revealed substantial data variability in Stages 1 and 2. Although there was still marked overall heterogeneity between sites in Stage 3 after complete harmonisation of the Irwin scoring scheme, heterogeneity was only moderate within functional domains. When comparing treatment groups versus vehicle, we found large effect sizes in the motor domain and subtle to moderate effects in the excitation-related and autonomic domains., Conclusion and Implications: The pronounced interlaboratory variability in Irwin datasets for the CNS-active compound MK-801 needs to be carefully considered when making decisions during drug development. While environmental and general study design had a minor impact, the study suggests that harmonisation of parameters and their scoring can limit variability and increase robustness., (© 2024 The Author(s). British Journal of Pharmacology published by John Wiley & Sons Ltd on behalf of British Pharmacological Society.)

Purpose: There is increasing recognition of the importance of transparency and reproducibility in scientific research. This study aimed to quantify the extent to which programming code is publicly shared in pharmacoepidemiology, and to develop a set of recommendations on this topic., Methods: We conducted a literature review identifying all studies published in Pharmacoepidemiology and Drug Safety (PDS) between 2017 and 2022. Data were extracted on the frequency and types of programming code shared, and other key open science practices (clinical codelist sharing, data sharing, study preregistration, and stated use of reporting guidelines and preprinting). We developed six recommendations for investigators who choose to share code and gathered feedback from members of the International Society for Pharmacoepidemiology (ISPE)., Results: Programming code sharing by articles published in PDS ranged from 1.8% in 2017 to 9.5% in 2022. It was more prevalent among articles with a methodological focus, simulation studies, and papers which also shared record-level data., Conclusion: Programming code sharing is rare but increasing in pharmacoepidemiology studies published in PDS. We recommend improved reporting of whether code is shared and how available code can be accessed. When sharing programming code, we recommend the use of permanent digital identifiers, appropriate licenses, and, where possible, adherence to good software practices around the provision of metadata and documentation, computational reproducibility, and data privacy., (© 2024 The Author(s). Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)

Purpose: T 1 mapping is a widely used quantitative MRI technique, but its tissue-specific values remain inconsistent across protocols, sites, and vendors. The ISMRM Reproducible Research and Quantitative MR study groups jointly launched a challenge to assess the reproducibility of a well-established inversion-recovery T 1 mapping technique, using acquisition details from a seminal T 1 mapping paper on a standardized phantom and in human brains., Methods: The challenge used the acquisition protocol from Barral et al. (2010). Researchers collected T 1 mapping data on the ISMRM/NIST phantom and/or in human brains. Data submission, pipeline development, and analysis were conducted using open-source platforms. Intersubmission and intrasubmission comparisons were performed., Results: Eighteen submissions (39 phantom and 56 human datasets) on scanners by three MRI vendors were collected at 3 T (except one, at 0.35 T). The mean coefficient of variation was 6.1% for intersubmission phantom measurements, and 2.9% for intrasubmission measurements. For humans, the intersubmission/intrasubmission coefficient of variation was 5.9/3.2% in the genu and 16/6.9% in the cortex. An interactive dashboard for data visualization was also developed: https://rrsg2020.dashboards.neurolibre.org., Conclusion: The T 1 intersubmission variability was twice as high as the intrasubmission variability in both phantoms and human brains, indicating that the acquisition details in the original paper were insufficient to reproduce a quantitative MRI protocol. This study reports the inherent uncertainty in T 1 measures across independent research groups, bringing us one step closer to a practical clinical baseline of T 1 variations in vivo., (© 2024 The Authors. Magnetic Resonance in Medicine published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.)

Abstract Repeatability is the cornerstone of science, and it is particularly important for systematic reviews. However, little is known on how researchers’ choice of database, and search platform influence the repeatability of systematic reviews. Here, we aim to unveil how the computer environment and the location where the search was initiated from influence hit results. We present a comparative analysis of time‐synchronized searches at different institutional locations in the world and evaluate the consistency of hits obtained within each of the search terms using different search platforms. We revealed a large variation among search platforms and showed that PubMed and Scopus returned consistent results to identical search strings from different locations. Google Scholar and Web of Science's Core Collection varied substantially both in the number of returned hits and in the list of individual articles depending on the search location and computing environment. Inconsistency in Web of Science results has most likely emerged from the different licensing packages at different institutions. To maintain scientific integrity and consistency, especially in systematic reviews, action is needed from both the scientific community and scientific search platforms to increase search consistency. Researchers are encouraged to report the search location and the databases used for systematic reviews, and database providers should make search algorithms transparent and revise access rules to titles behind paywalls. Additional options for increasing the repeatability and transparency of systematic reviews are storing both search metadata and hit results in open repositories and using Application Programming Interfaces (APIs) to retrieve standardized, machine‐readable search metadata.

Abstract Heart transplantation is a lifesaving procedure, which is limited by the availability of donor hearts. Using hearts from donors after circulatory death, which have sustained global ischemia, requires thorough studies on reliable and reproducible models that developing researchers may not have mastered. By combining the most recent literature and our recommendations based on observations and trials and errors, the methods here detail a sound in vivo heterotopic heart transplantation model for rats in which protective interventions on the ischemic heart can be studied, and thus allowing the scientific community to advance organ preservation research. Knowledge gathered from reproducible animal models allow for successful translation to clinical studies.

Abstract Plant–plant competition is ubiquitous in nature. However, studying below ground behavior of roots has always posed certain difficulties. Pea (Pisum sativum L.) has become a common study species for questions about how plant roots respond to neighboring plant roots and barriers in soil. However, published results point in several different directions. This has sometimes been interpreted as pea having sophisticated context dependent responses that can change in complex ways depending on its surroundings, but it could also just point to small statistical power resulting in type I or II statistical errors. To explore this further, here, we combine the result of five new experiments with published results to examine 18 unique experiments from 10 different studies and 6 cultivars of pea for a total of 254 replicate plants. We used a Bayesian hierarchical meta‐analysis approach to estimating the likely effect size from the available data, as well as quantify heterogeneity among different experiments, studies and cultivars. The posterior distributions show that, at the coarsest possible scale of total root production, it is unlikely that P. sativum root growth is influenced by either neighbors or pot volume that varies primarily by depth. We find no evidence of publication bias and conclude that this is simply due to statistical sampling error and the scientific method combined with frequentist statistics operating as intended. We suggest that further work on pea should consider repeating experiments that reported finer scale root plasticity at the rhizosphere scale or consider exploring different pot geometries such as volume that varies by depth or width. We also suggest that more diversity in study species are needed to better understand the neighbor‐volume response.

Abstract Transcranial magnetic stimulation (TMS) of the motor cortex can be used during a voluntary contraction to inhibit corticospinal drive to the muscle and consequently induce involuntary muscle relaxation. Our aim was to evaluate the reproducibility and the effect of varying experimental conditions (robustness) of TMS‐induced muscle relaxation. Relaxation of deep finger flexors was assessed in 10 healthy subjects (5 M, 5 F) using handgrip dynamometry with normalized peak relaxation rate as main outcome measure, that is, peak relaxation rate divided by (voluntary plus TMS‐evoked)force prior to relaxation. Both interday and interrater reliability of relaxation rate were high with intraclass correlation coefficient of 0.88 and 0.92 and coefficient of variation of 3.8 and 3.7%, respectively. Target forces of 37.5% of maximal voluntary force or higher resulted in similar relaxation rate. From 50% of maximal stimulator output and higher relaxation rate remained the same. Only the most lateral position (>2 cm from the vertex) rendered lower relaxation rate (mean ± SD: 11.1 ± 3.0 s−1, 95% CI: 9.0–13.3 s−1) compared to stimulation at the vertex (12.8 ± 1.89 s−1, 95% CI: 11.6–14.1 s−1). Within the range of baseline skin temperatures, an average change of 0.5 ± 0.2 s−1 in normalized peak relaxation rate was measured per 1°C change in skin temperature. In conclusion, interday and interrater reproducibility and reliability of TMS‐induced muscle relaxation of the finger flexors were high. Furthermore, this technique is robust with limited effect of target force, stimulation intensity, and coil position. Muscle relaxation is strongly affected by skin temperature; however, this effect is marginal within the normal skin temperature range. We deem this technique well suited for clinical and scientific assessment of muscle relaxation.

This study aimed to optimize the sampling of spin-lock times (TSLs) in quantitative T1ρ mapping for improved reproducibility. Two new TSL sampling schemes were proposed: (i) reproducibility-guided random sampling (RRS) and (ii) reproducibility-guided optimal sampling (ROS). They were compared to the existing linear sampling (LS) and precision-guided sampling (PS) schemes for T1ρ reproducibility through numerical simulations, phantom experiments, and volunteer studies. Each study evaluated the four sampling schemes with three commonly used T1ρ preparations based on composite and balanced spin-locking. Additionally, the phantom and volunteer studies investigated the impact of B 0 and B 1 field inhomogeneities on T1ρ reproducibility, respectively. The reproducibility was assessed using the coefficient of variation (CoV) by repeating the T1ρ measurements eight times for phantom experiments and five times for volunteer studies. Numerical simulations resulted in lower mean CoVs for the proposed RRS (1.74%) and ROS (0.68%) compared to LS (2.93%) and PS (3.68%). In the phantom study, the mean CoVs were also lower for RRS (2.7%) and ROS (2.6%) compared to LS (4.1%) and PS (3.1%). Furthermore, the mean CoVs of the proposed RRS and ROS were statistically lower (P < 0.001) compared to existing LS and PS schemes at a B 1 offset of 20%. In the volunteer study, consistently lower mean CoVs were observed in bilateral thigh muscles for RRS (9.3%) and ROS (9.2%) compared to LS (10.9%) and PS (10.2%), and the difference was more prominent at B 0 offsets higher than 50 Hz. The proposed sampling schemes improve the reproducibility of quantitative T1ρ mapping by optimizing the selection of TSLs. This improvement is especially beneficial for longitudinal studies that track and monitor disease progression and treatment response., (© 2024 John Wiley & Sons Ltd.)

Abstract Introduction Graph metrics have been proposed as potential biomarkers for diagnosis in clinical work. However, before it can be applied in a clinical setting, their reproducibility should be evaluated. Methods This study systematically investigated the effect of two denoising pipelines and different whole‐brain network constructions on reproducibility of subject‐specific graph measures. We used the multi‐session fMRI dataset from the Brain Genomics Superstruct Project consisting of 69 healthy young adults. Results In binary networks, the test–retest variability for global measures was large at low density irrespective of the denoising strategy or the type of correlation. Weighted networks showed very low test–retest values (and thus a good reproducibility) for global graph measures irrespective of the strategy used. Comparing the test–retest values for different strategies, there were significant main effects of the type of correlation (Pearson correlation vs. partial correlation), the (partial) correlation value (absolute vs. positive vs. negative), and weight calculation (based on the raw (partial) correlation values vs. based on transformed Z‐values). There was also a significant interaction effect between type of correlation and weight calculation. Similarly as for the binary networks, there was no main effect of the denoising pipeline. Conclusion Our results demonstrated that normalized global graph measures based on a weighted network using the absolute (partial) correlation as weight were reproducible. The denoising pipeline and the granularity of the whole‐brain parcellation used to define the nodes were not critical for the reproducibility of normalized graph measures.

Abstract Purpose To evaluate the intra and interobserver reproducibility of a new system that assesses the three-dimensional humero-scapulo-thoracic kinematics using wearable technology in an outpatient setting. To obtain normative data with the system for scapular angular motions in three planes. Methods The SHoW Motion 3D kinematic tracking system is a motion analysis system that uses wireless wearable non-invasive inertial-magnetic sensors to assess the three-dimensional kinematics of the shoulder girdle. The sensors are placed over the skin in the sternum, scapular spine and arm to precisely define angular motions of the humerus and the scapula with three Degrees of Freedom (DOF) for each segment. The system was used to measure the scapular angular motions in three planes (upward/downward rotation, internal/external rotation and anterior/posterior tilt) during two shoulder full-range movements (flexion/extension and abduction/abduction) in both shoulders of 25 healthy volunteers (13 males and 12 females, mean age: 37 [standard deviation 11.1] years). In a first measuring session one examiner made two evaluations alternating with another examiner that made a third evaluation. In a second session, one week apart, the first examiner made a fourth evaluation. A mean curve was computed from the normalized data for each measurement to obtain normative data for scapular angular kinematics. Intra and inter-observer reproducibility was evaluated using Root Mean Square Error Estimation (RMSE) and Coefficients for Multiple Correlations (CMC). Results Both shoulders of the 25 volunteers were evaluated four times. The two hundred resulting kinematic analyses were pooled to get normative values for relations between humeral elevation angles and the three angular movements of the scapula. The system showed at least very good (CMC > 0.90) intra and inter-observer reproducibility for scapular tilt and upward-downward rotations both in flexion and abduction. For scapular internal-external rotation the results were acceptable (CMC > 0.75) but not as good, especially for the abduction movement. RMSE calculations showed consistently good reproducibility with RSME

Abstract Background Multi‐segment foot models (MFMs) for assessing three‐dimensional segmental foot motions are calculated via various analytical methods. Although validation studies have already been conducted, we cannot compare their results because the experimental environments in previous studies were different from each other. This study aims to compare the kinematics, repeatability, and reproducibility of five MFMs in the same experimental conditions. Methods Eleven healthy males with a mean age of 26.5 years participated in this study. We created a merged 29‐marker set including five MFMs: Oxford (OFM), modified Rizzoli (mRFM), DuPont (DFM), Milwaukee (MiFM), and modified Shriners Hospital for Children Greenville (mSHCG). Two operators applied the merged model to participants twice, and then we analysed two relative angles of three segments: shank‐hindfoot (HF) and hindfoot‐forefoot (FF). Coefficients of multiple correlation (CMC) and mean standard errors were used to assess repeatability and reproducibility, and statistical parametric mapping (SPM) of the t‐value was employed to compare kinematics. Results HF varus/valgus of the MiFM and mSHCG models, which rotated the segment according to radiographic or goniometric measurements during the reference frame construction, were significantly more repeatable and reproducible, compared to other models. They showed significantly more dorsiflexed HF and plantarflexed FF due to their static offset angles. DFM and mSHCG showed a greater range of motion (ROM), and some models had significantly different FF points of peak angle. Conclusions Under the same conditions, rotating the segment according to the appropriate offset angle obtained from radiographic or goniometric measurement increased reliability, but all MFMs had clinically acceptable reliability compared to previous studies. Moreover, in some models, especially HF varus/valgus, there were differences in ROM and points of peak angle even with no statistical difference in SPM curves. Therefore, based on the results of this study, clinicians and researchers involved in the evaluation of foot and ankle dysfunction need an understanding of the specific features of each MFM to make accurate decisions.

Abstract Image‐based modeling, and more precisely, Structure from Motion (SfM) and Multi‐View Stereo (MVS), is emerging as a flexible, self‐service, remote sensing tool for generating fine‐grained digital surface models (DSMs) in the Earth sciences and ecology. However, drone‐based SfM + MVS applications have developed at a rapid pace over the past decade and there are now many software options available for data processing. Consequently, understanding of reproducibility issues caused by variations in software choice and their influence on data quality is relatively poorly understood. This understanding is crucial for the development of SfM + MVS if it is to fulfill a role as a new quantitative remote sensing tool to inform management frameworks and species conservation schemes. To address this knowledge gap, a lightweight multirotor drone carrying a Ricoh GR II consumer‐grade camera was used to capture replicate, centimeter‐resolution image datasets of a temperate, intensively managed grassland ecosystem. These data allowed the exploration of method reproducibility and the impact of SfM + MVS software choice on derived vegetation canopy height measurement accuracy. The quality of DSM height measurements derived from four different, yet widely used SfM‐MVS software—Photoscan, Pix4D, 3DFlow Zephyr, and MICMAC, was compared with in situ data captured on the same day as image capture. We used both traditional agronomic techniques for measuring sward height, and a high accuracy and precision differential GPS survey to generate independent measurements of the underlying ground surface elevation. Using the same replicate image dataset (n = 3) as input, we demonstrate that there are 1.7, 2.0, and 2.5 cm differences in RMSE (excluding one outlier) between the outputs from different SfM + MVS software using High, Medium, and Low quality settings, respectively. Furthermore, we show that there can be a significant difference, although of small overall magnitude between replicate image datasets (n = 3) processed using the same SfM + MVS software, following the same workflow, with a variance in RMSE of up to 1.3, 1.5, and 2.7 cm (excluding one outlier) for “High,” “Medium,” and “Low” quality settings, respectively. We conclude that SfM + MVS software choice does matter, although the differences between products processed using “High” and “Medium” quality settings are of small overall magnitude.

Abstract The minimal information for studies of extracellular vesicles (EVs, MISEV) is a field‐consensus rigour initiative of the International Society for Extracellular Vesicles (ISEV). The last update to MISEV, MISEV2018, was informed by input from more than 400 scientists and made recommendations in the six broad topics of EV nomenclature, sample collection and pre‐processing, EV separation and concentration, characterization, functional studies, and reporting requirements/exceptions. To gather opinions on MISEV and ideas for new updates, the ISEV Board of Directors canvassed previous MISEV authors and society members. Here, we share conclusions that are relevant to the ongoing evolution of the MISEV initiative and other ISEV rigour and standardization efforts.

Purpose: Arterial spin labeling (ASL) is a widely used contrast-free MRI method for assessing cerebral blood flow (CBF). Despite the generally adopted ASL acquisition guidelines, there is still wide variability in ASL analysis. We explored this variability through the ISMRM-OSIPI ASL-MRI Challenge, aiming to establish best practices for more reproducible ASL analysis., Methods: Eight teams analyzed the challenge data, which included a high-resolution T1-weighted anatomical image and 10 pseudo-continuous ASL datasets simulated using a digital reference object to generate ground-truth CBF values in normal and pathological states. We compared the accuracy of CBF quantification from each team's analysis to the ground truth across all voxels and within predefined brain regions. Reproducibility of CBF across analysis pipelines was assessed using the intra-class correlation coefficient (ICC), limits of agreement (LOA), and replicability of generating similar CBF estimates from different processing approaches., Results: Absolute errors in CBF estimates compared to ground-truth synthetic data ranged from 18.36 to 48.12 mL/100 g/min. Realistic motion incorporated into three datasets produced the largest absolute error and variability between teams, with the least agreement (ICC and LOA) with ground-truth results. Fifty percent of the submissions were replicated, and one produced three times larger CBF errors (46.59 mL/100 g/min) compared to submitted results., Conclusions: Variability in CBF measurements, influenced by differences in image processing, especially to compensate for motion, highlights the significance of standardizing ASL analysis workflows. We provide a recommendation for ASL processing based on top-performing approaches as a step toward ASL standardization., (© 2024 The Authors. Magnetic Resonance in Medicine published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.)

Purpose: Metadata for data dIscoverability aNd study rEplicability in obseRVAtional studies (MINERVA), a European Medicines Agency-funded project (EUPAS39322), defined a set of metadata to describe real-world data sources (RWDSs) and piloted metadata collection in a prototype catalogue to assist investigators from data source discoverability through study conduct., Methods: A list of metadata was created from a review of existing metadata catalogues and recommendations, structured interviews, a stakeholder survey, and a technical workshop. The prototype was designed to comply with the FAIR principles (findable, accessible, interoperable, reusable), using MOLGENIS software. Metadata collection was piloted by 15 data access partners (DAPs) from across Europe., Results: A total of 442 metadata variables were defined in six domains: institutions (organizations connected to a data source); data banks (data collections sustained by an organization); data sources (collections of linkable data banks covering a common underlying population); studies; networks (of institutions); and common data models (CDMs). A total of 26 institutions were recorded in the prototype. Each DAP populated the metadata of one data source and its selected data banks. The number of data banks varied by data source; the most common data banks were hospital administrative records and pharmacy dispensation records (10 data sources each). Quantitative metadata were successfully extracted from three data sources conforming to different CDMs and entered into the prototype., Conclusions: A metadata list was finalized, a prototype was successfully populated, and a good practice guide was developed. Setting up and maintaining a metadata catalogue on RWDSs will require substantial effort to support discoverability of data sources and reproducibility of studies in Europe., (© 2024 The Author(s). Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)

Neuroimaging data acquired using multiple scanners or protocols are increasingly available. However, such data exhibit technical artifacts across batches which introduce confounding and decrease reproducibility. This is especially true when multi-batch data are analyzed using complex downstream models which are more likely to pick up on and implicitly incorporate batch-related information. Previously proposed image harmonization methods have sought to remove these batch effects; however, batch effects remain detectable in the data after applying these methods. We present DeepComBat, a deep learning harmonization method based on a conditional variational autoencoder and the ComBat method. DeepComBat combines the strengths of statistical and deep learning methods in order to account for the multivariate relationships between features while simultaneously relaxing strong assumptions made by previous deep learning harmonization methods. As a result, DeepComBat can perform multivariate harmonization while preserving data structure and avoiding the introduction of synthetic artifacts. We apply this method to cortical thickness measurements from a cognitive-aging cohort and show DeepComBat qualitatively and quantitatively outperforms existing methods in removing batch effects while preserving biological heterogeneity. Additionally, DeepComBat provides a new perspective for statistically motivated deep learning harmonization methods., (© 2024 The Author(s). Human Brain Mapping published by Wiley Periodicals LLC.)

Purpose: Quantitative multi-parameter mapping (MPM) provides maps of physical quantities representing physiologically meaningful tissue characteristics, which allows to investigate microstructure-function relationships reflecting normal or pathologic processes in the brain. However, the achievable spatial resolution and stability of MPM for basic research or clinical applications is severely constrained by SNR limits of the MR acquisition process, resulting in relatively long acquisition times. To increase SNR, we denoise MPM acquisitions using principal component analysis along tensors exploiting the Marchenko-Pastur law (tMPPCA)., Methods: tMPPCA denoising was applied to three sets of MPM raw data before the quantification of maps of proton density, magnetization transfer saturation, R 1 , and R 2 *. The regional SNR gain for high-resolution MPM was investigated as well as reproducibility gains for clinically optimized protocols with moderate and high acceleration factors at different image resolutions., Results: Substantial noise reduction in raw data was achieved, resulting in reduced noise for quantitative mapping up to sixfold without introducing bias of mean values (below 1%). Scan-rescan fluctuations were reduced up to threefold. Denoising allowed to decrease the voxel volume fourfold at the same scan time or reduce the scan time twofold at same voxel volume without loss of sensitivity., Conclusions: tMPPCA denoising can (a) improve of fine spatial and temporal patterns, (b) considerably reduce scan time for clinical applications, or (c) increase resolution to potentially push cutting-edge MPM protocols from the upper to the lower limit of the mesoscopic scale., (© 2024 The Authors. Magnetic Resonance in Medicine published by Wiley Periodicals LLC on behalf of International Society for Magnetic Resonance in Medicine.)

Purpose: To reduce the inter-scanner variability of diffusion MRI (dMRI) measures between scanners from different vendors by developing a vendor-neutral dMRI pulse sequence using the open-source vendor-agnostic Pulseq platform., Methods: We implemented a standard EPI based dMRI sequence in Pulseq. We tested it on two clinical scanners from different vendors (Siemens Prisma and GE Premier), systematically evaluating and comparing the within- and inter-scanner variability across the vendors, using both the vendor-provided and Pulseq dMRI sequences. Assessments covered both a diffusion phantom and three human subjects, using standard error (SE) and Lin's concordance correlation to measure the repeatability and reproducibility of standard DTI metrics including fractional anisotropy (FA) and mean diffusivity (MD)., Results: Identical dMRI sequences were executed on both scanners using Pulseq. On the phantom, the Pulseq sequence showed more than a 2.5× reduction in SE (variability) across Siemens and GE scanners. Furthermore, Pulseq sequences exhibited markedly reduced SE in-vivo, maintaining scan-rescan repeatability while delivering lower variability in FA and MD (more than 50% reduction in cortical/subcortical regions) compared to vendor-provided sequences., Conclusion: The Pulseq diffusion sequence reduces the cross-scanner variability for both phantom and in-vivo data, which will benefit multi-center neuroimaging studies and improve the reproducibility of neuroimaging studies., (© 2024 International Society for Magnetic Resonance in Medicine.)

Purpose: Our objective is to describe how the U.S. Food and Drug Administration (FDA)'s Sentinel System implements best practices to ensure trust in drug safety studies using real-world data from disparate sources., Methods: We present a stepwise schematic for Sentinel's data harmonization, data quality check, query design and implementation, and reporting practices, and describe approaches to enhancing the transparency, reproducibility, and replicability of studies at each step., Conclusions: Each Sentinel data partner converts its source data into the Sentinel Common Data Model. The transformed data undergoes rigorous quality checks before it can be used for Sentinel queries. The Sentinel Common Data Model framework, data transformation codes for several data sources, and data quality assurance packages are publicly available. Designed to run against the Sentinel Common Data Model, Sentinel's querying system comprises a suite of pre-tested, parametrizable computer programs that allow users to perform sophisticated descriptive and inferential analysis without having to exchange individual-level data across sites. Detailed documentation of capabilities of the programs as well as the codes and information required to execute them are publicly available on the Sentinel website. Sentinel also provides public trainings and online resources to facilitate use of its data model and querying system. Its study specifications conform to established reporting frameworks aimed at facilitating reproducibility and replicability of real-world data studies. Reports from Sentinel queries and associated design and analytic specifications are available for download on the Sentinel website. Sentinel is an example of how real-world data can be used to generate regulatory-grade evidence at scale using a transparent, reproducible, and replicable process., (© 2024 John Wiley & Sons Ltd.)

Electronic health records (EHRs) and other administrative health data are increasingly used in research to generate evidence on the effectiveness, safety, and utilisation of medical products and services, and to inform public health guidance and policy. Reproducibility is a fundamental step for research credibility and promotes trust in evidence generated from EHRs. At present, ensuring research using EHRs is reproducible can be challenging for researchers. Research software platforms can provide technical solutions to enhance the reproducibility of research conducted using EHRs. In response to the COVID-19 pandemic, we developed the secure, transparent, analytic open-source software platform OpenSAFELY designed with reproducible research in mind. OpenSAFELY mitigates common barriers to reproducible research by: standardising key workflows around data preparation; removing barriers to code-sharing in secure analysis environments; enforcing public sharing of programming code and codelists; ensuring the same computational environment is used everywhere; integrating new and existing tools that encourage and enable the use of reproducible working practices; and providing an audit trail for all code that is run against the real data to increase transparency. This paper describes OpenSAFELY's reproducibility-by-design approach in detail., (© 2024 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)

Abstract This paper describes the design, setup, and performance of a new analytical system installed at Lancaster University, UK, to determine noble gas isotopes in gas samples using an NGX high‐resolution multiple collector noble gas mass spectrometer. The fixed position Faraday cups are configured to simultaneously measure all nine Xe isotopes whilst permitting all other noble gases, such as He, to be analyzed in peak jumping mode. The stability, sensitivity, and reproducibility of the measurements using this new noble gas analytical system were examined by repeating runs of atmospheric air. Reproducibility of the measurements obtained from 90 and 73 air standard runs are 4.2% and 0.4% for 3He/4He and 21Ne/20Ne, respectively. There is no significant difference in the 40Ar/36Ar ratio measurements with various 40Ar abundances, suggesting good sensitivity and pressure linearity of the NGX instrument. The standard errors for 124Xe/130Xe, 126Xe/130Xe, 128Xe/130Xe, 129Xe/130Xe, 131Xe/130Xe, 132Xe/130Xe, 134Xe/130Xe and 136Xe/130Xe in 46 air measurements are 0.782%, 0.804%, 0.053%, 0.029%, 0.029%, 0.029%, 0.030%, and 0.031%, respectively. In contrast, the corresponding reference values from Basford et al. (1973) are 0.342%, 0.505%, 0.148%, 0.139%, 0.153%, 0.076%, 0.156%, and 0.138%, respectively. This suggests that multicollection measurements of Xe isotopes on Faraday cups using the NGX have higher precision than the previous results except for 124Xe and 126Xe, where low abundances affect the measurement precision. Compared to traditional peak jumping single‐collector mass spectrometers, the NGX has significantly decreased analytical time in Xe analysis by allowing simultaneous measurements of all isotopes. As a consequence, high analytical precision can be achieved.

Abstract Background Broadly applicable reference intervals (RIs) for measurements of left atrial (LA) and left ventricular (LV) size and function generated prospectively using statistically appropriate methods are limited. Objectives To generate body size‐independent RIs for linear, area, and volume measurements of LA size and LV size and function. Animals Healthy adult dogs (n = 122) of variable size and somatotype. Methods Prospective study. All dogs underwent an echocardiogram performed by the same examiner. Effects of body weight, sex, age, and heart rate were evaluated by regression and correlation analyses. Scaling exponents and prediction intervals were generated for linear measurements using the allometric equation. After normalization to body weight, 95% RIs were determined using nonparametric methods with 2.5 and 97.5 percentiles serving as the lower and upper limits (each with 90% confidence intervals), respectively. Results Linear LA and LV measurements were strongly correlated (R2 ≥ 0.79) with body weight. Scaling exponents were close to the expected 1/3 (0.299‐0.392). Prediction intervals for linear measurements of LV chamber size were considerably narrower than previously reported. Weak correlations (r = −0.42 to −0.50) among LV fractional shortening, fractional area change, and ejection fraction and body weight were identified. No other meaningful relationships were identified between the measurements and sex, age, and heart rate. Conclusions and Clinical Importance Body size‐independent RIs for several linear, area, and volume measurements of LA and LV size and function were generated prospectively from a large and diverse reference population and are available for clinical use.

Abstract Background The ankle brachial index (ABI) is widely used in clinical practice as a non-invasive method to detect the presence and severity of peripheral arterial disease (PAD). Current guidelines suggest that it should be used to monitor potential progression of PAD in affected individuals. As such, it is important that the test is reliable when used for repeated measurements, by the same or different health practitioners. This systematic review aims to examine the literature to evaluate the inter- and intra-rater reliability of the ABI. Methods A systematic search of MEDLINE, EMBASE and CINAHL Complete was conducted to 20 January 2019. Two authors independently reviewed and selected relevant studies and extracted the data. Methodological quality was determined using the Quality Appraisal of Reliability (QAREL) Checklist. Results Fifteen studies of ABI reliability in a range of patient populations were identified as suitable for inclusion in the review: seven considered inter-rater reliability, four intra-rater reliability, and four studies evaluated both inter- and intra-rater reliability. Inter-rater reliability was found to be highly variable, with intraclass correlation coefficients (ICC’s) ranging from poor to excellent (ICC 0.42–1.00), while intra-rater also demonstrated considerable variation, with ICCs from 0.42–0.98. Meta-analysis was not possible due to the lack of statistical information reported. Conclusions Results of included studies suggest the inter- and intra-tester reliability of the ABI is acceptable. However, inconsistencies in obtaining systolic pressure measurements, calculating ABI values, and incomplete reporting of methodologies and statistical analysis make it difficult to determine the validity of the results of included studies. Further research, with more consistent reliability methodology, statistical analysis and reporting conducted in populations at risk of PAD is needed to conclusively determine the ABI reliability.

Abstract Purpose The objective of this study was to evaluate kinaesthetic proprioceptive deficit after knee anterior cruciate ligament (ACL) reconstruction in two populations of athletes, those in the post-surgery period and those in re-training during the intensive program-training phase. Methods We performed a prospective study in ACL-operated athletes without previous knee injuries, with 32 athletes in each group. Time since surgery in the operated athletes in the post-surgery group was 21 to 35 days, and between three and 9 months in the re-training group. We also analysed a control group of 32 uninjured non-operated subjects with a similar sporting level. Proprioception was evaluated using the threshold to detection of passive motion (TDPM) test with Biodex-type isokinetic equipment comparing operated knees, non-operated knees and control uninjured non-operated group. The control group was tested twice, 1 day apart to control reproducibility, using the intraclass correlation coefficient (ICC). The p-value threshold for statistical significance between different groups in hypothesis testing was

Abstract Background Electrocardiographic left ventricular hypertrophy (ECG‐LVH) represents preclinical cardiovascular disease and predicts cardiovascular disease morbidity and mortality. While the newly developed Peguero‐Lo Presti ECG‐LVH criteria have greater sensitivity for LVH than the Cornell voltage and Sokolow–Lyon criteria, its short‐term repeatability is unknown. Therefore, we characterized the short‐term repeatability of Peguero‐Lo Presti ECG‐LVH criteria and evaluate its agreement with Cornell voltage and Sokolow–Lyon ECG‐LVH criteria. Methods Participants underwent two resting, standard, 12‐lead ECGs at each of two visits one week apart (n = 63). We defined a Peguero‐Lo Presti index as a sum of the deepest S wave amplitude in any single lead and lead V4 (i.e., SD + SV4) and defined Peguero‐Lo Presti LVH index as ≥ 2,300 µV among women and ≥ 2,800 µV among men. We estimated repeatability as an intraclass correlation coefficient (ICC), agreement as a prevalence‐adjusted bias‐adjusted kappa coefficient (κ), and precision using 95% confidence intervals (CIs). Results The Peguero‐Lo Presti index was repeatable: ICC (95% CI) = 0.94 (0.91–0.97). Within‐visit agreement of Peguero‐Lo Presti LVH was high at the first and second visits: κ (95% CI) = 0.97 (0.91–1.00) and 1.00 (1.00–1.00). Between‐visit agreement of the first and second measurements at each visit was comparable: κ (95% CI) = 0.90 (0.80–1.00) and 0.93 (0.85–1.00). Agreement of Peguero‐Lo Presti and Cornell or Sokolow–Lyon LVH on any one of the four ECGs was slightly lower: κ (95% CI) = 0.71 (0.54–0.89). Conclusion The Peguero‐Lo Presti index and LVH have excellent repeatability and agreement, which support their use in clinical and epidemiological studies.

Abstract 1. Over the past decades, evidence‐based research has become increasingly important in restoration ecology. Evidence synthesis can be a powerful tool to identify the most effective strategies to conserve and restore ecosystems. However, reviews in the environmental sector have been described as non‐systematic and exhibit a diverse range of approaches. While it is known that environmental syntheses can substantially improve in quality, the reliability of restoration reviews (i.e. the level of confidence an end‐user may place in their methodology) remains poorly known. 2. Given the importance of literature reviews of restoration practice and outcomes for informing management and policy, as well as research, this systematic map protocol aims to scrutinize the peer‐reviewed literature for an assessment of the methodological reliability and reproducibility of restoration reviews. We will use bibliographic databases and search engines to collect studies published in peer‐reviewed journals dealing with the ecological restoration of terrestrial ecosystems. 3. Through a scoping exercise, a search string was developed which was based on a previously prepared test list. The search string was then tested for validity with one independent reference list. After searching, the screening process will be done on the title, abstract and full‐text level and consistency checking will be done on a random subsample by a second assessor, with decisions being compared using the kappa test of agreement. After retrieving studies and checking for relevance to the synthesis, we will appraise the methodological reliability of restoration reviews by applying the Collaboration for Environmental Evidence Assessment Tool – CEESAT. Lastly, we will collect bibliometric information to qualitatively describe the retrieved body of literature, and then key trends in data will be synthesized according to a range of generic questions. 4. To conduct the resulting review, we will follow the procedures specified in this protocol, considering guidelines from the Collaboration for Environmental Evidence and ROSES form. The resulting review will yield a useful overview of applying systematic reviews principles for various end users. At the same time, it will help restoration practitioners to identify critical points where restoration evidence syntheses must be improved to move forward.

Purpose: Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence., Methods: In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes., Results: Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization., Conclusions: The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE., (© 2024 The Authors. Pharmacoepidemiology and Drug Safety published by John Wiley & Sons Ltd.)