Search

Bernice Engeltjes,1,2 Ageeth Rosman,2 Loes CM Bertens,3 Eveline Wouters,4 Doug Cronie,2 Fedde Scheele1,5 1Athena Institute for Transdisciplinary Research, Faculty of Science, VU University, Amsterdam, the Netherlands; 2Department of Healthcare Studies, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands; 3Department of Obstetrics and Gynecology, Erasmus MC, University Medical Centre Rotterdam, Rotterdam, the Netherlands; 4Department of Tranzo, Tilburg School of Social and Behavioral Sciences, Tilburg University, Tilburg, the Netherlands; 5Department of Healthcare Education, OLVG Teaching Hospital, Amsterdam, the NetherlandsCorrespondence: Bernice EngeltjesDepartment of Healthcare Studies, Rotterdam University of Applied Sciences, Rotterdam, the NetherlandsTel +31 6 41804881Email engeltjesbernice@gmail.comBackground: Safety and efficiency of emergency care can be optimized with a triage system which uses urgency to prioritize care. The Dutch Obstetric Telephone Triage System (DOTTS) was developed to provide a basis for assessing urgency of unplanned obstetric care requests by telephone. Reliability and validity are important components in evaluating such (obstetric) triage systems.Objective: To determine the reliability of Dutch Obstetric Telephone Triage, by calculating the inter-rater and intra-rater reliability.Methods: To evaluate the urgency levels of DOTTS by testing inter-rater and intra-rater reliability, 90 vignettes of possible requests were developed. Nineteen participants, from hospitals where DOTTS had been implemented, rated in two rounds a set of ten vignettes. The five urgency levels and five presenting symptoms had an equal spread and had to be entered in accordance with DOTTS per vignette. Urgency levels were dichotomized into high urgency and intermediate urgency. Inter-rater reliability was rated as degree of agreement between two different participants with the same vignette. Intra-rater reliability was rated as agreement by the same participants at different moments in time. The degree of inter-rater and intra-rater reliability was tested using weighted Cohen’s Kappa and ICC.Results: The agreement of urgency level between participants in accordance with predefined urgency level per vignette was 90.5% (95% CI 87.5– 93.6) [335 of 370]. Agreement of urgency level between participants was 88.5% (95% CI 84.9– 93.0) [177 of 200] and 84.9% (95% CI 78.3– 91.4) after re-rating [101 of 119]. Inter-rater reliability of DOTTS expressed as Cohen’s Kappa was 0.77 and as ICC 0.87; intra-rater reliability of DOTTS expressed as Cohen’s Kappa was 0.70 and as ICC 0.82.Conclusion: Inter-rater and intra-rater reliability of DOTTS showed substantial correlation, and is comparable to other studies. Therefore, DOTTS is considered reliable.Keywords: obstetrics, triage system, inter-observer agreement, intra-observer agreement, undertriage and overtriage

Background The plasma cell disease is been studying by the whole-body MRI technology. However, the time requested to learn this radiological technique is unknown. Purpose To esteem, quantitatively and qualitatively, the essential time to learn the whole-body MRI diffusion-weighted imaging with background body signal suppression in patients with plasma cell disease. Materials and methods Between January 2015 and February 2017, three readers in-training with different levels of experience examined the anonymised and randomised whole-body MRI images of 52 patients with a diagnosis of plasma cell disease and analysed their morphological (T1w, T2w with and without fat suppression) and functional sequences. Reports of an expert radiologist were considered the standard of reference. Images were analysed in two sessions, during which each reader was timed. Readers reported the number of segments with lesions and staged the disease using the Durie–Salmon PLUS staging system. Weighted Cohen’s ĸ and Z-test were used to compare the trainees’ reports with those of the expert radiologist, and learning curves were drawn up to show changes between the two sessions. Results Weighted Cohen’s ĸ of number of lesioned segments increased from 0.536 ± 0.123 to 0.831 ± 0.129 (Prob > Z under 0.005), thus approaching the goal of ĸ > 0.8. Trainees reached the level of experienced radiologist in terms of time by the 33rd patient. Agreement concerning the Durie–Salmon PLUS increased from 0.536 ± 0.123 to 0.831 ± 0.129 (Prob > Z under 0.005). Conclusions The findings of this study demonstrate that whole-body MRI with DWIBS can be learned in about 80 reports and leads to a high level of inter-observer concordance when using the Durie–Salmon PLUS staging system.

Classification schemes for reporting thyroid cytology of fine needle aspiration (FNA) of thyroid nodules are largely used in clinical practice, but the level of inter-observer agreement among cytopathologists is poorly acknowledged. The present study aimed to explore the inter-observer agreement among the experienced authors of the 2014 Italian Consensus for Classification and Reporting of Thyroid Cytology (ICCRTC). A stratified randomization was performed in order to obtain a sample homogeneously distributed and representative of all ICCRTC classes. Four high-experience raters were randomly selected among the extensors of the Italian consensus. They independently reviewed 60 FNA samples blindly of the initial cytological report and clinical features. Their overall agreement was evaluated according to Fleiss' kappa. The overall inter-observer agreement was moderate (κ 0.46). Specifically, a good agreement was found when the samples were consistent for malignancy (TIR5) or were not adequate for diagnosis (TIR1) (κ 0.67 and κ 0.73, respectively). A moderate agreement was present for suspicious-for-malignant category (TIR4), and a fair agreement was recorded in the two intermediate ones (TIR3A and TIR3B) (κ 0.36 and κ 0.35, respectively). For clinical purposes, the agreement was good (κ 0.74) in differentiating cases with surgical indication (TIR4/TIR5) from those in which surgery is not essential or requires limited extension (TIR3B/TIR3A/TIR2). In conclusion, the present study confirms the reliability of ICCRTC. These data represent a reference for cytopathologists using this system and are useful for the practice of clinicians and surgeons.

Background Whole body magnetic resonance imaging (MRI) is now incorporated into international guidance for imaging patients with multiple myeloma. The aim of this study was to investigate inter-observer agreement of triple reported baseline whole-body MRI in myeloma and highlight potential pitfalls. Methods Fifty-seven patients with symptomatic myeloma at first presentation or relapse and planned for autologous stem cell transplant were included. All patients completed baseline whole body MRI within 2 weeks prior to starting treatment. Each scan was reported independently by 3 radiologists using a defined scoring system. Differences in observer scores were compared using analysis of variance (ANOVA) and inter-observer agreement assessed using intra class correlation coefficient (ICC). Results There was no significant difference in mean observer scores for whole skeleton and ICC demonstrated excellent inter-observer agreement at 0.91. ICC varied between skeletal regions with spine, pelvis and ribs showing good inter-observer agreement, whereas skull and long bones were moderate. Scans with variation in observer scores were re-examined and cause of discrepancies identified. This information was used to describe potential anatomical pitfalls in reporting . Conclusion Whole-body MRI has excellent inter-observer agreement in reporting symptomatic myeloma at baseline. Inter-observer agreement varied between skeletal regions highlighting specific areas of difficulty.

Summary: Background/Objective: A preoperative reliable classification system between clinical and computed tomography (CT) findings to better plan surgery in acute complicated diverticulitis (ACD) is lacking. We studied the inter-observer agreement of CT scan data and their concordance with the preoperative clinical findings and the adherence with the intraoperative status using a new classification of diverticular disease (CDD). Methods: 152 patients operated on for acute complicated diverticulitis (ACD) were retrospectively enrolled. All patients were studied with CT scan within 24 h before surgery and CT images were blinded reanalyzed by 2 couples of radiologists (A/B). Kappa value evaluated the inter-observer agreement between radiologists and the concordance between CDD, preoperative clinical findings and findings at operation. Univariate and multivariate analysis were used to evaluate the predicting values of CT classification and CDD stage at surgery on postoperative outcomes. Results: Overall inter-observer agreement for the CDD was high, with a kappa value of 0.905 (95% CI = 0.850–0.960) for observers A and B, while the concordance between radiological and surgical findings was weak (kappa values = 0.213 and 0,248, respectively and 95% CI = 0.106 to 0.319 and 95% CI = 0.142 to 0.355, respectively). When overall morbidity, mortality and the need of a terminal colostomy were considered as main endpoints no concordance was observed between surgical and radiological findings and the CDD (P=NS). Conclusions: The need for a more accurate classification of ACD, able to better stage this emergency, and to provide surgeons with reliable information for the best treatment is advocated. Keywords: Acute complicated diverticulitis, Inter-observer agreement, CDD, CT scanning, Operative findings, Predicting value

Background/purpose: Among patients suspected of pectus excavatum, visual examination is a key aspect of diagnosis and, moreover, guides work-up and treatment strategy. This study evaluated the inter observer and intra-observer agreement of visual examination and diagnosis of pectus excavatum among experts. Methods: Three-dimensional surface images of consecutive patients suspected of pectus excavatum were reviewed in a multi-center setting. Interactive three-dimensional images were evaluated for the presence of pectus excavatum, asymmetry, flaring, depth of deformity, cranial onset, overall severity and morphological subtype through a questionnaire. Observers were blinded to all clinical patient information, completing the questionnaire twice per subject. Agreement was analyzed by kappa statistics. Results: Fifty-eight subjects with a median age of 15.5 years (interquartile range: 14.1-18.2) were evaluated by 5 (cardio)thoracic surgeons. Pectus excavatum was visually diagnosed in 55% to 95% of cases by different surgeons, revealing considerable inter-observer differences (kappa: 0.50; 95%-confidence interval [CI]: 0.41-0.58). All other items demonstrated inter-observer kappa's of 0.25-0.37. Intra-observer analyses evaluating the presence of pectus excavatum demonstrated a kappa of 0.81 (95%-CI: 0.72-0.91), while all other items showed intra-observer kappa's of 0.36-0.68. Conclusions: Visual examination and diagnosis of pectus excavatum yields considerable inter-observer and intra-observer disagreements. As this variation in judgement could impact work-up and treatment strategy, objective standardization is urged. Levels of evidence: III. (c) 2021 Elsevier Inc. All rights reserved.

BACKGROUND The cytobrush technique is commonly used to sample the equine ocular surface. Impression cytology (IC) is an innovative noninvasive method, which allows for the collection of superficial layers of ocular epithelium. OBJECTIVES The aims of this study were to compare the cytobrush and IC techniques on healthy equine ocular surfaces, to assess the agreement between observers with different levels of expertise, and to test the preservability of filters over time. METHODS Twenty-four horses were sampled within 10 minutes of slaughter using IC on the left eye and the cytobrush technique on the right eye. May-Grunwald-Giemsa stained specimens were evaluated by two observers with different levels of expertise. Morphologic features were evaluated using a 4-grade system. The IC samples were re-evaluated after 6 months to examine filter preservation. RESULTS In IC samples, corneal and conjunctival cells were clearly separated. Goblet cells were found in five and 17 filters by observer 1 and 2, respectively. Using the cytobrush technique, corneal and conjunctival cells were present but mixed. Goblet cell cellularity, preservation, and enumeration were higher with the IC technique compared with the cytobrush technique (P = 0.013; P = 0.004; P = 0.031, respectively). The inter-observer agreement for the IC technique was moderate to fair. In 7/24 IC samples re-evaluated after 6 months, cellular morphology was impaired, and the overall score was significantly lower. CONCLUSIONS IC is an innovative noninvasive method, which allows for sample collection with higher cellularity and preservation. Moreover, the identification of goblet cells is easier. For these reasons, IC could be interesting and useful as a complementary diagnostic cytologic method in clinical practice.

Background Intestinal ultrasound [IUS] is a promising and non-invasive cross-sectional imaging modality in the diagnosis and monitoring of ulcerative colitis [UC]. Unlike endoscopy, where standardized scoring for evaluation of disease activity is widely used, scoring for UC with IUS is currently unavailable. Therefore, we conducted a study to assess the reliability of IUS in UC among expert sonographists in order to identify robust parameters. Methods Thirty patients with both clinically active [25] and quiescent [five] UC were included. Six expert sonographers first agreed upon key IUS parameters and grading, including bowel wall thickness [BWT], colour Doppler signal [CDS], inflammatory fat [i-fat], loss of bowel wall stratification [BWS], loss of haustrations and presence of lymph nodes. Thirty video-recorded cases were blindly reviewed. Results Inter-observer agreement was almost perfect for BWT (intra-class correlation coefficient [ICC]: 0.96) and substantial for CDS [κ = 0.63]. Agreement was moderate for presence of lymph nodes [κ = 0.41] and fair for presence of i-fat [κ = 0.36], BWS [κ = 0.24] and loss of haustrations [κ = 0.26]. Furthermore, there was substantial agreement for presence of disease activity on IUS [κ = 0.77] and almost perfect agreement for disease severity [ICC: 0.93]. Most individual parameters showed a strong association with IUS disease activity as measured by the six readers. Conclusion IUS is a reliable imaging modality to assess disease activity and severity in UC. Important individual parameters such as BWT and CDS are reliable and could be incorporated in a future UC scoring index. Standardized acquisition and assessment of UC utilizing IUS with established reliability is important to expand the use of IUS globally.

Background Use of 11C-Choline PET/CT is gaining ground in detecting hyperfunctioning parathyroid glands in primary hyperparathyroidism. The purpose of this study was to evaluate the robustness of 11C-Choline PET/CT by assessing intra- and inter-observer agreement to determine whether the method was reader sensitive and therefore should only be performed at highly specialised sites with a high number of cases. PET/CT images of 40 patients diagnosed with primary hyperparathyroidism were anonymised and evaluated three times by three readers: an expert reader and two non-experts (non-experts were experienced in PET/CT imaging, but not in 11C-Choline PET/CT in the setting of primary hyperparathyroidism). Number of hyperfunctioning parathyroid glands, location relative to the thyroid gland and confidence of each assessment (low, moderate or high) were noted, and intra- and inter-observer agreement calculated using Fleiss’ kappa method. Sensitivities and specificities of the non-experts were calculated using the expert reader as gold standard. Results Intra-observer agreement was ‘good’ to ‘near perfect’ for all readers. Inter-observer agreement was good between non-experts and the expert, with kappa values ≥ 0.74. Sensitivities between non-experts and the expert were high, > 81%, when assessing which side and 75% when assessing thyroid quadrant. All specificities were > 94%. Reader certainties were ‘high’ in > 80% of cases for the expert and > 70% and > 65%, respectively for the two non-experts. Conclusion 11C-Choline PET/CT is not reader sensitive for the localisation of hyperfunctioning parathyroid glands and may therefore be safely implemented at sites that have a moderate number of cases. Access to a cyclotron laboratory is, however, a necessity for the production of 11C-Choline. The study was conducted in accordance with the Helsinki 2 declaration and The International Council for Harmonisation Guideline for Good Clinical Practice (ICH_GCP) clinical trial, approved by the Research Ethics Committee of the Capital Region of Denmark (Journal-nr.:H-18012490, date of approval: 18 June 2018) and the Danish Medicines Agency (EudraCT no. 2018-000726-63, date of approval: 6 June 2018). The GCP unit in Eastern Denmark has carried out regular monitoring of the trial according to GCP (ID: 2018-1050).

In the constantly developing era of minimal diagnostic invasiveness, the role of colon capsule endoscopy in colonic examination is being increasingly recognised, especially in the context of curtailed endoscopy services due to the COVID-19 pandemic. It is a safe diagnostic tool with low adverse event rates. As with other endoscopic modalities, various colon capsule endoscopy scores allow the standardisation of reporting and reproducibility. As bowel cleanliness affects CCE’s diagnostic yield, a few operator-dependent scores (Leighton–Rex and CC-CLEAR scores) and a computer-dependent score (CAC score) have been developed to grade bowel cleanliness objectively. CCE can be used to monitor IBD mucosal disease activity through the UCEIS and the panenteric CECDAIic score for UC and CD, respectively. CCE may also have a role in CRC screening, given similar sensitivity and specificity rates to conventional colonoscopy to detect colonic polyps ≥ 10 mm and CRC. Given CCE’s diagnostic yield and reproducible clinical scores with high inter-observer agreements, CCE is fast becoming a suitable alternative to conventional colonoscopy in specific patient populations.

Objective: The aim of this research was to assess the quality and inter- and intra-observer agreement of tracings obtained by three different techniques for uterine contraction monitoring: the external tocodynamometer (TOCO), the intrauterine pressure catheter (IUPC) and a recently introduced method based on electrohysterography (EHG). Study design: We included 150 uterine activity registrations from a previous prospective observational study (W3 study), conducted at Máxima Medical Centre in Veldhoven, the Netherlands. Term singleton pregnant women were simultaneously monitored with TOCO, IUPC and EHG during labor. Six clinicians, blinded to the source (TOCO, IUPC, or EHG) and subject, evaluated all tracings that were subsequently presented in random order. They annotated contractions and assigned each tracing a score for interpretability of 2 (good), 1 (moderate) or 0 (poor). To evaluate inter-observer agreement, we calculated kappa values for the qualitative assessment, and intraclass correlation coefficients (ICC) for the number of contractions annotated by clinicians. Four clinicians repeated this procedure to evaluate intra-observer agreement. Results: IUPC tracings received the highest quality rating, with a mean score of 1.95, followed by a mean score of 1.60 for EHG and 0.80 for TOCO (p < 0.05). Mean weighted kappa values were 0.63 for TOCO and 0.45 for EHG. The average number of contractions that was picked up by clinicians was 59.8 for the intrauterine pressure catheter, 49.8 for EHG and 26.4 for TOCO. The ICC of the intrauterine pressure catheter was significantly higher than the external methods, regarding both inter- and intra-observer agreement (0.98 and 0.99 respectively). Conclusion: IUPC recordings scored best regarding quality, inter- and intra-observer agreement. However, due to safety issues, in many countries this technique is not used anymore. The quality of TOCO was rated as poor and many contractions were missed as compared to the gold standard. From a clinical interpretational point of view, EHG is favorable to TOCO. EHG recordings were assigned higher quality scores, but with less agreement between clinicians. An explanation could be that EHG is a relatively new technique, while IUPC and the TOCO are being used for decades. Building experience with EHG (training) is therefore recommended.

[EN] Currently, there is a high prevalence of cardiovascular diseases, being the leading cause of mortality worldwide. Thus, a rapid and effective evaluation of the heart is increasingly relevant to diagnose the patient early and achieve appropriate treatment. For this reason, this study aims to compare myocardial deformation parameters such as circumferential strain (segmental, base, mid, apical and global) and global torsion values by the CMR tissue tagging reference technique in SPAMM cine images and the promising feature-tracking technique in b-SSFP cine images, in 31 healthy subjects and 35 patients diagnosed with infarct. The results obtained showed that there is a high agreement in the inter-technique and inter-observer reproducibility for all myocardial parameters evaluated. Therefore, it can be determined the CMR featuretracking technique is highly reproducible for circumferential strain and torsion parameters and could currently be a reliable clinical alternative to assess myocardial function., DM acknowledges financial support from the Conselleria d'Educacio, Investigacio, Cultura i Esport, Generalitat Valenciana (grants AEST/2019/037 and AEST/2020/029), from the Agencia Valenciana de la Innovacion, Generalitat Valenciana (ref. INNCAD00/19/085), and from the Centro para el Desarrollo Tecnologico Industrial (Programa Eurostars-2, Interempresas Internacional), Spanish Ministerio de Ciencia, Innovacion y Universidades (ref. CIIP-20192020).

Background The METastasis Reporting and Data System for Prostate Cancer (MET-RADS-P) guidelines are designed to enable reproducible assessment in detecting and quantifying metastatic disease response using whole-body magnetic resonance imaging (WB-MRI) in patients with advanced prostate cancer (APC). The purpose of our study was to evaluate the inter-observer agreement of WB-MRI examination reports produced by readers of different expertise when using the MET-RADS-P guidelines. Methods Fifty consecutive paired WB-MRI examinations, performed from December 2016 to February 2018 on 31 patients, were retrospectively examined to compare reports by a Senior Radiologist (9 years of experience in WB-MRI) and Resident Radiologist (after a 6-months training) using MET-RADS-P guidelines, for detection and for primary/dominant and secondary response assessment categories (RAC) scores assigned to metastatic disease in 14 body regions. Inter-observer agreement regarding RAC score was evaluated for each region by using weighted-Cohen’s Kappa statistics (K). Results The number of metastatic regions reported by the Senior Radiologist (249) and Resident Radiologist (251) was comparable. For the primary/dominant RAC pattern, the agreement between readers was excellent for the metastatic findings in cervical, dorsal, and lumbosacral spine, pelvis, limbs, lungs and other sites (K:0.81–1.0), substantial for thorax, retroperitoneal nodes, other nodes and liver (K:0.61–0.80), moderate for pelvic nodes (K:0.56), fair for primary soft tissue and not assessable for skull due to the absence of findings. For the secondary RAC pattern, agreement between readers was excellent for the metastatic findings in cervical spine (K:0.93) and retroperitoneal nodes (K:0.89), substantial for those in dorsal spine, pelvis, thorax, limbs and pelvic nodes (K:0.61–0.80), and moderate for lumbosacral spine (K:0.44). Conclusions We found inter-observer agreement between two readers of different expertise levels to be excellent in bone, but mixed in other body regions. Considering the importance of bone metastases in patients with APC, our results favor the use of MET-RADS-P in response to the growing clinical need for monitoring of metastasis in these patients.

BACKGROUND: The accuracy of magnetic resonance imaging (MRI) for making the diagnosis of subscapularis tears presents wide variation in the literature and there are few prospective studies. OBJECTIVE: To compare the findings from MRI and arthroscopy for diagnosing subscapularis tears. DESIGN AND SETTING: Diagnostic test study performed in a tertiary care hospital. METHODS: We included patients who underwent arthroscopic rotator cuff repair and who had firstly undergone high magnetic field MRI without contrast. The images were independently evaluated by a shoulder surgeon and two musculoskeletal radiologists. Sensitivity, specificity, positive and negative predictive values, accuracy and inter and intra-observer agreement were calculated. RESULTS: MRIs on 200 shoulders were evaluated. The incidence of subscapularis tears was 69.5% (41.5% partial and 28.0% full-thickness). The inter and intra-observer agreement was moderate for detection of subscapularis tears. The shoulder surgeon presented sensitivity of 51.1% to 59.0% and specificity of 91.7% to 94.4%. The radiologists showed sensitivity of 83.5% to 87.1% and specificity of 41% to 45.9%. Accuracy ranged from 60.5% to 73.0%. CONCLUSION: The 1.5-T MRIs without contrast showed mean sensitivity of 70.2% and mean specificity of 61.9% for detection of subscapularis tears. Sensitivity was higher for the musculoskeletal radiologists, while specificity was higher for the shoulder surgeon. The mean accuracy was 67.6%, i.e. lower than that of rotator cuff tears overall.

Purpose: Contractions in non-pregnant uterine can be assessed by visual inspection of transvaginal ultrasound (TVUS). Many authors have used this method to extract features like contraction frequency and direction. However, visual inspection is a subjective method and the outcome is dependent on the sonographers and video analysts. In this study, we wanted to see which uterine feature is reproducible enough, in terms of inter-observer agreement, to serve as a reliable control for future research. Methods: Six observers assessed 80 TVUS videos, and rated video quality, contraction frequency, direction and timing. One observer assessed operating time. A Fleiss’ kappa (κ) or an intra-class correlation (ICC) was calculated to determine the inter-observer agreement of all features. Results: The inter-observer agreement in frequency was substantial (ICC = 0.68). Conversely, there was just slight to fair agreement in contraction timing and direction and in video quality: ICC = 0.26, κ = 0.17 and κ = 0.16, respectively. Overall, agreement among technical engineers was better than between medical professionals. The level of agreement was correlated with video quality, phase of the menstrual cycle and individual patient (all χ2 with p < 0.00). The time to analyze one video ranged between 6 and 20 min. Conclusions: This study shows that visual inspection of TVUS videos is a fairly reproducible method to assess contraction frequency. However, the operating time is too extensive to implement this method in daily practice. Automated methods could offer a solution for this problem in the future.

Purpose The clinical impact of the SIAPEC/SIE 2014 classification for thyroid cytology has been addressed in few studies that evaluated the malignancy rate and the relative prevalence of each category. No study analyzed its intra-observer and inter-observer reproducibility, so far. Methods We retrospectively collected all “indeterminate” lesions diagnosed before (2011–2014) and after (2015–2018) the application of the SIAPEC/SIE 2014 classification at our Institution. Their relative malignancy risks were calculated based on available histological diagnoses. Cytological and clinical features of TIR3A were compared with the surgical outcome. Finally, a large set of samples was re-evaluated in blind of the original cytological and histological diagnoses by two pathologists, independently. Results The prevalence of “indeterminate” diagnoses increased in years 2015–2018 (302/1482, 21% with 14% of TIR3A and 7% TIR3B categories) compared to years 2011–2014 (261/1680, 16%). Surgery was performed in 27% TIR3A and in 97% TIR3B cases. Malignancy rates were 40% for TIR3B and 17% for TIR3A, but were greatly influenced by the adoption of the WHO 2017 re-classification of encapsulated follicular-patterned lesions (decreasing to 28% and 6%, respectively). No criteria except for tumor size were associated to malignancy in TIR3A category. Intra-observer agreement of the experienced pathologist was 122/141 (86%), whereas inter-observer agreement between the expert and in-training pathologist was 95/141 (67%). Conclusions In this real-life experience, the sub-classification of TIR3A and TIR3B slightly increased the overall prevalence of “indeterminate” diagnoses. Malignancy rates were higher than estimated for both TIR3A and TIR3B categories. Agreement among observers highly depended on pathologist’s training.

Background: Conspicuous concentric ring artifacts in phantom reconstructions triggers retuning SPECT systems. These evaluations are visual, not quantitative. Our study was undertaken to determine the degree to which observers agree about SPECT concentric ring artifacts, and to test whether quantitative texture analysis metrics correspond to significant artifacts.Methods: Test data were acquired as part of quarterly quality assurance using standardized SPECT phantoms containing solid spheres, solid rods and volumes of uniform activity concentration loaded with 99mTc. Forty SPECT studies were identified as having concentric ring artifacts or were acquired to assess whether artifacts were resolved following camera retuning after obtaining an unacceptably non-uniform result. Transaxial reconstructions were reviewed independently by two medical physicists who graded severity of artifacts on a 5-point scale. Counts were tabulated in volumes of interest created in uniform phantom sections, from which were computed 72 radiomics image texture analysis metrics. Radial contrast (RContrast) derived from the radial profile of summed slices transformed into polar coordinates and radial noise-to-signal (RNSR) also calculated. Results: Artifacts were considered sufficiently severe to require camera retuning in 10 rods sections, 17 sphere sections, and 16 uniform sections. In uniform sections, there was “good agreement” for inter-observer and intra-rater assessments (κ = 0.66, Fisher exact p < 0.0001 and κ = 0.61, Fisher exact p = 0.001, respectively). While 3 radiomics image analysis features agreed significantly (p = 0.001) with visual detection of significant artifacts in uniform sections, the parameters most strongly associated with severe artifacts were RContrast > 4.75% and RNSR > 2.7%, for which ROC AUC accuracy = 88%±5%, sensitivity = 83%, specificity = 83%, p < 0.0001. Accuracy was 76%-78% for the 3 radiomics features, with significantly lower specificity (48%-61%, p < 0.05) than RContrast and RNSR. Increasing magnitude of RContrast and RNSR correlated significantly with increasingly severe artifact scores (ρ = 0.71-0.72, p < 0.0001). Conclusion: There was good agreement among physicists as to the presence of circular ring artifacts in uniform sections of SPECT quality assurance scans, with artifacts accurately detected by radial contrast and noise-to-signal ratio measurements.

Background: Colon capsule endoscopy is a promising technique for evaluation of the colon, but its reproducibility is still unknown. Objective: This study assesses intra and inter-observer agreement in evaluations of colon capsule endoscopy videos. Methods: Forty-two complete colon capsule endoscopy investigations were analysed by three experts and two beginners. Intra-observer agreement was assessed in paired readings of two experts and two beginners. Agreement was determined by the intraclass correlation coefficient: poor (0.9). Results: Agreement on ‘indication for a following colonoscopy’ based on the number and size of detected polyps and bowel cleansing quality was poor among all observers. Agreement among experts on the detection of large polyps and number of polyps was moderate, but agreement on bowel cleansing quality was poor. Beginners were in moderate agreement with the experts on polyp detection. Intra-observer agreement in experts was moderate to excellent for the detection of large polyps (≥10 mm), excellent for the number of polyps, and poor to moderate for bowel cleansing quality. Intra-observer agreement in beginners was poor to moderate for all variables. Conclusions: This study shows a poor agreement on ‘indication for a following colonoscopy’, but a high intra and inter-observer agreement for polyp detection among experts, as well as a moderate agreement between beginners and experts. Trial registration: NCT02303756.

• PURPOSE: To evaluate the interobserver agreement among uveitis experts on the diagnosis of the specific uveitic disease. • DESIGN: Interobserver agreement analysis. • METHODS: Five committees, each comprised of 9 individuals and working in parallel, reviewed cases from a preliminary database of 25 uveitic diseases, collected by disease, and voted independently online whether the case was the disease in question or not. The agreement statistic, κ, was calculated for the 36 pairwise comparisons for each disease, and a mean κ was calculated for each disease. After the independent online voting, committee consensus conference calls, using nominal group techniques, reviewed all cases not achieving supermajority agreement (> 75%) on the diagnosis in the online voting to attempt to arrive at a supermajority agreement. • RESULTS: A total of 5766 cases for the 25 diseases were evaluated. The overall mean κ for the entire project was 0.39, with disease-specific variation ranging from 0.23 to 0.79. After the formalized consensus conference calls to address cases that did not achieve supermajority agreement in the online voting, supermajority agreement overall was reached on approximately 99% of cases, with disease-specific variation ranging from 96% to 100%. • CONCLUSIONS: Agreement among uveitis experts on diagnosis is moderate at best but can be improved by discussion among them. These data suggest the need for validated and widely used classification criteria in the field of uveitis.

Sanitary risk inspection, an observation protocol for identifying contamination hazards around water sources, is promoted for managing rural water supply safety. However, it is unclear how far different observers consistently identify contamination hazards and consistently classify water source types using standard typologies. This study aimed to quantify inter-observer agreement in hazard identification and classification of rural water sources. Six observers separately visited 146 domestic water sources in Siaya County, Kenya, in wet and dry seasons. Each observer independently classified the source type and conducted a sanitary risk inspection using a standard protocol. Water source types assigned by an experienced observer were cross-tabulated against those of his colleagues, as were contamination hazards identified, and inter-observer agreement measures calculated. Agreement between hazards observed by the most experienced observer versus his colleagues was significant but low (intra-class correlation = 0.49), with inexperienced observers detecting fewer hazards. Inter-observer agreement in classifying water sources was strong (Cohen’s kappa = 0.84). However, some source types were frequently misclassified, such as sources adapted to cope with water insecurity (e.g. tanks drawing on both piped and rainwater). Observers with limited training and experience thus struggle to consistently identify hazards using existing protocols, suggesting observation protocols require revision and their implementation should be supported by comprehensive training. Findings also indicate that field survey teams struggle to differentiate some water source types based on a standard water source classification, particularly sources adapted to cope with water insecurity. These findings demonstrate uncertainties underpinning international monitoring and analyses of safe water access via household surveys. Electronic supplementary material The online version of this article (10.1007/s12403-019-00339-3) contains supplementary material, which is available to authorized users.

PURPOSE: To evaluate the optimal tracer uptake time, the minimal amount of radioactivity and the inter-observer agreement for 11C-choline positron emission tomography/computed tomography (PET/CT) in patients with primary hyperparathyroidism (pHPT).METHODS: Twenty-one patients with biochemically proven pHPT were retrospectively studied after injection of 6.3 ± 1.2 MBq/kg 11C-choline. PET data of the first nine patients, scanned for up to 60 min, were reconstructed in 10-min frames from 10- to 60-min postinjection (p.i.), mimicking varying 11C-choline uptake times. Parathyroid adenoma to background contrast ratios were calculated and compared, using standardized uptake values (SUVs). Data was reconstructed with varying scan durations (1, 2.5, 5, and 10 min) at 20-30-min p.i. (established optimal uptake time), mimicking less administered radioactivity. To establish the minimal required radioactivity, the SUVs in the shorter scan durations (1, 2.5, and 5 min) were compared to the 10-min scan duration to determine whether increased variability and/or statistical differences were observed. Four observers analyzed the 11C-choline PET/CT in four randomized rounds for all patients.RESULTS: SUVpeak of the adenoma decreased from 30 to 40 p.i. onwards. All adenoma/background contrast ratios did not differ from 20- to 30-min p.i. onwards. The SUVs of adenoma in the scan duration of 1, 2.5, and 5 min all differed significantly from the same SUV in the 10-min scan duration (all p = 0.012). However, the difference in absolute SUV adenoma values was well below 10% and therefore not considered clinically significant. The inter-observer analysis showed that the Fleiss' kappa of the 1-min scan were classified as "moderate," while these values were classified as "good" in the 2.5-, 5-, and 10-min scan duration. Observers scored lower certainty scores in the 1- and 2.5-min scans compared to the 5- and 10-min scan durations.CONCLUSION: The optimal time to start PET/CT scanning in patients with pHPT is 20 min after mean injection of 6.3 MBq/kg 11C-choline, with a recommended scan duration of at least 5 min. Alternatively, the radioactivity dose can be lowered by 50% while keeping a 10-min scan duration without losing the accuracy of 11C-choline PET/CT interpretation.

Brain metastases are seen in 20%-50% of patients with metastatic solid tumors. On the other hand, leptomeningeal disease (LMD) occurs more rarely. The gold standard for the diagnosis of LMD is serial cerebrospinal fluid (CSF) analyses, although in daily practice, the diagnosis of LMD is often made by neuroimaging. Leptomeningeal metastases (LM) have been a relative contra-indication to radiosurgery. It can be noted that focal LMD can be difficult to distinguish from a superficially located/cortical-based brain metastasis which is not a contra-indication for radiosurgery. Hence, justifying the need of a reliable diagnosis method. The goal of this study was to determine the inter-observer reliability of contrast-enhanced magnetic resonance imaging (gdMRI) in the differentiation of focal cortical-based metastases from leptomeningeal spread. This is a retrospective review of a prospectively collected database of patients with brain metastases. A total of 42 cases with superficial lesions were selected for review. Additionally, eight control cases demonstrating deep and/or white-matter based lesions were included in the study. Three neuroradiologists and three radiation oncologists were asked to review each study and score the presence of LM. Inter-observer agreement was calculated using group-derived agreement coefficients (Gwet's AC1 and Gwet's AC2). Pair-wise inter-observer agreement coefficients never reached substantial values for trichotomized outcomes (LMD, non-LMD or indeterminate) but did reach a substantial value in a minority of cases for dichotomised outcomes (LMD or non-LMD). The control subgroup analysis revealed substantial agreement between most pairs for both trichotomized and dichotomised outcomes. We observed low inter-observer agreement amongst specialists for the diagnosis of focal LMD by gdMRI. Neuroimaging should not be relied upon to make treatment decisions, notably to deny patients radiosurgery.

Our aim was to prospectively evaluate inter- and intra-observer agreement between Breast Imaging Reporting and Data System (BI-RADS) classifications and Tsukuba elasticity scores (TSs) of breast lesions. The study included 164 breast lesions (63 malignant, 101 benign). The BI-RADS classification and TS of each breast lesion was assessed by the examiner and twice by three reviewers at an interval of 2 months. Weighted κ values for inter-observer agreement ranged from moderate to substantial for BI-RADS classification (κ = 0.585–0.738) and was substantial for TS (κ = 0.608–0.779). Intra-observer agreement was almost perfect for ultrasound (US) BI-RADS (κ = 0.847–0.872) and TS (κ = 0.879–0.914). Overall, individual reviewers are highly self-consistent (almost perfect intra-observer agreement) with respect to BI-RADS classification and TS, whereas inter-observer agreement was moderate to substantial. Comprehensive training is essential for achieving high agreement and minimizing the impact of subjectivity. Our results indicate that breast US and real-time elastography can achieve high diagnostic performance.

Background Differentiating sepsis from the systemic inflammatory response syndrome (SIRS) in critical care patients is challenging, especially before serious organ damage is evident, and with variable clinical presentations of patients and variable training and experience of attending physicians. Our objective was to describe and quantify physician agreement in diagnosing SIRS or sepsis in critical care patients as a function of available clinical information, infection site, and hospital setting. Methods We conducted a post hoc analysis of previously collected data from a prospective, observational trial (N = 249 subjects) in intensive care units at seven US hospitals, in which physicians at different stages of patient care were asked to make diagnostic calls of either SIRS, sepsis, or indeterminate, based on varying amounts of available clinical information (clinicaltrials.gov identifier: NCT02127502). The overall percent agreement and the free-marginal, inter-observer agreement statistic kappa (κfree) were used to quantify agreement between evaluators (attending physicians, site investigators, external expert panelists). Logistic regression and machine learning techniques were used to search for significant variables that could explain heterogeneity within the indeterminate and SIRS patient subgroups. Results Free-marginal kappa decreased between the initial impression of the attending physician and (1) the initial impression of the site investigator (κfree 0.68), (2) the consensus discharge diagnosis of the site investigators (κfree 0.62), and (3) the consensus diagnosis of the external expert panel (κfree 0.58). In contrast, agreement was greatest between the consensus discharge impression of site investigators and the consensus diagnosis of the external expert panel (κfree 0.79). When stratified by infection site, κfree for agreement between initial and later diagnoses had a mean value + 0.24 (range − 0.29 to + 0.39) for respiratory infections, compared to + 0.70 (range + 0.42 to + 0.88) for abdominal + urinary + other infections. Bioinformatics analysis failed to clearly resolve the indeterminate diagnoses and also failed to explain why 60% of SIRS patients were treated with antibiotics. Conclusions Considerable uncertainty surrounds the differential clinical diagnosis of sepsis vs. SIRS, especially before organ damage has become highly evident, and for patients presenting with respiratory clinical signs. Our findings underscore the need to provide physicians with accurate, timely diagnostic information in evaluating possible sepsis. Electronic supplementary material The online version of this article (10.1186/s40560-019-0368-2) contains supplementary material, which is available to authorized users.

BackgroundGuidelines recommend maintaining cerebral perfusion pressure (CPP) between 60 and 70mmHg in patients with severe traumatic brain injury (TBI), but acknowledge that optimal CPP may vary depending on cerebral blood flow autoregulation. Previous retrospective studies suggest that targeting CPP where the pressure reactivity index (PRx) is optimized (CPPopt) may be associated with improved recovery.MethodsWe performed a retrospective cohort study involving TBI patients who underwent PRx monitoring to assess issues of feasibility relevant to future interventional studies: (1) the proportion of time that CPPopt could be detected; (2) inter-observer variability in CPPopt determination; and (3) agreement between manual and automated CPPopt estimates. CPPopt was determined for consecutive 6-h epochs during the first week following TBI. Sixty PRx-CPP tracings were randomly selected and independently reviewed by six critical care professionals. We also assessed whether greater deviation between actual CPP and CPPopt (CPP) was associated with poor outcomes using multivariable models.ResultsIn 71 patients, CPPopt could be manually determined in 985 of 1173 (84%) epochs. Inter-observer agreement for detectability was moderate (kappa 0.46, 0.23-0.68). In cases where there was consensus that it could be determined, agreement for the specific CPPopt value was excellent (weighted kappa 0.96, 0.91-1.00). Automated CPPopt was within 5mmHg of manually determined CPPopt in 93% of epochs. Lower PRx was predictive of better recovery, but there was no association between CPP and outcome. Percentage time spent below CPPopt increased over time among patients with poor outcomes (p=0.03). This effect was magnified in patients with impaired autoregulation (defined as PRx>0.2; p=0.003).ConclusionProspective interventional clinical trials with regular determination of CPPopt and corresponding adjustment of CPP goals are feasible, but measures to maximize consistency in CPPopt determination are necessary. Although we could not confirm a clear association between CPP and outcome, time spent below CPPopt may be particularly harmful, especially when autoregulation is impaired.

Background In early (T1) oesophageal adenocarcinoma (OAC), the histological profile of an endoscopic resection specimen plays a pivotal role in the prediction of lymph node metastasis and the potential need for oesophagectomy with lymphadenectomy. Objective To evaluate the inter-observer agreement of the histological assessment of submucosal (pT1b) OAC. Methods Surgical and endoscopic resection specimens with pT1b OAC were independently reviewed by three gastrointestinal pathologists. Agreement was determined by intraclass correlation coefficient for continuous variables, and Fleiss' kappa (κ) for categorical variables. Bland–Altman plots of the submucosal invasion depth were made. Results Eighty-five resection specimens with pT1b OAC were evaluated. The agreement was good for differentiation grade (κ=0.77, 95% confidence interval (CI) 0.68–0.87), excellent for lymphovascular invasion (κ=0.88, 95% CI 0.76–1.00) and moderate for submucosal invasion depth using the Paris and Pragmatic classifications (κ=0.60, 95% CI 0.49–0.72 and κ=0.42, 95% CI 0.33–0.51, respectively). Systematic mean differences between pathologists were detected for the measurement of submucosal invasion depth, ranging from 297 µm to 602 µm. Conclusions A substantial discordance was found between pathologists for the measurement of submucosal invasion depth in pT1b OAC. Differences may lead to an over- or underestimation of the lymph node metastasis risk, with grave implications for the treatment strategy. Review by a second gastrointestinal pathologist is recommended to improve differentiating between a favourable and an unfavourable histological profile.

BACKGROUND: Several studies have demonstrated a prognostic role for stromal tumour infiltrating lymphocytes (sTILs) in triple-negative breast cancer (TNBC). The reproducibility of scoring sTILs is variable with potentially excellent concordance being achievable using a software tool. We examined agreement between breast pathologists across Europe scoring sTILs on H&E-stained sections without software, an approach that is easily applied in clinical practice. The association between sTILs and response to anthracycline-taxane NACT was also examined. METHODOLOGY: Pathologists from the European Working Group for Breast Screening Pathology scored sTILs in 84 slides from 75 TNBCs using the immune-oncology biomarker working group guidance in two circulations. There were 16 participants in the first and 19 in the second circulation. RESULTS: Moderate agreement was achieved for absolute sTILs scores (intraclass correlation coefficient (ICC) = 0.683, 95% CI 0.601-0.767, p-value

Purpose: Evaluation of the performance of a computer-aided diagnosis (CAD) system based on the quantified color distribution in strain elastography imaging to evaluate the malignancy of breast tumors. Methods: The database consisted of 31 malignant and 52 benign lesions. A radiologist who was blinded to the diagnosis performed the visual analysis of the lesions. After six months with no eye contact on the breast images, the same radiologist and other two radiologists manually drew the contour of the lesions in B-mode ultrasound, which was masked in the elastography image. In order to measure the amount of hard tissue in a lesion, we developed a CAD system able to identify the amount of hard tissue, represented by red color, and quantify its predominance in a lesion, allowing classification as soft, intermediate, or hard. The data obtained with the CAD system were compared with the visual analysis. We calculated the sensitivity, specificity, and area under the curve (AUC) for the classification using the CAD system from the manual delineation of the contour by each radiologist. Results: The performance of the CAD system for the most experienced radiologist achieved sensitivity of 70.97%, specificity of 88.46%, and AUC of 0.853. The system presented better performance compared with his visual diagnosis, whose sensitivity, specificity, and AUC were 61.29%, 88.46%, and 0.829, respectively. The system obtained sensitivity, specificity, and AUC of 67.70%, 84.60%, and 0.783, respectively, for images segmented by Radiologist 2, and 51.60%, 92.30%, and 0.771, respectively, for those segmented by the Resident. The intra-class correlation coefficient was 0.748. The inter-observer agreement of the CAD system with the different contours was good in all comparisons. Conclusions: The proposed CAD system can improve the radiologist performance for classifying breast masses, with excellent inter-observer agreement. It could be a promising tool for clinical use.

Introduction Trauma constitutes a major public health problem. Ninty percent of world's fatalities on road occur in low and middle-income countries. Focused assessment with sonography in trauma (FAST) has a key role in trauma investigation, altering subsequent management in a significant number of patients. There is a rising trend of introducing FAST examination to non-radiologists and junior members of healthcare team to hasten triage of trauma patients. Objective To determine interobserver agreement on focused assessment with sonography for trauma in blunt abdominal injury between senior and junior residents. Methods This cross-sectional study was conducted at Aga Khan University Hospital. Three hundred patients with blunt abdominal trauma meeting inclusion criteria were enrolled. FAST was performed using standard curvilinear 3.5-5 MHz transducer. Agreement between junior and senior residents was measured and Kappa statistic was calculated. Results Mean age of the patients included in the study was 30.04 ± 18.09 years. Among these 237 (79%) were male and 63 (21%) were female. Sixteen (5.3%) were positive for intraperitoneal free fluid while the remaining 284 (94.7%) were negative. A total of 19 FAST examinations were reported positive by junior residents, of which 15 (78.9%) were confirmed by a senior resident to be correct, while four (21.05%) were falsely labeled positive by a junior resident. A total of 281 negative FAST examinations were reported by junior residents, of which 280 (99.6%) were confirmed by a senior resident, while only one (0.003%) was falsely labeled negative. Kappa statistic was calculated for inter-observer agreement on FAST examination findings, which showed a kappa value of 0.84 (very good agreement), with a p-value of

The reproducibility of the AO classification for distal radius fractures remains a topic of debate. Previous studies showed variable reproducibility results. Important treatment decisions depend on correct classification, especially in comminuted, intra-articular fractures. Therefore, reliable reproducibility results need to be undisputedly determined. Hence, the study objective was to assess inter- and intra-observer agreement of the AO classification for operatively treated distal radius fractures. A database of 54 radiographs of all AO types (A, B and C) and groups (A2-3, B1-3, and C1-3) of distal radius fractures was assessed in twofold. Likewise, a subset of 152 radiographs of solely C-type groups (C1-3) was assessed. All fractures were classified by six observers with different experience levels: three consultant trauma surgeons, one sixth-year trauma surgery resident, a consultant trauma radiologist, and an intern with limited experienced. The inter-observer agreement of both main types and groups was moderate (κ = 0.49 resp. κ = 0.48) in combination with a good intra-observer agreement (κ = 0.68 resp. κ = 0.70). The inter-observer agreement of the subset C-type fractures group was fair (κ = 0.27) with moderate intra-observer agreement (κ = 0.43). According to these results, the reproducibility of the AO classification of main types and groups of distal radius fractures based on conventional radiographs is insufficient (κ

Background Endoscopic diagnosis of atrophic gastritis can contribute to risk stratification and thereby tailored screening for gastric cancer. We aimed to evaluate the effect of training on inter-observer agreement in diagnosis and grading of endoscopic atrophic gastritis (EAG) according to the level of endoscopists' experience. Methods Twelve endoscopists (six less-experienced and six experienced) participated in this prospective study. The training session consisted of 1) four interventions with two-week intervals, and 2) a follow-up period (two follow-up assessments without feedback). EAG was categorized as C1 to O3 according to the Kimura-Takemoto classification. Kappa statistics were used to calculate inter-observer agreement. Results At baseline, kappa indexes were 0.18 in the less-experienced group and 0.32 in the experienced group, respectively. After four interventions with feedback, the kappa index improved in both groups and was sustained during the follow-up period. Overall diagnostic yields of EAG were 43.1% ± 10.7% in pre-intervention and 46.8% ± 5.9% in post-intervention. Variability in the rate of diagnosis of EAG significantly decreased in the less-experienced group (r = 0.04, P = 0.003). Conclusion Irrespective of experience level, inter-observer agreement for diagnosis and grading of EAG improved after training and remained stable after intervention., Graphical Abstract

Objectives: To measure changes in the visual interpretation of the EEG by the human expert for neonatal seizure detection when reducing the number of recording electrodes. Methods: EEGs were recorded from 45 infants admitted to the neonatal intensive care unit (NICU). Three experts annotated seizures in EEG montages derived from 19, 8 and 4 electrodes. Differences between annotations were assessed by comparing intra-montage with inter-montage agreement (K). Results: Three experts annotated 4464 seizures across all infants and montages. The inter-expert agreement was not significantly altered by the number of electrodes in the montage (p = 0.685, n = 43). Reducing the number of EEG electrodes altered the seizure annotation for all experts. Agreement between the 19-electrode montage (K-19,K-19 = 0.832) was significantly higher than the agreement between 19 and 8-electrode montages (dK = 0.114; p

Background The purpose of this study is to determine whether a qualitative approach or a semi-quantitative approach provides the most robust method for early response evaluation with 2′-deoxy-2′-[18F]fluoro-d-glucose (F-18-FDG) positron emission tomography combined with whole body computed tomography (PET/CT) in non-small cell lung cancer (NSCLC). In this study eight Nuclear Medicine consultants analyzed F-18-FDG PET/CT scans from 35 patients with locally advanced NSCLC. Scans were performed at baseline and after 2 cycles of chemotherapy. Each observer used two different methods for evaluation: (1) PET response criteria in solid tumors (PERCIST) 1.0 and (2) a qualitative approach. Both methods allocate patients into one of four response categories (complete and partial metabolic response (CMR and PMR) and stable and progressive metabolic disease (SMD and PMD)). The inter-observer agreement was evaluated using Fleiss’ kappa for multiple raters, Cohens kappa for comparison of the two methods, and intraclass correlation coefficients (ICC) for comparison of lean body mass corrected standardized uptake value (SUL) peak measurements. Results The agreement between observers when determining the percentage change in SULpeak was “almost perfect”, with ICC = 0.959. There was a strong agreement among observers allocating patients to the different response categories with a Fleiss kappa of 0.76 (0.71–0.81). In 22 of the 35 patients, complete agreement was observed with PERCIST 1.0. The agreement was lower when using the qualitative method, moderate, having a Fleiss kappa of 0.60 (0.55–0.64). Complete agreement was achieved in only 10 of the 35 patients. The difference between the two methods was statistically significant (p

Introduction Differences between radiologists and orthopaedic surgeons in the interpretation of MR images of the shoulder joint are experienced in daily clinical practice. This study set out to evaluate the inter-observer agreement between radiologists and orthopaedic surgeons in assessing pathology on MR imaging of the shoulder joint. Also, we determined the accuracy of the observers with arthroscopy as the standard of reference. Materials and methods Two radiologists and one orthopaedic surgeon reviewed 50 MR studies—25 conventional MR examinations and 25 MR arthrographies—of patients with shoulder complaints who had undergone MR imaging and subsequently arthroscopic surgery. The assessment was independent and blinded. All observers evaluated the MR examinations twice. Standard evaluation forms were used to score for pathology of rotator cuff, glenoid labrum, tendon of the long head of the biceps brachii and glenohumeral ligaments. The presence or absence of osteoarthritis, SLAP lesions, Bankart lesions, Hill-Sachs lesions or impingement was also noted. Intra- and inter-observer agreement, the sensitivity and specificity were calculated. Differences in percentages of correctly diagnosed lesions were tested for significance using McNemar’s test. Results There was a poor inter-observer agreement between the orthopaedic surgeon and the radiologists in assessing Bankart lesions and ligamentous lesions. We found significant differences between the radiologists and the orthopaedic surgeon in the assessment of osteoarthritis, Hill-Sachs lesions and impingement. Conclusion The orthopaedic surgeon and radiologists differed in their interpretation of what defines a Bankart lesion and what defines a ligamentous lesion. The orthopaedic surgeon was significantly more accurate in assessing impingement.

Background and purpose The use of Stereotactic Body Radiotherapy (SBRT) for bone metastases is increasing rapidly. Therefore, knowledge of the inter-observer differences in tumor volume delineation is essential to guarantee precise dose delivery. The aim of this study is to compare inter-observer agreement in bone metastases delineated on different imaging modalities. Material and methods Twenty consecutive patients with bone metastases treated with SBRT were selected. All patients received CT and MR imaging in treatment position prior to SBRT. Five observers from three institutions independently delineated gross tumor volume (GTV) on CT alone, CT with co-registered MRI and MRI alone. Four contours per imaging modality per patient were available, as one set of contours was shared by 2 observers. Inter-observer agreement, expressed in generalized conformity index [CIgen], volumes of contours and contours center of mass (COM) were calculated per patient and imaging modality. Results Mean GTV delineated on MR (45.9 ± 52.0 cm 3 ) was significantly larger compared to CT–MR (40.2 ± 49.4 cm 3 ) and CT (34.8 ± 41.8 cm 3 ). A considerable variation in CIgen was found on CT (mean 0.46, range 0.15–0.75) and CT–MRI (mean 0.54, range 0.17–0.71). The highest agreement was found on MRI (mean 0.56, range 0.20–0.77). The largest variations of COM were found in anterior–posterior direction for all imaging modalities. Conclusions Large inter-observer variation in GTV delineation exists for CT, CT–MRI and MRI. MRI-based GTV delineation resulted in larger volumes and highest consistency between observers.

Objectives This study aims to evaluate the reproducibility of femoropopliteal TASC II classification and to analyse the influence of an educational intervention on inter-observer agreement. Design This is a validation study. Materials This study included 200 consecutive angiograms of femoropopliteal arterial lesions. Methods Seven investigators evaluated the first 100 angiograms, independently aided by the available TASC guide. Thereafter, the intervention included a discussion of the 25 most problematic cases, initially by a panel of 22 vascular surgeons, and later by the seven investigators to clarify grading principles. In the second stage, the 100 remaining cases were evaluated independently. A multi-rater variation of Brennan and Prediger's free-marginal kappa (κfree) was used to calculate inter-observer agreement. Results There were lesions not fitting any of the TASC classes. Total agreement among all seven investigators was reached in 7% and 19% of the cases before and after the intervention, respectively. In the first stage, κfree was 0.32 between all observers (range between two observers κfree = 0.11–0.54). The intervention increased the agreement to κfree = 0.49 (range: 0.20–0.56). Agreement between the two observers was 38–69% (mean 49%) before the intervention and 51–73% (mean 61%) thereafter. Conclusions TASC II classification for femoropopliteal lesions allows individual interpretations, and the common use of this classification as a basis for decision making and reporting outcomes could therefore be questioned.

While surgery remains the cornerstone of rectal cancer treatment, organ-preservation is upcoming. Therefore, neo-adjuvant treatment should be optimized. By escalating doses, response can be increased. To limit toxicity of boost, accurate gross tumor volume (GTV) definition is required. MRI, especially undeformed fast spin echo diffusion-weighted MRI (DWI), looks promising for delineation. However, inconsistencies between observers should be quantified before clinical implementation. We aim to find which MRI sequence (T2w, DWI or combination) is optimal and clinically useful for GTV definition by evaluating inter-observer agreement.Locally advanced rectal cancer patients (tumors10 cm from anal verge) were scanned on 3T MRI transverse T2w and DWI (b=800 s/mm(2)). Three independent observers delineated T2w, DWI and combination (Combi) after training-set. Volumes, conformity index (CI), and maximum Hausdorff distance (HD) were calculated between any observer-pair per patient per modality.Twenty-four consecutive patients were included. One patient had cT2 (4.2%), 19 cT3 (79.1%) and 4 cT4 (16.7%), with 2 clinical node negative (8.3%), 4 cN1 (16.7%), and 18 cN2 (75.0%) on MRI. From 24 patients, 70 sequences were available (24x T2, 23x DWI, and 23x Combi). Between observers, no significant volume differences were observed per modality. T2 showed significantly largest volumes compared to DWI (mean difference 19.85 ml, SD 17.42, p0.0001) and Combi (mean difference 7.16 ml, SD 11.58, p0.0001). Mean CI was 0.70, 0.71 and 0.69 for T2, DWI and Combi respectively (p0.61). Average HD was largest on T2 (18.60mm, max 31.40 mm, min 9.20mm).Delineation on DWI resulted in delineation of the smallest volumes with similar consistency and mean distances, but with slightly lower Hausdorff distances compared to T2 and Combi. However, with lack of a gold standard it remains difficult to establish if delineations also represent true tumor. Study strengths were DWI adaptation to exclude geometrical distortions and training-set. DWI shows great potential for delineation purposes as long as sufficient experience exists and geometrical distortions are eliminated.

Objectives:To determine whether a novel method and device, called a variable attenuation plate (VAP), which equalizes chest radiographic appearance and allows for synchronization of manual image windowing with comparison studies, would improve consistency in interpretation.Materials and Methods:Research ethics board approved the prospective cohort pilot study, which included 50 patients in the intensive care unit (ICU) undergoing two serial chest radiographs with a VAP placed on each one of them. The VAP allowed for equalization of density and contrast between the patients’ serial chest radiographs. Three radiologists interpreted all the studies with and without the use of VAP. Kappa and percent agreement was used to calculate agreement between radiologists’ interpretations with and without the plate.Results:Radiologist agreement was substantially higher with the VAP method, as compared to that with the non-VAP method. Kappa values between Radiologists A and B, A and C, and B and C were 46%, 55%, and 51%, respectively, which improved to 73%, 81%, and 66%, respectively, with the use of VAP. Discrepant report impressions (i.e., one radiologist's impression of unchanged versus one or both of the other radiologists stating improved or worsened in their impression) ranged from 24 to 28.6% without the use of VAP and from 10 to 16% with the use of VAP (χ2= 7.454,P< 0.01). Opposing views (i.e., one radiologist's impression of improved and one of the others stating disease progression or vice versa) were reported in 7 (12%) cases in the non-VAP group and 4 (7%) cases in the VAP group (χ2= 0.85,P= 0.54).Conclusion:Numerous factors play a role in image acquisition and image quality, which can contribute to poor consistency and reliability of portable chest radiographic interpretations. Radiologists’ agreement of image interpretation can be improved by use of a novel method consisting of a VAP and associated software and has the potential to improve patient care.

ObjetivosValorar fiabilidad en términos de concordancia interobservador de las tomas de presión arterial (PA). Varios profesionales sanitarios y sistemas de medición. Influencia de la experiencia del observador.DiseñoEstudio observacional, descriptivo transversal.EmplazamientoCentro de salud urbano, Córdoba.ParticipantesUn total de 131 pacientes hipertensos, aleatorizados, pertenecientes a una unidad de atención funcional; se excluyó a 11.MedicionesPara disminuir variabilidad: curso sobre la toma correcta de la PA, otoscopia y comprobación de agudeza visual de los observadores, calibración y validación de aparatos de medida, tiempo limitado y cegamiento de medidas. Se realizaron 4 mediciones de PA por paciente, tres con esfigmomanómetro de mercurio (EMM) (dos simultáneas, una individual) y una con aparato automático. Se recogieron variables descriptivas, clínicas y somatométricas. La concordancia interobservador se evaluó mediante el coeficiente de correlación intraclase (CCI), método de la media de las diferencias (MMD) e índice de concordancia simple (ICS). Un CCI > de 0,75 se consideró aceptable. Una diferencia > 5 mmHg se consideró relevante clínicamente (MMD e ICS).Resultados principalesAcuerdo aceptable para el MMD: sólo, PAS y PAD de OBS-1/OBS-2, biauricular, (–6,1/+ 8,9 mmHg) y (–6,8/+ 5,8 mmHg). Resultados más desfavorables: para TAS y TAD: OBS-1/AUTO –20,9/25,0 y –16,4/15,1; OBS-2/AUTO –22,8/24,4 y –16,6/15,2. Resto de intervalos siempre > 10 mmHg; CCI > 0,75 en todas las comparaciones, excepto PAD OBS-1/AUTO y PAD OBS-2/AUTO (0,69 para ambos). Un 41% de comparaciones > 5 mmHg. Sin diferencias en profesional menos experto.ConclusionesInexactitud del método de medida de PA de referencia (EMM) para MMD e ICS. Conclusiones contradictorias según método de medida. Diferencias no aceptables clínicamente.ObjectivesTo assess reliability in terms of inter-observer agreement of blood pressure (BP) readings. Various health professionals and measuring systems. Influence of observer's experience.DesignObservational, descriptive, cross-sectional study.SettingUrban health centre, Córdoba.Participants131 hypertense, randomised patients, belonging to a functional care unit. 11 were excluded.MeasurementsTo reduce variability: course on the right way to take blood pressure, otoscope and verification of visual sharpness of observers, calibration and validation of measuring devices, limited time and blinding of measurements. 4 BP measurements per patient: 3 with mercury sphygmomanometer (2 simultaneously, one individual) and one with an automatic device. Descriptive, clinical and somatometric variables were gathered. Inter-observer agreement was evaluated through the intraclass correlation coefficient (ICC), the mean of differences method (MDM) and the simple concordance index (CI). An ICC > 0.75 was thought acceptable. A difference > 5 mmHg was thought clinically relevant (MDM and CI).Main resultsAcceptable consistency for MDM: alone, systolic and diastolic pressure of OBS 1/ OBS 2, bi-auricular, –6.1/+8.9 mmHg and –6.8/+5.8 mmHg. Less favourable results: for systolic and diastolic pressure: OBS 1/AUTO –20.9/25.0 and –16.4/15.1; OBS 2/AUTO –22.8/24.4 and –16.6/15.2. Remaining intervals always > 10 mmHg; CI > 0.75 in all comparisons except diastolic pressure OBS 1/AUTO and diastolic pressure OBS 2/AUTO (0.69 in both cases). 41% of comparisons were > 5 mmHg. No differences in less expert professionals were found.ConclusionsInaccuracy of the standard BP measurement method (mercury sphygmomanometer) for MDM and CI. Contradictory conclusions according to method of measurement. Differences not clinically acceptable.

Organic pig farming aims at maintaining a high health and welfare state of the animals through appropriate housing, management and feeding. Better knowledge of health and welfare indicators should help to identify critical points and hence to improve health and welfare as well as performance of organic pigs. This paper describes the health and welfare of organic pigs from 101 farms across six EU countries, using selected animal-based parameters from the Welfare Quality® protocol. Parameters were collected in sows, suckling and weaned piglets in 3 to 20 farms per country. Their assessment was trained before farm visits and inter-observer agreement determined after farm visits. The most prevalent problems identified in sows were thinness (median farm prevalence 18.8 %, range 0–81.0), injuries on the anterior part of the body (15.5 %, 0–66.7), injuries on hind part of body (7.9 %, 0–50), obesity (4.9 %, 0–50.0) and vulva lesions (3.5 %, 0–42.9). In suckling piglets, the median prevalence in terms of groups affected per farm was 0 % for all parameters but ‘> 50 % dirty piglets in group’, for which it was 10 %. Farm prevalence ranged from 0 to 100 % for ‘≥ 1 lame piglet in group’, presence of diarrhoea, and ‘> 50 % dirty piglets in group’. In weaned piglets, the median prevalence in terms of groups affected per farm was 0 % with a range of 0 to 100 % for all parameters. Based on the collected data, body condition, skin and vulva lesions in sows, lameness, diarrhoea and respiratory problems in piglets could be used as management and welfare indicators, with good potential for enhancement through farm improvement schemes like herd health planning. However, some definitions could be improved, especially lameness, diarrhoea and respiratory problems in piglets.

Background Parents and their parenting practices play an important role in shaping their children’s environment and energy-balance related behaviours (EBRBs). Measurement of parenting practices can be parent- or child-informed, however not much is known about agreement between parent and child perspectives. This study aimed to assess agreement between parent and child reports on parental practices regarding EBRBs across different countries in Europe and to identify correlates of agreement. Methods Within the ENERGY-project, a cross-sectional survey was conducted among 10–12 year old children and their parents in eight European countries. Both children and parents filled in a questionnaire on 14 parental practices regarding five different EBRBs (i.e. soft drink, fruit juice and breakfast consumption, sports activity and watching TV) and socio-demographic characteristics. Children’s anthropometric measurements were taken at school. We calculated percentages of agreement between children and their parents and weighted kappa statistics (for ordinal variables) per practice and country and assessed factors associated with agreement using multilevel linear regression. Results Reports of 6425 children and their parents were available for analysis. Overall mean agreement between parent and child reports was 43% and varied little among countries. The lowest agreement was found for questions assessing joint parent–child activities, such as sports (27%; Kappa (κ) = 0.14) or watching TV (30%;κ = 0.17), and for parental allowance of the child to have soft drinks (32%;κ = 0.24) or fruit juices (32%;κ = 0.19), or to watch TV (27%;κ = 0.17). Having breakfast products available at home or having a TV in the child’s bedroom were the only practices with moderate to good agreement (>60%;κ = 0.06 and 0.77, respectively). In general, agreement was lower for boys, younger children, younger parents, parents with less than 14 years of education, single parents, parents with a higher self-reported body mass index and parents who perceived their child to be underweight. Conclusions Parents and children perceive parental practices regarding dietary, physical activity and sedentary behaviours differently in all parts of Europe, with considerable variation across specific practices and countries. Therefore, future studies should assess both, parents and children’s view on parental practices. Electronic supplementary material The online version of this article (doi:10.1186/1471-2458-14-918) contains supplementary material, which is available to authorized users.

Background Accurate prediction of Helicobacter pylori infection status on endoscopic images can contribute to early detection of gastric cancer, especially in Asia. We identified the diagnostic yield of endoscopy for H. pylori infection at various endoscopist career levels and the effect of two years of training on diagnostic yield. Methods A total of 77 consecutive patients who underwent endoscopy were analyzed. H. pylori infection status was determined by histology, serology, and the urea breast test and categorized as H. pylori-uninfected, -infected, or -eradicated. Distinctive endoscopic findings were judged by six physicians at different career levels: beginner (5000). Diagnostic yield and inter- and intra-observer agreement on H. pylori infection status were evaluated. Values were compared between the two beginners after two years of training. The kappa (K) statistic was used to calculate agreement. Results For all physicians, the diagnostic yield was 88.9% for H. pylori-uninfected, 62.1% for H. pylori-infected, and 55.8% for H. pylori-eradicated. Intra-observer agreement for H. pylori infection status was good (K > 0.6) for all physicians, while inter-observer agreement was lower (K = 0.46) for beginners than for intermediate and advanced (K > 0.6). For all physicians, good inter-observer agreement in endoscopic findings was seen for atrophic change (K = 0.69), regular arrangement of collecting venules (K = 0.63), and hemorrhage (K = 0.62). For beginners, the diagnostic yield of H. pylori-infected/eradicated status and inter-observer agreement of endoscopic findings were improved after two years of training. Conclusions The diagnostic yield of endoscopic diagnosis was high for H. pylori-uninfected cases, but was low for H. pylori-eradicated cases. In beginners, daily training on endoscopic findings improved the low diagnostic yield.

Chronic endometritis (CE) is an unusual endometrial inflammation characterized by stromal plasmacyte infiltration. CE is easily missed due to its subtle symptoms and demanding histopathological examinations. Although the immunohistochemistry for the plasmacyte marker CD138 has facilitated the detection of endometrial stromal plasmacytes, the accuracy and biases of this method for CE diagnosis remain poorly understood. The aim of this study was to investigate the inter- and intra-observer variability in the immunohistochemical detection of stromal plasmacytes in the human endometrium. A total of 80 CE and 20 non-pathological archival hematoxylin-stained endometrial preparations with or without immunostaining for CD138 were evaluated independently by two experienced observers and two inexperienced observers. Endometrial stromal plasmacytes in unit areas were counted in the hematoxylin-stained and CD138-immunostained preparations. Each preparation was subdivided into 11 categories by every five plasmacyte counts. The second evaluation was performed four weeks after the first evaluation. The immunohistochemical detection method was superior to conventional histopathological evaluation in both the inter- and intra-observer agreement, irrespective of the experience level of the observers. The linear weighted κ coefficient for intra-observer agreement was higher in the experienced observers than in the inexperienced observers. The inter-observer agreement among the four observers by the immunohistochemical detection method was similarly good between the first and second evaluation. There was no significant inter- or intra-observer variability in the paired comparison of the individual samples. These findings validate the use of immunohistochemistry for CD138 as an accurate and less biased diagnostic tool for CE.

Humana Press Inc., In this series, laryngeal preneoplastic lesions were evaluated by the classifications of the World Health Organization (WHOC), Ljubljana (LC) and squamous intraepithelial neoplasia (SINC) by multiple observers. The inter-observer agreement (IA) by WHOC for laryngeal lesions had been previously evaluated, but to the best of our knowledge, there are no data for LC and SINC. H&E stained slides from 42 laryngeal biopsies were evaluated by fourteen participants according to WHOC and LC, and SINC was additionally applied by 6. The results were analyzed statistically. The diagnoses which were favored by most participants for each case, according to WHOC, were as follows: squamous cell hyperplasia (n = 5; 12%), mild dysplasia (n = 11; 26.2%), moderate dysplasia (n = 12; 28.6%), severe dysplasia (n = 7; 16.7%), carcinoma in situ (n = 5; 12%), and invasive squamous cell carcinoma (n = 2; 4.8%). There was a significant difference between the participants for all three classifications; some participants gave lower or higher scores than the others. The mean correlation coefficients (MCC) of the participants were higher for WHOC compared to LC (0.55 ± 0.15 and 0.48 ± 0.14, respectively). The mean linear-weighted kappa (wKappa) values of participants were not significantly different (0.42 ± 0.10, 0.41 ± 0.12 and 0.37 ± 0.07 for WHOC, LC and SINC, respectively). The kappa values in this series are in agreement with those in previous literature for WHOC, and the similar results obtained for LC and SINC are novel findings. Although the MCC of WHOC was higher, as the wkappa was not significantly different, the findings in this series are not in favor of any of the classifications for better IA for pre-neoplastic laryngeal lesions. © 2010 Humana., Sarioglu, S.; Department of Pathology, Dokuz Eylul University Faculty of Medicine, 35340 Inciralti, Izmir, Turkey; email: sulen.sarioglu@deu.edu.tr

STUDY QUESTION: Does the use of diagnostic criteria in the hysteroscopic diagnosis of a septate uterus improve inter-observer agreement? SUMMARY ANSWER: Pre-set diagnostic criteria slightly improve the inter-observer reproducibility of hysteroscopy in diagnosing a uterine septum, although agreement remains moderate. WHAT IS KNOWN ALREADY: The inter-observer agreement on the hysteroscopic diagnosis of the septate uterus has been reported to be poor. STUDY DESIGN, SIZE, DURATION: From April 2013 until May 2014, a randomized controlled comparative inter-observer study was performed. A total of 191 gynecologists from 43 countries took part. PARTICIPANTS/MATERIALS, SETTINGS, METHODS: Each gynecologist was asked to assess 10 video recordings of hysteroscopy procedures with a specific focus on the internal uterine shape. The hysteroscopies had been performed in subfertile women and women with recurrent miscarriage. The recordings contained images of uterine cavities primarily diagnosed as septate, arcuate or normal. Participating gynecologists were randomized into two groups: one group received diagnostic criteria for a septate uterus before assessment of the videos (DC group), whereas the other group assessed the recordings without instruction (no DC group). The inter-observer agreement, expressed as the intra-class correlation coefficient (ICC), was compared between groups. Main outcomes were the inter-observer agreement on the uterine shape and the necessity of surgical correction. MAIN RESULTS AND THE ROLE OF CHANCE: Eighty-six observers were randomized to the DC group and 105 to the no DC group. The ICCs in the diagnosis of a septum were 0.59 versus 0.52, in the DC group and the non-DC group, respectively (P-value: 0.002). The overall agreement on the need for surgical correction was found to be moderate (DC ICC 0.43 versus no DC 0.39, P-value: 0.70). Most importantly, once a septate uterus had been diagnosed, the agreement on the need for surgery was poor in both groups (DC ICC 0.05 versus no DC ICC 0.02, P-value: 0.78). LIMITATIONS, REASONS FOR CAUTION: We used video recordings rather than studying real-time hysteroscopic procedures, which may have influenced the accuracy of the assessments. WIDER IMPLICATIONS OF THE FINDINGS: The reproducibility of hysteroscopy for the diagnosis of a septate uterus is moderate, even with the use of standardized criteria. The fact that the agreement among physicians on both the diagnosis of a uterine septum, as well as the decision to resect such septum after hysteroscopy is moderate, may imply that hysteroscopy is insufficient as single tool to diagnose and decide on treatment of a septate uterus. STUDY FUNDING/COMPETING INTERESTS: No study funding was received and no competing interests are present.

Background: Recent developments in diagnostic techniques for malaria, particularly DNA probes and seroimmunology, have raised questions as to how these techniques might be used to facilitate malaria diagnosis at the most peripheral levels of the primary health care system. At present, malaria diagnosis is based on the standard microscopic examination of blood films in most field epidemiologic studies and is likely to remain so in the immediate future in Africa. The objective of this study was to assess inter-observer agreement for the examination of Giemsa-stained slides for Plasmodium falciparum parasites. Methods: Children aged 0 to 10 years were enrolled yearly in Bancoumana village (West Africa), mainly during the transmission season (June to October). The blood smears obtained from the persistently negative children in June 1996, August 1996, October 1996 and March 1997 were systematically re-examined. A stratified random sample (10%) proportional to the following parasite density classes 1–100, 101–5000, and 5001 and over was taken from the slides collected. The kappa statistics and the intra-class correlation were used as measures of agreement the first and the second slide examinations. Results: The weighted kappa statistic, widely used as a chance-corrected measure for nominal agreement, showed excellent inter-observer agreement (κw=0.7926; 95% CI [0.7588, 0.8263]; p=0.01). The intra-class correlation co-efficient had the same value of 0.7926 confirming the appropriateness of the weighted kappa statistic. Interobserver agreement for slides read as negative by one observer, or as containing more than 100 parasites per μl, was excellent: 97% (493/506) and 92% (145/158), respectively. In contrast, the inter-observer agreement for slides read by one observer as containing 1–100 parasites/μl was poor, 36% (96/268). Conclusions: In field conditions in Mali, there was a high reproducibility for slides reported as negative or as having more than 100 parasites per μl. However, smears with readings of 1–100 parasites per μl were less reproducible and should be re-examined carefully.