Search

Peer reviewed: True, Funder: National Institute for Health Research (NIHR) Research for Patient Benefit (RfPB); Grant(s): Grant Reference Number: PB-PG-0815-20048, OBJECTIVE: A qualitative sub-study was carried out within a larger phase II feasibility trial, to identify and describe the burden experienced by advanced melanoma patients participating in a clinical trial and the factors affecting their capacity to cope with the burden. METHODS: Semi-structured interviews were conducted with fourteen patients with advanced melanoma recruited from National Health Service hospitals in the United Kingdom. Qualitative analysis was undertaken using a framework analysis approach. Normalisation process theory was applied to the concept of research participation burden in order to interpret and categorise findings. RESULTS: Burdens of participation were identified as arising from making sense of the trial and treatment; arranging transport, appointment and prescriptions; enacting management strategies and enduring side effects; reflecting on trial documents and treatment efficacy, and emotional and mental effects of randomisation and treatment side effects. Factors reported as influencing capacity include personal attributes and skills, physical and cognitive abilities and support network. DISCUSSION: This is the first study to highlight the substantial burden faced by patients with advanced melanoma in a clinical trial and factors that may lessen or worsen the burden. Consideration of identified burdens during trial design and execution will reduce the burden experienced by research participants.

The expansive geography of Central Alberta presents many barriers to optimal care, including limited resources and access issues. In response to the COVID-19 pandemic, primary care networks (PCNs) within Central Alberta partnered with a technology provider to rapidly implement home health monitoring (HHM) for patients with chronic diseases. In the 37 patients evaluated in phase 1 (90 days), diabetes was most common (73%), followed by hypertension (38%), chronic obstructive pulmonary disease (27%), and heart failure (11%). Overall, patients were comfortable using the HHM technology, and >60% reported improved quality of life after follow-up. Patients also made fewer visits to their family physician/emergency department compared with the pre-enrolment period. In January 2021, the HHM initiative was expanded to a larger patient cohort (phase 2; n = 500). Interim results for 90 patients from eight PCNs up to the end of May 2021 show similar findings to phase 1.

Comprehensive participatory planning and evaluation (CPPE), a model used in community engagement research, has not been applied to patient engagement in research. We describe our methodology and interim results using CPPE in a project focused on improving research engagement of rural and distant patients and stakeholders. Specifically, we describe our development of a causal map and the subsequent use of the map to guide patient and stakeholder-driven evaluation.

Background/Aim In clinical trials, there is potential for bias from unblinded observers that may influence ascertainment of outcomes. This issue arose in the Strategies to Reduce Injuries and Develop Confidence in Elders trial, a cluster randomized trial to test a multicomponent intervention versus enhanced usual care (control) to prevent serious fall injuries, originally defined as a fall injury leading to medical attention. An unblinded nurse falls care manager administered the intervention, while the usual care arm did not involve contact with a falls care manager. Thus, there was an opportunity for falls care managers to refer participants reporting falls to seek medical attention. Since this type of observer bias could not occur in the usual care arm, there was potential for additional falls to be reported in the intervention arm, leading to dilution of the intervention effect and a reduction in study power. We describe the clinical basis for ascertainment bias, the statistical approach used to assess it, and its effect on study power. Methods The prespecified interim monitoring plan included a decision algorithm for assessing ascertainment bias and adapting (revising) the primary outcome definition, if necessary. The original definition categorized serious fall injuries requiring medical attention into Type 1 (fracture other than thoracic/lumbar vertebral, joint dislocation, cut requiring closure) and Type 2 (head injury, sprain or strain, bruising or swelling, other). The revised definition, proposed by the monitoring plan, excluded Type 2 injuries that did not necessarily require an overnight hospitalization since these would be most subject to bias. These injuries were categorized into those with (Type 2b) and without (Type 2c) medical attention. The remaining Type 2a injuries required medical attention and an overnight hospitalization. We used the ratio of 2b/(2b + 2c) in intervention versus control as a measure of ascertainment bias; ratios > 1 indicated the likelihood of falls care manager bias. We determined the effect of ascertainment bias on study power for the revised (Types 1 and 2a) versus original definition (Types 1, 2a, and 2b). Results The estimate of ascertainment bias was 1.14 (95% confidence interval: 0.98, 1.30), providing evidence of the likelihood of falls care manager bias. We estimated that this bias diluted the hazard ratio from the hypothesized 0.80 to 0.86 and reduced power to under 80% for the original primary outcome definition. In contrast, adapting the revised definition maintained study power at nearly 90%. Conclusion There was evidence of ascertainment bias in the Strategies to Reduce Injuries and Develop Confidence in Elders trial. The decision to adapt the primary outcome definition reduced the likelihood of this bias while preserving the intervention effect and study power.

Rakesh Ankit, India in the Interregnum: Interim Government, September 1946–August 1947, New Delhi: Oxford University Press, 2019, 376 pp.

Purpose: To characterize clinical outcomes of combined viscodilation of Schlemm’s canal and collector channels and 360° trabeculotomy as a standalone procedure or combined with cataract surgery in eyes with mild to moderate open-angle glaucoma (OAG). Methods: In this prospective case series, the OMNI glaucoma surgical platform (Sight Sciences, Menlo Park, CA) was utilized to perform the procedure either combined with phacoemulsification or as a standalone procedure. Changes from baseline in intraocular pressure (IOP) and IOP-lowering medications were evaluated through the first 12 months of a planned 24-month follow-up period. Results: Among 17 eyes of 15 subjects, mean IOP was reduced from 20.4 mmHg to 12.7–13.7 mmHg through 12 months of follow-up ( p Conclusion: Viscodilation of Schlemm’s canal and collector channels paired with ab interno trabeculotomy performed with a single integrated instrument (OMNI), whether as standalone or combined with phacoemulsification, effectively lowers both IOP and the need for IOP-lowering medications through 12 months of follow-up.

Interim assessment occurs throughout instruction to provide feedback about what students know and have achieved. Different from the current available cognitive diagnostic computerized adaptive testing (CD-CAT) design that focuses on assessment at a single time point, the authors discuss several designs of interim CD-CAT that are suitable in the learning context. The interim CD-CAT differs from the current available CD-CAT designs primarily because students’ mastery profile (i.e., skills mastery) changes due to learning, and new attributes are added periodically. Moreover, hierarchies exist among attributes taught sequentially and such information could be used during item selection. Two specific designs are considered: The first one is when new attributes are taught in Stage II, but the student mastery status of the previously taught attributes stays the same. The second design is when both new attributes are taught, and previously taught attributes can be further learned or forgotten in Stage II. For both designs, the authors propose an individual prior, which considers a person’s learning history and population learning model, to start an interim CD-CAT. Simulation results show that the Stage II CD-CAT using individual prior outperforms the methods using population priors. The GDINA (generalized deterministic inputs, noisy, “and” gate) diagnostic index (GDI) is extended to accommodate item hierarchies, and analytic results are provided to further illustrate the types of items that are most popular during item selection. As the first study that focuses on the application of CD-CAT in a learning context, the methods and results present herein showed the great promise of using CD-CAT to monitor learning.

The compassionate-love Jesus feels moves him to solve the problems of the suffering. Hence, everything Jesus thought, said, or did in his mission to salvage humankind was motivated by compassionate love. Jesus demonstrated that his mission-mandate should be done on the platform of genuine compassionate love. That is why, in the gospels, he was described as always being moved by compassion. Jesus demonstrated that his followers are to carry on the mission-mandate of the church in compassionate love. But in this era, the church has undergone a paradigm shift from this model of Jesus’ compassion. The problem of the church being less compassionate is hindering the contemporary church from achieving mission-desired goals. Consequently, this article studies the concept of compassion as an underlying theme in the gospel of Matthew and its implications for the mission-mandate of the church in Nigeria. Through the application of the redaction-criticism method of doing biblical exegesis, the study found that the church in Nigeria lacks the model of compassion which Jesus exemplified.

This project was designed to support the US Navy’s (USN) implementation of circadian- based watchbills by providing guidance on sleep management and shipboard practices which support crew endurance. In the first two years of the three-year effort, we identified specific training needs and developed instructor-led materials. We delivered the training to two audiences all active duty members of the US Navy or US Marine Corps: senior leaders and entry-level to mid-level personnel (N=353 participants; 15 training sessions total). Following the training, we administered a short questionnaire to obtain individual feedback. Responses from both audiences showed a high level of satisfaction with the training. Future efforts will focus on improving the training materials, further assessing training, and developing additional materials designed for specific military audiences at increased risk of insufficient sleep.

Background/Context Teachers’ practice of interim assessment data use is subject to an array of individual, social, and organizational contextual factors. While contemporary research situates interim assessments within teachers’ broader practice of data use, teachers do not make sense of interim assessment data in isolation. This interconnectedness of data sources serves as an important context for understanding how teachers make sense of interim assessment data. Purpose/Objective/Research Question/Focus of Study Teachers’ ability to connect data sources to inform a common instructional practice has important implications for their instructional decision making. The present study maps the interconnectedness of interim assessment with other data sources as one of two dimensions of teachers’ practice of data use. Furthermore, this article explores how those dimensions are related to similarities, differences, and patterns in how teachers make sense of interim assessment data. Population/Participants/Subjects Data come from 88 grade 3-5 mathematics or English language arts teachers across 10 elementary schools in two states. All participants completed a survey about their data use practices. A subsample of 23 teachers completed an extended survey and participated in an interview data about how they use interim assessment data. Research Design: We conducted a convergent mixed methods study of teachers’ practice of interim assessment data use. Data Collection and Analysis Quantitatively, we analyzed the interconnectedness of data sources across teachers within an epistemic network analytic framework. Qualitatively, we coded interview transcripts to gather evidence of how teachers use interim assessment data to make instruction decisions. We integrated our quantitative and qualitative findings and recoded interview transcripts to develop a nuanced perspective on the similarities, differences, and patterns in how teachers make sense of interim assessment data. Findings/Results Findings reveal teachers rarely interpret interim assessment data in isolation and meaningfully vary in the extent to which they perceive it informs their broader instructional practice. Taken together, these two dimensions of teachers’ practice of interim assessment data use are indicators of significant patterns in the authenticity of teachers’ data use with implications for their instructional decision making. Conclusions/Recommendations The present study maps the interconnectedness of data sources within teachers’ instructional practice and how it serves as one of two dimensions that explain similarities, differences, and patterns in how they make sense of interim assessment data. Our findings have implications for how instructional support coaches, administrators, interim assessment developers, and researchers can support teachers’ use of interim assessment data for instructional purposes.

The coronavirus disease 2019 pandemic has demonstrated a need for an infectious disease standard that will promote a safe and healthy work environment and assure business continuity. The current pandemic has revealed gaps in workplace preparedness and employee protections to microbial exposures. Federal and state government agencies have responded by providing interim guidelines and stop-gap measures that continue to evolve and vary in approach and required controls. This interim and inconsistent approach has resulted in confusion on the part of businesses as they work toward reopening during the pandemic and uncertainty as to the efficacy of required or suggested controls. Moving forward, the US Occupational Safety and Health Administration, with guidance from the US National Institute for Occupational Safety and Health, should establish consistent and effective strategies through a nationwide standard to address the potential microbial exposures in the workplace. Such a standard will require effective worker protections from infectious diseases and assure business continuity.

This article discusses the plight of the homeless during public health emergencies and the coronavirus disease of 2019 (COVID-19) pandemic. It reviews the role of public administrators that grounds their efforts by examining their foundational purpose to serve the most vulnerable in our society. Using subsidiarity principle as the context, it discusses homelessness in America and the role of the federal Department of Housing and Urban Development and their Continuum of Care program. It also highlights the role of the Centers for Disease Control and Prevention during public health emergencies and their interim guidelines for local governments in providing for the homeless during emergencies. Finally, through a case study on the city of Dallas, Texas, the article examines how local governments have responded to address the needs of the homeless during the COVID-19 pandemic. It concludes that it is imperative that public administrators at all levels of government explore areas of shared competence, cooperation, and allocate responsibility where it would yield the most efficient result.

Clinical trials in oncology often involve the statistical analysis of time-to-event data such as progression-free survival or overall survival to determine the benefit of a treatment or therapy. The log-rank test is commonly used to compare time-to-event data from two groups. The log-rank test is especially powerful when the two groups have proportional hazards. However, survival curves encountered in oncology studies that differ from one another do not always differ by having proportional hazards; in such instances, the log-rank test loses power, and the survival curves are said to have “non-proportional hazards”. This non-proportional hazards situation occurs for immunotherapies in oncology; immunotherapies often have a delayed treatment effect when compared to chemotherapy or radiation therapy. To correctly identify and deliver efficacious treatments to patients, it is important in oncology studies to have available a statistical test that can detect the difference in survival curves even in a non-proportional hazards situation such as one caused by delayed treatment effect. An attempt to address this need was the “max-combo” test, which was originally described only for a single analysis timepoint; this article generalizes that test to preserve type I error when there are one or more interim analyses, enabling efficacious treatments to be identified and made available to patients more rapidly.

Purpose: Many clinical trials of promising interventions fail to see anticipated effects at follow-up. To avoid carrying out a full trial when the tested treatment is likely to be inefficacious, researchers may include one or more interim analyses, assessing for trial futility before the final analysis, discontinuing the trial early if interim effects are less than expected. The current approach tests whether the trial is likely to show futility, not whether the treatment is likely to be inefficacious. Treatments that show efficacy but with wide confidence bounds around efficacy estimates may be stopped inappropriately for futility. We present an alternative approach which focuses on the expected efficacy of the treatment. The approach uses “futility regions” to determine whether to stop trials of treatments that are unlikely to show a clinically important effect at interim—a stopping rule which requires interim interval estimates to lie entirely within a “stopping zone” (current approaches only require estimates to cross into the region of null/futile effects). We contrast this with extant methods, highlighting how the proposed approach focuses on a clinically relevant question, and protects against undue stopping because of imprecise estimates. For a researcher wanting to gather evidence of efficacious treatments, this is a desirable characteristic; trials with precise interim estimates of null effects should be stopped, whereas those with imprecise estimates that include useful effect magnitudes should be allowed to continue. Method: We created a simulation study, testing 1000 permutations of six different parallel-design trial scenarios, all featuring one interim analysis, and stopping only for futility. The scenarios demonstrate a range of effect sizes and variances. We implemented four types of interval-based stopping rules—using Frequentist and Bayesian approaches—and compared their performance to the more commonly used O’Brien-Fleming and Pocock designs. We used a range of stopping rules, 15 in total, to demonstrate the limits of the approaches in the different trial scenarios. Results: All approaches discriminated well between trials with a final null effect, and those with a final large treatment effect. However, when the trial variance was different to that expected at the design stage, the interval-based approaches demonstrated favorable characteristics—continuing more permutations with imprecise but promising estimates, and stopping more of those with precise estimates of clinically trivial or null (futile) effect sizes (table). Empirical Bayesian approaches showed enhanced precision compared to the other methods, due to their property of allowing interim estimates to inform the prior distributions of the final effect estimate. Conclusion: A simple implementation of adaptive trial methodology, featuring only one interim analysis and using a modified version of interval estimate stopping rules demonstrated preferable performance characteristics for the researcher seeking to stop only trials that precisely estimate null effects than more current approaches. These characteristics have the benefit of producing two possible meaningful trial endpoints: completed trials that had continued to show promise at interim, and stopped trials with precise estimates of a null effect. This contrasts with a trial which stops early with an imprecise estimate of effect, about which few conclusions can be drawn.

The article looks at how jurisprudence has evolved in Indian courts on granting or refusing injunctions to patentees, especially when such patentees are not working/commercially using their patented product in India. The author also presents his views on the recent Eisai v. Dr Reddys case.

For some separatist movements, interim agreements offer a possible route to recognized statehood. However, such agreements require these movements to compromise on their demand for immediate independence and risk the preservation of the joint state. How is this reconciled with their claim to self-determination and how is it received by the community they claim to represent? This article examines four post-Cold War cases where an interim agreement has been accepted (New Caledonia, Bougainville, Montenegro and South Sudan). It finds that interim agreements are more easily accepted when the community is significantly divided on the issue of independence and when an inclusive and flexible construction of the community predominates. Somewhat paradoxically, this suggests that new states are more likely to emerge in cases without a determined, cohesive, ethnically defined demand for independence.

Sequential monitoring of efficacy and safety is an important part of clinical trials. A Group Sequential design allows researchers to perform interim monitoring after groups of patients have completed the study. Statistical literature is well developed for continuous and binary outcomes and relies on asymptotic normality of the test statistic. However, in the case of time-to-event data, existing methods of sample size calculation are done either assuming proportional hazards or assuming exponentially distributed lifetimes. In scenarios where these assumptions are not true, as evidenced from historical data, these traditional methods are restrictive and cannot always be used. As interim monitoring is driven by ethical, financial, and administrative considerations, it is imperative that sample size calculations be done in an efficient manner keeping in mind the specific needs of a clinical trial with a time-to-event outcome. To address these issues, a novel group sequential design is proposed using the concept of Proportional Time. This method utilizes the generalized gamma ratio distribution to calculate the efficacy and safety boundaries and can be used for all distributions that are members of the generalized gamma family using an error spending approach. The design incorporates features specific to survival data such as loss to follow-up, administrative censoring, varying accrual times and patterns, binding or non-binding futility rules with or without skips, and flexible alpha and beta spending mechanisms. Three practical examples are discussed, followed by discussion of the important aspects of the proposed design.

The study was conducted to understand the effects of having access to different training materials in the interim between training and first use of an injection device for 27 injection-naïve participants. Participants were told they would be giving themselves a real injection of saline in the return session, and were not told that they would use injection pads until after subjective data was gathered during session 2. This deception paradigm was used to increase the external validity of the study. Participants self-trained in one of three conditions based on what they were given to take home: instructions for use, mechanical trainer, and/or a training video. Participants self-trained in session 1, performed a simulated injection in the session 2 two weeks later, and completed three online surveys in the interim. In session 2, a majority of participants were able to achieve minimally acceptable performance to deliver the medication during the simulated use of the autoinjector. Participants who took home a mechanical trainer had fewer use deviations than those who did not have the trainer at home. This difference was only seen in injection specific subtasks; performance on pre- and post-injection subtasks was low across all groups. Intermittent practice with a mechanical trainer was seen to improve performance on subtasks involved in the motor performance of the injection itself but did not impact performance or compliance with other tasks important to the injection. More holistic training solutions may be required to impact other aspects of the injection beyond the injection itself. While performance differed, subjective ratings did not differ, showing an overconfidence bias that may affect a user’s ability to self-report competence in device use.

Objectives To address the need for long-term lasmiditan data, the GLADIATOR study evaluated the safety (primary) and efficacy (secondary) of lasmiditan for the intermittent, acute treatment of migraine attacks for up to 1 year. Methods In this prospective, randomized, open-label, Phase 3 study, patients who had completed either of two single-attack studies were offered the opportunity to be randomized 1:1 to lasmiditan 100 mg or 200 mg. Patients were asked to use lasmiditan as the first treatment for each new migraine attack of at least moderate severity. Assessments occurred at baseline and at prespecified time increments up to 48 hours after each dose of study drug using an electronic diary, and safety was assessed throughout the study. Migraine Disability Assessment (MIDAS) was assessed at each visit. Results As of the cut-off date for this interim analysis (6 March 2018), 1978 patients had received ≥ 1 lasmiditan dose and treated 19,058 migraine attacks. Overall, treatment-emergent adverse events (TEAEs) were similar to those in the single-attack studies and included dizziness (18.6%), somnolence (8.5%), and paresthesia (6.8%). The frequency of TEAEs generally decreased with subsequent attacks. No treatment-related serious adverse events and no cardiovascular TEAEs potentially due to vasoconstriction were observed. For both lasmiditan doses, efficacy measures were generally consistent over study quarters and treated attacks. Overall, across all treated attacks at 2 hours post-dose, pain freedom was observed in 26.9% of the attacks treated with lasmiditan 100 mg and 32.4% of the attacks treated with lasmiditan 200 mg. MIDAS total scores decreased over time. Conclusions The interim results of this long-term study showed intermittent lasmiditan (100 mg and 200 mg) to be generally well tolerated and efficacious for the acute treatment of migraine over a 1-year period. Trial registration number: NCT02565186; https://clinicaltrials.gov/ct2/show/NCT02565186

The coronavirus disease (COVID-19) pandemic has created unprecedented challenges in caring for individuals living with kidney disease. In response to a growing call for up-to-date information and evidence-informed advice, the Canadian Society of Nephrology has established a COVID-19 Rapid Response Team that will leverage existing evidence and national expertise to inform kidney care practices in the COVID-19 era. Given limited published evidence and compressed timelines, formal clinical practice guidelines are not feasible, and we have adopted rapid review methods to instead provide interim guidance across identified priority areas. In this article, we describe the methodological approach that was applied in developing a first iteration of guidance documents addressing clinical and operational aspects of care for patients treated with in-center hemodialysis, home dialysis, those with advanced chronic kidney disease, those with glomerulonephritis, and those with acute kidney injury. We further describe strategies for maintaining ongoing engagement with the renal community to elicit emerging needs and perspectives as the situation unfolds.

The interim briefing presents initial findings from a project exploring the support available to migrants with no recourse to public funds during the COVID-19 pandemic.The research included a survey of local authorities in England, and a call for evidence from migrant support organisations in England, Scotland and Wales. More than 90 percent of local authorities had not shared information about support for people with NRPF during the pandemic, and support organisations reported that service users had struggled to access food, shelter and subsistence support during the pandemic.

In 2003, in an effort to address St. Louis Public Schools’ (SLPS’) declining enrollment and debt, the interim superintendent closed 16 schools. In 2016, more than a decade since the school closings, the enrollment dwindled from 44,000 students to half that number. This study is an examination of the closed schools to determine if displaced African American children from the shuttered schools were better off as a result of attending the replacement schools. The findings suggest SLPS’ replacement schools were not better than the shuttered schools as was predicted in the SLPS 2003 Plan. Instead, we found displaced African American children who attended income imbalanced SLPS and charter schools in St. Louis had fewer children proficient on the state’s English Language Arts (ELA) assessment than their peers who participated in the Voluntary Interdistrict Transfer Program (VITP). The implications of these findings in the use of school closings as reform initiatives are discussed.

The interim councils and medical workers operating in besieged cities in opposition-held areas of Syria have faced enormous challenges in providing services to citizens. Despite the extremely diffi...

This descriptive, mixed methods study of one interim certification program explores first year urban teachers’ classroom management actions. This study investigates what strategies teachers implement to manage the classroom from programmatic surveys of 87 first-year teachers and interviews, field visits, video recordings, and journals of five case participants. Results indicate that teachers used behavioral, academic, and relational strategies to manage the classroom and they tend to refine several of these actions over time. Findings suggest that teacher preparation should promote beginning teachers to implement a range of classroom management strategies and support teachers in how to refine their actions.

This study proposes a framework for the interim evaluation of, and subsequent planning for, information systems (IS) in the public sector. We incorporate the considerations of cost efficiency and task effectiveness, which are critical for reaping a sustained benefit from IS. We recommend this double-sided analysis as the basis for interim evaluation. We also advise that this framework of evaluation can effectively indicate which subsequent course of action should be taken: maintain the status quo, upgrade functionality, redevelop, or discontinue the IS. We applied this framework to 118 ISs within the South Korean public sector with the k-means clustering method and provided both academic and practical implications.

With the death of first president of Uzbekistan Karimov, and Mirziyoyev’s appointment as interim president in 2016 and election to the presidency in December of the same year, speculation appeared in the media suggesting that Uzbekistan would move closer to Russia because of the close personal ties between the president and Russian elites. Others suggested that Uzbekistan would either follow the path designed by the first president or fall into isolationism like Turkmenistan. This article aims to explain the continuity and shifts in Uzbekistan’s foreign policy behaviour in the year following the death of the first president and the election of President Mirziyoyev.

The hierarchical item response theory (H-IRT) model is very flexible and allows a general factor and subfactors within an overall structure of two or more levels. When an H-IRT model with a large number of dimensions is used for an adaptive test, the computational burden associated with interim scoring and selection of subsequent items is heavy. An alternative approach for any high-dimension adaptive test is to reduce dimensionality for interim scoring and item selection and then revert to full dimensionality for final score reporting, thereby significantly reducing the computational burden. This study compared the accuracy and efficiency of final scoring for multidimensional, local multidimensional, and unidimensional item selection and interim scoring methods, using both simulated and real item pools. The simulation study was conducted under 10 conditions (i.e., five test lengths and two H-IRT models) with a simulated sample of 10,000 students. The study with the real item pool was conducted using item parameters from an actual 45-item adaptive test with a simulated sample of 10,000 students. Results indicate that the theta estimations provided by the local multidimensional and unidimensional item selection and interim scoring methods were relatively as accurate as the theta estimation provided by the multidimensional item selection and interim scoring method, especially during the real item pool study. In addition, the multidimensional method required the longest computation time and the unidimensional method required the shortest computation time.

Maintaining confidentiality of emerging data and ensuring the independence of Data Monitoring Committees are best practices of considerable importance to the ability of these committees to achieve their mission of safeguarding the interests of study participants and enhancing the integrity and credibility of clinical trials. Even with the wide recognition of these principles, there are circumstances where confidentiality issues remain challenging, controversial or inconsistently addressed. First, consider settings where a clinical trial’s interim data could provide the evidence regulatory authorities require for decisions about marketing approval, yet where such a trial would be continued post-approval to provide more definitive evidence about principal safety and/or efficacy outcomes. In such settings, data informative about the longer term objectives of the trial should remain confidential until pre-specified criteria for trial completion have been met. Second, for those other than Data Monitoring Committee members, access to safety and efficacy outcomes during trial conduct, even when presented as data pooled across treatment arms, should be on a limited “need to know” basis relating to the ability to carry out ethical or scientific responsibilities in the conduct of the trial. Third, Data Monitoring Committee members should have access to unblinded efficacy and safety data throughout the trial to enable timely and informed judgments about risks and benefits. Fourth, it should be recognized that a mediator potentially could be useful in rare settings where the Data Monitoring Committee would have serious ethical or scientific concerns about the sponsor’s dissemination or lack of dissemination of information. Data Monitoring Committee Contract Agreements, Indemnification Agreements and Charters should be developed in a manner to protect Data Monitoring Committee members and their independence, in order to enhance the Data Monitoring Committee’s ability to effectively address their mission.

Background: Data monitoring committees for randomized clinical trials have the responsibility of safeguarding interests of trial participants. To do so, the data monitoring committee must receive reports on safety and efficacy to assess risk/benefit and on trial conduct to ensure that the study can achieve its goals. This article outlines the key components of reports to the data monitoring committee and the important role of the unblinded statistician in preparing those reports. Methods: Most data monitoring committee meetings include open and closed sessions. For each session, there is a report of interim results. The open session is attended by the sponsor and lead investigators, including the statistician(s) responsible for the trial design. These investigators are blinded to the interim treatment comparisons. The closed session is attended by the data monitoring committee members and by the statistician(s) who prepared the closed report. These individuals are unblinded to interim treatment comparisons and therefore are not involved in study design changes. The optimal content of data monitoring committee reports and qualifications of the unblinded statistician(s) are discussed. Reports: Open reports should include responses to data monitoring committee recommendations, a synopsis of the protocol, a review of the protocol history and amendments, and information on enrollment, baseline characteristics, completeness of follow-up, and data quality. The open report is also a vehicle through which the sponsor and investigators should inform the data monitoring committee of relevant external information. Data in the open report are pooled over the treatment groups. The open report should not include data summaries by treatment group. The closed report should include a written summary with references to key tables and figures and methods used to prepare them. Tables and figures should summarize baseline characteristics, follow-up completeness, treatment adherence, and major safety and efficacy outcomes by treatment group. Text summaries should accompany the tables and figures. The data monitoring committee monitoring history (e.g. treatment differences at previous meetings) should be summarized. The unblinded statistician preparing the closed report should be familiar with the protocol and data collection plan and be capable of customizing the report to the current stage of the trial. This includes anticipating questions that may arise during the data monitoring committee review and pro-actively including data summaries to address these questions. Conclusions: There is considerable variation in the quality of open and closed data monitoring committee reports. Open and closed data monitoring committee reports should be concise, up to date, and informative. To achieve this, unblinded statisticians responsible for preparing closed data monitoring committee reports should be familiar with the statistical methods, the trial protocol, and the data collection plan. They should be capable of anticipating questions from the data monitoring committee and responding to requests for additional analyses.

Objective To describe the redactions in contemporary protocols for industry-sponsored randomised drug trials with patient relevant outcomes and to evaluate whether there was a legitimate rationale for the redactions. Design Cohort study. Under the Freedom of Information Act, we requested access to trial protocols approved by a research ethics committee in Denmark from October 2012 to March 2013. We received 17 consecutive protocols, which had been redacted before we got them, and nine protocols without redactions. In five additional cases, the companies refused to let the committees give us access, and in three other cases, documents were missing. Participants Not applicable. Setting Not applicable. Main outcome measure Amount and nature of redactions in 22 predefined key protocol variables. Results The redactions were most widespread in those sections of the protocol where there is empirical evidence of substantial problems with the trustworthiness of published drug trials: data analysis, handling of missing data, detection and analysis of adverse events, definition of the outcomes, interim analyses and premature termination of the study, sponsor’s access to incoming data while the study is running, ownership to the data and investigators’ publication rights. The parts of the text that were redacted differed widely, both between companies and within the same company. Conclusions We could not identify any legitimate rationale for the redactions. The current mistrust in industry-sponsored drug trials can only change if the industry offers unconditional access to its trial protocols and other relevant documents and data.

The authors study specialized personal incentives (SPIs), which are cash rewards granted to salespeople for meeting interim performance goals within the regular sales quota period (monthly, quarterly, etc.). Because firms often institute multiple SPIs, the authors are able to investigate whether different sales achievement trajectories have differential impacts on salespeople's period-end sales performance. The authors find that a steadily growing sales trajectory in a sales period is more strongly associated with period-end success than a sales trajectory that is relatively flat early but has a sharp spike later in the period. Furthermore, although salespeople who had high performance in the prior month (i.e., high-performance state) may be able to draw on superior selling strategies (compared with other salespeople), they too experience a boost in sales performance in the current month by earning SPIs. Notably, the authors also find that although earning SPIs benefits all salespeople, there is a U-shaped relationship between a salesperson's performance state and his or her month-end sales performance. For any specific number of SPIs earned, the probability of meeting and exceeding month-end quotas is boosted more for salespeople with low- and high-performance states than for salespeople with a medium-performance state.

Everyone knows that what happened in Flint is connected to race, but we lack the necessary frameworks to fully understand the multiple ways that race and racism contributed to the Flint Water Crisis. This article introduces the notion of Strategic–Structural racism, the manipulation of the forces of intentional racism, structural racism and unconscious bias for economic or political gain. This construct is applied to critical aspects of the Flint Water Crisis: the imposition of emergency management, the approval of the Karegnondi Water Authority (KWA) pipeline, the decision to use the Flint River as an interim source of drinking water and how municipal finance rules were manipulated to obtain financing for the KWA pipeline but not to fund necessary upgrades to the Flint Water Treatment Plant (WTP). Tragically, the strategic racism embedded in the KWA approval process created an environment of denial, cover-up and complicity as aspects of the public health crisis began to emerge.

Background: Sample size adjustment designs, which allow increasing the study sample size based on interim analysis of outcome data from a randomized clinical trial, have been increasingly promoted in the biostatistical literature. Although it is recognized that group sequential designs can be at least as efficient as sample size adjustment designs, many authors argue that a key advantage of these designs is their flexibility; interim sample size adjustment decisions can incorporate information and business interests external to the trial. Recently, Chen et al. (Clinical Trials 2015) considered sample size adjustment applications in the time-to-event setting using a design (CDL) that limits adjustments to situations where the interim results are promising. The authors demonstrated that while CDL provides little gain in unconditional power (versus fixed-sample-size designs), there is a considerable increase in conditional power for trials in which the sample size is adjusted. Methods: In time-to-event settings, sample size adjustment allows an increase in the number of events required for the final analysis. This can be achieved by either (a) following the original study population until the additional events are observed thus focusing on the tail of the survival curves or (b) enrolling a potentially large number of additional patients thus focusing on the early differences in survival curves. We use the CDL approach to investigate performance of sample size adjustment designs in time-to-event trials. Results: Through simulations, we demonstrate that when the magnitude of the true treatment effect changes over time, interim information on the shape of the survival curves can be used to enrich the final analysis with events from the time period with the strongest treatment effect. In particular, interested parties have the ability to make the end-of-trial treatment effect larger (on average) based on decisions using interim outcome data. Furthermore, in “clinical null” cases where there is no benefit due to crossing survival curves, the sample size adjustment design is shown to increase the probability of recommending an ineffective therapy. Conclusion: Access to interim information on the shape of the survival curves may jeopardize the perceived integrity of trials using sample size adjustment designs. Therefore, given the lack of efficiency advantage over group sequential designs, sample size adjustment designs in time-to-event settings remain unjustified.

Background With the rising cost of new oncology treatments, it is no longer sustainable to base initial drug funding decisions primarily on prospective clinical trials as their performance in real-life populations are often difficult to determine. In British Columbia, an approach in evidence building is to retrospectively analyse patient outcomes using observational research on an ad hoc basis. Methods The deliberative framework was constructed in three stages: framework design, framework validation and treatment programme characterization, and key informant interview. Framework design was informed through a literature review and analyses of provincial and national decision-making processes. Treatment programmes funded between 2010 and 2013 were used for framework validation. A selection concordance rate of 80% amongst three reviewers was considered to be a validation of the framework. Key informant interviews were conducted to determine the utility of this deliberative framework. Results A multi-domain deliberative framework with 15 assessment parameters was developed. A selection concordance rate of 84.2% was achieved for content validation of the framework. Nine treatment programmes from five different tumour groups were selected for retrospective outcomes analysis. Five contributory factors to funding uncertainties were identified. Key informants agreed that the framework is a comprehensive tool that targets the key areas involved in the funding decision-making process. Conclusions The oncology-based deliberative framework can be routinely used to assess treatment programmes from the major tumour sites for retrospective outcomes analysis. Key informants indicate this is a value-added tool and will provide insight to the current prospective funding model.

In 2015, more than one million migrants and refugees arrived in Europe. Commercial complementary foods, processed foods marketed for infants and young children 6-23 months of age, were distributed by various humanitarian actors along migrant routes and in European refugee camps. Unsolicited donations and distributions of commercial complementary food products were problematic and divergent from international policies on infant and young child feeding during humanitarian emergencies. Interim guidance regarding commercial complementary foods was published during the peak of the emergency but implemented differently by various humanitarian actors. Clearer and more technical specifications on commercial complementary foods are needed in order to objectively determine their suitability for operational contexts in Europe and emergency nutrition assistance in the future.

Background: Interim analyses of randomised controlled trials are sometimes published before the final results are available. In several cases, the treatment effects were noticeably different after patient recruitment and follow-up completed. We therefore conducted a literature review of peer-reviewed journals to compare the reported treatment effects between interim and final publications and to examine the magnitude of the difference. Methods: We performed an electronic search of MEDLINE from 1990 to 2014 (keywords: ‘clinical trial’ OR ‘clinical study’ AND ‘random*’ AND ‘interim’ OR ‘preliminary’), and we manually identified the corresponding final publication. Where the electronic search produced a final report in which the abstract cited interim results, we found the interim publication. We also manually searched every randomised controlled trial in eight journals, covering a range of impact factors and general medical and specialist publications (1996–2014). All paired articles were checked to ensure that the same comparison between interventions was available in both. Results: In all, 63 studies are included in our review, and the same quantitative comparison was available in 58 of these. The final treatment effects were smaller than the interim ones in 39 (67%) trials and the same size or larger in 19 (33%). There was a marked reduction, defined as a ≥20% decrease in the size of the treatment effect from interim to final analysis, in 11 (19%) trials compared to a marked increase in 3 (5%), p = 0.057. The magnitude of percentage change was larger in trials where commercial support was reported, and increased as the proportion of final events at the interim report decreased in trials where commercial support was reported (interaction p = 0.023). There was no evidence of a difference between trials that stopped recruitment at the interim analysis where this was reported as being pre-specified versus those that were not pre-specified (interaction p = 0.87). Conclusion: Published interim trial results were more likely to be associated with larger treatment effects than those based on the final report. Publishing interim results should be discouraged, in order to have reliable estimates of treatment effects for clinical decision-making, regulatory authority reviews and health economic analyses. Our work should be expanded to include conference publications and manual searches of additional journal publications.

More and more boards are tapping interim CEOs to temporarily fill the corner office. Prior research indicates the negative performance implications of this decision; yet, little is known about the rationale behind this decision. Our aim is to fill this research gap. Drawing on agency and human capabilities frameworks, we examined the contextual elements that influence a board’s decision to pursue temporary leadership. Within a sample of 375 successions occurring between 1998 and 2005, we found that boards of directors were more likely to select interim CEOs under certain succession conditions, namely, when the prior CEO was forced out and there was no heir apparent or when the prior CEO had served for a short tenure. Our results highlight the importance of context in succession selections and provide insights into why a board would pursue a decision with seemingly negative repercussions. Implications for theory and practice are discussed.

Background: Communicating with trial participants is an important aspect of study conduct, relevant for informed consent and respect for participants. Group teleconferences are one means to convey information to trial participants. We used group teleconferences during an ongoing large-scale clinical trial to communicate important trial updates. Methods: The National Institute of Neurological Disorders and Stroke Exploratory Trials in Parkinson’s Disease Longitudinal Study-1 trial studied creatine for treatment of early-stage Parkinson’s disease. A total of 1741 participants enrolled at 45 sites in the United States and Canada to take part in a double-blind randomized trial of 5 years of treatment with creatine versus placebo. The study leadership held two teleconferences with study participants and their caregivers after each of two pre-specified interim analyses, for a total of four teleconferences. Each agenda included a presentation by study leadership followed by an open question and answer period. Teleconference recordings were made available to all site personnel and trial participants. Recordings were reviewed and abstracted for themes and topics of the presentations, participant questions, and discussion. Number of participants, connection time for each participant, number of questions, and caller connection time were summarized using descriptive statistics. After the first teleconferences, participants who remained on the call until the end were invited to complete a voluntary, four-question survey about the teleconference process. During the second teleconferences, participants were notified of premature study closure. Results: There were 258 callers for the first pair of teleconferences and 604 callers for the second pair of teleconferences. Study leaders answered more than 110 questions from study participants and caregivers across all calls. The most frequently asked question themes related to study drug, Parkinson’s disease, side effects, future research, and data analysis. The initial teleconferences were well received by participants. Based on responses to the post-call survey, 98% (118/121) of participants found the call useful, 91% (115/127) were interested in future similar calls, 88% stated the call made them more likely to continue in the study (112/128), and 85% (90/106) were satisfied overall with study communications. Conclusion: Teleconferences provide a convenient way to communicate with trial participants and can be used during the conduct of clinical trials to convey study progress and other information. For multi-site trials, teleconferences enable participants to engage directly with study leadership and to ask questions. Survey respondents were highly satisfied with the group teleconference experience. Future research is needed to determine whether teleconferences improve participants’ satisfaction with clinical trial participation and improve retention.

Background: Futility (inefficacy) interim monitoring is an important component in the conduct of phase III clinical trials, especially in life-threatening diseases. Desirable futility monitoring guidelines allow timely stopping if the new therapy is harmful or if it is unlikely to demonstrate to be sufficiently effective if the trial were to continue to its final analysis. There are a number of analytical approaches that are used to construct futility monitoring boundaries. The most common approaches are based on conditional power, sequential testing of the alternative hypothesis, or sequential confidence intervals. The resulting futility boundaries vary considerably with respect to the level of evidence required for recommending stopping the study. Purpose: We evaluate the performance of commonly used methods using event histories from completed phase III clinical trials of the Radiation Therapy Oncology Group, Cancer and Leukemia Group B, and North Central Cancer Treatment Group. Methods: We considered published superiority phase III trials with survival endpoints initiated after 1990. There are 52 studies available for this analysis from different disease sites. Total sample size and maximum number of events (statistical information) for each study were calculated using protocol-specified effect size, type I and type II error rates. In addition to the common futility approaches, we considered a recently proposed linear inefficacy boundary approach with an early harm look followed by several lack-of-efficacy analyses. For each futility approach, interim test statistics were generated for three schedules with different analysis frequency, and early stopping was recommended if the interim result crossed a futility stopping boundary. For trials not demonstrating superiority, the impact of each rule is summarized as savings on sample size, study duration, and information time scales. Results: For negative studies, our results show that the futility approaches based on testing the alternative hypothesis and repeated confidence interval rules yielded less savings (compared to the other two rules). These boundaries are too conservative, especially during the first half of the study (50% of information) and may stop a trial even when there is a clinically meaningful treatment effect. The linear inefficacy boundary with three or more interim analyses provided the best results. For positive studies, we demonstrated that none of the futility rules would have stopped the trials. Conclusion: The linear inefficacy boundary futility approach is attractive from statistical, clinical, and logistical standpoints in clinical trials evaluating new anti-cancer agents.

We examined instructional processes in classrooms where students with and without disabilities received mathematics instruction to understand the relationship among key instructional process variables and achievement as measured by interim and end-of-year summative assessments. Teachers ( N = 78) completed instructional logs daily and administered easyCBM probes quarterly to 162 students with disabilities and 165 students without disabilities. Examination of instructional data indicated both groups of students had nearly equal opportunities to learn (OTLs) the same mathematics content, yet there were significant differences in these students’ mathematics achievement on interim and summative tests. Special education status and instructional practices were found to be significant predictors of achievement growth. Furthermore, grade level and special education status, along with OTL scores, accounted for significant variance in end-of-year mathematics scores. Discussion of results focuses on instructional practice implications and the role these practices play in achievement gaps.

Even in the age of abundant and fast computing resources, concurrency requirements for large-scale online testing programs still put an uninterrupted delivery of computer-adaptive tests at risk. In this study, to increase the concurrency for operational programs that use the shadow-test approach to adaptive testing, we explored various strategies aiming for reducing the number of reassembled shadow tests without compromising the measurement quality. Strategies requiring fixed intervals between reassemblies, a certain minimal change in the interim ability estimate since the last assembly before triggering a reassembly, and a hybrid of the two strategies yielded substantial reductions in the number of reassemblies without degradation in the measurement accuracy. The strategies effectively prevented unnecessary reassemblies due to adapting to the noise in the early test stages. They also highlighted the practicality of the shadow-test approach by minimizing the computational load involved in its use of mixed-integer programming.

This article addresses the issues involved in using compliance with probation supervision as an interim outcome measure in evaluation research. We address the complex nature of compliance and what it implies. Like much research on probation and criminal justice more generally, it was not possible to use random assignment to treatment and comparison groups in the case study we address, which evaluated the SEED training programme. We therefore compare two different data analysis methods to adjust for prior underlying differences between groups, namely regression adjustment of treatment covariates that are related to the outcome measure in the sample data and regression adjustment using propensity scores derived from a wide range of baseline variables. The propensity score method allows for control of a wider range of baseline variables, including those which do not differ significantly between the two groups.

Jerry Davis’s (2015) question “What is organizational research for?” is ill-served by the narrow answer “settled science.” Constraints of comprehension may give the illusion that organizational research represents settled science. But the experience of inquiring actually comprises a greater variety of actions that increase the meaning of present research experience and the contributions it makes. I discuss acts of conjecture, differentiation, attachment, affirmation, complication, discernment, interruption, and representation to illustrate that meaningful contributions are generated by actions associated with connecting perceptions to concepts. ASQ’s 60th anniversary is an opportune time to make these interim contributions more explicit.

This descriptive study explored how teachers implemented interactive practices during their vocabulary instruction after participating in a professional development initiative. The professional development was comprised of an initial workshop to provide an overview of vocabulary research and instruction, opportunities for teachers to receive targeted feedback on audiotaped classroom lessons, and an interim session where teachers discussed the process of implementation. Data were gathered through interviews and lesson transcripts. Teachers reported that they valued the feedback on classroom lessons and meetings with grade level colleagues. In addition, they reported positive changes in their instruction as a result of the professional development activities. Lesson transcripts showed that the instructional changes teachers reported corresponded to changes between pre- and post-intervention lesson transcripts.

Background Natalizumab is an effective treatment for relapsing multiple sclerosis. Return of disease activity upon natalizumab discontinuance creates the need for follow-up therapeutic strategies. Objective To assess the efficacy of teriflunomide following natalizumab discontinuance in relapsing multiple sclerosis patients. Methods Clinically stable relapsing multiple sclerosis patients completing 12 or more consecutive months of natalizumab, testing positive for anti-John Cunningham virus antibody, started teriflunomide 14 mg/day, 28 ± 7 days after their final natalizumab infusion. Physical examination, Expanded Disability Status Scale, laboratory assessments, and brain magnetic resonance imaging were performed at screening and multiple follow-up visits. Results Fifty-five patients were enrolled in the study. The proportion of patients relapse-free was 0.94, restricted mean time to first gadolinium-enhancing lesion was 10.9 months and time to 3-month sustained disability worsening was 11.8 months. The mean number of new or enlarging T2 lesions per patient at 12 months was 0.42. Exploratory analyses revealed an annualized relapse rate of 0.08, and a proportion of patients with no evidence of disease activity of 0.68. Forty-seven patients (85.5%) reported adverse events, 95% of which were mild to moderate. Conclusions Teriflunomide therapy initiated without natalizumab washout resulted in a low rate of return of disease activity. Clinicians may consider this a worthwhile strategy when transitioning clinically stable patients off natalizumab to another therapy. ClinicalTrials.gov Identifier: NCT01970410

In any clinical trial, it is essential to monitor the accumulating data to be sure that the trial continues to be safe for participants and that the trial is being conducted properly. Data monitoring committees, independent expert panels who undertake regular reviews of the data as the trial progresses, serve an important role in safeguarding the interests of research participants and ensuring trial integrity in many trials. Many pragmatic clinical trials, which aim to inform healthcare decisions by comparing alternate interventions in heterogeneous healthcare delivery settings, will warrant review by an independent data monitoring committee due to their potential impact on clinical practice. However, the very features that make a trial “pragmatic” may pose challenges in terms of which aspects of a trial to monitor and when it is appropriate for a data monitoring committee to intervene. Using the Pragmatic-Explanatory Continuum Indicator Summary tool that draws distinctions between pragmatic and explanatory clinical trials, we review characteristics of pragmatic clinical trials that may have implications for data monitoring committees and interim monitoring plans. These include broad eligibility criteria, a focus on subjective patient-centered outcomes, and in some cases a lack of standardized follow-up procedures across study sites. Additionally, protocol adherence is often purposefully not addressed in pragmatic trials in order to accurately represent the clinical practice setting and maintain practicability of implementation; there are differing viewpoints as to whether adherence should be assessed and acted upon by data monitoring committees in these trials. Some other issues not specifically related to the Pragmatic-Explanatory Continuum Indicator Summary criteria may also merit special consideration in pragmatic trials. Thresholds for early termination of a pragmatic clinical trial might be controversial. The distinguishing features of pragmatic clinical trials require careful consideration when developing interim data monitoring plans, and trial sponsors, investigators, and data monitoring committees should agree on a plan before trial inception. Finally, special expertise, such as an informatics, may be helpful on data monitoring committees for some pragmatic clinical trials. Patient representatives may provide particularly valuable insights in the monitoring process.

Pilot studies and other small clinical trials are often conducted but serve a variety of purposes and there is little consensus on their design. One paradigm that has been suggested for the design of such studies is Bayesian decision theory. In this article, we review the literature with the aim of summarizing current methodological developments in this area. We find that decision-theoretic methods have been applied to the design of small clinical trials in a number of areas. We divide our discussion of published methods into those for trials conducted in a single stage, those for multi-stage trials in which decisions are made through the course of the trial at a number of interim analyses, and those that attempt to design a series of clinical trials or a drug development programme. In all three cases, a number of methods have been proposed, depending on the decision maker’s perspective being considered and the details of utility functions that are used to construct the optimal design. EU project InSPiRe (Innovative methodology for small populations research); FP HEALTH 2013 – 602144

This article focuses on the role the African and Inter-American human rights systems in protecting the rights of detainees through interim measures. The need for regional human rights to complement protection at the national level is all the more pronounced when it comes to detainees, because they often lack democratic leverage at the national level. While complementing the universal system, regional systems have the added benefit of closer connection and binding judgments. Because the situation of detainees often requires urgent intervention, interim measures are required to ensure that detainees do not suffer undue harm. A comparative survey of interim measures in the two regional systems reveals that, while these measures had been issued in both systems, the Inter-American has used them much more frequently. One of the reasons for this difference is the fact that the Inter-American system does not make the submission of a formal complaint a prerequisite for issuing interim measures. The African Commission should amend its Rules of Procedure to allow for a similar possibility; and it should devote more resources and institutionalize this procedure. Despite the inclusion of collective rights in the African system, the African Commission has infrequently dealt with urgent measures affecting groups of detainees. Both systems suffer from some defects. There is a lack of substantiated reasoning in decisions dealing with interim measures, especially when requests are rejected; information is not made available systematically about the fate of all requests for interim measures; and a lack of compliance besets both systems.

In this issue, the Research Report, entitled "Results of an Online Refresher Course to Build Braille Transcription Skills in Professionals and Volunteers," by Herzberg and Rosenblum, uses pretest and posttest scores on a 20-item multiple-choice test to compare the participants' knowledge before and after taking an online course. In this Statistical Sidebar, I will discuss the realm of pretest-posttest measures as a way to assess change. This approach is used frequently in scholarly research, given the logic that measuring performance or knowledge before some sort of training and then measuring it again after the instruction should provide an idea of the effectiveness of the intervention. The method of using pretest-posttest measures, however, can be subject to a number of subtle flaws. As in any experimental design, random assignment of participants to experimental and control groups makes the study stronger (for more information on this topic, please see the Statistical Sidebar, "Control Groups and Experimental Groups: It Is All in the Numbers," which was published in the January-February 2015 issue). Unfortunately, it is not always possible to use random assignment, and even when participants are randomly placed in experimental and control groups, the pretest-posttest design is subject to threats to internal validity. The particular type of pretest-posttest design used in this report was a "one group pre-posttest design," in which there was no control group and, therefore, no random assignment of participants. Although the pretest-posttest model is quite common, this type of research design is open to threats of history, maturation, testing, and instrumentation. Since time passes between the pretest and posttest measures, participants experience other events in the interim. This threat of "history" means that participants might be influenced by factors outside of the study that affect their performance on the posttest. If enough time passes between the pretest and posttest, then the threat of "maturation" might be a factor, in which biological changes in the participants might influence posttest scores. There is also the possibility that, since the pretest and posttest are measuring the same knowledge or skill, the act of taking the pretest elevates knowledge or performance, meaning that the training is not fully responsible for the measured change. If the exact same assessment is used for both the pretest and the posttest, when participants take the posttest, they have already answered the questions or performed the skills it requires, which might artificially inflate performance on the posttest. …