Search

In an opening paper Leon Feinstein reviews methodological criticism of his influential research into the relationship between early cognitive development and socioeconomic grades, based on UK 1970 Birth Cohort Study data. The points he raises are then debated in commentaries by John Jerrim and Anna Vignoles, Harvey Goldstein and Robert French, Elizabeth Washbrook and RaeHyuck Lee and Ruth Lupton.Leon Feinstein's response to these comments will be published in the next issue of the journal.

OBJECTIVES: Our aim was to estimate the rate of data linkage error in Hospital Episode Statistics (HES) by testing the HESID pseudoanonymisation algorithm against a reference standard, in a national registry of paediatric intensive care records. SETTING: The Paediatric Intensive Care Audit Network (PICANet) database, covering 33 paediatric intensive care units in England, Scotland and Wales. PARTICIPANTS: Data from infants and young people aged 0-19 years admitted between 1 January 2004 and 21 February 2014. PRIMARY AND SECONDARY OUTCOME MEASURES: PICANet admission records were classified as matches (records belonging to the same patient who had been readmitted) or non-matches (records belonging to different patients) after applying the HESID algorithm to PICANet records. False-match and missed-match rates were calculated by comparing results of the HESID algorithm with the reference standard PICANet ID. The effect of linkage errors on readmission rate was evaluated. RESULTS: Of 166,406 admissions, 88,596 were true matches (where the same patient had been readmitted). The HESID pseudonymisation algorithm produced few false matches (n=176/77,810; 0.2%) but a larger proportion of missed matches (n=3609/88,596; 4.1%). The true readmission rate was underestimated by 3.8% due to linkage errors. Patients who were younger, male, from Asian/Black/Other ethnic groups (vs White) were more likely to experience a false match. Missed matches were more common for younger patients, for Asian/Black/Other ethnic groups (vs White) and for patients whose records had missing data. CONCLUSIONS: The deterministic algorithm used to link all episodes of hospital care for the same patient in England has a high missed match rate which underestimates the true readmission rate and will produce biased analyses. To reduce linkage error, pseudoanonymisation algorithms need to be validated against good quality reference standards. Pseudonymisation of data 'at source' does not itself address errors in patient identifiers and the impact these errors have on data linkage.

SummaryA striking feature of UK public services in the 1990s was the rise of performance monitoring (PM), which records, analyses and publishes data in order to give the public a better idea of how Government policies change the public services and to improve their effectiveness.PM done well is broadly productive for those concerned. Done badly, it can be very costly and not merely ineffective but harmful and indeed destructive.Performance indicators (PIs) for the public services have typically been designed to assess the impact of Government policies on those services, or to identify well performing or underperforming institutions and public servants. PIs’ third role, which is the public accountability of Ministers for their stewardship of the public services, deserves equal recognition. Hence, Government is both monitoring the public services and being monitored by PIs.Especially because of the Government's dual role, PM must be done with integrity and shielded from undue political influence, in the way that National Statistics are shielded. It is in everyone's interest that Ministers, Parliament, the professions, practitioners and the wider public can have confidence in the PM process, and find the conclusions from it convincing.Before introducing PM in any public service, a PM protocol should be written. This is an orderly record not only of decisions made but also of the reasoning or calculations that led to those decisions. A PM protocol should cover objectives, design considerations and the definition of PIs, sampling versus complete enumeration, the information to be collected about context, the likely perverse behaviours or side-effects that might be induced as a reaction to the monitoring process, and also the practicalities of implementation. Procedures for data collection, analysis, presentation of uncertainty and adjustment for context, together with dissemination rules, should be explicitly defined and reflect good statistical practice. Because of their usually tentative nature, PIs should be seen as ‘screening devices’ and not overinterpreted. If quantitative performance targets are to be set, they need to have a sound basis, take account of prior (and emerging) knowledge about key sources of variation, and be integral to the PM design.Aspirational targets have a distinctive role, but one which is largely irrelevant in the design of a PM procedure; motivational targets which are not rationally based may demoralize and distort. Anticipated and actual side-effects of PM, including on individuals’ behaviour and priorities, may need to be monitored as an intrinsic part of the PM process.Independent scrutiny of PM schemes for the public services should be set up and must report publicly. The extent and nature of this scrutiny should be related to the assessed drawbacks and benefits, reflect ethical concerns, and conform with good statistical practice.Research is needed into the merits of different strategies for identifying institutions or individuals in the public release of PM data, into how new PM schemes should be evaluated, and into efficient designs for evaluating a series of new policies which are monitored by PIs.The Royal Statistical Society considers that attempts to educate the wider public, as well as policy makers, about the issues surrounding the use of PIs are very important. High priority should be given to sponsoring well-informed public debate, and to disseminating good practices by implementing them across Government.

Background: Congenital heart defects (CHDs) are a significant cause of death in infancy. Although contemporary management ensures that 80% of affected children reach adulthood, post-infant mortality and factors associated with death during childhood are not well-characterised. Using data from a UK-wide multicentre birth cohort of children with serious CHDs, we observed survival and investigated independent predictors of mortality up to age 15 years. Methods: Data were extracted retrospectively from hospital records and death certificates of 3,897 children (57% boys) in a prospectively identified cohort, born 1992–1995 with CHDs requiring intervention or resulting in death before age one year. A discrete-time survival model accounted for time-varying predictors; hazards ratios were estimated for mortality. Incomplete data were addressed through multilevel multiple imputation. Findings: By age 15 years, 932 children had died; 144 died without any procedure. Survival to one year was 79.8% (95% confidence intervals [CI] 78.5, 81.1%) and to 15 years was 71.7% (63.9, 73.4%), with variation by cardiac diagnosis. Importantly, 20% of cohort deaths occurred after age one year. Models using imputed data (including all children from birth) demonstrated higher mortality risk as independently associated with cardiac diagnosis, female sex, preterm birth, having additional cardiac defects or non-cardiac malformations. In models excluding children who had no procedure, additional predictors of higher mortality were younger age at first procedure, lower weight or height, longer cardiopulmonary bypass or circulatory arrest duration, and peri-procedural complications; non-cardiac malformations were no longer significant. Interpretation: We confirm the high mortality risk associated with CHDs in the first year of life and demonstrate an important persisting risk of death throughout childhood. Late mortality may be underestimated by procedure-based audit focusing on shorter-term surgical outcomes. National monitoring systems should emphasise the importance of routinely capturing longer-term survival and exploring the mechanisms of mortality risk in children with serious CHDs.

BACKGROUND: Linkage of electronic healthcare records is becoming increasingly important for research purposes. However, linkage error due to mis-recorded or missing identifiers can lead to biased results. We evaluated the impact of linkage error on estimated infection rates using two different methods for classifying links: highest-weight (HW) classification using probabilistic match weights and prior-informed imputation (PII) using match probabilities. METHODS: A gold-standard dataset was created through deterministic linkage of unique identifiers in admission data from two hospitals and infection data recorded at the hospital laboratories (original data). Unique identifiers were then removed and data were re-linked by date of birth, sex and Soundex using two classification methods: i) HW classification - accepting the candidate record with the highest weight exceeding a threshold and ii) PII-imputing values from a match probability distribution. To evaluate methods for linking data with different error rates, non-random error and different match rates, we generated simulation data. Each set of simulated files was linked using both classification methods. Infection rates in the linked data were compared with those in the gold-standard data. RESULTS: In the original gold-standard data, 1496/20924 admissions linked to an infection. In the linked original data, PII provided least biased results: 1481 and 1457 infections (upper/lower thresholds) compared with 1316 and 1287 (HW upper/lower thresholds). In the simulated data, substantial bias (up to 112%) was introduced when linkage error varied by hospital. Bias was also greater when the match rate was low or the identifier error rate was high and in these cases, PII performed better than HW classification at reducing bias due to false-matches. CONCLUSIONS: This study highlights the importance of evaluating the potential impact of linkage error on results. PII can help incorporate linkage uncertainty into analysis and reduce bias due to linkage error, without requiring identifiers.