Search

In the context of the current “replication crisis” across the sciences, failures to reproduce a finding are often viewed as discrediting it. This paper shows how such a conclusion can be incorrect. In 1981, Schuman and Presser showed that including the word “freedom” in a survey question significantly increased approval of allowing a speech against religion in the USA. New experiments in probability sample surveys (n = 23,370) in the USA and 10 other countries showed that the wording effect replicated in the USA and appeared in four other countries (Canada, Germany, Taiwan, and the Netherlands) but not in the remaining countries. The effect appeared only in countries in which the value of freedom is especially salient and endorsed. Thus, public support for a proposition was enhanced by portraying it as embodying a salient principle of a nation’s culture. Instead of questioning initial findings, inconsistent results across countries signal limits on generalizability and identify an important moderator.

The day of the week on which sample members are invited to participate in a web survey might influence propensity to respond, or to respond promptly (within two days from the invitation). This effect could differ between sample members with different characteristics. We explore such effects using a large-scale experiment implemented on the Understanding Society Innovation Panel, in which some people received an invitation on a Monday and some on a Friday. Specifically, we test whether any effect of the invitation day is moderated by economic activity status (which may result in a different organisation of time by day of the week), previous participation in the panel, or whether the invitation was sent only by post or by post and email simultaneously. Overall, we do not find any effect of day of invitation in survey participation or in prompt participation. However, sample members who provided an email address, and, thus, were contacted by email in addition to postal letter, are less likely to participate if invited on Friday (email reminders: Sunday and Tuesday) as opposed to Monday (email reminders: Wednesday and Friday). Given that no difference between the two protocols is found for prompt response, the effect seems to be due to the day of mailing of reminders. With respect to sample members' economic activity status, those not having a job and the retired are less likely to participate when invited on a Friday; this result holds also for prompt participation, but only for retired respondents. Also, sample members who work long hours are less likely to participate when invited on a Friday; however, no effect is found for prompt response.

Background To our knowledge, we report the first case of endocarditis with root abscess causing compressive superior vena cava (SVC) obstruction. Case summary An 84-year-old gentleman with previous tissue aortic valve replacement presented with fevers and systemic upset. Blood cultures grew Streptococcus anginosus and transoesophageal echocardiogram identified prosthetic valve vegetations with an associated root abscess. Antibiotics were commenced and referral made for surgical consideration. Several days into treatment the patient developed clinical signs of SVC obstruction and computed tomography demonstrated an enlarging root abscess with SVC compression. The patient was discussed with local cardiothoracic centres, but surgery was not an option primarily due to abscess size and vascular involvement. Priority moved from active to palliative treatment given no improvement with antibiotics, unsuitability for surgery, and patient discomfort. Within several weeks, symptoms/signs of SVC obstruction actually improved, likely due to collateral venous circulation formation and the patient was discharged home with palliative care input. Discussion There are previous reports of SVC obstruction related to infected SVC thrombus, indwelling intravascular devices, and para-aortic abscess, but none related to infective endocarditis. Streptococcus anginosus endocarditis is rare but often associated with abscess formation, and male gender, increasing age, and previous surgery are recognized risk factors.

The article examines two important aspects of data quality in self-completion surveys of young people, taking advantage of a unique data source: Understanding Society: the United Kingdom Household Longitudinal Study. Young persons aged 10–15 are asked to complete a self-administered paper questionnaire at annual intervals. The number of completed interviews varies over waves from 4,049 to 5,020. Data are also collected from parents, providing important explanatory covariates for our analysis. Stronger parent–child relationship and higher mother’s involvement in education were associated with lower item nonresponse rate and lower inconsistency throughout waves. We also found some evidence for a negative panel conditioning effect with an increase of social desirability bias and measurement errors in the subsequent waves. There was a higher level of inconsistent responses and a higher probability of social desirability bias throughout waves in more sensitive items.

Prospect theory states that the influential power of avoiding negative outcomes is stronger than that of achieving positive outcomes. In a survey context, this theory has been tested with respect to not only participation in a CATI survey, but also giving consent to data linkage in CATI surveys. No study, however, has tested the theory with respect to participation in a CAPI or web survey. This study does so in a mixed-mode panel context; it also tests the moderating effects of time-in-panel, response history, and mode protocol. Results show that the framing of the survey participation request influences participation propensity in a way consistent with prospect theory, but only for relatively recent panel entrants. The opposite effect is found for long-term panel participants. No difference is found between mode protocols.

Complex sampling designs involving features such as stratification, cluster sampling, and unequal selection probabilities are often used in large-scale longitudinal surveys to improve cost-effectiveness and ensure adequate sampling of small or under-represented groups. However, complex sampling designs create challenges when there is a need to account for non-random attrition; a near inevitability in social science longitudinal studies. In this article we discuss these challenges and demonstrate the application of weighting approaches to simultaneously account for non-random attrition and complex design in a large UK-population representative survey. Using an auto-regressive latent trajectory model with structured residuals (ALT-SR) to model the relations between relationship satisfaction and mental health in the Understanding Society study as an example, we provide guidance on implementation of this approach in both R and Mplus is provided. Two standard error estimation approaches are illustrated: pseudo-maximum likelihood robust estimation and Bootstrap resampling. A comparison of unadjusted and design-adjusted results also highlights that ignoring the complex survey designs when fitting structural equation models can result in misleading conclusions.

This article is concerned with the extent to which the propensity to participate in a web face-to-face sequential mixed-mode survey is influenced by the ability to communicate with sample members by e-mail in addition to mail. Researchers may be able to collect e-mail addresses for sample members and to use them subsequently to send survey invitations and reminders. However, there is little evidence regarding the value of doing so. This makes it difficult to decide what efforts should be made to collect such information and how to subsequently use it efficiently. Using evidence from a randomized experiment within a large mixed-mode national survey, we find that using a respondent-supplied e-mail address to send additional survey invites and reminders does not affect survey response rate but is associated with an increased proportion of responses by web rather than face-to-face and, hence, lower survey costs.

Adaptive survey designs can be used to allocate sample elements to alternative data collection protocols in order to achieve a desired balance between some quality measure and survey costs. We compare four alternative methods for allocating sample elements to one of two data collection protocols. The methods differ in terms of the quality measure that they aim to optimize: response rate, R-indicator, coefficient of variation of the participation propensities, or effective sample size. Costs are also compared for a range of sample sizes. The data collection protocols considered are CAPI single-mode and web-CAPI sequential mixed-mode. We use data from a large experiment with random allocation to one of these two protocols. For each allocation method we predict outcomes in terms of several quality measures and costs. Although allocating the whole sample to single-mode CAPI produces a higher response rate than allocating the whole sample to the mixed-mode protocol, we find that two of the targeted allocations achieve a better response rate than single-mode CAPI at a lower cost. We also find that all four of the targeted designs out-perform both single-protocol designs in terms of representativity and effective sample size. For all but the smallest sample sizes, the adaptive designs bring cost savings relative to CAPI-only, though these are fairly modest in magnitude.

Sequential mixed-mode designs are increasingly considered as an alternative to interviewer-administered data collection, allowing researchers to take advantage of the benefits of each mode. We assess the effects of the introduction of a sequential web-face-to-face mixed-mode design over three waves of a longitudinal survey in which members were previously interviewed face-to-face. Findings are reported from a large-scale randomised experiment carried out on the UK Household Longitudinal Study. No differences are found between the mixed-mode design and face-to-face design in terms of cumulative response rates and only minimal differences in terms of sample composition. On the other hand, potential cost savings are evident.

In multi-national surveys, different countries usually implement different sample designs. The sample designs affect the variance of estimates of differences between countries. When making such estimates, analysts often fail to take sufficient account of sample design. This failure occurs sometimes because variables indicating stratification, clustering, or weighting are unavailable, partially available, or in a form that is unsuitable for cross-national analysis. In this article, we demonstrate how complex sample design should be taken into account when estimating differences between countries, and we provide practical guidance to analysts and to data producers on how to deal with partial or inappropriately-coded sample design indicator variables. Using EU-SILC as a case study, we evaluate the inverse misspecification effect (imeff ) that results from ignoring clustering or stratification, or both in a between-country comparison where countries’ sample designs differ. We present imeff for estimates of between-country differences in a number of demographic and economic variables for 19 European Union Member States. We assess the magnitude of imeff and the associated impact on standard error estimates. Our empirical findings illustrate that it is important for data producers to supply appropriate sample design indicators and for analysts to use them.

Background Improving response rates in epidemiologic studies is important for the generalizability of the outcome. The aim of this study was to examine whether it can be advantageous for participation to target different versions of the cover letters to different sample subgroups. Methods A randomized trial was incorporated in a cross-sectional health survey in Denmark (n = 25,000) where a motivational sentence in the cover letter intended to heighten perceptions of relevance of the survey was varied among 11 sample subgroups (treatment groups). Ten different versions of a sentence outlining questionnaire themes were tested: each mentioned three out of five themes: stress, alcohol, sex, sleep problems, and contact with family and friends. An eleventh group, the control group, omitted this sentence. Results On average, the additional motivational sentence resulted in a significantly lower response rate overall compared to the control group. However, the additional motivational sentence was found to have heterogeneous effects on survey response. Furthermore, the nature of the heterogeneity differed between the versions of the sentence. Specifically, the additional sentence tended to produce a higher response rate among the youngest age group and a lower response rate in the oldest age group compared to the generic letter. The use of alcohol in the motivational sentence tended to have a positive effect on response in the age group 16–24 years, and stress tended to have a positive effect in the age group ≥65 years. On the contrary, sex tended to have a negative effect in the age groups 45–64 years and ≥ 65 years. However, a significant interaction was only found between the use of stress and age group (p =

Previous research into survey invitation letters and prenotification letters has sought to identify optimal features of a standard letter that can be sent to all sample members. In the spirit of adaptive design, this study seeks to establish whether it can be advantageous to target different versions of a letter to different sample subgroups. Specifically, a paragraph intended to heighten perceptions of relevance of the survey is varied among six subgroups in a panel survey. Random allocation to control and treatment groups is crossed with variants of two other design features: time in sample and data collection mode. This enables analysis of the effects of the targeted letter in different survey contexts. The targeted version of the letter improved response rates for two operationally important low-response propensity groups, but only in certain survey design contexts.

This is a protocol for a Cochrane Review (Methodology). The objectives are as follows: To evaluate the effectiveness of design-based methods to influence the completeness of item response to self-administered questionnaires. This will be achieved by assessing the effects of aspects of style, appearance and layout of self-administered questionnaires on the proportion of items completed in returned questionnaires.

Linking survey responses and the electoral register is a core part of data collection in political science. This experiment shows that there is no difference in consent rates if the motivation for data linkage is changed to a more truthful one. This is encouraging for researchers of registration and turnout; survey respondents do not object to their data being linked for checking potential errors in their reporting. It also is consistent with the idea that consent to link survey data to information from the electoral register is driven by a consideration of the linkage itself, rather than being overly reliant on question wording. This view is supported by the additional finding that there is also no difference in the rates of opt-out consent between two alternate motivations. The second finding at this stage is that far fewer people opt out of consent than refuse their consent when they are asked to opt in.

Background: Surveys of immigrants face challenges of coverage, representativeness, and response rates. Longitudinal studies of immigrants and ethnic minorities, which have potential to address pressing issues in demographic research, are rare or partial. In the absence of register data, the highest quality approach is argued to be probability sampling using household screening. Objective: To describe the design and implementation of a nationally representative probability sample of immigrants and ethnic minorities in the United Kingdom. Methods: We boosted a nationally representative sample by using small-area census data to identify areas that covered the majority of immigrant and target ethnic minority populations and oversampled addresses from those areas using varying sampling fractions. Households were screened for eligibility based on whether they included a target immigrant/ethnic minority member. If so, all adult members were interviewed. Results: We anticipated the main challenges would be: fewer eligible households than predicted in sampled areas due to geographical mobility; refusal of those screened to provide information on household eligibility; nonparticipation of eligible households. All these issues were found to some degree. We describe how we addressed them and with what success. Conclusions: A careful design and robust fieldwork practices can enable a two-stage probability sampling to achieve good coverage and a much more representative sample of immigrants and ethnic minorities than with more ad hoc methods. The potential research payoffs are substantial. Contribution: We demonstrate the potential for careful two-stage sampling on the back of an existing study for creating a high-quality multi-purpose survey of immigrants.

Purpose Successes in care of congenital heart disease (CHD) have resulted in an increasing number of our patients considering pregnancy. For some, pregnancy could be a significant undertaking. We believe that we counsel our patients about contraception. Our aim was to assess contraception advice from a patient’s perspective. Methods An audit questionnaire was designed and given to women of child-bearing age attending the Scottish Adult Congenital Cardiology outpatient clinic. Results 20 questionnaires were returned (mean age: 27 years). 17 (85%) were currently sexually active, 2 (10%) declined to comment. Contraceptive methods frequency is demonstrated in the table below: 17 (85%) were either unsure or not actively planning a pregnancy. 8 (40%) were unsure if their current medications were safe to continue if they were to become pregnant. 6 (30%) had previously been informed pregnancy was not advisable and all were using some form of contraception. GPs were the most common source of contraceptive information (n=10 [50%]). Only 9 (45%) recalled receiving contraceptive advice from our team. 19 (95%) thought specific information on contraception for women with CHD would be useful. Conclusion Compared with our audit completed in 2011 there was an increased retention of contraceptive advice. For further improvement, we have developed more accessible information about safe and effective methods of contraception for adults with CHD and will repeat the audit.

Longitudinal studies have a number of challenges in terms of data collection and analysis including sample attrition, panel conditioning, coverage error, time and cost. In addition, variability exists among European nations as to the availability and coverage of sampling frames, laws and regulations that restrict aspects of survey practice, availability and capacity of survey research organisations, cultural and behavioural norms, language(s) spoken, geographical dispersal of the study populations. Furthermore, research involving children and young people requires ethical considerations including how the children and young people can and should be involved in a study with reference to the degree of control and participation that they have.

A 78-year-old female underwent a right pneumonectomy for a neuroendocrine tumour. In the early postoperative phase, she developed a paralytic ileus and went on to develop breathlessness and orthostatic desaturation, which characterizes platypnoea–orthodeoxia. She was found to have an intra-atrial shunt on bubble echo, with equal atrial pressures. This was managed by device closure of the atrial connection, following which her condition improved.

This special issue is devoted to discussion of probability-based survey panels that collect data either solely or partly through online questionnaires. Panels of this kind have been around for a long time, though they have been few in number, but recent years have seen several new panels start up in Europe. This has led to renewed interest in the methodology of such panels and also to deeper questioning of the role of these panels. On one hand, the probability-based panels are to some extent competing against cheaper non-probability access panels. On the other hand, the probability-based panels are increasingly being seen as possible alternatives to more expensive probability-based survey methods. In both cases, clients and data users want to better understand the relative advantages and disadvantages of the probability-based panels.

Web questionnaires, including those used in mixed-mode surveys, generally produce higher levels of item nonresponse than interviewer-administered questionnaires. Item nonresponse is generally seen as having a detrimental impact on data quality. The current study examines using motivational statements to reduce item nonresponse in a web survey component of a mixed-mode design. The effects of alternative implementations are compared, both for web surveys and for mixed-mode surveys. In addition, the mixed-mode results are compared to a face-to-face survey. The current study adds to the literature on the use of motivational statements by using a unique large-scale randomized experiment to examine the impact of the timing of the motivational statement, and to compare with the same survey in an interviewer-administered context. Findings show that a motivational statement following immediately after an item is left unanswered greatly outperforms either the control or a motivational statement at a later point in the survey. Using this immediate reactive prompt reduces item nonresponse to levels equivalent to a face-to-face version. Conversely, the control (no statement) and later placed motivational statement lead to significantly greater item nonresponse. Point estimates for the tested variables are not affected by the additional responses obtained. The results suggest practical design implications to reduce item nonresponse when using a web design, specifically the use of a reactive motivational prompt in a planned way.

Over time, some respondents have been recruited to participate in longitudinal research may stop responding to survey requests. This phenomenon is known as panel attrition and it occurs when a longitudinal panel fails to retain all recruited members. While there are many potential causes of this attrition, there are a number of concrete steps that researchers that manage panels can take to tackle the problem of panel attrition. This chapter provides an overview of the causes and consequences of panel attrition. It also highlights strategies that may be effective in addressing this problem by reviewing innovations and best practices for minimizing panel attrition and also reducing the statistical impacts of attrition when it does occur.

This article describes the application of the total survey error paradigm to longitudinal surveys. Several aspects of survey error, and of the interactions between different types of error, are distinct in the longitudinal survey context. Furthermore, error trade-off decisions in survey design and implementation are subject to some unique considerations. Previous literature on total survey error mostly fails to explicitly consider uniquely longitudinal issues. We aim to show how uniquely longitudinal sources of error in surveys should be understood within the Total Survey Error framework and we provide examples of studies of some of the unique interactions between errors.

We assess whether the probability of a sample member cooperating at a particular wave of a panel survey is greater if the same interviewer is deployed as at the previous wave. Previous research on this topic mainly uses nonexperimental data. Consequently, a) interviewer change is generally nonrandom, and b) continuing interviewers are more experienced by the time of the next wave. Our study is based on a balanced experiment in which both interviewer continuity and experience are controlled. Multilevel multiple membership models are used to explore the effects of interviewer continuity on refusal rate as well as interactions of interviewer continuity with other variables. We find that continuity reduces refusal propensity for younger respondents but not for older respondents, and that this effect depends on the age of the interviewer. This supports the notion that interviewer continuity may be beneficial in some situations, but not necessarily in others.

This article considers the effect that a wave of mixed-mode data collection (telephone and face-to-face), in an otherwise face-to-face survey, can have on panel attrition and the extent to which this effect is dependent on the nature of the mode-switch protocol. Findings are reported from an experiment carried out at wave 2 of the UK Household Longitudinal Study Innovation Panel, a survey in which the objective is to interview each adult member of the household. One protocol involves making intensive efforts to interview each household member by telephone before switching to face-to-face, and the other involves switching a household to face-to-face as soon as it is apparent that an interviewer visit will be needed for at least one household member. I assess effects on response at waves 2, 3, and 4. With both protocols response rate is lower at wave 2 than with face-to-face single-mode data collection, but with the protocol involving intensive efforts, this response differential is eroded by wave 4, whereas with the other protocol the difference remains. This difference in effect may be caused by intra-household communications.

We examine the effects on survey estimates of extended interviewer efforts to gain survey response, including refusal conversion attempts and attempts to make contact with difficult-to-contact sample members. Previous research on this topic has identified that extended efforts do appear to affect estimates, and in ways that seem consistent with bias reduction. We extend the previous research in three ways. First, we provide the first study of changes over time in the effects of extended efforts on estimates. We study change in the UK over a ten-year period. Second, we use a more precise measure of the difficulty of contact and third, we assess the effects of extended efforts conditional on weight adjustments for non-response estimates as well as on unweighted sample statistics.

Sampling is a complex discipline, yet it is of primary importance in many studies. Sampling methods for scientific study should be objective and should maximise accuracy of estimation. Scientific sampling is to provide a means of making inferences about the population of interest using observations made on the sample. There are two basic approaches to inference such as the design‐based approach, and the model‐based approach. With a much larger sample, it may be more difficult to find enough high‐quality interviewers and infeasible to brief them personally. The procedures and mechanisms that collectively constitute the method of sample selection are known as the sample design. Probability sampling or random sampling is often thought the only defensible selection method for serious scientific study unless it is simply not feasible. An important element of sample design is the determination of the sample size. Adaptive sampling is frequently used to sample rare or elusive populations.