Search

With the rise of mobile surveys comes the need for shorter questionnaires. We investigate the modularization of an existing questionnaire in the Longitudinal Internet Study for the Social Sciences (LISS) Panel in the Netherlands. We randomly divided respondents into a normal length survey condition, a condition where the same survey was split into 3 parts, and a condition where the survey was split into 10 parts. Respondents received the parts consecutively at regular intervals over a 1-month period. We discuss response rates, data quality measures, and respondents’ evaluation of the questionnaire. Our results indicate higher start rates when the survey is cut into smaller parts but also higher dropout rates. However, the fraction of missing information is lower in the 3- and 10-part conditions. More respondents use their mobile phone for survey completion when the survey is shorter. We find fewer item missings and satisficing in shorter surveys. We find no effect on neutral and extreme responding nor on estimates of the validity of answers. People with low and high education and young and old evaluate shorter surveys better than the normal length survey.

In this paper we discuss the implications of using mobile devices for online survey completion. With more and more people accessing online surveys on mobile devices, online surveys need to be redesigned in order to be able to meet the characteristics of mobile device usage, such as small screens and short messaging. We discuss mobile friendly design by focussing on survey layout, the length of the survey, special features, and the decision of making the survey app or browser based. Further, we discuss the different sensors that can be used to augment or replace survey questions, and respondents’ willingness to share sensor data. We end with three examples of surveys conducted by Statistics Netherlands, where sensors are used for active and passive measurement in mobile surveys.

Smartphones enable passive collection of sensor data alongside survey participation. Location data add context to people’s reports about their time use. In addition, linking global positioning system data to self-reported time use surveys (TUSs) can be valuable for understanding how people spend their time. This article investigates whether and how passive collection of geographical locations (coordinates) proves useful for deriving respondents’ functional locations. Participants of the ongoing Children of Immigrants Longitudinal Survey in the Netherlands were invited to participate in a TUS administered with a smartphone app that also unobtrusively tracked respondents’ locations. Respondents reported their activities per 10-min interval in a smartphone diary app ( n = 1,339) and shared their geographical location data ( n = 1,264). The correspondence between the functional locations derived from the time use data and those derived from the geographical location data was assessed by calculating the percentage of intervals in which both measures are similar. Overall, results show that home locations can be automatically assigned reliably but that respondent information is required to reliably assign work or school locations. In addition, location tracking data contain many measurement errors, making it difficult to record valid locations. Multilevel models show that the variability in correct classifications is intrapersonal and largely predicted by phone type, which determines location measurement frequency.

The increasing use of smartphones opens up opportunities for novel ways of survey data collection, but also poses new challenges. Collecting more and different types of data means that studies can become increasingly intrusive. We risk over-asking participants, leading to nonresponse. This study documents nonresponse and nonresponse bias in a smartphone-only version of the Dutch Time Use Survey (TUS). Respondents from the Dutch LISS panel were asked to perform five sets of tasks to complete the whole TUS: 1) accept an invitation to participate in the study and install an app, 2) fill out a questionnaire on the web, 3) participate in the smartphone time use diary on their smartphone, 4) answer pop-up questions and 5) give permission to record sensor data (GPS locations and call data). Results show that 42.9% of invited panel members responded positively to the invitation to participate in a smartphone survey. However, only 28.9% of these willing panel members completed all stages of the study. Predictors of nonresponse are somewhat different at every stage. In addition, respondents who complete all smartphone tasks are different from groups who do not participate at some or any stage of the study. By using data collected in previous waves we show that nonresponse leads to nonresponse bias in estimates of time use. We conclude by discussing implications for using smartphone apps in survey research., Survey Research Methods, Vol 13 No 2 (2019)

In this study, we investigate whether mobile device use in surveys can be predicted. We aim to identify possible motives for device use and build a model by drawing on theory from technology acceptance research and survey research. We then test this model with a Structural Equation Modeling approach using data of seven waves of the GESIS panel. We test whether our theoretical model fits the data by focusing on measures of fit, and by studying the standardized effects of the model. Results reveal that intention to use a particular device can predict actual use quite well. Ease of smartphone use is the most meaningful variable: if people use a smartphone for specific tasks, their intention to use a smartphone for survey completion is also more likely. In conclusion, investing in ease of use of mobile survey completion could encourage respondents to use mobile devices. This can foremost be established by building well-designed surveys for mobile devices.

With the rise of mobile surveys comes the need for shorter questionnaires. We investigate the modularization of an existing questionnaire in the Longitudinal Internet Study for the Social Sciences (LISS) Panel in the Netherlands. We randomly divided respondents into a normal length survey condition, a condition where the same survey was split into 3 parts, and a condition where the survey was split into 10 parts. Respondents received the parts consecutively at regular intervals over a 1-month period. We discuss response rates, data quality measures, and respondents’ evaluation of the questionnaire. Our results indicate higher start rates when the survey is cut into smaller parts but also higher dropout rates. However, the fraction of missing information is lower in the 3- and 10- part conditions. More respondents use their mobile phone for survey completion when the survey is shorter. We find fewer item missings and satisficing in shorter surveys. We find no effect on neutral and extreme responding nor on estimates of the validity of answers. People with low and high education and young and old evaluate shorter surveys better than the normal length survey.

In deze studie onderzochten we of er verschillende ontwikkelingstrajecten bestaan in het gebruik van seksueel expliciet internetmateriaal (SEIM) van adolescenten en welke factoren voorspellend zijn voor deze trajecten. Een combinatie van latente klasseanalyse met groeicurves en multinominale regressieanalyse werd toegepast op 4-wave longitudinale gegevens van 441 adolescenten. Onder jongens waren vier ontwikkelingstrajecten in SEIM-gebruik te onderscheiden: ‘Stabiel geen/weinig gebruik’, ‘Sterk toenemend gebruik’, ‘Licht afnemend gebruik’ en ‘Stabiel veel gebruik’. Voor meisjes werd slechts één traject ‘Stabiel geen/weinig gebruik’ gevonden. Jongens in trajecten gekenmerkt door een hogere mate van SEIM-gebruik bleken bij de aanvang van de studie meer seksueel permissief en seksueel geïnteresseerd, zij communiceerden vaker met hun ouders over seks, konden vaker in privacy internetten en hadden een hogere realiteitsperceptie ten aanzien van seksuele media vergeleken met jongens in het traject ‘Stabiel geen/weinig gebruik’.

The present study examined the joint development of substance use and externalizing problems in early and middle adolescence. First, it was tested whether the relevant groups found in previous studies i.e., those with an early onset, a late onset, and no onset or low levels of risk behavior could be identified, while using a developmental model of a single, underlying construct of risk behavior. Second, departing from Moffitt's taxonomy of antisocial behavior, it was tested if early, but not late, onset risk behavior is predicted by a problematic risk profile in childhood. Data were used from TRAILS, a population based cohort study, starting at age 11 with two follow-ups at mean ages of 13.6 and 16.3 years. Latent transition analyses demonstrated that, both in early and middle adolescence, a single underlying construct of risk behavior, consisting of two classes (labeled as low and high risk behavior), adequately represented the data. Respondents could be clearly classified into four possible transition patterns from early to middle adolescence, with a transition from high to low being almost non-existent (2.5 %), low to low (39.4 %) and low to high (41.8 %) being the most prevalent, and high to high (16.2 %) substantial. As hypothesized, only the high-high group was characterized by a clear adverse predictor profile in late childhood, while the low-high group was not. This study demonstrates that the development of substance use is correlated with externalizing problems and underscores the theory that etiologies of early and later onset risk behavior are different. © The Author(s) 2012.

Web surveys are no longer completed on just a desktop or laptop computer. Respondents increasingly use mobile devices, such as tablets and smartphones to complete web surveys. In this article, we study how respondents in the American Life Panel complete surveys using varying devices. We show that about 30 percent of respondents sometimes complete surveys on a mobile device, and about 12 percent always use a mobile device. We study the characteristics of these “mobile-only” web survey respondents and find that they share many characteristics of typically hard-to-recruit survey respondents. They are more likely to be non-white, young, and not have a higher education. In terms of voting behavior, we find no differences between the groups of survey respondents who use different devices. This suggests that biases in political polls conducted through web-surveys are unlikely to occur when mobile-only respondents are underrepresented.

Different types of nonresponse threaten the validity of longitudinal studies: first, initial nonresponse during the recruitment in the baseline survey and second, successive dropout at each time point. Dropout in longitudinal surveys has three separate sources: failure to locate research participants, failure to contact participants, and failure to achieve cooperation. In this entry, specific attention is given to existing knowledge on the correlates of dropout and strategies to limit the problem. To limit nonresponse, a total design approach is advocated with specific attention to each source. That is to limit both noncontact (i.e., failure to locate and subsequent failure to contact a located research participant) and noncooperation.

Five specific studies investigate how the methodology of panel surveys can be improved. 1) Propensity Score Matching to separate different sources of measurement error. Errors of non-observation and measurement errors are easily confounded in mixed-mode surveys. I propose propensity score matching as a method to deal with this problem 2 and 3) Dependent Interviewing (DI) to improve data quality of measures of change. In Dependent Interviewing, data from the previous interviews are used in the current interview to check whether the old situation of the respondent has changed. It is unclear whether DI leads to higher data quality, or rather, that correlated errors decrease data quality. Two papers study this question. 4) What is change? When a questionnaire is administered twice, we cannot simply compare the scores on the tests from both measurements. Errors of measurements in the questions result change estimates to be biased. A study that combined open interviews (qualitatively), with an analysis of measurement equivalence (quantitatively) helps us to understand how change estimates can be biased, making it impossible to study change scores. 5) methods to separate different types of dropout in a panel surveys and possibe ways to deal with dropout. Attrition or dropout models cannot at the same time study variation among individuals and the process of attrition over time. I show how Growth Mixture Models (GMM) can be used to study how the attrition process over individuals. There are several distinct groups of attriters, that have clearly different background characteristics

The analysis of measurement invariance of latent constructs is important in research across groups, or across time. By establishing whether factor loadings, intercepts and residual variances are equivalent in a factor model that measures a latent concept, we can assure that comparisons that are made on the latent variable are valid across groups or time. Establishing measurement invariance involves running a set of increasingly constrained structural equation models, and testing whether differences between these models are significant. This paper provides a step-by-step guide to analysing measurement invariance.

A primary objective of panel studies is to analyze change. The same questionnaire is used to compare data recorded at various times. Panel designs assume that the meaning of the questions and the concept of interest are stable over time. Analyses of measurement invariance often show the contrary. A qualitative part supplementing a panel survey can help us understand this phenomenon. In this study, 261 first-year psychology students completed questionnaires about their study motivation on two occasions; we interviewed some students as well. The survey showed that study motivation is not invariant over time. The qualitative data converged with the quantitative outcomes and explained the lack of invariance by the students’ overall transition during the first study year.We conclude that mixing quantitative and qualitative research methods for panel studies helps us understand change in constructs over time. We can study change at the macrolevel and better understand such change at the microlevel.

The present study investigated whether a conceptual distinction between two components of self-control (inhibitory and initiatory self-control) is empirically valid. To that purpose, a series of confirmative factor analyses were employed in two samples (total N = 577), providing support for a distinction between inhibitory and initiatory self-control. In addition, the predictive validity of the two components of self-control was examined by regression analyses with (un)desired health/academic behavior as dependent variables, showing that inhibitory self-control was a superior predictor of undesired behavior and initiatory self-control a better predictor of desired behavior.

In mixed-mode surveys, it is difficult to separate sample selection differences from mode-effects that can occur when respondents respond in different interview settings. This paper provides a framework for separating mode effects from selection effects by matching very similar respondents from different survey modes using propensity score matching. The answer patterns of the matched respondents are subsequently compared. We show that matching can explain differences in nonresponse and coverage in two Internet samples. When we repeat this procedure for a telephone and Internet sample however, differences persist between the samples after matching. This indicates the occurrence of mode effects in telephone and Internet surveys. Mode effects can be problematic; hence we conclude with a discussion of designs that can be used to explicitly study mode effects.

Many internet-panels consist of self-selected respondents and hence cover a relatively small part of the population. Estimates based on Internet-panels therefore may suffer from non-coverage and self-selection bias. One way to correct for these biases is to use adjustment weighting(Lee, 2006). However, when Internet-panel respondents are intrinsically different from the general population, previous studies showed that weighting may result in an increase in bias (for example, see Loosveldt and Sonck, 2008). How can we show that panel-members are intrinsically different from respondents that take part in a conventional random-sample survey? To answer this question we compared the results of a volunteer Internet-panel to the results of a web-interview (WI) based on a random sample of the same population. First, differences in population coverage are studied. Secondly, we test if significant differences in coverage predict differences on dependent variables. Finally, we use propensity matching to test for self-selection bias. This contribution sheds light on the extent of coverage bias relative to self-selection bias in random- and volunteer opt-in Internet surveys. We use propensity score matching to answer our question. Propensity scores summarize the conditional probability of a respondent to be member of either the random or volunteer sample based on a set of covariates. When the propensity score includes relevant covariates, respondents with the same propensity scores can be matched. Remaining differences between dependent variables after matching cannot be caused by coverage errors, and are indicative for the size of self-selection bias