Search

In this case study, we examine a novel aspect of data collected in a typical probability and a typical nonprobability panel: mobile app data. The data were collected in Great Britain in 2018, using the Innovation Panel of the UK Household Longitudinal Study and the Lightspeed online access panel. Respondents in each panel were invited to participate in a month-long study, reporting all their daily expenditures in the app. In line with most of the research on nonprobability and probability-based panel data, our results indicate differences in the data gathered from these data sources. For example, more female, middle-aged, and highly educated people with higher digital skills and a greater interest in their finances participated in the nonprobability app study. Our findings also show that resulting differences in the app spending data are difficult to eliminate by weighting. The only data quality aspect for which we do not find evidence of differences between the nonprobability and probability-based panel is behavior in using the spending app. This finding is contrary to the argument that nonprobability online panel participants try to maximize their monetary incentive at the expense of data quality. However, this finding is in line with some of the scarce existing literature on response behavior in surveys, which is inconclusive regarding the question of whether nonprobability online panel participants answer questions less conscientiously than probability-based panel respondents. Since the two panels in our case study differ in more aspects than the sample selection procedure, more research in different contexts is necessary to establish generalizability and causality. [ABSTRACT FROM AUTHOR]

Experiments are frequently used in the travel and hospitality literature to provide stronger evidence of causal relationships between various constructs. Recently, despite the use of online platforms for experimental studies, such studies have often failed to find significant results, as expected. To increase the probability of obtaining significant results for experiments using online panels, this paper suggests six practical recommendations across three categories: (i) handling less homogenous online participants; (ii) understanding and managing different motivations and abilities in online participants; and (iii) using effective and transparent experimental designs and procedures. This paper provides the results of three empirical investigations to support these recommendations. [ABSTRACT FROM AUTHOR]

Despite its importance in terms of survey participation, the literature is sparse on how face-to-face interviewers differentially affect specific groups of sample units. This paper demonstrates how an alternative parametrization of the random components in multilevel models, so-called separate coding, delivers valuable insights into differential interviewer effects for specific groups of sample members. In the example of a face-to-face recruitment interview for a probability-based online panel, we detect small interviewer effects regarding survey participation for non-Internet households, whereas we find sizable interviewer effects for Internet households. We derive practical guidance for survey practitioners to address differential interviewer effects based on the proposed variance decomposition. [ABSTRACT FROM AUTHOR]

This database consists of the Peruvian media diet in a post-pandemic context. Specifically, it examines how Peruvians define and utilise media to create environments for information, learning, and entertainment. Since the pandemic, the relationship that users have developed with the media has intensified and changed, fostering new uses and interactions. However, our data demonstrates that the Peruvian public maintains a critical role towards mass media (broadcasting model) [1], which motivates them to seek out media and platforms that cater to their interests and expectations. In this regard, we compare three axes of analysis (information, education, and entertainment), taking socio demographic variables, to produce a baseline that is useful for specific research projects. Through a quantitative approach we explored the expectations, decisions, forms of interaction, and satisfactions that users obtain in their relationship with the media, offering new theoretical insights based on emerging data, such as the level of planning in content consumption or the role that traditional media play in different age and socioeconomic groups

We investigate panel conditioning effects in a long-running probability-based online panel of the general population through a large-scale experiment conducted in 2020. Our experiment was specifically designed to study the effect of intensifying the surveying frequency for the treatment group (N = 5,598 panel members) during a 16-week corona study while keeping the control group (N = 799 panel members) at the usual bi-monthly surveying frequency. Our results show that panel conditioning is only a minor issue when increasing surveying frequency and only matters for survey questions directly related to the corona study. With this study, we contribute to the general panel conditioning literature, which is still unclear about the conditions under which conditioning effects may occur. In addition, while many panel surveys have conducted add-on studies with increased surveying frequency during the COVID-19 pandemic, ours is among the very few which can provide experimental evidence regarding the potential impact of the add-on study on the underlying panel data infrastructure.

If respondents recognize repeated survey questions and remember their previous responses, this can result in measurement error. Most studies to date that have investigated respondents’ recall of their prior answers have done so in the context of repeated measurements within one cross-sectional survey. The present study extends this research to a longitudinal panel context by investigating whether respondents remember their previous responses to different types of survey questions (beliefs, attitudes, and behaviors) from a previous wave in a probability-based online panel in Germany. We find evidence that some respondents remember their responses from a previous panel wave even after four months, but at a considerably lower rate than previous studies found within a single cross-sectional survey. Respondents who could not remember their response were most commonly off by only a single scale point. Respondents remembered their responses to different types of questions at different rates and were more likely remember an extreme response. Female respondents were more likely to remember their responses, but we find no link to age, education, perceived response burden, survey enjoyment or online panel experience. As respondents could not remember their previous responses in most cases and we find little evidence for a systematic variation of memory effects across groups of respondents, we conclude that the potential for measurement error due to memory effects across panel waves is low after four or more than four months.

Recent years have seen a growing number of studies investigating the accuracy of nonprobability online panels; however, response quality in nonprobability online panels has not yet received much attention. To fill this gap, we investigate response quality in a comprehensive study of seven nonprobability online panels and three probability-based online panels with identical fieldwork periods and questionnaires in Germany. Three response quality indicators typically associated with survey satisficing are assessed: straight-lining in grid questions, item nonresponse, and midpoint selection in visual design experiments. Our results show that there is significantly more straight-lining in the nonprobability online panels than in the probability-based online panels. However, contrary to our expectations, there is no generalizable difference between nonprobability online panels and probability-based online panels with respect to item nonresponse. Finally, neither respondents in nonprobability online panels nor respondents in probability-based online panels are significantly affected by the visual design of the midpoint of the answer scale. [ABSTRACT FROM AUTHOR]

Abstract The outbreak of the COVID-19 pandemic has led to a vast increase in the demand for fast, frequent, and multi-faceted data to study the impact of the pandemic on people’s lives. Existing data collection infrastructures had to be adapted quickly during the early phase of the pandemic to meet this data demand. Our research group contributed to this by conducting the Mannheim Corona Study (MCS), a longitudinal probability-based online survey, in a daily rotating panel design that took place from March 20 through July 10, 2020. The fast-and-frequent panel data collection design of the MCS had numerous consequences for designing its questionnaires and choosing its measurement instruments. This included designing new instruments on the fly in the ever-changing pandemic environment, making efficient use of limited questionnaire space, and deciding on measurement frequencies in a structured manner under uncertain external conditions. In this report, we document the MCS approach to choosing measurement instruments fit for the purpose of fast and frequent data collection during the early phase of COVID-19 in Germany. We particularly highlight three examples of measurement instruments in the MCS and reflect on their measurement properties.

We explored the feasibility, suitability, and reliability of using controls recruited among members of a non-probabilistic online panel ('panel controls') in a case–control study (CCS) to investigate a Salmonella Braenderup outbreak in Germany. For comparison, another control group was recruited via random digit dialling ('classical controls'). Panel members received questionnaires by email; classical controls were interviewed by phone. Both control groups were frequency-matched to cases by age and sex; the classical controls also by federal state. Cases and controls were queried mainly about fruit consumption since melons were the suspected infection vehicle. We calculated adjusted odds ratios (aOR) and 95% confidence intervals (CIs) using single-variable and multivariable logistic regression. The study included 32 cases, 81 panel controls and 110 classical controls. Analyses identified melons, particularly Galia melons, as the most likely infection vehicle using either control group (panel controls – aOR 12, CI 2.7–66; classical controls – aOR 55, CI 8–1100). Recruitment of panel versus classical controls required substantially less person-time (8 vs. 111 hours) and was about 10 times less expensive. We recommend this timely and reliable control recruitment method when investigating diffuse foodborne outbreaks with CCS. [ABSTRACT FROM AUTHOR]

A crucial assumption of survey measurements is that respondents carefully perceive, reflect upon, and provide a response to a given question and that this process is independent of respondents’ memory of their responses to previous questions. A violation of this assumption may considerably affect parameter estimations. Despite this possibility, there is little research on when and to what extent such violations occur. To shed light on such memory effects, we investigate the ability of respondents to recall their answers to three types of survey questions (attitudes, behaviors, and beliefs) within one wave of an online panel survey. We find that respondents’ ability to recall their answers is highest for behavior questions and lowest for belief questions, and higher for respondents with extreme responses than respondents with non-extreme responses. We find no difference across respondents with different levels of panel experience and the devices used to complete the survey.

A growing trend in UX research is the use of Online Panels (OPs), namely people enrolled in a web platform who have agreed to participate regularly in online studies and/or in the execution of simple and repetitive operations. The effect of the participation of such "professional respondents" on data quality has been questioned in a variety of fields (e.g., Psychology and Marketing). Notwithstanding the increasing use of OPs in UX research, there is a lack of studies investigating the bias affecting usability assessments provided by this type of respondents. In this paper we have addressed this issue by comparing the usability evaluations provided by professional respondents commonly involved in debugging activities, non-professional respondents, and naive people not belonging to any OP. In a set of three studies, we have addressed both the effect of expertise and type of task (debugging vs. browsing) on the usability assessments. A total of 138 individuals participated in these studies. Results showed that individuals who performed the debugging test provided more positive usability ratings regardless of their skills, conversely, professional respondents provided more severe and critical ratings of perceived usability than non-professionals. Finally, the comparison between the online panelists and naive users allowed us to better understand whether professional respondents can be involved in usability evaluations without jeopardizing them. [ABSTRACT FROM AUTHOR]

As ever more surveys are conducted, recruited respondents are more likely to already have previous survey experience. Furthermore, it has become more difficult to convince individuals to participate in surveys, and thus, incentives are increasingly used. Both previous survey experience and participation in surveys due to incentives have been discussed in terms of their links with response quality. This study aims to assess whether previous web survey experience and survey participation due to incentives are linked with three indicators of response quality: item non‐response, primacy effect and non‐differentiation. Analysing data of the probability‐based CROss‐National Online Survey panel covering Estonia, Slovenia and Great Britain, we found that previous web survey experience is not associated with item non‐response and the occurrence of a primacy effect but is associated with non‐differentiation. Participating due to the incentive is not associated with any of the three response quality indicators assessed. Hence, overall, we find little evidence that response quality is linked with either previous web survey experience or participating due to the incentive. [ABSTRACT FROM AUTHOR]

Driven by innovations in the digital space, surveys have started to move towards online data collection across the world. However, evidence is needed to demonstrate that online data collection strategy will produce reliable data which could be confidently used to inform policy decisions. This issue is even more pertinent in cross‐national surveys, where the comparability of data is of the utmost importance. Due to differences in internet coverage and willingness to participate in online surveys across Europe, there is a risk that any strategy to move existing surveys online will introduce differential coverage and nonresponse bias. This paper explores representativeness across waves in the first cross‐national online probability‐based panel (CRONOS) by employing R‐indicators that summarize the representativeness of the data across a range of variables. The analysis allows comparison of the results over time and across three countries (Estonia, Great Britain and Slovenia). The results suggest that there are differences in representativeness over time in each country and across countries. Those with lower levels of education and those who are in the oldest age category contribute more to the lack of representativeness in the three countries. However, the representativeness of CRONOS panel does not become worse when compared to the regular face‐to‐face interviewing conducted in the European Social Survey (ESS). [ABSTRACT FROM AUTHOR]

The outbreak of COVID‐19 has sparked a sudden demand for fast, frequent and accurate data on the societal impact of the pandemic. This demand has highlighted a divide in survey data collection: Most probability‐based social surveys, which can deliver the necessary data quality to allow valid inference to the general population, are slow, infrequent and ill‐equipped to survey people during a lockdown. Most non‐probability online surveys, which can deliver large amounts of data fast, frequently and without interviewer contact, however, cannot provide the data quality needed for population inference. Well aware of this chasm in the data landscape, at the onset of the pandemic, we set up the Mannheim Corona Study (MCS), a rotating panel survey with daily data collection on the basis of the long‐standing probability‐based online panel infrastructure of the German Internet Panel (GIP). The MCS has provided academics and political decision makers with key information to understand the social and economic developments during the early phase of the pandemic. This paper describes the panel adaptation process, demonstrates the power of the MCS data on its own and when linked to other data sources, and evaluates the data quality achieved by the MCS fast‐response methodology. [ABSTRACT FROM AUTHOR]

Established methods of recruiting population controls for case–control studies in infectious disease outbreak investigations are resource- and time-intensive, and are often subject to bias. The online panel have recently gained interest as an easy and timely method to select controls. We examined the feasibility, suitability and reliability of using an online panel to select controls for case–control studies as part of investigations of diffuse food and waterborne outbreaks. In January 2019, we deployed a web survey by email to the 277 members of a non-probabilistic online panel in Lower Saxony, Germany. We questioned them on basic sociodemographic characteristics and eating habits. They were frequency matched to cases on sex and age. Their food exposures were compared to those of traditionally recruited controls of four historical case–controls studies, which successfully investigated food and waterborne outbreaks. We used logistic regressions to assess the association between the food exposures and the disease (odds ratios). The use of a control panel successfully led to the identification of the food items in three of the four historical outbreak investigations, and their recruitment benefitted from increased speed and limited costs. Timely outbreak investigations would enable rapidly implementing control measures. We recommend the further evaluation of using panellists as controls in parallel case–control studies and case–panel studies. [ABSTRACT FROM AUTHOR]

The term "prevention paradox" focuses on the notion that more aggregate harm is experienced by low-risk individuals even though high-risk individuals experience the greatest amount of harm per individual. This paper examines whether the prevention paradox in relation to gambling harms exists in Massachusetts. The analysis is drawn from two population surveys and the distribution of harms across four gambling severity groups is examined. The results show that because of the larger size of the three lower severity groups, even the much smaller average number of harms endorsed by members of these groups accounts for nearly three-quarters (72.9%) of the aggregate number of harms across all groups. While almost all individuals in the highest severity group report one or more harms, any individual reporting one or more harms is more likely to be in a lower severity group. Financial, health, and emotional/psychological harms are more common and more broadly distributed across the gambling severity groups compared to relationship, work/school, and illegal harms. In contrast to a similar study in Finland, which found that the most severe group accounted for over 50% of the harms in the health, relationship, and illegal harm domains, the prevention paradox is supported across all harm domains in Massachusetts. [ABSTRACT FROM AUTHOR]

The use of non-probability Internet panels and crowdsource websites is increasing in gambling research. These paid online sampling methods offer a convenient and inexpensive recruitment strategy. The quality of data may be questionable due to careless responding and identity misrepresentation which can bias study results. It is necessary to investigate data quality given the important implications of gambling research in guiding policy decisions, public health initiatives, and treatments. In this review article, we 1) critically analyze the advantages and limitations of paid online recruitment methods, including associated threats to data quality in the gambling literature; 2) present findings from a rapid review of gambling studies using online panel and crowdsource data; and 3) outline recommendations for maximizing data quality and trustworthiness of findings. Substantially overinflated problem gambling rates were found in the 63 gambling studies we reviewed; less than one-quarter had incorporated data quality checks and reported participation rates. Future studies should incorporate pre-registration of methodology and analysis plans, robust participant screening procedures, mid-survey attention and response consistency items, and an analysis of response quality post data collection. Applying these recommendations to nonrepresentative online panel and crowdsource-based studies may enhance the replicability of findings in additional studies using representative samples [ABSTRACT FROM AUTHOR]

A number of countries in Europe and beyond have established online or mixed mode panels with a web component based upon probability samples. This paper aims to assess the first ever attempt to do this cross-nationally using an input harmonised approach representing a major innovation in cross-national survey methodology. The European Social Survey established a panel using the face-to-face interview as the basis for recruitment to the panel. This experiment was conducted in Estonia, Slovenia and Great Britain using an input harmonised approach to the design throughout something never done before across multiple countries simultaneously. The paper outlines how the experiment was conducted. It then moves on to compare the web panel respondents to the ESS achieved face-to-face sample in each country, as well as comparing the web panel achieved sample to external benchmarks. Most importantly, since the literature is very scarce, the differences in attitudinal and behavioural characteristics are also assessed. By comparing the answers of the total achieved sample in the ESS to the subset who also answered the CRONOS web panel we assess changes in representativeness and substantive answers without confounding the findings with other changes such as mode effects. This approach is only possible where ‘piggybacking’ recruitment has been used. This in itself is rare at the national level but this is the first time survey methodologists have employed this cross-nationally allowing such an analytical approach. Our findings suggest that the CRONOS sample is not too divergent from the target population and to the ESS with the exception of the oldest age groups. However, there are cross-national differences suggesting cross-national comparability might be different when compared to estimates from a face-to-face survey.

While online panels offer numerous advantages, they are often criticized for excluding the offline population. Therefore, some probability-based online panels have developed offline population inclusion strategies. Two dominant approaches prevail: providing internet equipment and offering an alternative survey participation mode. We investigate the impact of these approaches on two probability-based online panels in Germany: the German Internet Panel, which provides members of the offline population with internet equipment, and the GESIS Panel, which offers members of the offline population to participate via postal mail surveys. In addition, we explore the impact of offering an alternative mode only to non-internet users versus also offering the alternative mode to internet users who are unwilling to provide survey data online. Albeit lower recruitment and/or panel wave participation probabilities among offliners than onliners, we find that including the offline population has a positive long-term impact on sample accuracy in both panels. In the GESIS Panel, the positive impact is particularly strong when offering the alternative participation mode to non-internet users and internet users who are unwilling to provide survey data online. [ABSTRACT FROM AUTHOR]

Abstract Declining response rates worldwide have stimulated interest in understanding what may be influencing this decline and how it varies across countries and survey populations. In this paper, we describe the development and validation of a short 9-item survey attitude scale that measures three important constructs, thought by many scholars to be related to decisions to participate in surveys, that is, survey enjoyment, survey value, and survey burden. The survey attitude scale is based on a literature review of earlier work by multiple authors. Our overarching goal with this study is to develop and validate a concise and effective measure of how individuals feel about responding to surveys that can be implemented in surveys and panels to understand the willingness to participate in surveys and improve survey effectiveness. The research questions relate to factor structure, measurement equivalence, reliability, and predictive validity of the survey attitude scale. The data came from three probability-based panels: the German GESIS and PPSM panels and the Dutch LISS panel. The survey attitude scale proved to have a replicable three-dimensional factor structure (survey enjoyment, survey value, and survey burden). Partial scalar measurement equivalence was established across three panels that employed two languages (German and Dutch) and three measurement modes (web, telephone, and paper mail). For all three dimensions of the survey attitude scale, the reliability of the corresponding subscales (enjoyment, value, and burden) was satisfactory. Furthermore, the scales correlated with survey response in the expected directions, indicating predictive validity.

Plain English summary Evidence shows that public and patient involvement in research has a positive effect on its quality and end-results. Thus, public and patient involvement in all stages of research is becoming commonplace. There are limited detailed examples however, that describe how to make this possible, especially for those doing PhD research. Doctoral researchers are often new to research practice or have limited experience and are often bound by strict time and financial constraints. It is also not usually a requirement of the award to involve public and patients in their research. Hence, they may not feel confident or motivated to involve or engage with public and patients during their research. We, four doctoral researchers, share examples from our own research studies that have included different approaches to public and patient involvement. Two studies formed public and patient advisory groups who helped design the research questions, data collection tools and recruitment methods. One enlisted the help of an online public and patient panel from a local hospital. A different study worked with patients from an established group to help define key medical words. We did face some challenges, such as the need to develop good group work skills and to apply for grants to cover reimbursement, but we all found it beneficial to involve patients in our studies. We noticed a positive effect on each study’s progression and an improvement in our own self-esteem. In addition, having public and patient involvement helped reduce the isolation we felt as doctoral researchers. Thus, we strongly encourage more doctoral researchers to involve public and patients in their studies. Abstract Public and patient involvement (PPI) has been shown to have a positive impact on health and social care research. However, adequate examples describing how to operationalise effective PPI, especially in doctoral studies, are lacking. Hence, doctoral researchers new to research, or those with limited experience, can be discouraged from facilitating PPI in their research. This paper aims to describe and discuss in detail the approaches used by four doctoral researchers to incorporate PPI at different stages of their research studies from study design to disseminating findings. We aim to inform other doctoral researchers about the challenges and limitations relating to PPI that we faced. Through these, we share pragmatic recommendations for facilitating PPI during doctoral studies. The description of four case studies demonstrated that PPI could be incorporated at various stages during doctoral research. This has had a beneficial impact on our research study progression, researcher self-esteem and lastly, helped alleviate researcher isolation during doctoral studies.

A growing trend in UX research is the use of Online Panels (OPs), namely people enrolled in a web platform who have agreed to participate regularly in online studies and/or in the execution of simple and repetitive operations. The effect of the participation of such “professional respondents” on data quality has been questioned in a variety of fields (e.g., Psychology and Marketing). Notwithstanding the increasing use of OPs in UX research, there is a lack of studies investigating the bias affecting usability assessments provided by this type of respondents. In this paper we have addressed this issue by comparing the usability evaluations provided by professional respondents commonly involved in debugging activities, non-professional respondents, and naive people not belonging to any OP. In a set of three studies, we have addressed both the effect of expertise and type of task (debugging vs. browsing) on the usability assessments. A total of 138 individuals participated in these studies. Results showed that individuals who performed the debugging test provided more positive usability ratings regardless of their skills, conversely, professional respondents provided more severe and critical ratings of perceived usability than non-professionals. Finally, the comparison between the online panelists and naive users allowed us to better understand whether professional respondents can be involved in usability evaluations without jeopardizing them.

Crowdsourcing offers fast and cost-effective access to human labor for business projects as well as to research participants for scientific projects. Due to the loose links between crowdsourcing employers and workers, quality control is even more important than in the off-line realm. We developed and validated the web-delivered attention test attentiveWeb in two versions (1) to come up with advance filters to identify workers who produce low-quality results and (2) to gauge the attention of workers who pass the advance filter. We apply attentiveWeb in three parallel user studies: one in the crowdsource Microworkers (N = 539), another one in Figure Eight (N = 333), and a third one in the online panel WiSoPanel (N = 1,837). The user studies confirm that it is useful to apply advance filtering to screen out poor workers. We propose an easily computed filter based on objective user behavior involving attentiveWeb. With regard to attention, despite the more severe advance filtering with Microworkers, their attention was lowest, followed by workers from Figure Eight, and it was highest in WiSoPanel. The platform differences in attention were not entirely explained by known differences—demographic and others—among the users of the three platforms. The attention test attentiveWeb has high Cronbach's α and split-half reliability. The first version of attentiveWeb predicted performance of the same crowdworkers in the second version of attentiveWeb 2 years later. We release attentiveWeb for assessing crowdworkers' attention into the research community and the wider public domain. The attention test attentiveWeb is open-source and can be used for free. [ABSTRACT FROM AUTHOR]

The outbreak of the COVID-19 pandemic has a massive impact on society. To curb the spread of the SARS-CoV-2 virus, unprecedented containment measures are being taken by governments around the world. These measures and the fear of the disease itself are likely affecting the economy, social inequality, mental and physical health, and even people’s perception of good democratic governance. Equally unprecedented is the speed at which these massive changes take place and the lack of statistical evidence that accompanies them. Within days of the first containment measures in Germany, the German Internet Panel (GIP) launched the Mannheim Corona Study (MCS), a daily rotating panel study of the general adult population of approximately 3,600 respondents. Its data and reports now inform the crisis cabinet of the German government and are the basis for groundbreaking social and economic research. This paper gives insights into the MCS methodology and data quality.

Copyright of Comunicação Pública is the property of Escola Superior de Comunicacao Social and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

One of the methods for evaluating online panels in terms of data quality is comparing the estimates that the panels provide with benchmark sources. For probability-based online panels, high-quality surveys or government statistics can be used as references. If differences among the benchmark and the online panel estimates are found, these can have several causes. First, the question wordings can differ between the sources, which can lead to differences in measurement. Second, the reference and the online panel may not be comparable in terms of sample composition. Finally, since the reference estimates are usually collected face-to-face or by telephone, mode effects might be expected. In this article, we investigate mode system effects, an alternative to mode effects that does not focus solely on measurement differences between the modes, but also incorporates survey design features into the comparison. The data from a probability-based offline-recruited online panel is compared to the data from two face-to-face surveys with almost identical recruitment protocols. In the analysis, the distinction is made between factual and attitudinal questions. We report both effect sizes of the differences and significances. The results show that the online panel differs from face-to-face surveys in both attitudinal and factual measures. However, the reference surveys only differ in attitudinal measures and show no significant differences for factual questions. We attribute this to the instability of attitudes and thus show the importance of triangulation and using two surveys of the same mode for comparison.

Background: Reliability of nonprobability online volunteer panels for epidemiological purposes has rarely been studied. Objectives: To assess the quality of a questionnaire on sexual and reproductive health (SRH) administered in a nonprobability Web panel and in a random telephone survey (n = 8,992; n = 8,437, age 16–49 years). Especially, we were interested in the possible difference in the association of sociodemographic variables and some outcome variables in the two surveys that are in the reliability of analytical epidemiological studies conducted in such panels. Methods: Interventions to increase response rate were used in both surveys (four e-mail reminders, high number of call attempts and callbacks to refusals). Both were calibrated on the census population. Sociodemographic composition, effects of reminders, and prevalence were compared to their telephone counterpart. In addition, the associations of sociodemographic and sexual behaviors were compared in the two samples in multivariate logistic regressions. Results: The online survey had a lower response rate (20.0 percent vs. 44.8 percent) and a more distorted sociodemographic structure although the reminders improved the representativeness as did the analogous interventions on the telephone survey. Prevalences of SRH variables were similar for the common behaviors but higher online for the stigmatized behaviors, depending on gender. Overall, 29 percent of the 63 interactions studied were significant for males and 11 percent for women, although opposite effects of sociodemographic variables were rare (5 percent of the 171 tested for each gender). Conclusion: Nonprobability online panels are to be used with caution to monitor SRH and conduct analytical epidemiological studies, especially among men. [ABSTRACT FROM AUTHOR]

The aim of the article is to identify and to study trends of application and forming of sampling selection of participants, control of survey quality and methods of conducting online panels and online communities. The results of the analysis. The article provides the analysis of the modern tendencies of marketing researches on the Internet. The development of Internet technologies and the spread of the Internet to all spheres of human life lead to the relocation of marketing research of consumer behavior to the Internet. The structure of the Ukrainian population in accordance with the structure of the Internet audience leads to the representativeness of samples of marketing research on the Internet. We analyzed the types of respondents' recruiting for online panels and online communities, their advantages and disadvantages. We revealed areas of application of online panels and online communities. Online panels are applicable in two cases: the target group that consists exclusively of Internet users and the target group amounting not less than 50% of Inter net users. Exceptionally in these cases, we can extrapolated the results of panel studies to the entire population. Online communities are used for qualitative research and there are no strict requirements for the population and sample for them. Online communities are classified into three groups: project, syndicated and permanent. Each group is characterized by the goals of its creation, the volumes of samples being generated and the methods of operation. We analyzed and systematized methods of quality control online survey. Quality control of the survey consists of three stages: monitoring during the interview, data quality control, and panel management. At each stage, a system of indicators is formed, which confirms the quality of the survey. We considered further perspectives of Internet research, which don`t need Internet surveys. An increase in information about consumers (geolocation, activity on the Internet, profiles in social networks) can contribute to this. Conclusions and direction of further researches. The study we identified the main trends of surveys on the Internet, both quantitatively by online panels, and in qualitative by online communities. We analyzed the advantages and disadvantages of surveys on the Internet. We determined areas of application of online panels and online communities. We presented the methods of sampling for online panel research. We systematized the factors by which a decision about the possibility of using online panels is made. Further research should be focus on the psychological aspects of online community members, as this direction of research is only developing in Ukraine. [ABSTRACT FROM AUTHOR]

The past decade has seen a rise in the use of online panels for conducting survey research. However, the popularity of online panels, largely driven by relatively low implementation costs and high rates of Internet penetration, has been met with criticisms regarding their ability to accurately represent their intended target populations. This criticism largely stems from the fact that (1) non-Internet (or offline) households, despite their relatively small size, constitute a highly selective group unaccounted for in Internet panels, and (2) the preeminent use of nonprobability-based recruitment methods likely contributes a self-selection bias that further compromises the representativeness of online panels. In response to these criticisms, some online panel studies have taken steps to recruit probability-based samples of individuals and providing them with the means to participate online. Using data from one such study, the German Internet Panel, this article investigates the impact of including offline households in the sample on the representativeness of the panel. Consistent with studies in other countries, we find that the exclusion of offline households produces significant coverage biases in online panel surveys, and the inclusion of these households in the sample improves the representativeness of the survey despite their lower propensity to respond. [ABSTRACT FROM AUTHOR]

Background: Public and stakeholder engagement can improve the quality of both research and policy decision making. However, such engagement poses significant methodological challenges in terms of collecting and analysing input from large, diverse groups. Objective: To explain how online approaches can facilitate iterative stakeholder engagement, to describe how input from large and diverse stakeholder groups can be analysed and to propose a collaborative learning framework (CLF) to interpret stakeholder engagement results. Methods: We use ‘A National Conversation on Reducing the Burden of Suicide in the United States’ as a case study of online stakeholder engagement and employ a Bayesian data modelling approach to develop a CLF. Results: Our data modelling results identified six distinct stakeholder clusters that varied in the degree of individual articulation and group agreement and exhibited one of the three learning styles: learning towards consensus, learning by contrast and groupthink. Learning by contrast was the most common, or dominant, learning style in this study. Conclusion: Study results were used to develop a CLF, which helps explore multitude of stakeholder perspectives; identifies clusters of participants with similar shifts in beliefs; offers an empirically derived indicator of engagement quality; and helps determine the dominant learning style. The ability to detect learning by contrast helps illustrate differences in stakeholder perspectives, which may help policymakers, including Patient‐Centered Outcomes Research Institute, make better decisions by soliciting and incorporating input from patients, caregivers, health‐care providers and researchers. Study results have important implications for soliciting and incorporating input from stakeholders with different interests and perspectives. [ABSTRACT FROM AUTHOR]

Identifying ways to efficiently maximize the response rate to surveys is important in survey-based research. However, evidence on the response rate effect of donation incentives and especially altruistic and egotistic text appeal interventions is sparse and ambiguous. Via a randomized survey experiment among 6,162 members of an online survey panel, this article shows how low-cost incentives and cost-free text appeal interventions may affect the survey response rate in online panels. The experimental treatments comprise (a) a cash prize lottery incentive, (b) two donation incentives that promise a monetary donation to a good cause in return for survey response, (c) an egotistic text appeal, and (d) an altruistic text appeal. Relative to a control group, we find higher response rates among recipients of the egotistic text appeal and the lottery incentive. Donation incentives yield lower response rates. [ABSTRACT FROM AUTHOR]

Probability-based online panels are beginning to replace traditional survey modes for existing established surveys in Europe and the United States. In light of this, current standards for panel response rate calculations are herein reviewed. To populate these panels cost-effectively, more diverse recruitment methods, such as, mail, telephone, and recruitment modules added to existing surveys are being used, either alone or in combinations. This results in panel member cohorts from different modes complicating panel response rate calculations. Also, as a panel ages with inevitable attrition, multiple cohorts result from panel refreshment and growth strategies. Formulas are presented to illustrate how to handle multiple cohorts for panel metrics. Additionally, drawing on relevant metrics used for a panel response rate, we further demonstrate a computational tool to assist planners in building a probability-based panel. This provides a means to estimate the recruitment effort required to build a panel of a predetermined size. [ABSTRACT FROM AUTHOR]

The purpose of this study was to compare the influence of two survey modes – telephone random digit dialling (RDD) and stratified sampling of an online panel – on Canadian Problem Gambling Index (CPGI) scores and co-morbidities. An identical gambling survey was administered to 4000 South Koreans selected via random dialling of cell phone numbers and to 4330 South Korean members of an online panel. The online panelists were found to have significantly higher levels of tobacco use, drug or alcohol problems and mental health problems, higher employment status and lower happiness levels. However, even when controlling for these variables, CPGI scores for the online panel were significantly higher than those for the telephone interview. This appears to be due to higher levels of pathology among online panelists combined with potentially more valid self-reports due to enhanced anonymity. [ABSTRACT FROM PUBLISHER]

This article looks into the processes and outcomes of setting up and maintaining a probability-based longitudinal online survey, which is recruited face-to-face and representative of both the online and the offline population aged 16-75 in Germany. This German Internet Panel studies political and economic attitudes and reform preferences through bimonthly longitudinal online interviews of individuals. The results presented here demonstrate that a carefully designed and implemented online panel can produce high- quality data at lower marginal costs than existing panels that operate solely in a face-to-face mode. Analyses into the representativeness of the online sample showed no major coverage or nonresponse biases. Finally, including offline households in the panel is important as it improves the representation of the older and female segments of the population. [ABSTRACT FROM AUTHOR]

To examine the effects of lotteries on response behavior, five experiments were conducted in a nonprofit online panel. The type of lottery offered for study participation was a cash lottery, a voucher lottery, or a lottery of surprise gifts of either known or unknown value. The control group was not offered a lottery. Dependent measures were response, retention, and several facets of response quality. None of the lotteries significantly increased response, retention, or response quality. However, looking at effect sizes reveals a pattern across the five experiments that can inform and refine the practice of employing lotteries in nonprofit online panels. This pattern suggests that lotteries are more effective with panelists low in motivation than high in motivation. [ABSTRACT FROM AUTHOR]

Two incentive experiments on response behavior were conducted in a nonprofit online panel. Experiment 1 examined effects of a lottery and of the lottery’s splitting into multiple prizes. Two cash lotteries and a no-incentive control group were compared. One lottery was announced to be paid out in one lump sum, whereas the other lottery was split into multiple smaller prizes. Dependent variables were two facets of response quantity, namely response and retention, and two facets of response quality, namely nondifferentiation and item nonresponse. Moreover, panelists’ characteristics were tested whether they moderated the lottery effects. The lottery and its splitting did not significantly affect response behavior; however, in terms of effect sizes, splitting the lottery mildly decreased response quantity. Experiment 2 was in part aimed at replication. In addition, it examined the effect of offering study results. Dependent variables were response, retention, and nondifferentiation. The cash lottery mildly enhanced response quantity and quality, whereas splitting the lottery tended to decrease response quantity. Offering study results had no impact on response behavior, both as a stand-alone incentive and in combination with a lottery. The two experiments revealed moderating effects, but these were not stable across both studies. A share of invitees in Experiment 1 was reinvited in Experiment 2, thus enabling analysis of whether their lottery condition in Experiment 1 influenced their response behavior 5 months later in Experiment 2. No such longitudinal effects were found. [ABSTRACT FROM PUBLISHER]

A cross-national, longitudinal study of Canadian and US respondents examined impacts of the Vancouver Olympic Games (VOG) on images of the host country, Canada, and the Olympic Games (OG) itself. Questionnaires were completed before and after the VOG on four distinct attitude objects: Canada as a vacation destination, the country/people of Canada, and the VOG as a destination and an event. Repeated measures MANOVA indicated positive impacts of the VOG but differences across the two populations. Little change was seen in OG images and interest among Americans, but major improvements occurred among Canadians. Differences in perceptions of the domestic and foreign populations towards the host country appeared, confirming earlier research on domestic bias effects. Results indicated hosting the OG contributed to improved images for Americans of Canadians, Canada as a country and as a destination. Canadians evidenced increased pride in their own country. Findings indicated mutual positive benefits for Canada and the OG images and interest. [ABSTRACT FROM AUTHOR]

Evidence-based insight on the effectiveness of reminders in web-based data collection in online panels is scarce. Thirty-eight studies were conducted in three different online panels to examine the effect of reminders on response rates, retention rates, and two facets of response quality (i.e., item omissions and response nondifferentiation). The studies were summarized using a meta-analytical approach including moderator tests (N = 246,040). Results indicated that reminders in web-based data collection increase response (i.e., from 49.5% to 65.6%, ESpd = .152). However, reminders somewhat reduce retention (i.e., from 98.0% among prereminder respondents to 97.2% among reminded respondents, ESpd = −.008), which might be explained by differences in motivation and computer literacy between unreminded and reminded respondents. No differences were found between prereminder respondents and reminded respondents in item omissions and response nondifferentiation. [ABSTRACT FROM PUBLISHER]

Based on self-determination and social exchange theory, this study investigates the effects of extrinsic motivation, intrinsic motivation, and trust in anonymity on enduring survey participation over a period of 2 years. Trust in anonymity was expected to act as a moderator between motivations and the likelihood of repeated survey participation. Participants were N = 227 adolescent members of an academic online panel for youth research. Results of longitudinal logistic regression analyses demonstrated a steady decline in the probability of survey participation over time. Extrinsic but not intrinsic motivation significantly increased the probability of initial survey participation, whereas both, extrinsic and intrinsic motivation, buffered the declining probability of survey participation over time; however, only if trust in anonymity was comparably low. These results suggest that the beneficial effects of extrinsic and intrinsic motivations on enduring survey participation are especially prevalent if trust in anonymity is of low to medium size. [ABSTRACT FROM AUTHOR]

Abstract: To date there have been few large-scale studies empirically examining context factors and the relationships between food choice and meal occasion. This study was conducted online and used 24-h dietary recall to capture the foods eaten by a sample of 25 to 49-year-old Spanish people (n =831), as well as some of the situational and social contexts of those meal occasions. Canonical correspondence analysis of the 3086 recorded eating occasions was used to map food and contexts and reveal associations between food categories and the meal occasions in which they were consumed. The study confirmed, quantitatively, knowledge in the extant literature, for example, that foods eaten vary by time of day and that meal occasion vary by level of formality. The canonical analysis also mapped general associations between specific types of food and certain contexts, for example between categories ‘sandwich’ and ‘sweets and chocolate’ and working on the computer. The scope for uncovering specific associations between a focal food category and the contextual characteristics of meals that it is consumed in was demonstrated in the case of fruit. In this sample of 25 to 49-year-old Spanish people, the majority (66%) of eating occasions involving fruit were categorised as main meals, of which fruit was not the main component. The times of day when fruit was most often eaten was around 3p.m. and from around 7p.m. and later. Fruit was also eaten more often during the week than during weekends. Overall, the findings from this study demonstrate the value of research that empirically seeks to understand the role of context effects on food choice. The use of canonical correspondence analysis was new to meals research, as was the collection of dietary recall data online. In addition to its methodological contributions, this study also offers a detailed picture of dietary patterns in the sample. [Copyright &y& Elsevier]

A five-wave experiment examined the influence of incentives on response in online panels. One group of panelists was sent an advance gift after registering with the panel; the other group did not receive a gift. For participating in the studies, half of the panelists were repeatedly offered redeemable loyalty points. The others were offered inclusion in cash lotteries. At the outset, the advance gift significantly increased participation, but this effect dwindled linearly throughout the waves of the study. Initially, there was no difference in response between people with loyalty points and those offered to be included in the cash lottery. Over time, however, loyalty points relative to the cash lotteries became more attractive. In wave 1, the advance gift was especially useful when combined with the lottery. This effect faded in the course of the longitudinal study. Low-income panelists were more susceptible to the advance gift. [ABSTRACT FROM AUTHOR]

A longitudinal experiment examined the influence of a lottery of gift certificates on response and retention in a four-wave study conducted in an online panel. Independent of the lottery, people who responded in a given wave were more likely to respond in the next wave. This process was characterized to follow a first-order Markov chain. There was a direct positive effect of the lottery on response only at the first wave of the study. However, mediated by the Markov process, the positive effect of the lottery on response at the first wave was carried over into later waves. The lottery did not have any effect on retention. Furthermore, it was found that retention at a given wave is a reliable predictor for response at the next wave. This information could be used by survey managers to diagnose and act on any impending nonresponse. [ABSTRACT FROM AUTHOR]

Six incentive experiments were conducted in a nonprofit online panel. In each experiment, the incentive offered for participation was a cash lottery. The control group was not offered any incentive. The cash lottery was offered in two versions: Either the total payout of the lottery was mentioned, or the lottery was split into multiple prizes. Dependent measures included response and retention rates. The results of the six individual experiments were meta-analytically summarized. Cash lotteries relative to no incentives did not reliably increase response or retention; neither did it make a significant difference if one large prize or multiple smaller prizes were raffled. [ABSTRACT FROM AUTHOR]

• In the Netherlands: Fresh water recreational angling has been slowly decreasing between 2009 and 2017, whereas this trend was not found in marine angling. • Participation rate was much lower in marine angling than in fresh water. • Children had high participation rates in fresh water angling, but less so in marine angling. • Participation rates was higher for males than for females. • Lower educated had higher participation rates compared to higher educated. Since 2009 the Netherlands has conducted extensive online screening surveys to establish the number, trends and demographic profile of recreational anglers, resulting in a large dataset of almost 500.000 data records between 2009 and 2017. Participation in both marine and fresh water recreational angling were analysed using general linear models (GLM). Results showed a steady decline in the participation rate in fresh water angling. The participation rate in marine angling was smaller, and declined from 2009 to 2011, but remained similar afterwards. Analysis of demographics (age, gender, education and region) showed that males were overall much more likely to participate in recreational angling than females. Additionally, the age distribution differed for marine and fresh water. In marine water young adult males (age group (25,45]) had the highest participation rate, whereas in fresh water the youngest age group, (5,15], had the highest participation rate, closely followed by young adult males (age group (25,45]). Additionally, lower educated persons were more likely to participate in recreational angling than higher educated persons. This study provides more insight in the culture of Dutch recreational angling. Furthermore, the participation model can be used to predict future angling participation. [ABSTRACT FROM AUTHOR]

Plain English summary: Evidence shows that public and patient involvement in research has a positive effect on its quality and end-results. Thus, public and patient involvement in all stages of research is becoming commonplace. There are limited detailed examples however, that describe how to make this possible, especially for those doing PhD research. Doctoral researchers are often new to research practice or have limited experience and are often bound by strict time and financial constraints. It is also not usually a requirement of the award to involve public and patients in their research. Hence, they may not feel confident or motivated to involve or engage with public and patients during their research. We, four doctoral researchers, share examples from our own research studies that have included different approaches to public and patient involvement. Two studies formed public and patient advisory groups who helped design the research questions, data collection tools and recruitment methods. One enlisted the help of an online public and patient panel from a local hospital. A different study worked with patients from an established group to help define key medical words. We did face some challenges, such as the need to develop good group work skills and to apply for grants to cover reimbursement, but we all found it beneficial to involve patients in our studies. We noticed a positive effect on each study's progression and an improvement in our own self-esteem. In addition, having public and patient involvement helped reduce the isolation we felt as doctoral researchers. Thus, we strongly encourage more doctoral researchers to involve public and patients in their studies. Public and patient involvement (PPI) has been shown to have a positive impact on health and social care research. However, adequate examples describing how to operationalise effective PPI, especially in doctoral studies, are lacking. Hence, doctoral researchers new to research, or those with limited experience, can be discouraged from facilitating PPI in their research. This paper aims to describe and discuss in detail the approaches used by four doctoral researchers to incorporate PPI at different stages of their research studies from study design to disseminating findings. We aim to inform other doctoral researchers about the challenges and limitations relating to PPI that we faced. Through these, we share pragmatic recommendations for facilitating PPI during doctoral studies. The description of four case studies demonstrated that PPI could be incorporated at various stages during doctoral research. This has had a beneficial impact on our research study progression, researcher self-esteem and lastly, helped alleviate researcher isolation during doctoral studies. [ABSTRACT FROM AUTHOR]