Search

BackgroundEcological momentary assessment (EMA) has the potential to minimize recall bias by having people report on their experiences in the moment (momentary model) or over short periods (coverage model). This potential hinges on the assumption that participants provide their ratings based on the reporting time frame instructions prescribed in the EMA items. However, it is unclear what time frames participants actually use when answering the EMA questions and whether participant training improves participants’ adherence to the reporting instructions. ObjectiveThis study aims to investigate the reporting time frames participants used when answering EMA questions and whether participant training improves participants’ adherence to the EMA reporting timeframe instructions. MethodsTelephone-based cognitive interviews were used to investigate the research questions. In a 2×2 factorial design, participants (n=100) were assigned to receive either basic or enhanced EMA training and randomized to rate their experiences using a momentary (at the moment you were called) or a coverage (since the last phone call) model. Participants received five calls over the course of a day to provide ratings; after each rating, participants were immediately interviewed about the time frame they used to answer the EMA questions. A total of 2 raters independently coded the momentary interview responses into time frame categories (Cohen κ=0.64, 95% CI 0.55-0.73). ResultsThe results from the momentary conditions showed that most of the calls referred to the period during the call (57/199, 28.6%) or just before the call (98/199, 49.2%) to provide ratings; the remainder were from longer reporting periods. Multinomial logistic regression results indicated a significant training effect (χ21=16.6; P

BackgroundInterest in the measurement of the temporal dynamics of people’s emotional lives has risen substantially in psychological and medical research. Emotions fluctuate and change over time, and measuring the ebb and flow of people’s affective experiences promises enhanced insights into people’s health and functioning. Researchers have used a variety of intensive longitudinal assessment (ILA) methods to create measures of emotion dynamics, including ecological momentary assessments (EMAs), end-of-day (EOD) diaries, and the day reconstruction method (DRM). To date, it is unclear whether they can be used interchangeably or whether ostensibly similar emotion dynamics captured by the methods differ in meaningful ways. ObjectiveThis study aims to examine the extent to which different ILA methods yield comparable measures of intraindividual emotion dynamics. MethodsData from 90 participants aged 50 years or older were collected in a probability-based internet panel, the Understanding America Study, and analyzed. Participants provided positive and negative affect ratings using 3 ILA methods: (1) smartphone-based EMA, administered 6 times per day over 1 week, (2) web-based EOD diaries, administered daily over the same week, and (3) web-based DRM, administered once during that week. We calculated 11 measures of emotion dynamics (addressing mean levels, variability, instability, and inertia separately for positive and negative affect, as well as emotion network density, mixed emotions, and emotional dialecticism) from each ILA method. The analyses examined mean differences and correlations of scores addressing the same emotion dynamic across the ILA methods. We also compared the patterns of intercorrelations among the emotion dynamics and their relationships with health outcomes (general health, pain, and fatigue) across ILA methods. ResultsEmotion dynamics derived from EMAs and EOD diaries demonstrated moderate-to-high correspondence for measures of mean emotion levels (ρ≥0.95), variability (ρ≥0.68), instability (ρ≥0.51), mixed emotions (ρ=0.92), and emotional dialecticism (ρ=0.57), and low correspondence for measures of inertia (ρ≥0.17) and emotion network density (ρ=0.36). DRM-derived measures showed correlations with EMAs and EOD diaries that were high for mean emotion levels and mixed emotions (ρ≥0.74), moderate for variability (ρ=0.38-.054), and low to moderate for other measures (ρ=0.03-0.41). Intercorrelations among the emotion dynamics showed high convergence across EMAs and EOD diaries, and moderate convergence between the DRM and EMAs as well as EOD diaries. Emotion dynamics from all 3 ILA methods produced very similar patterns of relationships with health outcomes. ConclusionsEMAs and EOD diaries provide corresponding information about individual differences in various emotion dynamics, whereas the DRM provides corresponding information about emotion levels and (to a lesser extent) variability, but not about more complex emotion dynamics. Our results caution researchers against viewing these ILA methods as universally interchangeable.

BackgroundDaily diaries are extensively used for examining participants’ daily experience in behavioral and medical science. However, little attention is paid to whether participants recall their experiences within the time frames prescribed by the task. ObjectiveThis study aimed to describe survey respondents’ self-reported recall time frames and to evaluate the impact of different daily diary items on respondents’ reported affective states. MethodsIn this study, 577 participants completed a mood survey with one of the following 4 time frame instructions: (1) today, (2) since waking up today, (3) during the last 24 hours, or (4) in the last day. They were also asked to indicate the periods they considered when answering these items and to recall the instructional phrases associated with the items. ResultsAlmost all participants in the today (141/146, 96.6%) and since waking up today (136/145, 93.8%) conditions reported using periods consistent with our expectations, whereas a lower proportion was observed in the during the last 24 hours (100/145, 69.0%) condition. A diverse range of responses was observed in the in the last day condition. Furthermore, the instructions influenced the levels of some self-reported affects, although exploratory analyses were not able to identify the mechanism underlying this finding. ConclusionsOverall, these results indicate that today and since waking up today are the most effective instructional phrases for inquiring about daily experience and that investigators should use caution when using the other 2 instructional phrases.

Mental processes underlying people's responses to Ecological Momentary Assessments (EMA) have rarely been studied. In cognitive psychology, one of the most popular and successful mental process models is the drift diffusion model. It decomposes response time (RT) data to distinguish how fast information is accessed and processed ('drift rate'), and how much information is accessed and processed ('boundary separation'). We examined whether the drift diffusion model could be successfully applied to people's RTs for EMA questions and could shed light on between- and within-person variation in the mental process components underlying momentary reports. We analyzed EMA data (up to 6 momentary surveys/day for one week) from 954 participants in the Understanding America Study (29,067 completed measurement occasions). An item-response-theory diffusion model was applied to RTs associated with 5 momentary negative affect ratings. As hypothesized, both diffusion model parameters showed moderate stability across EMA measurement occasions. Drift rate and boundary separation together explained a majority of the variance in the observed RTs and demonstrated correspondence across different sets of EMA items, both within and between individuals. The parameters related in theoretically expected ways to within-person changes in activities (momentary work and recreation) and person-level characteristics (neuroticism and depression). Drift rate increased and boundary separation decreased over the study, suggesting that practice effects in EMA consist of multiple distinctive cognitive processes. The results support the reliability and validity of the diffusion model parameters derived from EMA and provide initial evidence that the model may enhance understanding of process underlying EMA affect ratings., Author(s): Stefan Schneider [sup.1] [sup.2] [sup.3] [sup.4] , Raymond Hernandez [sup.1] , Doerte U. Junghaenel [sup.1] [sup.2] [sup.3] [sup.4] , Bart Orriens [sup.4] , Pey-Jiuan Lee [sup.1] , Arthur A. [...]

Author(s): Arthur A. Stone [sup.1] [sup.2] , Stefan Schneider [sup.1] [sup.2] , Joshua M. Smyth [sup.3] , Doerte U. Junghaenel [sup.1] [sup.2] , Mick P. Couper [sup.4] , Cheng Wen [...], Participant selection bias is of concern to researchers conducting surveys of all types. For momentary data capture studies, such as Ecological Momentary Assessment, the level of burden associated with these techniques and the possibility of low uptake rates makes the concerns especially salient. This study invited 3,000 individuals to participate in a study of health and mood and recorded the uptake rates at various points in the process. Respondents expressing interest in participating in general were randomized into a one-time survey, a low-burden momentary study, or a high-burden momentary study. Overall, 85.9% of the sample did not respond to the study invitation (including confirming non-interest); 6.9% of the sample expressed interest in the study by completing a brief survey; 2.1% agreed to participate in the study when the protocol specifics were described (none of the study protocols were actually run). Whites were more likely to complete the survey. Of those completing the survey, individuals who reported higher income, a more "open" personality, better typing skills, better computer skills, who viewed the research topic as important, and who expressed interest in research on daily feelings more likely consented to being enrolled in the experiment. The number of prior surveys taken had an inverted-U shaped association with participation in this study. Finally, all individuals randomized to the one-time survey group agreed to participate compared to two-thirds of individuals in the momentary groups. These results suggest that participant selection bias may affect both one-time survey and momentary data capture studies, with the caveat that the degree of such bias will be related to a study's hypotheses.

Researchers have become increasingly interested in response times to survey items as a measure of cognitive effort. We used machine learning to develop a prediction model of response times based on 41 attributes of survey items (e.g., question length, response format, linguistic features) collected in a large, general population sample. The developed algorithm can be used to derive reference values for expected response times for most commonly used survey items.

To ensure the accuracy of self-reported data, it is important to reduce potential sources of bias such as the unwanted influence of prior questions on subsequent questions, the so-called item context effect. This article attempts to replicate the finding that evaluative subjective well-being was affected by a preceding item, a question about the political atmosphere in the country; it also examines manipulations that could mitigate the impact of the context-inducing item on well-being. Study 1 used a sample of 4,500 participants recruited from Amazon's Mechanical Turk; it examined the effect of three manipulations based on adding buffer questions or adding text to reorient participants' attention. A context effect was found, and one manipulation mitigated the context effect. Study 2 used a nationally representative sample (n = 906); it only replicated the context effect. These results reaffirm the importance of carefully considering item context effects in survey research.

Introduction: Delayed responses are a common yet often overlooked aspect of participant compliance in ecological momentary assessment (EMA) research. This study investigated whether response delays introduce selection bias in the moments captured by EMA. Methods: Participants (n = 339) self-reported their physical activity behaviors using EMA five times a day over 7 days while wearing a continuous physical activity monitor. The continuous activity monitor data provided an objective reference value to evaluate potential biases in delayed EMA self-reports. Results: Results showed that participants were significantly more likely to delay EMA responses when they were prompted during higher levels of physical activity, and they subsequently reduced their activity levels, postponing their response until they were in a significantly less active state. There was no significant evidence that response delays systematically biased the levels of EMA reported activities, although delayed responses were associated with significantly more random errors in EMA reports (with small effect sizes). Discussion: The results suggest that respondents self-select the moments for answering EMA surveys based on their current activity levels, but brief response delays do not translate into marked reductions in the quality of EMA data. [ABSTRACT FROM AUTHOR]

Objectives Self-reported survey data are essential for monitoring the health and well-being of the population as it ages. For studies of aging to provide precise and unbiased results, it is necessary that the self-reported information meets high psychometric standards. In this study, we examined whether the quality of survey responses in panel studies of aging depends on respondents' cognitive abilities. Methods Over 17 million survey responses from 157,844 participants aged 50 years and older in 10 epidemiological studies of aging were analyzed. We derived 6 common statistical indicators of response quality from each participant's data and estimated the correlations with participants' cognitive test scores at each study wave. Effect sizes (correlations) were synthesized across studies, cognitive tests, and waves using individual participant data meta-analysis methods. Results Respondents with lower cognitive scores showed significantly more missing item responses (overall effect size ρ ^ = −0.144), random measurement error (⁠ ρ ^ = −0.192), Guttman errors (⁠ ρ ^ = −0.233), multivariate outliers (⁠ ρ ^ = −0.254), and acquiescent responses (⁠ ρ ^ = −0.078); the overall effect for extreme responses (⁠ ρ ^ = −0.045) was not significant. Effect sizes were consistent across studies, modes of survey administsration, and different cognitive functioning domains, although some cognitive domain specificity was also observed. Discussion Lower-quality responses among respondents with lower cognitive abilities add random and systematic errors to survey measures, reducing the reliability, validity, and reproducibility of survey study results in aging research. [ABSTRACT FROM AUTHOR]

Psychology has witnessed a dramatic increase in the use of intensive longitudinal data (ILD) to study within-person processes, accompanied by a growing number of indices used to capture individual differences in within-person dynamics (WPD). The reliability of WPD indices is rarely investigated and reported in empirical studies. Unreliability in these indices can bias parameter estimates and yield erroneous conclusions. We propose an approach to (a) estimate the reliability and (b) correct for sampling error of WPD indices using "Level-1 variance-known" (V-known) multilevel models (Raudenbush & Bryk, 2002). When WPD indices are calculated for each individual, the sampling variance of the observed WPD scores is typically falsely assumed to be zero. V-known models replace this "zero" with an approximate sampling variance fixed at Level 1 to estimate the true variance of the index at Level 2, following random effects meta-analysis principles. We demonstrate how V-known models can be applied to a broad range of emotion dynamics commonly derived from ILD, including indices of the average level (mean), variability (intraindividual standard deviation), instability (probability of acute change), bipolarity (correlation), differentiation (intraclass correlation), inertia (autocorrelation), and relative variability (relative standard deviation) of emotions. A simulation study shows the usefulness of V-known models to recover the true reliability of these indices. Using a 21-day diary study, we illustrate the implementation of the proposed approach to obtain reliability estimates and to correct for unreliability of WPD indices in real data. The techniques may facilitate psychometrically sound inferences from WPD indices in this burgeoning research area. [ABSTRACT FROM AUTHOR]