Search

We examined knowledge-based inference in 6-, 8- and 10-year-olds. Participants listened to texts where the number of clues for an inference was manipulated and then judged whether single-word probes (target inference, competing inference, literal word from the text and an unrelated concept) were related to the story. Accuracy and response times were analyzed with mixed-effects models. Inference accuracy was higher with three clues, and older children responded to the inference more accurately and quickly than younger children. The inference and literal probes were compared: 10-year-olds responded more accurately and quickly to the inference; however, 6-year-olds were more accurate for the literal. We propose that by 10-years of age, children are able to activate knowledge-based inferences while listening to text. All age groups had some inhibition difficulty, with less accurate and slower responses for the competing inference than the unrelated concept. Inference instruction should be sensitive to developmental differences in inferential processing.

Small but robust differences in cognition exist between the sexes in adult populations. Studying sex differences in children's cognition can bring insight into when, where and how these differences might emerge in development. Here, we focus on differences in working memory because of its importance in underpinning a wide range of complex cognitive tasks and developmental outcomes for children. Using two levels of difficulty on a standard test of working memory (N-Back), data from 104 6- to 7-year-olds in Cuba showed that boys have quicker reaction times, but girls provide more accurate responses. With a comparable true positive rate between boys and girls, the sex differences in both accuracy and reaction times were limited to false-positive responses. Sex differences were consistent across levels of task difficulty and persisted after speed-accuracy trade-offs were considered. We argue that avoiding false positives requires a particularly strong role for inhibitory control and that this emerges in development according to a different maturational schedule for girls than it does for boys, underpinned by quantitative and qualitative differences in the development of brain areas that support this function.

For autistic students receiving special education services, little is known about their relative strengths, weaknesses, and enjoyment across different math content areas; their overall math interest and persistence are also not well-studied. Using the 2017 eighth-grade National Assessment of Education Progress data, this study finds, relative to general education peers with the same math proficiency level, autistic students scored higher and exhibited faster speed in solving visuospatial problems (e.g. identifying figures), but scored lower on math word problems with complex language or social context. Autistic students reported a higher level of enjoyment in solving math problems related to finding areas of shapes or figures but a lower level of persistence than their non-autistic, general education peers. Our work points out the need to help autistic students overcome their weaknesses in word problems and develop their mathematical persistence.

The present study examined whether scaling direction and perceptual modality affect children's spatial scaling. Children aged 6-8 years (N = 201) were assigned to a visual, visuo-haptic, and haptic condition in which they were presented with colourful, embossed graphics. In the haptic condition, they were asked to wear a blindfold during the test trials. Across several trials, children were asked to learn about the position of a target in a map and to localize a disc at the same location in a referent space. Scaling factor was manipulated systematically, so that children had to either scale up or scale down spatial information. Their absolute deviations from the correct target location, reversal and signed errors, and response times served as dependent variables. Results revealed higher absolute deviations and response times for the haptic modality as opposed to the visual modality. Children's signed errors, however, showed similar response strategies across the perceptual conditions. Therefore, it seems that a functional equivalence between vision and touch seems to emerge slowly across development for spatial scaling. With respect to scaling directions, findings showed that absolute deviations were affected by scaling factors, with symmetric increases in scaling up and scaling down in the haptic condition. Conversely, children showed an unbalanced pattern in the visual conditions, with higher accuracy in scaling down as opposed to scaling up. Overall, our findings suggest that visibility seems to factor into children's scaling process.

When a highly salient distractor is present in a search array, it speeds target absent visual search and increases errors during target present visual search, suggesting lowered quitting thresholds (Moher in Psychol Sci 31(1):31-42, 2020). Missing a critical target in the presence of a highly salient distractor can have dire consequences in real-world search tasks where accurate target detection is crucial, such as baggage screening. As such, the current study examined whether emphasizing either accuracy or speed would eliminate the distractor-generated quitting threshold effect (QTE). Three blocks of a target detection search task which included a highly salient distractor on half of all trials were used. In one block, participants received no instructions or feedback regarding performance. In the remaining two blocks, they received instructions and trial-by-trial feedback that either emphasized response speed or response accuracy. Overall, the distractor lowered quitting thresholds, regardless of whether response speed or response accuracy was emphasized in a block of trials. However, the effect of the distractor on target misses was smaller when accuracy was emphasized. It, therefore, appears that while the distractor QTE is not easily eradicated by explicit instructions and feedback, it can be shifted. As such, future research should examine the applicability of these and similar strategies in real-world search scenarios.

Purpose: Individuals with developmental language disorder (DLD) often exhibit slower processing on time-based tasks in comparison with age-matched peers. Processing speed has been linked to various linguistic skills and might serve as a global indicator of individual differences in language abilities. Despite an extensive literature on processing speed in DLD, it remains unclear whether slower processing is domain general or restricted to linguistic and/or auditory tasks. Method: This meta-analysis used robust variance estimation to compare response/reaction times (RTs) of DLD and age-matched groups (N = 812 DLD, 870 neurotypical; M[subscript age] [DLD] = 8.9 years, range: 4.3--22.7 years). Moderators included task (simple RT, choice RT, naming, congruent/baseline conditions of interference control tasks), stimulus type (linguistic/nonlinguistic), stimulus modality (auditory/nonauditory), and response modality (verbal/nonverbal). Age and publication year were covariates. Results: The overall effect based on 46 studies and 144 estimates indicated longer mean RTs in DLD groups (g = 0.47, p < 0.001, 95% CI [0.38, 0.55]). Moderator analyses indicated larger effects when tasks required verbal as opposed to nonverbal responses. No other moderators approached significance. All subgroup analyses were significant, indicating longer mean RTs in DLD groups across tasks, stimulus types, stimulus modalities, and response modalities. Conclusions: Individuals with DLD exhibit longer RTs across verbal and nonverbal tasks, which may contribute to observed difficulties in language, motor skills, and executive functioning. Simple processing speed measures should be included in screening for language delays but may not be suitable for differential diagnosis, given that slower processing may occur across multiple disorders.

Background: Examinees may not make enough effort when responding to test items if the assessment has no consequence for them. These disengaged responses can be problematic in low-stakes, large-scale assessments because they can bias item parameter estimates. However, the amount of bias, and whether this bias is similar across administrations, is unknown. This study compares the degree of disengagement (i.e., fast and non-effortful responses) and the impact of disengagement on item parameter estimates in the Programme for International Student Assessment (PISA) across the 2015 and 2018 administrations. Method: We detected disengaged responses at the item level based on response times and response behaviors. We used data from the United States and analyzed 51 computer-based mathematics items administered in both PISA 2015 and PISA 2018. We compared the percentage of disengaged responses and the average scores of the disengaged responses for the 51 common items. We filtered disengaged responses at the response- and examinee-levels and compared item difficulty (P+ and b) and item discrimination (a) before and after filtering. Results: Our findings suggested that there were only slight differences in the amount of disengagement in the U.S. results for PISA 2015 and PISA 2018. In both years, the amount of disengagement was less than 5.2%, and the average scores of disengaged responses were lower than the average scores of engaged responses. We did not find any serious impact of disengagement on item parameter estimates when we applied response-level filtering; however, we found some bias, particularly on item difficulty, when we applied examinee-level filtering. Conclusions: This study highlights differences in the amount of disengagement in PISA 2015 and PISA 2018 as well as the implications of the decisions made for handling disengaged responses on item difficulty and discrimination. The results of this study provide important information for reporting trends across years.

Given the ongoing development of computer-based tasks, there has been increasing interest in modelling students' behaviour indicators from log file data with contextual variables collected via questionnaires. In this work, we apply a latent regression model to analyse the relationship between latent constructs (i.e., performance, speed, and exploration behaviour) and contextual variables among Scandinavian students (i.e., Norway, Sweden, and Denmark) during their completion of three interactive mathematics tasks in the 2012 Program for International Student Assessment (PISA). The purpose of this study is threefold: (1) to investigate whether a factor model is adequate for explaining the relationship between extracted process indicators from log files and students' performance on the three items; (2) to evaluate whether measurement invariance holds for the three analysed countries; and (3) to determine how well contextual variables [i.e., a student's background, availability, use of information and communication technology (ICT) resources, and learning conditions for mathematical literacy] correlate with the model's latent constructs. Our findings reveal that a three-factor CFA model is well-suited to the analysed data and that a weak measurement invariance model performs best. By including contextual variables in the modelling, we also highlight the differences in Scandinavian students' profiles. Specifically, higher economic social cultural status (ESCS) consistently led to higher math proficiency across all three countries. Norway did not show any gender differences in math proficiency, response time, or exploration behaviours. Experience with math tasks at school led to more exploration behaviours across all three countries. Swedish and Norwegian students who had more experience with pure math tasks at school were likely to obtain higher mathematics achievement scores and tended to interact more frequently with the testing platform when answering the three math items. When Danish students had higher ICT scores (i.e., more ICT resources available at school and home), they had lower response accuracy. There was, however, no effect of ICT on the three factors among Swedish and Norwegian students. Finally, we also discuss the implications and limitations of this study.

In this article, we explore the extraction of recursive nested structure in the processing of binary sequences. Our aim was to determine whether humans learn the higher-order regularities of a highly simplified input where only sequential-order information marks the hierarchical structure. To this end, we implemented a sequence generated by the Fibonacci grammar in a serial reaction time task. This deterministic grammar generates aperiodic but self-similar sequences. The combination of these two properties allowed us to evaluate hierarchical learning while controlling for the use of low-level strategies like detecting recurring patterns. The deterministic aspect of the grammar allowed us to predict precisely which points in the sequence should be subject to anticipation. Results showed that participants' pattern of anticipation could not be accounted for by "flat" statistical learning processes and was consistent with them anticipating upcoming points based on hierarchical assumptions. We also found that participants were sensitive to the structure constituency, suggesting that they organized the signal into embedded constituents. We hypothesized that the participants built this structure by merging recursively deterministic transitions.

Frequency and proficiency have been found to play an important role in second language (L2) phrasal processing. However, existing research has largely focused on English and other European languages, with other commonly used languages, such as Chinese, being largely disregarded. To fill this gap, we carried out two experiments to investigate how frequency--operationalized as a dichotomy (collocation vs. control) and a continuum (log-transformed corpus frequency)--and language proficiency affect phrasal processing in first language (L1) and L2 speakers of Chinese. The results of a grammaticality judgment task (Experiment 1) and an eye-tracking experiment (Experiment 2) largely converged to show that not only L1 speakers but also L2 learners are sensitive to phrase frequency manipulations, with collocations and higher frequency phrases being processed faster than controls and lower frequency phrases, respectively. Additionally, the analysis of eye movements--but not reaction times--showed a relationship between proficiency and phrase frequency. Finally, the use of the eye-tracking methodology allowed us to tap into the mechanisms associated with earlier and later stages of phrasal processing, and to analyze multiple interest areas.

Listeners can access information about a speaker such as age, gender identity, socioeconomic status, and their linguistic background upon hearing their speech. However, it is still not clear if listeners use these factors to assess speakers' speech. Here, an audio-visual (matched-guise) test is used to measure whether listeners' accentedness judgments can be modulated depending on the type of face that they see. American and Indian English were used as different English varieties and presented with either a White female face or a South Asian female face. Results show that listeners' accentedness judgments increased for Indian English compared to American English. Importantly, the increase in accentedness judgments was also observed when both American English and Indian English were presented with a South Asian face compared to a White face. These findings suggest that linguistic evaluations are modulated by non-linguistic factors and that speech perception is socially gated.

The capacities of detecting visual regularities were tested among twenty typical (age 11.1 ± 0.32), and twenty poor (age 11.03 ± 0.28) native-Arab readers. Two stages were implemented, passive exposure to visual regularities and forced decision task. In the first stage, the participants were passively presented with four shapes; each shape was displayed with unique invariant features, which present the regularities of the shape in addition to variant features. In the second stage, the participants were presented with forced decision task and were asked to make an acceptance or rejection of presented 160 shapes according to their own preferences regarding the shapes familiarity. Eighty of the presented shapes were compatible with the visual regularities as was presented in the passive learning stage where the other shapes presented the non-compatible condition. The results indicated that typical readers showed significantly higher levels of true responses in the decision task. In addition, among the typical readers group shorter response times were significantly recorded for the compatible items compared to non-compatible items. Such differences in response times were not observed among the poor readers. The results support the assumption that poor orthographic learning among poor readers could be associated with inefficient statistical learning capacities.

Successful word recognition requires that listeners attend to differences that are phonemic in the language while also remaining flexible to the variation introduced by different voices and accents. Previous work has demonstrated that American-English-learning 19-month-olds are able to balance these demands: although one-off one-feature mispronunciations typically disrupt English-learning toddlers' lexical access, they no longer do after toddlers are exposed to a novel accent in which these changes occur systematically. The flexibility to deal with different types of variation may not be the same for toddlers learning different first languages, however, as language structure shapes early phonological biases. We examined French-learning 19-month-olds' sensitivity and adaptation to a novel accent that shifted either the standard pronunciation of /a/ from [a] to [open-mid front unrounded vowel] (Experiment 1) or the standard pronunciation of /p/ from [p] to [t] (Experiment 2). In Experiment 1, French-learning toddlers recognized words with /a/ produced as [open-mid front unrounded vowel], regardless of whether they were previously exposed to an accent that contained this vowel shift or not. In Experiment 2, toddlers did not recognize words with /p/ pronounced as [t] at test unless they were first familiarized with an accent that contained this consonant shift. These findings are consistent with evidence that French-learning toddlers privilege consonants over vowels in lexical processing. Together with previous work, these results demonstrate both differences and similarities in how French- and English-learning children treat variation, in line with their language-specific phonological biases.

Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.

Acting upon target stimuli from the environment becomes faster when the targets are preceded by a warning (alerting) cue. Accordingly, alerting is often used to support action in safety-critical contexts (e.g., honking to alert others of a traffic situation). Crucially, however, the benefits of alerting for action have been established using laboratory tasks assessing only simple choice reactions. Real-world actions are considerably more complex and mainly consist of sensorimotor sequences of several sub-actions. Therefore, it is still unknown if the benefits of alerting for action transfer from simple choice reactions to such sensorimotor sequences. Here, we investigated how alerting affected performance in a sequential action task derived from the Trail-Making-Test, a well-established neuropsychological test of cognitive action control (Experiment 1). In addition to this task, participants performed a classic alerting paradigm including a simple choice reaction task (Experiment 2). Results showed that alerting sped up responding in both tasks, but in the sequential action task, this benefit was restricted to the first action of a sequence. This was the case, even when multiple actions were performed within a short time (Experiment 3), ruling out that the restriction of alerting to the first action was due to its short-lived nature. Taken together, these findings reveal the existence of an interface between phasic alertness and action control that supports the next action.

Individual differences in cognitive performance depend on age, skill, and type of task. Nonetheless, whether performance is measured with accuracy (ACC) or with the trade-off between responding speed and accuracy (SAT) could render subtle different relationships. Age and skill might associate more strongly with SAT performance in reasoning tasks, whereas they might relate more similarly with either ACC or SAT in knowledge tasks. These expectations were evaluated here with data from the cognitively taxing domain of chess (n = 259). Age was associated more strongly with the SAT than with the ACC measures in reasoning tasks. In contrast, skill related more robustly with the ACC than with the SAT measures in knowledge tasks. The main findings suggest that the associations of age and skill with performance vary depending on the type of task, but also on whether considering accuracy or speed-accuracy measures of cognitive performance.

In low-stakes assessment settings, students' performance is not only influenced by students' ability level but also their test-taking engagement. In computerized adaptive tests (CATs), disengaged responses (e.g., rapid guesses) that fail to reflect students' true ability levels may lead to the selection of less informative items and thereby contaminate item selection and ability estimation procedures. To date, researchers have developed various approaches to detect and remove disengaged responses after test administration is completed to alleviate the negative impact of low test-taking engagement on test scores. This study proposes an alternative item selection method based on Maximum Fisher Information (MFI) that considers test-taking engagement as a secondary latent trait to select the most optimal items based on both ability and engagement. The results of post-hoc simulation studies indicated that the proposed method could optimize item selection and improve the accuracy of final ability estimates, especially for low-ability students. Overall, the proposed method showed great promise for tailoring CATs based on test-taking engagement. Practitioners are encouraged to consider incorporating engagement into the item selection algorithm to enhance the validity of inferences made from low-stakes CATs.

Within the last two decades, researchers have begun to investigate how L2 learners process syntactic, morpho-syntactic, and lexical information during the comprehension of L2 sentences. The present study aimed to add to research by investigating how L1 influences L2 processing of sentences indicating plurality in constructions involving numerals. More specifically, the research investigated whether Persian speaking low-proficient L2 learners of English showed L1 transfer effects performing a self-paced online reading task. To address this issue, employing the IBEX software, reaction times on critical regions and accuracy rates of learners' performance were measured on four types of structures (i.e., "Numeral + Count Noun," "Numeral + Classifier + Mass Noun," "Numeral + Classifier + Count Noun," and "Numeral + Non-referential Noun + Noun") and two sentence types ("ill-formed" vs. "well-formed"). Statistical analysis indicated the effect of both structure and sentence type on reaction times on the critical regions studied. Results also indicated traces of L1 effect in processing "Numeral + Classifier + Mass Noun" and "Numeral + Classifier + Count Noun" structures. Concerning the "Numeral + Count Noun" and "Numeral + Classifier + Non-referential Noun + Noun," no clear evidence for L1 transfer effect was observed. Further studies employing a larger sample size, investigating the issue at higher proficiency levels, and having native English speakers as control group are suggested.

Common response-time-based approaches for non-effortful response behavior (NRB) in educational achievement tests filter responses that are associated with response times below some threshold. These approaches are, however, limited in that they require a binary decision on whether a response is classified as stemming from NRB; thus ignoring potential classification uncertainty in resulting parameter estimates. We developed a response-time-based probabilistic filtering procedure that overcomes this limitation. The procedure is rooted in the principles of multiple imputation. Instead of creating multiple plausible replacements of missing data, however, multiple data sets are created that represent plausible filtered response data. We propose two different approaches to filtering models, originating in different research traditions and conceptualizations of response-time-based identification of NRB. The first approach uses Gaussian mixture modeling to identify a response time subcomponent stemming from NRB. Plausible filtered data sets are created based on examinees' posterior probabilities of belonging to the NRB subcomponent. The second approach defines a plausible range of response time thresholds and creates plausible filtered data sets by drawing multiple response time thresholds from the defined range. We illustrate the workings of the proposed procedure as well as differences between the proposed filtering models based on both simulated data and empirical data from PISA 2018.

Response times (RTs) have recently attracted a significant amount of attention in the literature as they may provide meaningful information about item preknowledge. In this study, a new model, the Deterministic Gated Lognormal Response Time (DG-LNRT) model, is proposed to identify examinees with item preknowledge using RTs. The proposed model was applied to two different data sets and performance was assessed with false-positive rates, true-positive rates, and precision. The results were compared with another recently proposed Z-statistic. Follow-up simulation studies were also conducted to examine model performance in settings similar to the real data sets. The results indicate that the proposed model is viable and can help detect item preknowledge under certain conditions. However, its performance is highly dependent on the correct specification of the compromised items.

Understanding the cognitive processes, skills and strategies that examinees use in testing is important for construct validity and score interpretability. Although response processes evidence has long been included as an important aspect of validity (i.e., "Standards for Educational and Psychological Tests," 1999), relevant studies are often lacking, especially in large scale educational and psychological testing. An important method for studying response processes involves explanatory mathematical modeling of item responses and item response times from variables that represent sources of cognitive complexity. For many item types, examinees may differ in strategies applied to responding to items. Mixture class item response theory models can identify latent classes of examinees with different processes, skills and strategies based on their pattern of item responses. This study will illustrate the use of response times in conjunction with explanatory item response theory models and mixture models, to provide information relevant to test validity and, hence, to score interpretations.

In this study, I examined how 34 Spanish-speaking English language learners (ELLs) used their linguistic resources (English and Spanish) and language modes (oral and written language) to demonstrate their knowledge of proportional reasoning in a dual language mathematics assessment task. The assessment allows students to see the item in both English and Spanish, to listen to someone read aloud the item in English or Spanish, to provide oral or written responses in English, Spanish, or both for selected constructed-response items. The assessment was piloted in three algebra classrooms at a middle school and a high school in the United States. Data included students' responses, the amount of time it took students to complete each item, the language that the students used in each item, the language mode they used to complete the constructed-response items, the number of times the students listened to the read alouds, and responses to a short survey. This study reports on how Spanish-speaking ELLs performed on the assessment task, the linguistic resources and the language modes they used to access and to respond to the task, and how language and mode use impacted performance and the time spend on the test items.

Research Findings: The study investigated whether preschool code-related home literacy experiences had direct associations with regular and irregular word reading in the first year of primary school as well as exploring whether there were indirect associations between these experiences and later word reading via children's language skills or inhibitory control. The parents of 274 preschool children completed a home learning questionnaire at time 1 (M[subscript age] = 3:11). At time 2, the children completed phonological awareness, vocabulary, inhibitory control and nonverbal reasoning assessments (M[subscript age] = 4:3) and at time 3 a word reading assessment (M[subscript age] = 5:3). Letter-sound interactions (a code-related home literacy index that included discussions about letter-sound associations) bore significant associations with children's word reading, whereas letter activities (a code-related index that was less focussed on letter-sound links) did not. Path analyses indicated that letter-sound interactions directly predicted regular word reading and predicted regular and irregular word reading indirectly via children's phonological awareness. These findings highlight that different aspects of code-related home literacy experiences are differentially associated with later word reading skills. Practice and Policy: The findings suggest that parents' integration of interactive, age-appropriate discussions that focus on letter-sound associations into children's everyday experiences may support emerging word decoding skills.

When an audience member becomes immersed, their attention shifts towards the media and story, and they allocate cognitive resources to represent events and characters. Here, we investigate whether it is possible to measure immersion using continuous behavioural and physiological measures. Using television and film clips, we validated dual-task reaction times, heart rate, and skin conductance against self-reported narrative engagement. We find that reaction times to a secondary task were strongly positively correlated with self-reported immersion: slower reaction times were indicative of greater immersion, particularly emotional engagement. Synchrony in heart rate across participants was associated with self-reported attentional and emotional engagement with the story, although we found no such relationship with skin conductance. These results establish both dual-task reaction times and heart rate as candidate measures for the real-time, continuous, assessment of audience immersion.

Preventing panel members from attriting is a fundamental challenge for panel surveys. Research has shown that response behavior in earlier waves (response or nonresponse) is a good predictor of panelists' response behavior in upcoming waves. However, response behavior can be described in greater detail by considering the time until the response is returned. In the present study, we investigated whether respondents who habitually return their survey late and respondents who switch between early and late response in multiple waves are more likely to attrit from a panel. Using data from the GESIS Panel, we found that later response is related to a higher likelihood of attrition (AME = 0.087) and that response-time stability is related to a lower likelihood of attrition (AME = -0.013). Our models predicted most cases of attrition; thus, survey practitioners could potentially predict future attriters by applying these models to their own data.

To develop theories of how comprehenders extract the message from a linguistic stream, it is critical to understand how they conceptually represent referents. The experiments reported here focus on singular collective nouns (e.g., "committee," "team"), which introduce a single group into the discourse and test whether they nonetheless are conceptually plural (i.e., construed as consisting of multiple entities) by using the spatial-numerical association of response codes (SNARC) paradigm (Dehaene & Changeux, 1993). In this paradigm, participants are typically faster to respond to smaller numbers or numerical stimuli when making a response on their left and faster to respond to larger numbers or numerical stimuli when making a response on their right. In three experiments, participants saw German words on a computer screen and decided whether each one described a single entity or something that could be subdivided into multiple entities (Experiment 1) or whether they would use "ist" or "sind" ("is" or "are") in combination with the word if it were the subject of a sentence (Experiments 2 and 3). The mapping of responses to participants' left and right hands was counterbalanced. Experiment 1 failed to show a grammatical SNARC effect. Experiments 2 and 3 showed a grammatical SNARC effect that extended to collective noun phrases. The results of these experiments suggest that collective noun phrases are instantiated as conceptually plural in comprehenders' minds. We discuss the differential task effects and the implications of these data on theories of language comprehension.

There has been considerable interest in what components of decision-making change when speed or accuracy is stressed. In many early studies, quite strict assumptions were made about parameter invariance across experimental conditions (sometimes called selective influence). Here we fit the standard diffusion model to the data from four large experiments with speed--accuracy instructions (with over a million total responses), allowing all model parameters to vary freely between the speed and accuracy conditions. Results show that most of the observed differences between speed and accuracy conditions appear in the boundary separation parameter, followed by nondecision time, with small effects on drift rates. However, changes in drift rates are accompanied by changes in across-trial variability in drift rate, which cancels out the effect of drift rate on accuracy and response time. Another analysis in which across-trial variance in drift rate was kept the same in fits to speed and accuracy conditions produced no difference in drift rates. Generally, if speed is stressed moderately, then both boundary separation and nondecision time are reduced and any changes in drift rate are compensated for by changes in the across-trial variance in drift rates. If speed is stressed to a high degree (Starns et al., 2012), boundary separation, nondecision time, and drift rates are reduced. This is because (we hypothesize) encoding is restricted leading to a lower degree of perceptual information or match with memory.

Questioning individuals about future behavior influences the subsequent performance of that behavior. This question-behavior effect may be enhanced when questions include positive self-identity prompts. To evaluate the effectiveness of this intervention in a testing context, we randomly assigned examinees to one of three conditions prior to completing a low-stakes test: answering five questions regarding intended effort prior to the test, answering five questions regarding intended effort that referenced positive self-identity prior to the test, or a control condition. We then administered two multiple-choice tests and collected two measures of test-taking effort: self-reported effort and response time effort. Answering questions about effort prior to completing the tests resulted in higher self-reported effort and perceived test importance. Moreover, in the question conditions, significantly fewer examinees were identified to be filtered from the dataset due to low effort. The easy strategy of asking examinees to report their intended effort appears to increase test-taking effort.

Purpose: Tests measuring speech comprehension and listening effort for cochlear implant (CI) users may reflect important aspects of real-world speech communication. In this study, we describe the development of a multiple-talker, English-language sentence verification task (SVT) for use in adult CI users to measure speech comprehension and listening effort. We also examine whether talker differences affect speech comprehension and listening effort. Method: Thirteen experienced adult CI users participated in the study and underwent testing using a newly developed multiple-talker SVT. Participants were sequentially presented with audio recordings of unique sentences spoken in English by six different talkers. Participants classified each sentence as either true or false. Accuracy of classification and the response time (RT) for correct responses were used as measures of comprehension and listening effort, respectively. The effect of talker on the results was further analyzed. Results: All 13 participants successfully completed the SVT. The mean verification accuracy for participants was 87.2% ± 8.8%. The mean RT for correct responses across participants was 1,050 ms ± 391 ms. When stratified by talker, verification accuracy ranged from 83.7% to 95.2% and mean RTs across participant ranged from 786 ms to 1,254 ms. Talker did not have a significant effect on sentence classification accuracy, but it did have a significant effect on RTs (p < 0.001). Conclusions: The SVT is an easily implemented test that can assess speech comprehension and listening effort simultaneously. CI users may experience increased effort for comprehending certain talkers' speech, despite showing similar levels of comprehension accuracy.

In this study, we asked how the emotional status, i.e., valence and arousal, and concreteness of idioms contribute to their processing. Additionally, we asked whether the contribution of emotional factors and concreteness is modulated by other linguistic constraints, specifically idiom familiarity and decomposability, that has been shown to impact idiom processing. Participants read short idiomatic phrases (e.g., he kicked the bucket), word-by-word and for comprehension while their reaction time was recorded. The results showed that the emotional status of idioms contribute to their processing and this contribution is modulated by familiarity and decomposability levels of idioms in different ways. In particular, the impact of valence (i.e., the degree an idiom is pleasant/unpleasant) was modulated by familiarity, and the impact of arousal was modulated by decomposability. We did not find strong evidence for the contribution of concreteness for idiom processing. Our findings are aligned with theories of semantic representation, which suggest that besides linguistic information, sensory-motor and affective information are fundamental in representing meaning.

Learning from videos is becoming an important part of educational activities. Whereas outlines have great potential, particularly during information seeking (IS), few studies have so far explored the effect of their presence in videos. The two aims of the present study were thus to 1) evaluate the influence of outlines on IS and learning, and 2) analyze the extent to which an IS task can optimize the use of outlines during learning. As outlines are known to have a positive impact on video representation during IS tasks, we predicted that performing an IS task prior to a learning task featuring an outline would result in better content representation by learners. A total of 103 participants were divided into four groups (2 × 2 factorial design): with or without an outline, and with or without IS questions before learning. Results indicated that the presence of an outline promotes topic recall in terms of both the number and organization of the topics recalled, and performing an IS task with an outline before learning promotes content memorization to a small extent. Results are discussed in terms of engagement and metacognitive processes.

Visual (perceptual) reasoning is a critical skill in many medical specialties, including pathology, diagnostic imaging, and dermatology. However, in an ever-compressed medical curriculum, learning and practicing this skill can be challenging. Previous studies (including work with pigeons) have suggested that using reward-feedback-based activities, novices can gain expert levels of visual diagnostic accuracy in shortened training times. But is this level of diagnostic accuracy a result of image recognition (categorization) or is it the acquisition of diagnostic expertise? To answer this, the authors measured electroencephalographic data (EEG) and two components of the human event-related brain potential (reward positivity and N170) to explore the nature of visual expertise in a novice-expert study in pathology visual diagnosis. It was found that the amplitude of the reward positivity decreased with learning in novices (suggesting a decrease in reliance on feedback, as in other studies). However, this signal remained significantly different from the experts whose reward positivity signal did not change over the course of the experiment. There were no changes in the amplitude of the N170 (a reported neural marker of visual expertise) in novices over time. Novice N170 signals remained statistically and significantly lower in amplitude compared to experts throughout task performance. These data suggest that, while novices gained the ability to recognize (categorize) pathologies through reinforcement learning as quantified by the change in reward positivity, increased accuracy, and decreased time for responses, there was little change in the neural marker associated with visual expertise (N170). This is consistent with the multi-dimensional and complex nature of visual expertise and provides insight into future training programs for novices to bridge the expertise gap.

When designing or modifying a test, an important challenge is controlling its speededness. To achieve this, van der Linden (2011a, 2011b) proposed using a lognormal response time model, more specifically the two-parameter lognormal model, and automated test assembly (ATA) via mixed integer linear programming. However, this approach has a severe limitation, in that the two-parameter lognormal model lacks a slope parameter. This means that the model assumes that all items are equally speed sensitive. From a conceptual perspective, this assumption seems very restrictive. Furthermore, various other empirical studies and new data analyses performed by us show that this assumption almost never holds in practice. To overcome this shortcoming, we bring together the already frequently used three-parameter lognormal model for response times, which contains a slope parameter, and the ATA approach for controlling speededness by van der Linden. Using multiple empirically based illustrations, the proposed extension is illustrated, including complete and documented R code. Both the original van der Linden approach and our newly proposed approach are available to practitioners in the freely available R package eatATA.

In multidimensional computerized adaptive testing (MCAT), item selection strategies are generally constructed based on responses, and they do not consider the response times required by items. This study constructed two new criteria (referred to as DT-inc and DT) for MCAT item selection by utilizing information from response times. The new designs maximize the amount of information per unit time. Furthermore, these two new designs were extended to the DT[subscript S]-inc and DT[subscript S] designs to efficiently estimate intentional abilities. Moreover, the EAP method for ability estimation was also equipped with response time. The performances of the response-time-based EAP (RT-based EAP) and the new designs were evaluated in simulation and empirical studies. The results showed that the RT-based EAP significantly improved the ability estimation precision compared with the EAP without using response time, and the new designs dramatically saved testing times for examinees with a small sacrifice of ability estimation precision and item pool usage.

With the increased restrictions on physical distancing due to the COVID-19 pandemic, remote proctoring has emerged as an alternative to traditional onsite proctoring to ensure the continuity of essential assessments, such as computer-based medical licensing exams. Recent literature has highlighted the significant impact of different proctoring modalities on examinees' test experience, including factors like response-time data. However, the potential influence of these differences on test performance has remained unclear. One limitation in the current literature is the lack of a rigorous learning analytics framework to evaluate the comparability of computer-based exams delivered using various proctoring settings. To address this gap, the current study aims to introduce a machine-learning-based framework that analyzes computer-generated response-time data to investigate the association between proctoring modalities in high-stakes assessments. We demonstrated the effectiveness of this framework using empirical data collected from a large-scale high-stakes medical licensing exam conducted in Canada. By applying the machine-learning-based framework, we were able to extract examinee-specific response-time data for each proctoring modality and identify distinct time-use patterns among examinees based on their proctoring modality.

We have investigated the temporal patterns of algebra (N = 606) and calculus (N = 507) introductory physics students practicing multiple basic physics topics several times throughout the semester using an online mastery homework application called science, technology, engineering, and mathematics (STEM) fluency aimed at improving basic physics skills. For all skill practice categories, we observed an increase in measures of student accuracy, such as a decrease in the number of questions attempted to reach mastery, and a decrease in response time per question, resulting in an overall decrease in the total time spent on the assignments. The findings in this study show that there are several factors that impact a student's performance and evolution on the mastery assignments throughout the semester. For example, using linear mixed modeling, we report that students with lower math preparation for the physics class start with lower accuracy and slower response times on the mastery assignments than students with higher math preparation. However, by the end of the semester, the less prepared students reach similar performance levels to their more prepared classmates on the mastery assignments. This suggests that STEM fluency is a useful tool for instructors to implement to refresh student's basic math skills. Additionally, gender and procrastination habits impact the effectiveness and progression of the student's response time and accuracy on the STEM fluency assignments throughout the semester. We find that women initially answer more questions in the same amount of time as men before reaching mastery. As the semester progresses and students practice the categories more, this performance gap diminishes between males and females. In addition, we find that students who procrastinate (those who wait until the final few hours to complete the assignments) are spending more time on the assignments despite answering a similar number of questions as compared to students who do not procrastinate. We also find that student mindset (growth vs fixed mindset) was not related to a student's progress on the online mastery assignments. Finally, we find that STEM fluency practice improves performance beyond the effects of other components of instruction, such as lectures, group-work recitations, and homework assignments.

This study aims to assess student motivation to provide valid responses to the PISA student questionnaire. This was done by modeling response times using a three-component finite mixture model, comprising two satisficing response styles (rapid and idle) and one optimizing response style. Each participant's motivation was operationalized as their probability of providing an optimizing response to questionnaire items. Overall, the model offered a good fit to the data. Results indicate that most responders were motivated to optimize, with a slight decline toward the end. Further, results showed a positive effect of questionnaire-taking motivation on PISA performance, suggesting a positive relationship to test-taking motivation. In conclusion, response times can be valuable indicators for assessing survey response quality and may serve as a proxy for test-taking motivation.

Processing rational numbers is difficult for many children. The natural number bias is one possible explanation for why children struggle with rational numbers. It refers to the tendency to overgeneralize the properties of natural numbers. In this study, it is argued that in order to be successful in rational number tasks, individuals need to inhibit or suppress their unwanted impulses (in this case the tendency to apply natural number properties). It was investigated whether inhibition plays a role in the occurrence of the natural number bias among primary school children by administering two rational number tasks, two Stroop tasks and a questionnaire measuring inhibitory skills. The results indicated that primary school children were hampered by the natural number bias both in terms of accuracy rates and response times. Additionally, the results did not yield strong evidence for a relationship between inhibition and the occurrence of the natural number bias.

Process data are becoming more and more popular in education research. In the field of computer-based assessments of collaborative problem solving (ColPS), process data have been used to identify students' test-taking strategies while working on the assessment, and such data can be used to complement data collected on accuracy and overall performance. Such information can be used to understand, for example, whether students are able to use a range of styles and strategies to solve different problems, given evidence that such cognitive flexibility may be important in labor markets and societies. In addition, process information might help researchers better identify the determinants of poor performance and interventions that can help students succeed. However, this line of research, particularly research that uses these data to profile students, is still in its infancy and has mostly been centered on small- to medium-scale collaboration settings between people (i.e., the human-to-human approach). There are only a few studies involving large-scale assessments of ColPS between a respondent and computer agents (i.e., the human-to-agent approach), where problem spaces are more standardized and fewer biases and confounds exist. In this study, we investigated students' ColPS behavioral patterns using latent profile analyses (LPA) based on two types of process data (i.e., response times and the number of actions) collected from the Program for International Student Assessment (PISA) 2015 ColPS assessment, a large-scale international assessment of the human-to-agent approach. Analyses were conducted on test-takers who: (a) were administered the assessment in English and (b) were assigned the "Xandar" unit at the beginning of the test. The total sample size was N = 2,520. Analyses revealed two profiles (i.e., Profile 1 [95%] vs. Profile 2 [5%]) showing different behavioral characteristics across the four parts of the assessment unit. Significant differences were also found in overall performance between the profiles.

Questionnaire designers are encouraged to write questions as complete sentences. In self-administered surveys, incomplete question stems may reduce visual clutter but may also increase burden when respondents need to scan the response options to fully complete the question. We experimentally examine the effects of three categories of incomplete question stems (incomplete conversational, incomplete ordinal, and incomplete nominal questions) versus complete question stems on 53 items in a probability web-mail survey. We examine item nonresponse, response time, selection of the first and last response options, and response distributions. We find that incomplete question stems take slightly longer to answer and slightly reduce the selection of the last response option but have no effect on item nonresponse rates or selection of the first response option. We conclude that questionnaire designers should follow current best practices to write complete questions, but deviations from complete questions will likely have limited effects.

Older adults show a progressive cognitive decline, and although language processing appears to resist advancing age, studies in word retrieval report that elders show important difficulties. Previous research reports that such failures increase from age 70 years, which suggests that during the fourth age word retrieval would exhibit even stronger constraints. At the same time, extant evidence suggests that only retrieval of a specific word might decline in advanced aging, and not the recovery of multiple pieces associated with a given semantic category. However, those studies did not explicitly assess these phenomena in a group of older adults in their fourth age, and thus cannot be considered conclusive with regards to this particular group. In the present study, we examined word retrieval in three distinctive advanced age groups (60-69, 70-79, and 80+ years) in two production tasks: a picture naming task, and a lexical availability task. We compared the fourth-age group against the other two third-age group in their reaction time (RT) and accuracy as well as, on the lexical availability index (LAI) and total words retrieved in a lexical availability task. Fourth-age group exhibited longer RT in the picture naming task, yet a high level of accuracy. They also showed a reduced number of retrieved words, as well as, a reduced LAI, relative to the control groups. We discuss our results in the context of the decline of fluid intelligence, and the information transmission deficits hypothesis.

Response time models (RTMs) are of increasing interest in educational and psychological testing. This article focuses on the lognormal model for response times, which is one of the most popular RTMs. Several existing statistics for testing normality and the fit of factor analysis models are repurposed for testing the fit of the lognormal model. A simulation study and two real data examples demonstrate the usefulness of the statistics. The Shapiro-Wilk test of normality and a "z"-test for factor analysis models were the most powerful in assessing the misfit of the lognormal model. [For the corresponding grantee submission, see ED603049.]