Search

We employ a mathematical model (a phase oscillator model) to describe the deterministic and stochastic features of frog choruses in which male frogs attempt to avoid call overlaps. The mathematical model with a general interaction term is identified using a Bayesian approach, and it qualitatively reproduces the stationary and dynamical features of the empirical data. In addition, we quantify the magnitude of attention paid among the male frogs from the identified model, and then analyse the relationship between attention and behavioural parameters using a statistical approach. Our analysis demonstrates a negative correlation between attention and inter-frog distance, and also suggests a behavioural strategy in which male frogs selectively attend to a less attractive male frog (i.e. a male producing calls at longer intervals) in order to more effectively advertise their superior relative attractiveness to females.

Interactions using various sensory cues produce sophisticated behaviour in animal swarms, e.g. the foraging behaviour of ants and the flocking of birds and fish. Here, we investigate the behavioural mechanisms of frog choruses from the viewpoints of mathematical modelling and its application. Empirical data on male Japanese tree frogs demonstrate that (1) neighbouring male frogs avoid call overlaps with each other over a short time scale and (2) they collectively switch between the calling state and the silent state over a long time scale. To reproduce these features, we propose a mathematical model in which separate dynamical models spontaneously switch due to a stochastic process depending on the internal dynamics of respective frogs and also the interactions among the frogs. Next, the mathematical model is applied to the control of a wireless sensor network in which multiple sensor nodes send a data packet towards their neighbours so as to deliver the packet to a gateway node by multi-hop communication. Numerical simulation demonstrates that (1) neighbouring nodes can avoid a packet collision over a short time scale by alternating the timing of data transmission and (2) all the nodes collectively switch their states over a long time scale, establishing high network connectivity while reducing network power consumption. Consequently, this study highlights the unique dynamics of frog choruses over multiple time scales and also provides a novel bio-inspired technology that is applicable to the control of a wireless sensor network.

Synthetically generated speech (SGS) has become an integral part of our oral communication in a wide variety of contexts. It can be generated instantly at a low cost and allows precise control over multiple aspects of output, all of which can be highly appealing to second language (L2) assessment developers who have traditionally relied upon human voice actors for recording audio materials. Nevertheless, SGS is not widely used in L2 assessments. One major concern in this use case lies in its potential impact on test-taker performance: Would the use of SGS (as opposed to using human voice actor recordings) change how test takers respond to an item? In this study, we investigated using SGS as stimuli for English L2 listening assessment items on test-taker performance. The data came from a pilot administration of multiple new task types and included 653 test takers' responses to two versions of the same 13 items, differing only in terms of their listening stimuli: a version using human voice actor recordings and the other version with SGS files. Multifaceted comparisons between test takers' responses across the two versions showed that the two versions elicited remarkably comparable performance. The comparability provides strong empirical evidence for the use of SGS as a viable alternative for human voice actor recordings in the immediate domain of L2 assessment as well as related domains such as learning material and research instrument development.

"Biometrics" are physical or behavioral human characteristics that can be used to identify a person. It is widely known that keystroke or typing dynamics for short, fixed texts (e.g., passwords) could serve as a behavioral biometric. In this study, we investigate whether keystroke data from essay responses can lead to a reliable biometric measure, with implications for test security and the monitoring of writing fluency and style changes. Based on keystroke data collected from a high-stakes writing testing setting, we established a preliminary biometric benchmark for detecting test-taker identity by using features extracted from their writing process logs. We report a benchmark keystroke biometric accuracy of equal error rate of 4.7% for identifying same versus different individuals on an essay task. In particular, we show that the inclusion of writing process features (e.g., features designed to describe the writing process) in addition to the widely used typing-timing features (e.g., features based on the time intervals between two-letter key sequences) improves the accuracy of the keystroke biometrics. The proposed keystroke biometrics can have important implications for the writing assessments administered through the remotely proctored tests that have been widely adopted during the COVID pandemic.

Although sleep is tightly regulated by multiple neuronal circuits in the brain, nonneuronal cells such as glial cells have been increasingly recognized as crucial sleep regulators. Recent studies have shown that microglia may act to maintain wakefulness. Here, we investigated the possible involvement of microglia in the regulation of sleep quantity and quality under baseline and stress conditions through electroencephalography (EEG)/electromyography (EMG) recordings, and by employing pharmacological methods to eliminate microglial cells in the adult mouse brain. We found that severe microglial depletion induced by the colony-stimulating factor 1 receptor (CSF1R) antagonist PLX5622 (PLX) reversibly decreased the total wake time and the wake episode duration and increased the EEG slow-wave power during wakefulness under baseline conditions. To examine the role of microglia in sleep/wake regulation under mental stress, we used the acute social defeat stress (ASDS) paradigm, an ethological model for psychosocial stress. Sleep analysis under ASDS revealed that microglial depletion exacerbated the stress-induced decrease in the total wake time and increase in anxiety-like behaviors in the open field test. These results demonstrate that microglia actively modulate sleep quantity and architecture under both baseline and stress conditions. Our findings suggest that microglia may potentially provide resilience against acute psychosocial stress by regulating restorative sleep.

Prey pursuit by an echolocating bat was studied theoretically and experimentally. First, a mathematical model was proposed to describe the flight dynamics of a bat and a single prey. In this model, the flight angle of the bat was affected by [Formula: see text] angles related to the flight path of the single moving prey, that is, the angle from the bat to the prey and the flight angle of the prey. Numerical simulation showed that the success rate of prey capture was high, when the bat mainly used the angle to the prey to minimize the distance to the prey, and also used the flight angle of the prey to minimize the difference in flight directions of itself and the prey. Second, parameters in the model were estimated according to experimental data obtained from video recordings taken while a Japanese horseshoe bat (Rhinolphus derrumequinum nippon) pursued a moving moth (Goniocraspidum pryeri) in a flight chamber. One of the estimated parameter values, which represents the ratio in the use of the [Formula: see text] angles, was consistent with the optimal value of the numerical simulation. This agreement between the numerical simulation and parameter estimation suggests that a bat chooses an effective flight path for successful prey capture by using the [Formula: see text] angles. Finally, the mathematical model was extended to include a bat and [Formula: see text] prey. Parameter estimation of the extended model based on laboratory experiments revealed the existence of bat's dynamical attention towards [Formula: see text] prey, that is, simultaneous pursuit of [Formula: see text] prey and selective pursuit of respective prey. Thus, our mathematical model contributes not only to quantitative analysis of effective foraging, but also to qualitative evaluation of a bat's dynamical flight strategy during multiple prey pursuit.

Using item-response theory to model rater effects provides an alternative solution for rater monitoring and diagnosis, compared to using standard performance metrics. In order to fit such models, the ratings data must be sufficiently connected in order to estimate rater effects. Due to popular rating designs used in large-scale testing scenarios, there tends to be a large proportion of missing data, yielding sparse matrices and estimation issues. In this article, we explore the impact of different types of connectedness, or linkage, brought about by using a linkage set--a collection of responses scored by most or all raters. We also explore the impact of the properties and composition of the linkage set, the different connectedness yielded from different rating designs, and the role of scores from automated scoring engines. In designing monitoring systems using the rater response version of the generalized partial credit model, the study results suggest use of a linkage set, especially a large one that is comprised of responses representing the full score scale. Results also show that a double-human-scoring design provides more connectedness than a design with one human and an automated scoring engine. Furthermore, scores from automated scoring engines do not provide adequate connectedness. We discuss considerations for operational implementation and further study.

Keystroke logs provide a comprehensive record of observable writing processes. Previous studies examining the keystroke logs of young L1 English writers performing experimental writing tasks have identified writing processes features predictive of the quality of responses. Contrarily, large-scale studies on the dynamic and temporal nature of L2 writing process are scarce, especially in an assessment setting. This study utilized the keystroke logs of adult English as a foreign language (EFL) learners responding to assessment tasks to examine the usefulness of the process features in this new context. We evaluated the features in terms of stability, explored factor structures for their correlations, and constructed models to predict response quality. The results showed that most of the process features were stable and that their correlations could be efficiently represented with a five-factor structure. Moreover, we observed improved response quality prediction over a baseline by up to 48%. These findings have implications for the evaluation and understanding of writing process features and for the substantive understanding of writing processes under assessment conditions.

Stakeholders of language tests are often interested in subscores. However, reporting a subscore is not always justified; a subscore should provide reliable and distinct information to be worth reporting. When a subscore is used for decisions across multiple levels (e.g., individual test takers and schools), it needs to be justified for its reliability and distinctiveness at every relevant level. In this study, we examined whether reporting seven Reading and Listening subscores of the "TOEFL Primary"® test, a standardized English proficiency test for young English as a foreign language learners, could be justified for reporting at individual and school levels. We analyzed data collected in pilot administrations, in which 4776 students from 51 schools participated. We employed the classical test theory (CTT) based approaches of Haberman (2008) and Haberman, Sinharay, and Puhan (2009) for the individual and school-level investigations, respectively. We also supplemented the CTT-based approaches with a factor analytic approach for the individual level analysis and a multilevel modeling approach for the school-level analysis. The results differed across the two levels: we found little support for reporting the subscores at the individual level, but strong evidence supporting the added-value of the school-level subscores when the sample size for each school exceeds 50.

Rater training is essential in ensuring the quality of constructed response scoring. Most of the current knowledge about rater training comes from experimental contexts with an emphasis on short-term effects. Few sources are available for empirical evidence on whether and how raters become more accurate as they gain scoring experiences or what long-term effects training can have. In this study, we addressed this research gap by tracking how the accuracies of new raters change through experience and by examining the impact of an additional training session on their accuracies in scoring calibration and monitoring essays. We found that, on average, raters' accuracy improved with scoring experience and that individual raters differed in their accuracy trajectories. The estimated average effect of the training was an approximately six percent increase in the calibration essay accuracy. On the other hand, we observed a smaller impact on the monitoring essay accuracy. Our follow-up analysis showed that this differential impact of the additional training on the calibration and monitoring essay accuracy could be accounted for by successful gatekeeping through calibration.

Allowance for multiple chances to answer constructed response questions is a prevalent feature in computer-based homework and exams. We consider the use of item response theory in the estimation of item characteristics and student ability when multiple attempts are allowed but no explicit penalty is deducted for extra tries. This is common practice in online formative assessments, where the number of attempts is often unlimited. In these environments, some students may not always answer-until-correct, but may rather terminate a response process after one or more incorrect tries. We contrast the cases of graded and sequential item response models, both unidimensional models which do not explicitly account for factors other than ability. These approaches differ not only in terms of log-odds assumptions but, importantly, in terms of handling incomplete data. We explore the consequences of model misspecification through a simulation study and with four online homework data sets. Our results suggest that model selection is insensitive for complete data, but quite sensitive to whether missing responses are regarded as informative (of inability) or not (e.g., missing at random). Under realistic conditions, a sequential model with similar parametric degrees of freedom to a graded model can account for more response patterns and outperforms the latter in terms of model fit.

Scaffolding, an instructional strategy for providing support until learners can perform a task on their own, holds a potential to improve assessments for test takers who need support to demonstrate their abilities, such as English learners (ELs). In this study, we evaluated the benefits of including scaffolded tasks in standardized speaking assessments for young ELs. Our focus was on scaffolded retell tasks that require test takers to retell a given story with procedural scaffolds. We collected responses from 233 third grade ELs to two scaffolded retell tasks, and investigated the relationship between the pre- and post-scaffolding retell performances to learn the impact of the scaffolding on the post-scaffolding performance. We also examined the psychometric added-value of the scaffolding steps and post-scaffolding responses through an application of item response theory (IRT) measurement models. The findings showed that, conditional on the pre-scaffolding performance, the EL test takers' performance on the scaffolding steps were positively associated with their post-scaffolding retell performance. Moreover, the responses to the scaffolding steps and the post-scaffolding retell provided additional information about test takers' oral English proficiency. These findings provide empirical support for the benefits of using scaffolding for EL assessments in a standardized setting.

This study examined whether reporting subscores for groups of items within a test section assessing a second-language modality (specifically reading or listening comprehension) added value from a measurement perspective to the information already provided by the section scores. We analyzed the responses of 116,489 test takers to reading and listening items from operational administrations of two large-scale international tests of English as a foreign language. To "strengthen" the reliability of the subscores, and thus improve their added value, we applied a score augmentation method (Haberman, 2008). In doing so, our aim was to examine whether reporting augmented subscores for specific groups of reading and listening items could improve the added value of these subscores and consequently justify providing more fine-grained information about test taker performance. Our analysis indicated that in general, there was lack of support for reporting subscores from a psychometric perspective, and that score augmentation marginally improved the added value of the subscores. We discuss several implications of our findings for test developers wishing to report more fine-grained information about test performance. We conclude by arguing that research on how to best report such refined feedback should remain the focus of future efforts related to second-language proficiency tests.

Pragmatics has been a key component of language competence frameworks. While the majority of second/foreign language (L2) pragmatics tests have targeted productive skills, the assessment of receptive pragmatic skills remains a developing field. This study explores validation evidence for a test of receptive L2 pragmatic ability called the American English Sociopragmatic Comprehension Test (AESCT), which is a Web-based assessment consisting of 54 tasks measuring knowledge of speech acts, routine formulae, and culture-dependent lexical differences. The AESCT is intended to be used as a learning-oriented assessment in university-level applied linguistics classes. This study collected evidence on construct validity supporting the AESCT design as a measure of pragmatic comprehension and for providing feedback in low-stakes learning contexts; 97 university-level English language learners took the AESCT along with the Cambridge Placement Test and a background questionnaire on their exposure to target language input. Descriptive statistics, correlation analyses, and linear regression were used to analyze aspects of construct validity. Results indicate that the AESCT is sufficiently reliable. Overall, learners were found to perform as previous research suggests: sociopragmatic knowledge was related to L2 exposure and L2 proficiency. Alongside future research into L2 pragmatics test validation, implications for instruction-oriented utilization of the AESCT are discussed.

Language proficiency constitutes a crucial barrier for prospective international teaching assistants (ITAs). Many US universities administer screening tests to ensure that ITAs possess the required academic oral English proficiency for their TA duties. Such ITA screening tests often elicit a sample of spoken English, which is evaluated in terms of multiple aspects by trained raters. In this light, ITA screening tests provide an advantageous context in which to gather rich information about test taker performances. This study introduces a systematic way of extracting meaningful information for major stakeholders from an ITA screening test administered at a US university. In particular, this study illustrates how academic oral English proficiency profiles can be identified based on test takers' subscale score patterns, and discusses how the resulting profiles can be used as feedback for ITA training and screening policy makers, the ITA training program of the university, ESL instructors, and test takers. The proficiency profiles were identified using finite mixture modeling based on the subscale scores of 960 test takers. The modeling results suggested seven profile groups. These groups were interpreted and labeled based on the characteristic subscale score patterns of their members. The implications of the results are discussed, with the main focus on how such information can help ITA policy makers, the ITA training program, ESL instructors, and test takers make important decisions.