Search

Word segmentation is crucial for reading in Chinese, where the absence of explicit word boundaries poses a distinct challenge. Previous studies in Chinese have examined how lexical and sub-lexical variables affect word segmentation. The present study investigated whether higher-level semantic information affects word segmentation using a primed word segmentation task with Overlapping Ambiguous Strings (OAS). An OAS is a three-character string in Chinese (e.g. ABC [in Latin letters]) where the middle character can constitute a word with both the left (word AB) and right (word BC) characters. The OAS was preceded by a semantic or repetition prime (presented for 42, 83, or 200 ms, across participants), priming either AB or BC. The semantic priming effect occurred at the 200-ms Stimulus Onset Asynchrony (SOA), whereas the repetition priming effect occurred at both 83 and 200-ms SOAs. These findings demonstrate that semantic information can affect word segmentation in Chinese within 200 ms. [ABSTRACT FROM AUTHOR]

The present study explored the processing units of high-proficiency second language (L2) Chinese learners in on-line reading in an eye-tracking experiment. The critical aim was to investigate how learners segment continuous characters into words without the aid of word boundary demarcations. Based on previous studies, the embedded words of 2- and 3-character incremental words were manipulated to be either plausible or implausible with the preceding verbs, while the incremental words themselves were always plausible. The results revealed an effect of the plausibility manipulation, which suggested that L2 Chinese learners activated embedded words first and integrated embedded words with previous sentence context as soon as they read them. [ABSTRACT FROM AUTHOR]

The absence of explicit word boundaries is a distinctive characteristic of Chinese script, setting it apart from most alphabetic scripts, leading to word boundary disagreement among readers. Previous studies have examined how this feature may influence reading performance. However, further investigations are required to generate more ecologically valid and generalizable findings. In order to advance our understanding of the impact of word boundaries in Chinese reading, we introduce the Chinese Word Segmentation Agreement (CWSA) corpus. This corpus consists of 500 sentences, comprising 9813 character tokens and 1590 character types, and provides data on word segmentation agreement at each character position. The data revealed a high level of overall segmentation agreement (92%). However, participants disagreed on the position of word boundaries in 8.96% of the cases. Moreover, about 85% of the sentences contained at least one ambiguous word boundary. The character strings with high levels of disagreement were tentatively classified into three categories, namely the morphosyntactic type (e.g., “反映–了”), modifier–head type (e.g., “科學–教育”), and others (e.g., “大力–支持”). Finally, the agreement scores also significantly influenced reading behaviors, as evidenced by analyses with published eye movement data. Specifically, a high level of disagreement was associated with longer single fixation durations. We discuss the implications of these results and highlight how the CWSA corpus can facilitate future research on word segmentation in Chinese reading. [ABSTRACT FROM AUTHOR]

Previous research has shown that when domain‐general transitional probability (TP) cues to word segmentation are in conflict with language‐specific stress cues, English‐learning 5‐ and 7‐month‐olds rely on TP, whereas 9‐month‐olds rely on stress. In two artificial languages, we evaluated English‐learning infants' sensitivity to TP cues to word segmentation vis‐a‐vis language‐specific vowel phonotactic (VP) cues—English words do not end in lax vowels. These cues were either consistent or conflicting. When these cues were in conflict, 10‐month‐olds relied on the VP cues, whereas 5‐month‐olds relied on TP. These findings align with statistical bootstrapping accounts, where infants initially use domain‐general distributional information for word segmentation, and subsequently discover language‐specific patterns based on segmented words. Research Highlights: Research indicates that when transitional probability (TP) conflicts with stress cues for word segmentation, English‐learning 9‐month‐olds rely on stress, whereas younger infants rely on TP.In two artificial languages, we evaluated English‐learning infants' sensitivity to TP versus vowel phonotactic (VP) cues for word segmentation.When these cues conflicted, 10‐month‐olds relied on VPs, whereas 5‐month‐olds relied on TP.These findings align with statistical bootstrapping accounts, where infants first utilize domain‐general distributional information for word segmentation, and then identify language‐specific patterns from segmented words. [ABSTRACT FROM AUTHOR]

We reported a large-scale Internet-based experiment to investigate the impact of emotion information on Chinese word segmentation, in which participants completed an overlapping ambiguous string (OAS) segmentation task and the Chinese version of Beck Depression Inventory-II in a counterbalanced order. OAS is a three-character string (ABC) in which the middle character can form a distinct word with both the character on its left side (word AB) and the character on its right side (word BC). Participants were presented with isolated OASs and were asked to report the word they identified first. Emotional OAS was constructed by a combination of a neutral word and an emotional word, with the neutral and emotional words sharing character B. We orthogonally manipulated the valence of the emotional words (positive vs. negative) and their position in the OAS (left-side vs. right-side). The results showed that compared with neutral words, both positive and negative words were more likely to be segmented, and this segmentation outcome was not affected by readers with different depression tendencies. These findings suggest that emotion information can influence word segmentation, and that both positive and negative words take precedence over neutral words in the word segmentation process. This study provides a new perspective and evidence to understand the impact of emotion information on word processing. [ABSTRACT FROM AUTHOR]

This study examined whether intelligibility-enhancing hyperarticulated clear speaking styles improve word segmentation during real-time speech processing in quiet and in noise. English-speaking listeners heard clearly and conversationally spoken sentences in which the target (e.g. ham) was temporarily ambiguous with a competitor (e.g. hamster) across a word boundary (e.g. ham starting) while their eye fixations to target and competitor images were recorded. Relative to conversational speech, clear speech led listeners to fixate the target image over the competitor image to a greater degree, indicating facilitation of word segmentation. Such facilitation emerged in quiet and in noise even before disambiguating segmental information (e.g. /ɑ/ in starting) was available. A parallel clear speech benefit was not found when the disyllabic word (e.g. hamster) was the target. The findings suggest that improved word segmentation partly underlies the well-documented clear speech perceptual and cognitive benefits and may arise from the enhancements of multiple word boundary cues. [ABSTRACT FROM AUTHOR]

One of the first tasks in language acquisition is word segmentation, a process to extract word forms from continuous speech streams. Statistical approaches to word segmentation have been shown to be a powerful mechanism, in which word boundaries are inferred from sequence statistics. This approach requires the learner to represent the frequency of units from syllable sequences, though accounts differ on how much statistical exposure is required. In this study, we examined the computational limit with which words can be extracted from continuous sequences. First, we discussed why two occurrences of a word in a continuous sequence is the computational lower limit for this word to be statistically defined. Next, we created short syllable sequences that contained certain words either two or four times. Learners were presented with these syllable sequences one at a time, immediately followed by a test of the novel words from these sequences. We found that, with the computationally minimal amount of two exposures, words were successfully segmented from continuous sequences. Moreover, longer syllable sequences providing four exposures to words generated more robust learning results. The implications of these results are discussed in terms of how learners segment and store the word candidates from continuous sequences. [ABSTRACT FROM AUTHOR]

College-aged readers use efficient strategies to segment and recognize words in naturally unspaced Chinese text. Whether this capability changes across the adult lifespan is unknown, although segmenting words in unspaced text may be challenging for older readers due to visual and cognitive declines in older age, including poorer parafoveal processing of upcoming characters. Accordingly, we conducted two eye movement experiments to test for age differences in word segmentation, each with 48 young (18–30 years) and 36 older (65+ years) native Chinese readers. Following Zhou and Li (2021), we focused on the processing of "incremental" three-character words, like 幼儿园 (meaning "kindergartens"), which contain an embedded two-character word (e.g., 幼儿, meaning "children"). In Experiment 1, either the three-character word or its embedded word was presented as the target word in sentence contexts where the three-character word always was plausible, and the embedded word was either plausible or implausible. Both age groups produced similar plausibility effects, suggesting age constancy in accessing the embedded word early during ambiguity processing before ultimately assigning an incremental word analysis. Experiment 2 provided further evidence that both younger and older readers access the embedded word early during ambiguity processing, but rapidly select the appropriate (incremental) word. Crucially, the findings suggest that word segmentation strategies do not differ with age. Public Significance Statement: The ability to segment unspaced text into words is fundamental for reading in writing systems that do not use spaces to indicate the boundaries between words in text. This includes character-based scripts, like Chinese and Japanese; alphabetic scripts that do not contain spacing, like Thai; and languages in which long words are composed of multiple units of meaning, like Finnish. Given typical age-related declines in perception and some aspects of cognition, our study focused on whether older readers have difficulty relative to the young in segmenting unspaced text into words. The findings suggest that the ability to segment words is retained by older readers. [ABSTRACT FROM AUTHOR]

Words are essential parts for understanding classical Chinese poems. We report a collection of 32,399 classical Chinese poems that were annotated with word boundaries. Statistics about the annotated poems support a few heuristic experiences, including the patterns of lines and a practice for the parallel structures (對仗), that researchers of Chinese literature discuss in the literature. The annotators were affiliated with two universities, so they could annotate the poems as independently as possible. Results of an inter-rater agreement study indicate that the annotators have consensus over the identified words 93 per cent of the time and have perfect consensus for the segmentation of a poem 42 per cent of the time. We applied unsupervised classification methods to annotate the poems in several different settings, and evaluated the results with human annotations. Under favorable conditions, the classifier identified about 88 per cent of the words, and segmented poems perfectly 22 per cent of the time. [ABSTRACT FROM AUTHOR]

Background and Objectives Social media has made online care facility reviews popular. By analyzing online reviews of nursing homes (NHs), managers and designers can acquire insight into the perceptions of the older adults and their families. This study aims to help improve the care and environment of NHs. Research Design and Methods This study employed a mixed-methods approach to analyze online NH reviews. An inductive thematic analysis was utilized to identify and develop themes, followed by a detailed content analysis using Jieba, a Python-based program. Also, a regression analysis was conducted between the sentiment level of each subtheme and the final star ratings. Results Online reviews of NHs could be classified into 6 main themes, 18 subthemes, and 53 initial themes. Among the main themes, "service quality" received the most reviews, followed by "physical space environment." Of the 53 initial themes, "attitude and caring" received the most feedback, followed by "general impression of the space environment," and "meals and nutrition." Regression analysis using 18 subthemes revealed that, except for the "facility scale," all 17 subthemes were significantly connected with the final star rating. "Personal and property security" had the highest regression coefficients, followed by "service attitude" and "space." Discussion and Implications Online reviews provide a valuable supplement to conventional NH quality assessment criteria, enhancing person-centered care delivery. Based on the findings, recommendations for NH management and design are proposed to improve care quality, environment, and satisfaction for older adults and families. [ABSTRACT FROM AUTHOR]

Infants' sensitivity to transitional probabilities (TPs) supports language development by facilitating mapping high-TP (HTP) words to meaning, at least up to 18 months of age. Here we tested whether this HTP advantage holds as lexical development progresses, and infants become better at forming word–referent mappings. Two groups of 24-month-olds (N = 64 and all White, tested in the United States) first listened to Italian sentences containing HTP and low-TP (LTP) words. We then used HTP and LTP words, and sequences that violated these statistics, in a mapping task. Infants learned HTP and LTP words equally well. They also learned LTP violations as well as LTP words, but learned HTP words better than HTP violations. Thus, by 2 years of age sensitivity to TPs does not lead to an HTP advantage but rather to poor mapping of violations of HTP word forms. Public Significance Statement: Learning words is a fundamental aspect of early language development. This experiment sheds light on how the mechanisms that support word learning change across time, and suggests that experience with patterns in speech relevant to finding word forms play an important role in mapping word forms to meaning. [ABSTRACT FROM AUTHOR]

Infants are sensitive to statistics in spoken language that aid word‐form segmentation and immediate mapping to referents. However, it is not clear whether this sensitivity influences the formation and retention of word‐referent mappings across a delay, two real‐world challenges that learners must overcome. We tested how the timing of referent training, relative to familiarization with transitional probabilities (TPs) in speech, impacts English‐learning 23‐month‐olds' ability to form and retain word‐referent mappings. In Experiment 1, we tested infants' ability to retain TP information across a 10‐min delay and use it in the service of word learning. Infants successfully mapped high‐TP but not low‐TP words to referents. In Experiment 2, infants readily mapped the same words even when they were unfamiliar. In Experiment 3, high‐ and low‐TP word‐referent mappings were trained immediately after familiarization, and infants readily remembered these associations 10 min later. In sum, although 23‐month‐old infants do not need strong statistics to map word forms to referents immediately, or to remember those mappings across a delay, infants are nevertheless sensitive to these statistics in the speech stream, and they influence mapping after a delay. These findings suggest that, by 23 months of age, sensitivity to statistics in speech may impact infants' language development by leading word forms with low coherence to be poorly mapped following even a short period of consolidation. [ABSTRACT FROM AUTHOR]

The environment in which infants learn language is multimodal and rich with social cues. Yet, the effects of such cues, such as eye contact, on early speech perception have not been closely examined. This study assessed the role of ostensive speech, signalled through the speaker's eye gaze direction, on infants' word segmentation abilities. A familiarisation‐then‐test paradigm was used while electroencephalography (EEG) was recorded. Ten‐month‐old Dutch‐learning infants were familiarised with audio‐visual stories in which a speaker recited four sentences with one repeated target word. The speaker addressed them either with direct or with averted gaze while speaking. In the test phase following each story, infants heard familiar and novel words presented via audio‐only. Infants' familiarity with the words was assessed using event‐related potentials (ERPs). As predicted, infants showed a negative‐going ERP familiarity effect to the isolated familiarised words relative to the novel words over the left‐frontal region of interest during the test phase. While the word familiarity effect did not differ as a function of the speaker's gaze over the left‐frontal region of interest, there was also a (not predicted) positive‐going early ERP familiarity effect over right fronto‐central and central electrodes in the direct gaze condition only. This study provides electrophysiological evidence that infants can segment words from audio‐visual speech, regardless of the ostensiveness of the speaker's communication. However, the speaker's gaze direction seems to influence the processing of familiar words. Research Highlights: We examined 10‐month‐old infants' ERP word familiarity response using audio‐visual stories, in which a speaker addressed infants with direct or averted gaze while speaking.Ten‐month‐old infants can segment and recognise familiar words from audio‐visual speech, indicated by their negative‐going ERP response to familiar, relative to novel, words.This negative‐going ERP word familiarity effect was present for isolated words over left‐frontal electrodes regardless of whether the speaker offered eye contact while speaking.An additional positivity in response to familiar words was observed for direct gaze only, over right fronto‐central and central electrodes. [ABSTRACT FROM AUTHOR]

We have previously shown that bilingual Spanish and English-learning infants can segment English iambs, two-syllable words with final stress (e.g., guiTAR), earlier than their monolingual peers. This is consistent with accelerated development in bilinguals and was attributed to bilingual infants' increased exposure to iambs through Spanish; about 10% of English content words start with an unstressed syllable, compared to 40% in Spanish. Here, we evaluated whether increased exposure to a stress pattern alone is sufficient to account for acceleration in bilingual infants. In English, 90% of content words start with a stressed syllable (e.g., KINGdom), compared to 60% in Spanish. However, we found no evidence for accelerated segmentation of Spanish trochees by Spanish-English bilingual infants compared to their monolingual Spanish-learning peers. Based on this finding, we argue that merely increased exposure to a linguistic feature in one language does not result in accelerated development in the other. Instead, only the acquisition of infrequent patterns in one language may be accelerated due to the additive effects of the other language. [ABSTRACT FROM AUTHOR]

Word segmentation is a crucial step in children's vocabulary learning. While computational models of word segmentation can capture infants' performance in small‐scale artificial tasks, the examination of early word segmentation in naturalistic settings has been limited by the lack of measures that can relate models' performance to developmental data. Here, we extended CLASSIC (Chunking Lexical and Sublexical Sequences in Children; Jones et al., 2021), a corpus‐trained chunking model that can simulate several memory and phonological and vocabulary learning phenomena to allow it to perform word segmentation using utterance boundary information, and we have named this extended version CLASSIC utterance boundary (CLASSIC‐UB). Further, we compared our model to the performance of children on a wide range of new measures, capitalizing on the link between word segmentation and vocabulary learning abilities. We showed that the combination of chunking and utterance‐boundary information used by CLASSIC utterance boundary allowed a better prediction of English‐learning children's output vocabulary than did other models. A one‐page Accessible Summary of this article in non‐technical language is freely available in the Supporting Information online and at https://oasis‐database.org [ABSTRACT FROM AUTHOR]

For Chinese readers, reading from left to right is the norm, while reading from right to left is unfamiliar. This study comprises two experiments investigating how format familiarity and word frequency affect reading by Chinese people. Experiment 1 examines the roles of format familiarity (reading from left to right is the familiar Chinese format, and reading from right to left is the unfamiliar Chinese format) and word frequency in vocabulary recognition. Forty students read the same Chinese sentences from left to right and from right to left. Target words were divided into high and low frequency words. In Experiment 2, participants engaged in right-to-left reading training for 10 days to test whether their right-to-left reading performance could be improved. The study yields several main findings. First, format familiarity affects vocabulary recognition. Participants reading from left to right had shorter fixation times, higher skipping rates, and viewing positions closer to word center.. Second, word frequency affects vocabulary recognition in Chinese reading. Third, right-to-left reading training could improve reading performance. In the early indexes, the interaction effect of format familiarity and word frequency was significant. There was also a significant word-frequency effect from left to right but not from right to left. Therefore, word segmentation and vocabulary recognition may be sequential in Chinese reading. [ABSTRACT FROM AUTHOR]

For geoscience text, rich domain corpora have become the basis of improving the model performance in word segmentation. However, the lack of domain-specific corpus with annotation labelled has become a major obstacle to professional information mining in geoscience fields. In this paper, we propose a corpus augmentation method based on Levenshtein distance. According to the technique, a geoscience dictionary of 20,137 words was collected and constructed by crawling the keywords from published papers in China National Knowledge Infrastructure (CNKI). The dictionary was further used as the main source of synonyms to enrich the geoscience corpus according to the Levenshtein distance between words. Finally, a Chinese word segmentation model combining the BERT, Bi-gated recurrent neural network (Bi-GRU), and conditional random fields (CRF) was implemented. Geoscience corpus composed of complex long specific vocabularies has been selected to test the proposed word segmentation framework. CNN-LSTM, Bi-LSTM-CRF, and Bi-GRU-CRF models were all selected to evaluate the effects of Levenshtein data augmentation technique. Experiments results prove that the proposed methods achieve a significant performance improvement of more than 10%. It has great potential for natural languages processing tasks like named entity recognition and relation extraction. [ABSTRACT FROM AUTHOR]

Infants born prematurely are at a high risk of developing linguistic deficits. In the current study, we compare how full-term and healthy preterm infants without neuro-sensorial impairments segment words from fluent speech, an ability crucial for lexical acquisition. While early word segmentation abilities have been found in monolingual infants, we test here whether it is also the case for French-dominant bilingual infants with varying non-dominant languages. These bilingual infants were tested on their ability to segment monosyllabic French words from French sentences at 6 months of (postnatal) age, an age at which both full-term and preterm monolinguals are able to segment these words. Our results establish the existence of segmentation skills in these infants, with no significant difference in performance between the two maturation groups. Correlation analyses failed to find effects of gestational age in the preterm group, as well as effects of the language dominance within the bilingual groups. These findings indicate that monosyllabic word segmentation, which has been found to emerge by 4 months in monolingual French-learning infants, is a robust ability acquired at an early age even in the context of bilingualism and prematurity. Future studies should further probe segmentation abilities in more extreme conditions, such as in bilinguals tested in their non-dominant language, in preterm infants with medical issues, or testing the segmentation of more complex word structures. [ABSTRACT FROM AUTHOR]

Word spacing is important in guiding eye movements during spaced alphabetic reading. Chinese is unspaced and it remains unclear as to how Chinese readers segment and identify words in reading. We conducted two parallel experiments to investigate whether the positional probabilities of the initial and the final characters of a multicharacter word affected word segmentation and identification in Chinese reading. Two-character words were selected as targets. In Experiment 1, the initial character's positional probability was manipulated as being either high or low, and the final character was kept identical across the two conditions. In Experiment 2, an analogous manipulation was made for the final character of the target word. We recorded adults' and children's eye movements when they read sentences containing these words. In Experiment 1, reading times on targets did not differ in the two conditions for both children and adults, providing no evidence that a word initial character's positional probability contributes to word segmentation. In Experiment 2, adults had shorter reading times and made fewer refixations on targets that comprised final characters with high relative to low positional probabilities; a similar effect was observed in children, but this effect had a slower time course. The results demonstrate that the positional probability of the final (but not the initial) character of a word influences segmentation commitments in reading. It suggests that Chinese readers identify where a currently fixated word ends, and via this commitment, by default, they identify where the subsequent word begins. [ABSTRACT FROM AUTHOR]

There is a constant rise in the amount of data being copied or plagiarized because of the abundance of content and information freely available across the internet. Even though the systems try to check documents for the plagiarism, there have been trials to overcome these system checks. In this paper, the concept of character injection is used to trick plagiarism checker is presented. It is also showcased that how does the similarity check algorithms based on k-grams fail to detect the character injection. In order to eradicate the problem or error in similarity rates caused due to the problem of character injection, image processing based approach of multiple histogram projections are used. An application is developed to detect the character injection in the document and produce the accurate similarity rate. The results are shown with some test documents and the proposed method eliminates any kind of character injected in the document that tricks plagiarism. The proposed method has addressed the problem of character injection with image processing based changes in the existing methods of document-similarity check algorithms using k-grams. The proposed method can detect 100% injected character be it any alphabet of any language, The processing time for conversion, histogram projections and applying winnowing algorithm takes 1.2 sec per page on average when experimented on multiple types of document varying in size from 2 KB to 10 MB. [ABSTRACT FROM AUTHOR]

Spoken sentences are continuous streams of sound, without reliable acoustic cues to word boundaries. We have previously proposed that language learners identify words via an implicit statistical learning mechanism that computes transitional probabilities between syllables. Neuroimaging studies in healthy young adults associate this learning with left inferior frontal gyrus, left arcuate fasciculus, and bilateral striatum. Here, we test the effects of healthy aging and left hemisphere (LH) injury on statistical learning. Following 10-minute exposure to an artificial language, participants rated familiarity of Words, Part-words (sequences spanning word boundaries), and Non-words (unfamiliar sequences). Young controls (N = 14) showed robust learning, rating Words > Part-words > Non-words. Older controls (N = 28) showed this pattern to a weaker degree. Stroke survivors (N = 24) as a group showed no learning. A lesion comparison examining individual differences revealed that "non-learners" are more likely to have anterior lesions. Together, these findings demonstrate that word segmentation is sensitive to healthy aging and LH injury. [ABSTRACT FROM AUTHOR]

Gender identification at the word level in a multi-script environment is challenging due to variations posed by free-style handwriting of individuals and geographical differences in writing styles. This paper presents a new approach, Multi-Orientation-Scale Gabor Response Fusion (MOSGF), for gender identification at the word level using handwritten text. Our method has two steps: (i) word segmentation from unconstrained lines and (ii) gender identification at the word level. In the first step, the method explores the number of zero crossing points and gradient information for word segmentation from handwritten text lines. In the second step, employs Gabor responses at different orientations and scales to detect fine details in female and male handwriting. For each Gabor response, the proposed model estimates the correlation between average templates of all Gabor responses and the individual Gabor response to extract global consistency in writing. To strengthen correlation features, the proposed method uses the Mahalanobis distance measure, which extracts local similarity. Further, the proposed approach fuses correlation coefficient and distance-based features in a novel way. The fused features are then fed to a Neural Network (NN) for gender identification. Experiments on our dataset, which comprises Roman (English), Chinese, Farsi (Persian), Arabic, and Indian scripts, and a benchmark dataset, namely, IAM which includes English text, KHATT which includes Arabic, and QUWI which includes both English and Arabic, show that the proposed system outperforms the existing methods in terms of word segmentation and gender identification. [ABSTRACT FROM AUTHOR]

Two classes of cognitive mechanisms have been proposed to explain segmentation of continuous sensory input into discrete recurrent constituents: clustering and boundary-finding mechanisms. Clustering mechanisms are based on identifying frequently co-occurring elements and merging them together as parts that form a single constituent. Bracketing (or boundary-finding) mechanisms work by identifying rarely co-occurring elements that correspond to the boundaries between discrete constituents. In a series of behavioral experiments, I tested which mechanisms are at play in the visual modality both during segmentation of a continuous syllabic sequence into discrete word-like constituents and during recognition of segmented constituents. Additionally, I explored conscious awareness of the products of statistical learning—whole constituents versus merged clusters of smaller subunits. My results suggest that both online segmentation and offline recognition of extracted constituents rely on detecting frequently co-occurring elements, a process likely based on associative memory. However, people are more aware of having learnt whole tokens than of recurrent composite clusters. [ABSTRACT FROM AUTHOR]

Aiming at the problem of ambiguity caused by boundary segmentation errors caused by word segmentation in Chinese relation extraction, and the problem of multiple relations involved in the overlapping of entity pairs, this paper proposed joint extraction method based on word mixing. Firstly, for the word segmentation boundary problem, the embedding layer combined word vectors on the basis of word vectors, and added position information to ensure the correct order between words. Secondly, the model introduced a hybrid expanded convolutional network for feature extraction with different granularity and longer distance. Finally, it used the hierarchical labeling method to mark the corresponding relationship and object entity through the obtained main entity information, and each main entity could correspond to multiple relations and object entities. Compared with other relation extraction methods on the same Chinese data set, the experimental results show that the extraction effect of this method is the best and also shows better stability. [ABSTRACT FROM AUTHOR]

Sanskrit processing has seen a surge in the use of data-driven approaches over the past decade. Various tasks such as segmentation, morphological parsing, and dependency analysis have been tackled through the development of state-of-the-art models despite working with relatively limited datasets compared to other languages. However, a significant challenge lies in the availability of annotated datasets that are lexically, morphologically, syntactically, and semantically tagged. While syntactic and semantic tags are preferable for later stages of processing such as sentential parsing and disambiguation, lexical and morphological tags are crucial for low-level tasks of word segmentation and morphological parsing. The Digital Corpus of Sanskrit (DCS) is one notable effort that hosts over 650,000 lexically and morphologically tagged sentences from around 250 texts but also comes with its limitations at different levels of a sentence like chunk, segment, stem and morphological analysis. To overcome these limitations, we look at alternatives such as Sanskrit Heritage Segmenter (SH) and Saṃsādhanī tools, that provide information complementing DCS’ data. This work focuses on enriching the DCS dataset by incorporating analyses from SH, thereby creating a dataset that is rich in lexical and morphological information. Furthermore, this work also discusses the impact of such datasets on the performances of existing segmenters, specifically the Sanskrit Heritage Segmenter. [ABSTRACT FROM AUTHOR]

There is growing evidence that COVID-19 brought changes that immediately affected early language development. Little is known for how long these COVID-19 related changes shaped development. The current study systematically and longitudinally addressed this issue, following up children's language development throughout the first 2.5 years. The present study follows up on the sample from Frota et al. (2022), which demonstrated that 7–9-month-old infants born and raised during the pandemic do not segment words unlike pre-pandemic peers. Four studies were conducted: (1) word segmentation task at 12 months of age (N pandemic = 15); (2) word learning task at 20 months of age (N pandemic = 20); (3) language and communication development up to 30 months of age, via CDI and CSBS parental reports (N pandemic ranged 25–74); (4) overall development at 30 months of age using the Griffiths Developmental Scales (N pandemic = 16). The pandemic sample consistently underperformed in all four studies in comparison to pre-pandemic data. There was no evidence of developed word segmentation abilities at 12 months of age, and no successful word learning at 20 months of age. Lexical development between 12 and 24 months of age was lower than in the pre-pandemic sample, while social communication did not seem to be affected. At 30 months of age, the pandemic sample showed lower scores and lower mental age on the Language and Communication Griffiths' subscale, in comparison to the pre-pandemic data. Infants born and raised during the pandemic have a poorer language development, that persists at least until 30 months of age. • We examined language development in children born and raised during COVID-19. • Findings were compared to data from pre-pandemic time. • Children born during COVID showed no word segmentation abilities at 12 months. • They failed in a word learning task at 20 months of age. • They also showed poorer vocabulary and communication skills until 30 months of age. [ABSTRACT FROM AUTHOR]

In two experiments, we investigated the correspondences between off‐line word segmentation and on‐line segmentation processing during Chinese reading. In Experiment 1, participants were asked to read sentences which contained critical four‐character strings, and then, they were required to segment the same sentences into words in a later off‐line word segmentation task. For each item, participants were split into 1‐word segmenters (who segmented four‐character strings as a single word) and 2‐word segmenters (who segmented four‐character strings as 2 two‐character words). Thus, we split participants into two groups (1‐word segmenters and 2‐word segmenters) according to their off‐line segmentation bias. The data analysis showed no reliable group effect on all the measures. In order to avoid the heterogeneity of participants and stimuli in Experiment 1, two groups of participants (1‐word segmenters and 2‐word segmenters) and three types of critical four‐character string (1‐word strings, ambiguous strings, and 2‐word strings) were identified in a norming study in Experiment 2. Participants were required to read sentences containing these critical strings. There was no reliable group effect in Experiment 2, as was the case in Experiment 1. However, in Experiment 2, participants spent less time and made fewer fixations on 1‐word strings compared to ambiguous and 2‐word strings. These results indicate that the off‐line word segmentation preferences do not necessarily reflect on‐line word segmentation processing during Chinese reading and that Chinese readers exhibit flexibility such that word, or multiple constituent, segmentation commitments are made on‐line. [ABSTRACT FROM AUTHOR]

While Parkinson's Disease (PD) impacts the production of prosody and may lead to dysprosody, its effect on the perception of prosody is less clear. In the current study, we investigated how people with PD (PwPD) segment continuous speech using prosodic cues. We used phonemically identical and prosodically different sequences in French. Twenty-three PwPD and 30 controls took part in the study. PwPD showed similar performance to controls (mean difference in terms of correct responses = 2%, 95% confidence interval = [−4%; 8%]). Using Bayesian statistics, our data is 3.6 times more in favour of the null model compared to the alternative model (i.e. difference between PwPD and controls). It thus seems unlikely that PD impacts the perception of prosody systematically. Furthermore, the cognitive performance of PwPD predicted their performance in our segmentation task. This suggests interesting pathways for future research on the mechanisms underlying the impact of PD on speech processing. Clinically, our findings suggest that adequate evaluation of the cognitive capacity of PwPD would help speech and language therapists in assessing speech processing skills in PwPD and in managing their speech impairments. [ABSTRACT FROM AUTHOR]

Infants are sensitive to syllable co‐occurrence probabilities when segmenting words from fluent speech. However, segmenting two languages overlapping at the syllabic level is challenging because the statistical cues across the languages are incongruent. Successful segmentation, thus, relies on infants' ability to separate language inputs and track the statistics of each language. Here, we report three experiments investigating how infants statistically segment words from two overlapping languages in a simulated language‐mixing bilingual environment. In the first two experiments, we investigated whether 9.5‐month‐olds can use French and English phonetic markers to segment words from two overlapping artificial languages produced by one individual. After showing that infants could segment the languages when the languages were presented in isolation (Experiment 1), we presented infants with two interleaved languages differing in phonetic cues (Experiment 2). Both monolingual and bilingual infants successfully segmented words from one of the two languages—the language heard last during familiarization. In Experiment 3, a conceptual replication, we replicated the findings of Experiment 2 with a different population and with different cues. As before, when 12‐month‐old monolingual infants heard two interleaved languages differing in English and Finnish phonetic cues, they learned only the last language heard during familiarization. Together, our findings suggest that segmenting words in a language‐mixing environment is challenging, but infants possess a nascent ability to recruit phonetic cues to segment words from one of two overlapping languages in a bilingual‐like environment. A video abstract of this article can be viewed at https://www.youtube.com/watch?v=92pNcpxZguw. [ABSTRACT FROM AUTHOR]

A classic problem in spoken language comprehension is how listeners perceive speech as being composed of discrete words, given the variable time‐course of information in continuous signals. We propose a syllable inference account of spoken word recognition and segmentation, according to which alternative hierarchical models of syllables, words, and phonemes are dynamically posited, which are expected to maximally predict incoming sensory input. Generative models are combined with current estimates of context speech rate drawn from neural oscillatory dynamics, which are sensitive to amplitude rises. Over time, models which result in local minima in error between predicted and recently experienced signals give rise to perceptions of hearing words. Three experiments using the visual world eye‐tracking paradigm with a picture‐selection task tested hypotheses motivated by this framework. Materials were sentences that were acoustically ambiguous in numbers of syllables, words, and phonemes they contained (cf. English plural constructions, such as "saw (a) raccoon(s) swimming," which have two loci of grammatical information). Time‐compressing, or expanding, speech materials permitted determination of how temporal information at, or in the context of, each locus affected looks to, and selection of, pictures with a singular or plural referent (e.g., one or more than one raccoon). Supporting our account, listeners probabilistically interpreted identical chunks of speech as consistent with a singular or plural referent to a degree that was based on the chunk's gradient rate in relation to its context. We interpret these results as evidence that arriving temporal information, judged in relation to language model predictions generated from context speech rate evaluated on a continuous scale, informs inferences about syllables, thereby giving rise to perceptual experiences of understanding spoken language as words separated in time. A classic problem in cognitive science concerns how listeners perceive and understand speech as comprised of discrete words. We propose a Syllable Inference account of spoken word recognition and segmentation, under which alternative hierarchical models of syllables, words, and phonemes are dynamically posited from cues that include current and past speech rate, with a goal of maximal prediction of sensory input. Three experiments using the Visual World eye‐tracking paradigm provide evidence supporting our proposal. [ABSTRACT FROM AUTHOR]

How frequently a character appears in a word (positional character frequency) is used as a cue in word segmentation when reading aloud in the Chinese language. In this study we created 176 sentences with a target word in the center of each. Participants were 76 college students (mature readers) and 76 third-grade students (beginner readers). Results show an interaction effect of age and positional frequency of the initial character in the word on gaze duration. Further analysis shows that the third-grade students' gaze duration was significantly longer in high, relative to low, positional character frequency of the target words. This trend was consistent with refixation duration, and there was a marginally significant interaction between age and total fixation time. Overall, positional character frequency was an important cue for word segmentation in oral reading in the Chinese language, and third-grade students relied more heavily on this cue than did college students. [ABSTRACT FROM AUTHOR]

Bilingual infants must navigate the similarities and differences between their languages to achieve native proficiency in childhood. Bilinguals learning to find individual words in fluent speech face the possibility of conflicting cues to word boundaries across their languages. Despite this challenge, bilingual infants typically begin to segment and learn words in both languages around the same time as monolinguals. It is possible that early bilingual experience may support infants' abilities to track regularities relevant for word segmentation separately across their languages. In a dual speech stream statistical word segmentation task, we assessed whether 16‐month‐old infants could track syllable co‐occurrence regularities in two artificial languages despite conflicting information across the languages. We found that bilingual, but not monolingual, infants were able to segment the dual speech streams using statistical regularities. Although the two language groups did not differ on secondary measures of cognitive and linguistic development, bilingual infants' real‐world experience with bilingual speakers was predictive of their performance in the dual language statistical segmentation task. [ABSTRACT FROM AUTHOR]

Because of the cumbersome content of traditional Chinese medicine literature, many professional terms, and various writing methods like classical Chinese, the use of word segmentation in general fields is less effective. In order to solve this problem, this paper constructed the BilSTM-CRF model to segment the literature in the field of Chinese medicine, especially the classical Chinese medicine literature. This paper compared the results of the BilSTM-CRF model and the Bi-lSTM model and the segmentation results of the mainstream common Chinese word segmenters jieba and Ansj in the literature of traditional Chinese medicine. The results show that the segmentation based on BilSTM-CRF model achieves better classification performance and robustness. [ABSTRACT FROM AUTHOR]

In the Chinese writing system, there are no interword spaces to mark word boundaries. To understand how Chinese readers conquer this challenge, we constructed an integrated model of word processing and eye-movement control during Chinese reading (CRM). The model contains a word-processing module and an eye-movement control module. The word-processing module perceives new information within the perceptual span around a fixation. The model uses the interactive activation framework (McClelland & Rumelhart, 1981) to simulate word processing, but some new assumptions were made to address the word segmentation problem in Chinese reading. All the words supported by characters in the perceptual span are activated and they compete for a winner. When one word wins the competition, it is identified and it is simultaneously segmented from text. The eye-movement control module makes the decision regarding when and where to move the eyes using the activation information of word units and character units provided by the word-processing module. The model estimates how many characters can be processed during a fixation, and then makes a saccade to somewhere beyond this point. The model successfully simulated important findings on the relation between word processing and eye-movement control, how Chinese readers choose saccade targets, how Chinese readers segment words with ambiguous boundaries, and how Chinese readers process information with parafoveal vision during Chinese sentence reading. The current model thus provides insights on how Chinese readers address some important challenges, such as word segmentation and saccade-target selection. [ABSTRACT FROM AUTHOR]

The current study investigated how the prior context influences word segmentation of overlapping ambiguous strings when reading Chinese. Chinese readers' eye movements were recorded as they read sentences containing a three-character overlapping ambiguous string (ABC), where both AB and BC were two-character words. In the informative condition, prior contexts provided syntactic information that supported either the first word segmentation (AB-C) or the second word segmentation (A-BC). The neutral condition did not provide syntactic constraint for word-segmentation. The post-target contexts were syntactically consistent with either the first word (AB-C) or the second word (A-BC) segmentation. The results showed that there were higher skipping rates and shorter first-fixation durations on the overlapping ambiguous string region in the informative AB-C condition than those in the informative A-BC condition, whereas no difference between the AB-C and A-BC segmentation types was found in the neutral condition. Readers still made regressions into the overlapping ambiguous string region in the informative condition. These results imply that readers use sentence context information immediately to segment the overlapping ambiguous words, but they do not use the context information fully. The first word (AB) has processing advantages over the second word (BC), suggesting a left-side word advantage. [ABSTRACT FROM AUTHOR]

This paper proposes a model which is used to recognize online handwritten Bangla words. This word recognition module comprises of different modules: preprocessing of the word samples, segmentation of words into basic strokes, recognizing the basic strokes using multilayer perceptron, followed by recognition of words using Hidden Markov Model (HMM) aided by Language Model (LM). For stroke recognition, two different feature extraction techniques (point-based and curvature-based procedures) are used using late fusion technique. Top 5 stroke recognition choices are used to construct HMM for the prediction of word sample. An N-gram LM is applied as a post-processing step to rectify the HMM outcomes if required. A total of 50 different word samples with 110 instances each are used to evaluate the proposed model. The overall stroke-level and word-level recognition accuracies obtained by this model are 95.4% and 90.3%, respectively. The proposed model can be extended to recognize online handwritten words written in other script like Devanagari, Assamese, and Gurumukhi, etc. The methodologies described in the manuscript can also be applied for offline word recognition purpose. [ABSTRACT FROM AUTHOR]

Background: To provide satisfying answers, medical QA system has to understand the intentions of the users' questions precisely. For medical intent classification, it requires high-quality datasets to train a deep-learning approach in a supervised way. Currently, there is no public dataset for Chinese medical intent classification, and the datasets of other fields are not applicable to the medical QA system. To solve this problem, we construct a Chinese medical intent dataset (CMID) using the questions from medical QA websites. On this basis, we compare four intent classification models on CMID using a case study.Methods: The questions in CMID are obtained from several medical QA websites. The intent annotation standard is developed by the medical experts, which includes four types and 36 subtypes of users' intents. Besides the intent label, CMID also provides two types of additional information, including word segmentation and named entity. We use the crowdsourcing way to annotate the intent information for each Chinese medical question. Word segmentation and named entities are obtained using the Jieba and a well-trained Lattice-LSTM model. We loaded a Chinese medical dictionary consisting of 530,000 for word segmentation to obtain a more accurate result. We also select four popular deep learning-based models and compare their performances of intent classification on CMID.Results: The final CMID contains 12,000 Chinese medical questions and is organized in JSON format. Each question is labeled the intention, word segmentation, and named entity information. The information about question length, number of entities, and are also detailed analyzed. Among Fast Text, TextCNN, TextRNN, and TextGCN, Fast Text and TextCNN models have achieved the best results in four types and 36 subtypes intent classification, respectively.Conclusions: In this work, we provide a dataset for Chinese medical intent classification, which can be used in medical QA and related fields. We performed an intent classification task on the CMID. In addition, we also did some analysis on the content of the dataset. [ABSTRACT FROM AUTHOR]

The present study investigated whether word-boundary information, provided by alternating colors (consistent or inconsistent with word-boundary information) in a Chinese sentence would facilitate the reading of second-language (L2) learners. Thirty-three Korean students were recruited in the eye-movement experiment. Relative to a baseline (i.e., mono-colors) condition, incorrect word segmentation produced closer fixation location toward the beginning of words, longer fixation duration, higher refixation rate, and slower reading speed. In contrast, word segmentation with alternating colors produced further fixation location toward the center of words, shorter fixation duration, lower refixation rate, and faster reading speed. These results indicate that L2 readers are capable of making use of word-boundary knowledge for saccade generation, which can result in a facilitation of reading efficiency. [ABSTRACT FROM AUTHOR]

As a routine of daily life handwriting gives clues about mood, personality traits and some diseases . In addition, many disciplines such as forensic medicine, medicine and archeology often use handwriting in their own fields. For example, forensic medicine can reach information from the handwriting, such as age range and which hand is used in some cases. In this study, a handwriting system was proposed. The proposed system subtracted a series of features from the handwriting that would characterize the handwriting, classifying them according to gender that used in machine learning techniques. With the tablet in which handwriting is recorded, the feature is extracted from the curve movements that the pen follows in the air both in case the pen is in contact with the tablet and in the transition between letters and words. These features are the speed of the pen, acceleration, jerking movements while writing, the angle of inclination, the curves in the writing, the rate of pen stay in the air, the pressure value of the pen and the height angle of the pen. Decision Trees (DT), Naive Bayes (NB), Support Vector Machines (SVM) and k-Nearest Neighbor (k-NN) approaches were used as the classifier. There are 410 samples in the data set used in the experimental studies and the performances of the experimental studies were evaluated with accuracy criterion. According to the results that have been observed, the best performance was obtained with SVM and the accuracy value was 85.1%. [ABSTRACT FROM AUTHOR]

This paper proposes a novel character-level neural machine translation model which can effectively improve the Neural Machine Translation (NMT) by fusing word and character attention information. In our work, the bidirectional Gated Recurrent Unit (GRU) network is utilized to compose word-level information from the input sequence of characters automatically. Contrary to traditional NMT models, two kinds of different attentions are incorporated into our proposed model: One is the character-level attention which pays attention to the original input characters; The other is the word-level attention which pays attention to the automatically composed words. With the two attentions, the model is able to encode the information from the character level and word level simultaneously. We find that the composed word-level information is compatible and complementary to the original input character-level information. Experimental results on Chinese-English translation tasks show that the proposed model can offer a boost of up to +1.92 BLEU points over the traditional word based NMT models. Furthermore, our translation performance is also comparable to the latest outstanding models, including the state-of-the-art. [ABSTRACT FROM AUTHOR]

Two eye-movement experiments are reported in which a boundary paradigm was used to manipulate the presence versus absence of boundaries for high-frequency and low-frequency target words in the parafovea. In Experiment 1, this was done by introducing a blank space after the target words, whereas in Experiment 2 this was done by rendering the target words in red. In both experiments, higher frequency targets engendered longer saccades, whereas the presence of parafoveal word boundaries engendered shorter saccades. This pattern suggests the operation of two countermanding saccade-targeting mechanisms: one that uses parafoveal processing difficulty to adjust saccade lengths and a second that uses word boundaries to direct the eyes toward specific saccade targets. The implications of these findings for models of eye-movement control during reading are discussed, as are suggestions for integrating dynamic-adjustment and default-targeting accounts. [ABSTRACT FROM AUTHOR]

Research has shown that contoids (phonetically defined consonants) may provide more robust and reliable cues to syllable and word boundaries than vocoids (phonetically defined vowels). Recent studies of Danish, a language characterized by frequent long sequences of vocoids in speech, have suggested that the reduced occurrence of contoids may make speech in it intrinsically harder to segment than in closely related languages such as Norwegian. We addressed this hypothesis empirically in an artificial language learning experiment with native speakers of Danish, Norwegian, and English. We tested whether artificial speech consisting of concatenated contoid–vocoid syllables is easier to segment than speech consisting of vocoid–vocoid syllables where the first segment is a semivowel and the second a full vowel. Contrary to what was expected, we found no effect of the phonetic makeup of the syllables on speech segmentability. Possible interpretations and implications of this result are discussed. Open Practices: This article has been awarded Open Materials and Open Data badges. All materials and data are publicly accessible via the Open Science Framework at https://osf.io/rmjtg. Learn more about the Open Practices badges from the Center for Open Science: https://osf.io/tvyxz/wiki. [ABSTRACT FROM AUTHOR]

Segmentation is an important step for developing any optical character recognition (OCR) system, which has to be redesigned for each script having, non-uniform nature/property. It is used to decompose the image into its sub-units, which act as a basis for character recognition. Brahmi is a non-cursive ancient script, in which characters are not attached to each other and have some spacing between them. This study analyses various segmentation methods for different scripts to develop the best suitable segmentation method for Brahmi. MATLAB software was used for segmentation purpose in the experiment. The sample data belongs to Brahmi script-based 'Rumandei inscription'. In this paper, we discuss a segmentation methodology for distinct components, namely text lines, words and characters of Rumandei inscription, written in Brahmi script. For segmenting distinct components of inscription different approach were used like horizontal projection profile, vertical projection profile and Relative minima approach. This is fundamental research on an inscription based on Brahmi script, which acts as a foundation for developing a segmentation module of an OCR solution/system of similar scripts in future. Information search and retrieval is an important activity of a library. So, to ensure this support for digitised documents written in ancient script, their character recognition is mandatory through the OCR system. [ABSTRACT FROM AUTHOR]

Infants' ability to segment words in fluent speech is affected by their language experience. In this study we investigated the conditions under which infants can segment words in a non-native language. Using the Head-turn Preference Procedure, we found that monolingual English-learning 8-month-olds can segment bisyllabic words in Spanish (trochees and iambs) but not French (iambs). Our results are incompatible with accounts that rely on distributional learning, language rhythm similarity, or target word prosodic shape alone. Instead, we show that monolingual English-learning infants are able to segment words in a non-native language as long as words have stress, as is the case in English. More specifically, we show that even in a rhythmically different non-native language, English-learning infants can find words by detecting stressed syllables and treating them as word onsets or offsets. [ABSTRACT FROM AUTHOR]

Infants are sensitive to statistical regularities (i.e., transitional probabilities, or TPs) relevant to segmenting words in fluent speech. However, there is debate about whether tracking TPs results in representations of possible words. Infants show preferential learning of sequences with high TPs (HTPs) as object labels relative to those with low TPs (LTPs). Such findings could mean that only the HTP sequences have a word‐like status, and they are more readily mapped to a referent for that reason. But these findings could also suggest that HTP sequences are easier to encode, just like any other predictable sequence. Here we aimed to distinguish between these explanations. To do so, we built on findings that infants become resistant to learning labels that are not typical of their native language as they approach 2 years of age and add words to their lexicons. If tracking TPs in speech results in identifying candidate words, at this age TPs may have reduced power to confer lexical status when they yield a unit that is very dissimilar to word forms that are typical of infants' native language. Indeed, we found that at 20 months, English‐learning infants with relatively small vocabularies learned HTP Italian words (but not LTP words) as object labels, while infants with larger vocabularies resisted learning HTP Italian words. These findings suggest that the HTP sequences may be represented as candidate words, and more broadly, that TP statistics are relevant to word learning. [ABSTRACT FROM AUTHOR]

During sentence reading, low spatial frequency information afforded by spaces between words is the primary factor for eye guidance in spaced writing systems, whereas saccade generation for unspaced writing systems is less clear and under debate. In the present study, we investigated whether word-boundary information, provided by alternating colors (consistent or inconsistent with word-boundary information) influences saccade-target selection in Chinese. In Experiment 1, as compared to a baseline (i.e., uniform color) condition, word segmentation with alternating color shifted fixation location towards the center of words. In contrast, incorrect word segmentation shifted fixation location towards the beginning of words. In Experiment 2, we used a gaze-contingent paradigm to restrict the color manipulation only to the upcoming parafoveal words and replicated the results, including fixation location effects, as observed in Experiment 1. These results indicate that Chinese readers are capable of making use of parafoveal word-boundary knowledge for saccade generation, even if such information is unfamiliar to them. The present study provides novel support for the hypothesis that word segmentation is involved in the decision about where to fixate next during Chinese reading. [ABSTRACT FROM AUTHOR]

Abstract: Recent work asked if visual cues facilitate word segmentation in adults and infants (Thiessen, 2010). While adults showed better word segmentation when presented with a regular visual cue (consistent mapping between words and objects), infants did not. This difference was attributed to infants' lack of understanding that objects have labels. Alternatively, infants’ performance could reflect their difficulty with tracking and integrating multiple multimodal cues. We contrasted these two accounts by looking at the effect of visual cues on word segmentation in adults and across childhood (6–12 years). We found that older children (Mage 10;7) benefitted from the regular visual cues, but younger children (Mage 7;10), who already knew that objects have labels, did not. Knowing that objects have labels was not enough to use visual cues as an aid for segmentation. These findings show that the ability to integrate multimodal cues develops during childhood, and it is not yet adult‐like in children. [ABSTRACT FROM AUTHOR]

Abstract: Statistical learning is often taken to lie at the heart of many cognitive tasks, including the acquisition of language. One particular task in which probabilistic models have achieved considerable success is the segmentation of speech into words. However, these models have mostly been tested against English data, and as a result little is known about how a statistical learning mechanism copes with input regularities that arise from the structural properties of different languages. This study focuses on statistical word segmentation in Arabic, a Semitic language in which words are built around consonantal roots. We hypothesize that segmentation in such languages is facilitated by tracking consonant distributions independently from intervening vowels. Previous studies have shown that human learners can track consonant probabilities across intervening vowels in artificial languages, but it is unknown to what extent this ability would be beneficial in the segmentation of natural language. We assessed the performance of a Bayesian segmentation model on English and Arabic, comparing consonant‐only representations with full representations. In addition, we examined to what extent structurally different proto‐lexicons reflect adult language. The results suggest that for a child learning a Semitic language, separating consonants from vowels is beneficial for segmentation. These findings indicate that probabilistic models require appropriate linguistic representations in order to effectively meet the challenges of language acquisition. [ABSTRACT FROM AUTHOR]

The word and character segmentation in ancient handwritten Devanagari (Indian) and Maithili (Indian) documents in order to develop any optical character recognition (OCR) systems for the ancient documents are still considered as challenging tasks in the field of document analysis and recognition. The presence of horizontal lines over each character, variable spacing between two adjacent characters or words, and text skewness in these two scripts make these tasks more challenging. This article proposes a novel method to segment any textline of ancient handwritten Devanagari and Maithili documents into different words and each word into different characters by dividing the textline into three different horizontal zones—upper, middle, and lower. The middle zone of the textline has been extracted by constructing a linear regression curve passing through the middle zone. The gaps between two adjacent characters or words present in the middle zone have then been utilized to segment one textline into various words as well as one word into various characters. The performance of the proposed method has been evaluated on the self-generated datasets of ancient handwritten textline images in Devanagari and Maithili scripts. The proposed method has achieved accuracies of 96.31% and 98.35% for the word segmentation and character segmentation respectively for ancient Devanagari documents. For ancient handwritten documents in Maithili script, accuracies of 97.39% and 98.65% have been obtained from the proposed method for word segmentation and character segmentation respectively. The experimental results demonstrate that the proposed word and character segmentation system outperforms the state-of-the-art methods in this regard. • Proposing a novel method of text segmentation in ancient handwritten documents. • Documents written in two ancient Indian scripts have been considered. • The segmentations have been done by extracting the middle zone of any text line. • Linear regression curve based middle zone extraction has been done. [ABSTRACT FROM AUTHOR]

• The first word segmenter for Pashto language. • Using CRF for Pashto word segmentation. • Correction of whitespace in noisy Pashto text. • Annotated text corpus for the low-resource language Pashto. • Pashto text proofing using CRF. Word segmentation is the process of splitting up the text into words. In English and most European languages, word boundaries are identified by whitespace, while in Pashto, there is no explicit word delimiter. Pashto uses whitespace for word separation but not consistently, and it cannot be considered a reliable word-boundary identifier. This inconsistency makes the Pashto word segmentation unique and challenging. Moreover, Pashto is a low-resource, non-standardized language with no established rules for the correct usage of whitespace that leads to two typical spelling errors, space-omission, and space-insertion. These errors significantly affect the performance of the word segmenter. This study aims to develop a state-of-the-art word segmenter for Pashto, with a proofing tool to identify and correct the position of space in a noisy text. The CRF algorithm is incorporated to train two machine learning models for these tasks. For models' training, we have developed a text corpus of nearly 3.5 million words, annotated for the correct positions of spaces and explicit word boundary information using a lexicon-based technique, and then manually checked for errors. The experimental results of the model are very satisfactory, where the F1-scores are 99.2% and 96.7% for the proofing model and word segmenter, respectively. [ABSTRACT FROM AUTHOR]