Search

In linguistics and natural language processing, measuring text features is crucial for representing and revealing the properties of texts in terms of topics, genres, sentiment and more. Current methods predominantly rely on the frequency of linguistic units and rarely consider the syntagmatic properties of these units, which can reflect deeper linguistic characteristics (e.g. syntax and pragmatics). This paper proposes a general method for calculating the sequence complexity of text. By combining the features of word length and word frequency, two specific implementations of this method are provided. Using these two formulas, word sequence complexity of text based on the Brown Corpus and the Gutenberg Corpus is calculated. The results show that word sequence complexity of text indicates a characteristic of gradual stabilization. By comparing random texts with natural texts, it is found that word sequence complexity is influenced by syntactic rules rather than sentence order, and natural texts tend to alternate between words of varying lengths and frequencies. Classification experiment results indicate that the proposed word sequence complexity outperforms commonly used quantitative indicators for representing the genre of text, such as TTR, information entropy, Zipf's law parameters and motifs. [ABSTRACT FROM AUTHOR]

Word order is one of the most important grammatical devices and the basis for language understanding. However, as one of the most popular NLP architectures, Transformer does not explicitly encode word order. A solution to this problem is to incorporate position information by means of position encoding/embedding (PE). Although a variety of methods of incorporating position information have been proposed, the NLP community is still in want of detailed statistical researches on position information in real-life language. In order to understand the influence of position information on the correlation between words in more detail, we investigated the factors that affect the frequency of words and word sequences in large corpora. Our results show that absolute position, relative position, being at one of the two ends of a sentence and sentence length all significantly affect the frequency of words and word sequences. Besides, we observed that the frequency distribution of word sequences over relative position carries valuable grammatical information. Our study suggests that in order to accurately capture word–word correlations, it is not enough to focus merely on absolute and relative position. Transformers should have access to more types of position-related information which may require improvements to the current architecture. [ABSTRACT FROM AUTHOR]

Two phonemes that may induce minimal pairs constitute a phonemic contrast. Some phonemic contrasts may disappear for various reasons, which, nevertheless, does not seem to seriously impede linguistic communication. Does it mean that the disappeared phonemic contrasts are unimportant? In our study, we calculated the proportions (here termed degree of contrast) of minimal pairs to the words in which the two contrastive phonemes occur and explored the role of phonemic contrasts in the phonemic combinations. The degree of contrast of phonemes reflects the relation between phonemes. Our results indicate that (1) the average degree of contrast of Chinese phonemes declines exponentially with the increase in the number of syllables, rapidly approaching zero; (2) the average degree of contrast of Chinese consonants that differ from each other in only one distinctive feature and of the consonants that are absent in some Chinese dialects is significantly higher than that of other consonants; (3) the degree of contrast of Chinese consonants that differ from each other in only one distinctive feature is not significantly different from that of the consonants absent in some Chinese dialects; (4) Chinese phonemic combinations exhibit high degree of sparsity, which increases exponentially with the number of syllables and rapidly approaches 1. All these results show that the high degree of sparsity and the low degree of contrast of human languages not only leave enough room for new words, new dialects and new languages to appear but also contribute to effective and reliable communication, because a few phonemic mistakes are not likely to cause wrong decoding (sound recognition) and failed communication., (Copyright © 2012 S. Karger AG, Basel.)