Homogeneity of Token Probability Distributions in ChatGPT and Human Texts

Authors :: Dragica Ljubisavljevic
Marko Koprivica
Aleksandar Kostic
Vladan Devedžic
Source :: International Association for Development of the Information Society. 2023.
Publication Year :: 2023
Abstract: This paper delves into statistical disparities between human-written and ChatGPT-generated texts, utilizing an analysis of Shannon's equitability values, and token frequency. Our findings indicate that Shannon's equitability can potentially be a differentiating factor between texts produced by humans and those generated by ChatGPT. Additionally, we uncover substantial distinctions when studying the most frequent tokens. [For the full proceedings, see ED636095.]

Language :: English
Database :: ERIC
Journal :: International Association for Development of the Information Society
Publication Type :: Conference
Accession number :: ED636600
Document Type :: Speeches/Meeting Papers<br />Reports - Research