Search

BACKGROUND: About one-third of older adults aged 65 years and older often have mild cognitive impairment or dementia. Acoustic and psycho-linguistic features derived from conversation may be of great diagnostic value because speech involves verbal memory and cognitive and neuromuscular processes. The relative decline in these processes, however, may not be linear and remains understudied. OBJECTIVE: This study aims to establish associations between cognitive abilities and various attributes of speech and natural language production. To date, the majority of research has been cross-sectional, relying mostly on data from structured interactions and restricted to textual versus acoustic analyses. METHODS: In a sample of 71 older (mean age 83.3, SD 7.0 years) community-dwelling adults who completed qualitative interviews and cognitive testing, we investigated the performance of both acoustic and psycholinguistic features associated with cognitive deficits contemporaneously and at a 1-2 years follow up (mean follow-up time 512.3, SD 84.5 days). RESULTS: Combined acoustic and psycholinguistic features achieved high performance (F1-scores 0.73-0.86) and sensitivity (up to 0.90) in estimating cognitive deficits across multiple domains. Performance remained high when acoustic and psycholinguistic features were used to predict follow-up cognitive performance. The psycholinguistic features that were most successful at classifying high cognitive impairment reflected vocabulary richness, the quantity of speech produced, and the fragmentation of speech, whereas the analogous top-ranked acoustic features reflected breathing and nonverbal vocalizations such as giggles or laughter. CONCLUSIONS: These results suggest that both acoustic and psycholinguistic features extracted from qualitative interviews may be reliable markers of cognitive deficits in late life.

BACKGROUND: Although uncertainties exist regarding implementation, artificial intelligence-driven generative language models (GLMs) have enormous potential in medicine. Deployment of GLMs could improve patient comprehension of clinical texts and improve low health literacy. OBJECTIVE: The goal of this study is to evaluate the potential of ChatGPT-3.5 and GPT-4 to tailor the complexity of medical information to patient-specific input education level, which is crucial if it is to serve as a tool in addressing low health literacy. METHODS: Input templates related to 2 prevalent chronic diseases-type II diabetes and hypertension-were designed. Each clinical vignette was adjusted for hypothetical patient education levels to evaluate output personalization. To assess the success of a GLM (GPT-3.5 and GPT-4) in tailoring output writing, the readability of pre- and posttransformation outputs were quantified using the Flesch reading ease score (FKRE) and the Flesch-Kincaid grade level (FKGL). RESULTS: Responses (n=80) were generated using GPT-3.5 and GPT-4 across 2 clinical vignettes. For GPT-3.5, FKRE means were 57.75 (SD 4.75), 51.28 (SD 5.14), 32.28 (SD 4.52), and 28.31 (SD 5.22) for 6th grade, 8th grade, high school, and bachelors, respectively; FKGL mean scores were 9.08 (SD 0.90), 10.27 (SD 1.06), 13.4 (SD 0.80), and 13.74 (SD 1.18). GPT-3.5 only aligned with the prespecified education levels at the bachelors degree. Conversely, GPT-4s FKRE mean scores were 74.54 (SD 2.6), 71.25 (SD 4.96), 47.61 (SD 6.13), and 13.71 (SD 5.77), with FKGL mean scores of 6.3 (SD 0.73), 6.7 (SD 1.11), 11.09 (SD 1.26), and 17.03 (SD 1.11) for the same respective education levels. GPT-4 met the target readability for all groups except the 6th-grade FKRE average. Both GLMs produced outputs with statistically significant differences (P

BackgroundPatients find technology tools to be more approachable for seeking sensitive health-related information, such as reproductive health information. The inventive conversational ability of artificial intelligence (AI) chatbots, such as ChatGPT (OpenAI Inc), offers a potential means for patients to effectively locate answers to their health-related questions digitally.ObjectiveA pilot study was conducted to compare the novel ChatGPT with the existing Google Search technology for their ability to offer accurate, effective, and current information regarding proceeding action after missing a dose of oral contraceptive pill.MethodsA sequence of 11 questions, mimicking a patient inquiring about the action to take after missing a dose of an oral contraceptive pill, were input into ChatGPT as a cascade, given the conversational ability of ChatGPT. The questions were input into 4 different ChatGPT accounts, with the account holders being of various demographics, to evaluate potential differences and biases in the responses given to different account holders. The leading question, "what should I do if I missed a day of my oral contraception birth control?" alone was then input into Google Search, given its nonconversational nature. The results from the ChatGPT questions and the Google Search results for the leading question were evaluated on their readability, accuracy, and effective delivery of information.ResultsThe ChatGPT results were determined to be at an overall higher-grade reading level, with a longer reading duration, less accurate, less current, and with a less effective delivery of information. In contrast, the Google Search resulting answer box and snippets were at a lower-grade reading level, shorter reading duration, more current, able to reference the origin of the information (transparent), and provided the information in various formats in addition to text.ConclusionsChatGPT has room for improvement in accuracy, transparency, recency, and reliability before it can equitably be implemented into health care information delivery and provide the potential benefits it poses. However, AI may be used as a tool for providers to educate their patients in preferred, creative, and efficient ways, such as using AI to generate accessible short educational videos from health care provider-vetted information. Larger studies representing a diverse group of users are needed.

This article provides three major contributions to the literature: we provide granular information on the development of student argumentative writing across secondary school; we replicate the MacArthur et al. model of Natural Language Processing (NLP) writing features that predict quality with a younger group of students; and we are able to examine the differences for students across language status. In our study, we sought to find the average levels of text length, cohesion, connectives, syntactic complexity, and word-level complexity in this sample across Grades 7-12 by sex, by English learner status, and for essays scoring above and below the median holistic score. Mean levels of variables by grade suggest a developmental progression with respect to text length, with the text length increasing with grade level, but the other variables in the model were fairly stable. Sex did not seem to affect the model in meaningful ways beyond the increased fluency of women writers. We saw text length and word level differences between initially designated and redesignated bilingual students compared to their English-only peers. Finally, we see that the model works better with our higher scoring essays and is less effective explaining the lower scoring essays.

The price of oil is highly complex to predict as it is impacted by global demand and supply, geopolitical events, and market sentiment. The accuracy of such predictions, however, has far-reaching implications for supply chain performance, portfolio management, and expected stock market returns. This paper contributes to the oil price prediction literature by evaluating the predictive impact of the US President's communication on Twitter, while benchmarking various Natural Language Processing (NLP) techniques, including Term Frequency-Inverse Document Frequency (TF-IDF), Word2Vec, Doc2Vec, Global Vectors for Word Representation (GloVe), and Bidirectional Encoder Representations from Transformers (BERT). These techniques are combined with a deep neural network Long Short-Term Memory (LSTM) architecture using a five-day lag for both the oil price and the textual Twitter data. The data was collected during the term of US President Donald Trump, resulting in 1449 days of crude oil price prediction and a total of 16,457 tweets. The study is validated for Brent and West Texas Intermediate blends, using the daily price of a barrel of crude oil as the target variable. The results confirm that including the US President's tweets significantly increases the predictive power of oil price prediction models, and that an LSTM architecture with BERT as NLP technique has the best performance. [ABSTRACT FROM AUTHOR]

Our research strives to examine how simulation models of logistics systems can be produced automatically from verbal descriptions in natural language and how human experts and artificial intelligence (AI)-based systems can collaborate in the domain of simulation modelling. We demonstrate that a framework constructed upon the refined GPT-3 Codex is capable of generating functionally valid simulations for queuing and inventory management systems when provided with a verbal explanation. As a result, the language model could produce simulation models for inventory and process control. These results, along with the rapid improvement of language models, enable a significant simplification of simulation model development. Our study offers guidelines and a design of a natural language processing-based framework on how to build simulation models of logistics systems automatically, given the verbal description. In generalised terms, our work offers a technological underpinning of human-AI collaboration for the development of simulation models. [ABSTRACT FROM AUTHOR]

BackgroundThe growing availability of big data spontaneously generated by social media platforms allows us to leverage natural language processing (NLP) methods as valuable tools to understand the opioid crisis.ObjectiveWe aimed to understand how NLP has been applied to Reddit (Reddit Inc) data to study opioid use.MethodsWe systematically searched for peer-reviewed studies and conference abstracts in PubMed, Scopus, PsycINFO, ACL Anthology, IEEE Xplore, and Association for Computing Machinery data repositories up to July 19, 2022. Inclusion criteria were studies investigating opioid use, using NLP techniques to analyze the textual corpora, and using Reddit as the social media data source. We were specifically interested in mapping studies' overarching goals and findings, methodologies and software used, and main limitations.ResultsIn total, 30 studies were included, which were classified into 4 nonmutually exclusive overarching goal categories: methodological (n=6, 20% studies), infodemiology (n=22, 73% studies), infoveillance (n=7, 23% studies), and pharmacovigilance (n=3, 10% studies). NLP methods were used to identify content relevant to opioid use among vast quantities of textual data, to establish potential relationships between opioid use patterns or profiles and contextual factors or comorbidities, and to anticipate individuals' transitions between different opioid-related subreddits, likely revealing progression through opioid use stages. Most studies used an embedding technique (12/30, 40%), prediction or classification approach (12/30, 40%), topic modeling (9/30, 30%), and sentiment analysis (6/30, 20%). The most frequently used programming languages were Python (20/30, 67%) and R (2/30, 7%). Among the studies that reported limitations (20/30, 67%), the most cited was the uncertainty regarding whether redditors participating in these forums were representative of people who use opioids (8/20, 40%). The papers were very recent (28/30, 93%), from 2019 to 2022, with authors from a range of disciplines.ConclusionsThis scoping review identified a wide variety of NLP techniques and applications used to support surveillance and social media interventions addressing the opioid crisis. Despite the clear potential of these methods to enable the identification of opioid-relevant content in Reddit and its analysis, there are limits to the degree of interpretive meaning that they can provide. Moreover, we identified the need for standardized ethical guidelines to govern the use of Reddit data to safeguard the anonymity and privacy of people using these forums.

Large-scale text data from public sources, including social media or online platforms, can expand urban planners’ ability to monitor and analyze urban conditions in near real-time. To overcome scalability challenges of manual techniques for qualitative data analysis, researchers and practitioners have turned to computer-automated methods, such as natural language processing (NLP) and deep learning. However, the benefits, challenges, and trade-offs of these methods remain poorly understood. How much meaning can different NLP techniques capture and how do their results compare to traditional manual techniques? Drawing on 90,000 online rental listings in Los Angeles County, this study proposes and compares manual, semi-automated, and fully automated methods for identifying context-informed topics in unstructured, user-generated text data. We find that fully automated methods perform best with more-structured text, but struggle to separate topics in free-flow text and when handling nuanced language. Introducing a manual technique first on a small data set to train a semi-automated method, however, improves accuracy even as the structure of the text degrades. We argue that while fully automated NLP methods are attractive replacements for scaling manual techniques, leveraging the contextual understanding of human expertise alongside efficient computer-based methods like BERT models generates better accuracy without sacrificing scalability.

BACKGROUND: Pathology reports contain key information about the patients diagnosis as well as important gross and microscopic findings. These information-rich clinical reports offer an invaluable resource for clinical studies, but data extraction and analysis from such unstructured texts is often manual and tedious. While neural information retrieval systems (typically implemented as deep learning methods for natural language processing) are automatic and flexible, they typically require a large domain-specific text corpus for training, making them infeasible for many medical subdomains. Thus, an automated data extraction method for pathology reports that does not require a large training corpus would be of significant value and utility. OBJECTIVE: To develop a language model-based neural information retrieval system that can be trained on small datasets and validate it by training it on renal transplant-pathology reports to extract relevant information for two predefined questions: (1) What kind of rejection does the patient show?; (2) What is the grade of interstitial fibrosis and tubular atrophy (IFTA)? METHODS: Kidney BERT was developed by pre-training Clinical BERT on 3.4K renal transplant pathology reports and 1.5M words. Then, exKidneyBERT was developed by extending Clinical BERTs tokenizer with six technical keywords and repeating the pre-training procedure. This extended the models vocabulary. All three models were fine-tuned with information retrieval heads. RESULTS: The model with extended vocabulary, exKidneyBERT, outperformed Clinical BERT and Kidney BERT in both questions. For rejection, exKidneyBERT achieved an 83.3% overlap ratio for antibody-mediated rejection (ABMR) and 79.2% for T-cell mediated rejection (TCMR). For IFTA, exKidneyBERT had a 95.8% exact match rate. CONCLUSION: ExKidneyBERT is a high-performing model for extracting information from renal pathology reports. Additional pre-training of BERT language models on specialized small domains does not necessarily improve performance. Extending the BERT tokenizers vocabulary library is essential for specialized domains to improve performance, especially when pre-training on small corpora.

Background: NLPs such as ChatGPT are novel sources of online healthcare information that are readily accessible and integrated into internet search tools. The accuracy of NLP-generated responses to health information questions is unknown. Methods: We queried four NLPs (ChatGPT 3.5 and 4, Bard, and Claude 2.0) for responses to simulated patient questions about inguinal hernias and their management. Responses were graded on a Likert scale (1 poor to 5 excellent) for relevance, completeness, and accuracy. Responses were compiled and scored collectively for readability using the Flesch–Kincaid score and for educational quality using the DISCERN instrument, a validated tool for evaluating patient information materials. Responses were also compared to two gold-standard educational materials provided by SAGES and the ACS. Evaluations were performed by six hernia surgeons. Results: The average NLP response scores for relevance, completeness, and accuracy were 4.76 (95% CI 4.70–4.80), 4.11 (95% CI 4.02–4.20), and 4.14 (95% CI 4.03–4.24), respectively. ChatGPT4 received higher accuracy scores (mean 4.43 [95% CI 4.37–4.50]) than Bard (mean 4.06 [95% CI 3.88–4.26]) and Claude 2.0 (mean 3.85 [95% CI 3.63–4.08]). The ACS document received the best scores for reading ease (55.2) and grade level (9.2); however, none of the documents achieved the readibility thresholds recommended by the American Medical Association. The ACS document also received the highest DISCERN score of 63.5 (57.0–70.1), and this was significantly higher compared to ChatGPT 4 (50.8 [95% CI 46.2–55.4]) and Claude 2.0 (48 [95% CI 41.6–54.4]). Conclusions: The evaluated NLPs provided relevant responses of reasonable accuracy to questions about inguinal hernia. Compiled NLP responses received relatively low readability and DISCERN scores, although results may improve as NLPs evolve or with adjustments in question wording. As surgical patients expand their use of NLPs for healthcare information, surgeons should be aware of the benefits and limitations of NLPs as patient education tools. [ABSTRACT FROM AUTHOR]

This article proposes a joint application of online network analysis and NLP techniques to explore dynamics of "polarized intersectionality"—that is, how (mis)representations of women that develop online and at the intersection of different axes of discrimination entwine with ideological and affective polarization. We look in particular at whether and how digital (mis)representations change together with political scenarios in which political parties, leaders, and partisan communities more in general swing from collaboration to hostility. Our analysis of two Twitter conversations that put women at the center of attention show that changing political scenarios generate different digital conversations which, in turn, reflect patterns of alliances and rivalry. Regardless of these changes, women are invariantly (mis)represented in narratives that are often weaponized against political enemies in ways that foster both ideological and affective political polarization. [ABSTRACT FROM AUTHOR]

Social media has revolutionized the manner in which our society is interconnected. While this extensive connectivity offers numerous benefits, it is also accompanied by significant drawbacks, particularly in terms of the proliferation of fake news and the vast dissemination of hate speech. Identifying offensive comments is a critical task for ensuring the safety of users, which is why industry and academia have been working on developing solutions to this problem. Prior research on hate speech detection has predominantly focused on the English language, with few studies devoted to other languages such as Portuguese. This paper introduces the Offensive Language Identification Dataset for Brazilian Portuguese (OLID-BR), a high-quality NLP dataset for offensive language detection, which we make publicly available. The dataset contains 6,354 (extendable to 13,538) comments labeled using a fine-grained three-layer annotation schema compatible with datasets in other languages, which allows the training of multilingual/cross-lingual models. The five NLP tasks available in OLID-BR allow the detection of offensive comments, the classification of the types of offenses such as racism, LGBTQphobia, sexism, xenophobia, and so on, the identification of the type and the target of offensive comments, and the extraction of toxic spans of offensive comments. All those tasks can enhance the capabilities of content moderation systems by providing deep contextual analysis or highlighting the spans that make a text toxic. We further experiment with and evaluate the dataset using state-of-the-art BERT-based and NER models, which demonstrates the usefulness of OLID-BR for the development of toxicity detection systems for Portuguese texts. [ABSTRACT FROM AUTHOR]

To prevent ordinary people from being harmed by natural language processing (NLP) technology, finding ways to measure the extent to which a language model is biased (e.g., regarding gender) has become an active area of research. One popular class of NLP bias measures are bias benchmark datasets—collections of test items that are meant to assess a language model's preference for stereotypical versus non-stereotypical language. In this paper, we argue that such bias benchmarks should be assessed with models from the psychometric framework of item response theory (IRT). Specifically, we tie an introduction to basic IRT concepts and models with a discussion of how they could be relevant to the evaluation, interpretation and improvement of bias benchmark datasets. Regarding evaluation, IRT provides us with methodological tools for assessing the quality of both individual test items (e.g., the extent to which an item can differentiate highly biased from less biased language models) as well as benchmarks as a whole (e.g., the extent to which the benchmark allows us to assess not only severe but also subtle levels of model bias). Through such diagnostic tools, the quality of benchmark datasets could be improved, for example by deleting or reworking poorly performing items. Finally, in regards to interpretation, we argue that IRT models' estimates for language model bias are conceptually superior to traditional accuracy-based evaluation metrics, as the former take into account more information than just whether or not a language model provided a biased response. [ABSTRACT FROM AUTHOR]

Understanding the modus operandi of adversaries aids organizations to employ efficient defensive strategies and share intelligence in the community. This knowledge is often present in unstructured natural language text within threat analysis reports. A translation tool is needed to interpret the modus operandi explained in the sentences of the threat report and convert it into a structured format. This research introduces a methodology named TTPXHunter for automated extraction of threat intelligence in terms of Tactics, Techniques, and Procedures (TTPs) from finished cyber threat reports. It leverages cyber domain-specific state-of-the-art natural language model to augment sentences for minority class TTPs and refine pinpointing the TTPs in threat analysis reports significantly. We create two datasets: an augmented sentence-TTP dataset of \(39,296\) sentence samples and a \(149\) real-world cyber threat intelligence report-to-TTP dataset. Further, we evaluate TTPXHunter on the augmented sentence and report datasets. The TTPXHunter achieves the highest performance of \(92.42\%\) f1-score on the augmented dataset, and it also outperforms existing state-of-the-art TTP extraction method by achieving an f1-score of \(97.09\%\) when evaluated over the report dataset. TTPXHunter significantly improves cybersecurity threat intelligence by offering quick, actionable insights into attacker behaviors. This advancement automates threat intelligence analysis and provides a crucial tool for cybersecurity professionals to combat cyber threats. [ABSTRACT FROM AUTHOR]

This study measures boilerplate disclosure (BD) of Corporate Social Responsibility (CSR) reports published by Chinese companies using natural language processing (NLP) technology, and then investigates the impact of Environmental, Social, and Governance (ESG) incidents on BD of CSR reports. The results indicate that companies experiencing ESG incidents tend to reduce BD to save their reputation. Further analysis shows that more detailed CSR reports are truly helpful for firms to obtain higher ESG ratings. [ABSTRACT FROM AUTHOR]

Automating the job hazard analysis (JHA) process is an urgent requirement in the construction safety management field due to limitations of the conventional process. The manual nature of conducting the JHA and the dynamic environment of construction sites make it necessary to perform the analysis before commencing the job and to then regularly update it in accordance with changes in the construction plans. With this in mind, this research aims to develop an automated approach to support safety personnel during the JHA process. In seeking to automate the JHA process, the nature of construction accidents, hazards and risk assessment needs to be studied in light of the theoretical knowledge on accident causation. Thus, this research was designed according to the constructive research approach to develop a job hazard analysis knowledge graph (JHAKG) to automate the JHA process. The JHAKG incorporated an ontology (O-JHAKG) built according to the systematic ontology development method, METHONTOLOGY, which formalises both explicit and implicit knowledge inherent in the JHA process. The data were imported to the JHAKG from an incident database using rule-based natural language processing (NLP) which helped to extract implicit information not evident in the traditional JHA document. The validation of the JHAKG was conducted in two stages: the first stage validated the information extraction process by calculating performance metrics, while the second stage validated the data population process and the JHAKG's reasoning capability. The overall research resulted in a comprehensive JHAKG with advanced inferencing capabilities which can assist safety personnel in effectively executing the JHA process. [ABSTRACT FROM AUTHOR]

Event detection on social media platforms, especially Twitter, poses significant challenges due to the dynamic nature and high volume of data. The rapid flow of tweets and the varied ways users express thoughts complicate the identification of relevant events. Accurately identifying and interpreting events from this noisy and fast-paced environment is crucial for various applications, including crisis management and market analysis. This paper presents a novel unsupervised framework for event detection on social media, designed to enhance the accuracy and efficiency of identifying significant events from Twitter data. The framework incorporates several innovative techniques, including dynamic bandwidth adjustment based on local data density, Mahalanobis distance integration, adaptive kernel density estimation, and an improved Louvain-MOMR method for community detection. Additionally, a new scoring system is implemented to accurately extract trending words that evoke strong emotions, improving the identification of event-related keywords. The proposed framework demonstrates robust performance across three diverse datasets: FACup, Super Tuesday, and US Election, showcasing its effectiveness in capturing temporal and semantic patterns within tweets. [ABSTRACT FROM AUTHOR]

Detecting errors in Knowledge Graphs is challenging due to the scarcity of ground-truth labels and the unpredictable nature of error patterns, leading to noticeable noise affecting downstream tasks. KG's repositories contain billions of triplets representing relationships as instances (head entity, relation, tail entity). This distinctive and symbolic architecture facilitates advanced knowledge exploitation techniques aimed at improving the accuracy of other intelligence-driven applications. Making logical rules to validate triples is a traditional approach, yet it lacks generalizability due to the variability of rules across knowledge graphs stemming from domain-specific knowledge. Recent models such as TransE, TransH, TransR, and TransD have approached the task of knowledge graph completion by treating relationships as translations from head entities to tail entities. However, these models exhibit limitations in capturing the complexity and diversity of entities and relations within knowledge graphs, particularly with regards to symmetric, one-to-many, and many-to-many relations. To address these limitations, this paper introduces a novel model called TransDAE for dynamic asymmetrical embedding in knowledge graph completion, building upon the foundation of TransDR. Unlike TransDR, which focuses less on asymmetrical relations and overlooks the imbalanced characteristics of relationships, such as one-to-many and many-to-one, TransDAE considers the distinct properties of head and tail entities in similar relations. Specifically, TransDAE incorporates two vectors for each triple relationship, weighting each entity vector by its corresponding head and tail relation vectors in the relation embedding space. This enhances the model's flexibility and improves its ability to interpret and uncover latent attributes of entities and relations. Through experimental evaluation on tasks including triplet classification and link prediction, TransDAE demonstrates superior performance compared to previous models, particularly excelling in link prediction on benchmark datasets FB15K and WN18. The performance (Hits@10) of TransDAE models in predicting both head and tail entities for various relation categories in KG was presented. Across different categories, TransDAE(U) and TransDAE(B) achieved high accuracy, with TransDAE(U) generally outperforming TransDAE(B) in most cases. [ABSTRACT FROM AUTHOR]