Search

In this article, we provide a brief overview of generative artificial intelligence (GenAI) and large language models (LLMs). We then propose eight critical questions that libraries should ask when exploring this technology and its implications for their communities. We argue that libraries have a unique role in facilitating informed and responsible use of GenAI, as well as safeguarding and promoting the values of access, privacy, and intellectual freedom. [ABSTRACT FROM AUTHOR]

Since ChatGPT's release in November 2022, artificial intelligence has come into the spotlight. Inspiring both fascination and fear, chatbots have stirred debates among researchers, developers, and policy makers. The concerns range from concrete and tangible ones—which include replication of existing biases and discrimination at scale, harvesting personal data, and spreading misinformation—to more existential fears that their development will lead to machines with human-like cognitive abilities. Understanding how chatbots work and the human labor and data involved can better help evaluate the validity of concerns surrounding these systems, which although innovative, are hardly the stuff of science fiction. [ABSTRACT FROM AUTHOR]

Multiple research problems specific to advertising emerge from there, including whether and how the advertising industry should address the ethical and legal issues surrounding the possibility of AI-generated advertising content containing false or misleading information as well as strategies for designing and implementing AI-powered advertising in a socially responsible and ethical manner. In her editorial, the previous editor, Shelly Rodgers ([16]) proposed an AI classification schema to systematically understand and develop subdomains in the emerging AI advertising research field along the following three dimensions: AI type, AI function, and learning type. The lack of transparency and consumers' limited ability to understand AI technology and the mechanism and processes of AI-powered advertising would affect advertising outcomes. Since ChatGPT was released on November 30, 2022, followed by Microsoft's announcement of its artificial intelligence (AI)-powered new Bing search engine on February 7, 2023, and Google's Bard release on March 21, 2023, it seems AI generally has taken over conversations across all sectors of society. [Extracted from the article]

Purpose: To assess the diagnostic performance of two chatbots, ChatGPT and Glass, in uveitis diagnosis compared to renowned uveitis specialists, and evaluate clinicians' perception about utilizing artificial intelligence (AI) in ophthalmology practice. Methods: Six cases were presented to uveitis experts, ChatGPT (version 3.5 and 4.0) and Glass 1.0, and diagnostic accuracy was analyzed. Additionally, a survey about the emotions, confidence in utilizing AI-based tools, and the likelihood of incorporating such tools in clinical practice was done. Results: Uveitis experts accurately diagnosed all cases (100%), while ChatGPT achieved a diagnostic success rate of 66% and Glass 1.0 achieved 33%. Most attendees felt excited or optimistic about utilizing AI in ophthalmology practice. Older age and high level of education were positively correlated with increased inclination to adopt AI-based tools. Conclusions: ChatGPT demonstrated promising diagnostic capabilities in uveitis cases and ophthalmologist showed enthusiasm for the integration of AI into clinical practice. [ABSTRACT FROM AUTHOR]

This article describes an experiment to use the ChatGPT Large Language Model as a tool to refine Schema.org metadata. ChatGPT was asked to give suggestions to improve a preexisting package of Schema.org structured metadata in the NMSU Library homepage for search engine optimization. A package of reformatted metadata based on ChatGPT's recommendations was used to replace the preexisting metadata for seven weeks and relevant web stats are compared to an equivalent seven-week period from the preceding semester. This article discusses ChatGPT's recommendations in some depth and examines the outcomes from a theoretical perspective. Implications of the experiment are outlined along with future areas for research. [ABSTRACT FROM AUTHOR]

Large language models such as ChatGPT enable users to automatically produce text but also raise ethical concerns, for example about authorship and deception. This paper analyses and discusses some key philosophical assumptions in these debates, in particular assumptions about authorship and language and—our focus—the use of the appearance/reality distinction. We show that there are alternative views of what goes on with ChatGPT that do not rely on this distinction. For this purpose, we deploy the two phased approach of deconstruction and relate our finds to questions regarding authorship and language in the humanities. We also identify and respond to two common counter-objections in order to show the ethical appeal and practical use of our proposal. [ABSTRACT FROM AUTHOR]

A large language model (LLM) is an artificial intelligence (AI) model that uses natural language processing (NLP) to understand, interpret, and generate human-like language responses from unstructured text input. Its real-time response capabilities and eloquent dialogue enhance the interactive user experience in human–AI communication like never before. By gathering several sources on the internet, LLM chatbots can interact and respond to a wide range of queries, including problem solving, text summarization, and creating informative notes. Since ophthalmology is one of the medical fields integrating image analysis, telemedicine, AI, and other technologies, LLMs are likely to play an important role in eye care in the near future. This review summarizes the performance and potential applicability of LLMs in ophthalmology according to currently available publications. [ABSTRACT FROM AUTHOR]

Introduction: OpenAI recently introduced the ability to create custom generative pre-trained transformers (cGPTs) using text-based instruction and/or external documents using retrieval-augmented generation (RAG) architecture without coding knowledge. This study aimed to analyze the features of ophthalmology-related cGPTs and explore their potential utilities. Methods: Data collection took place on January 20 and 21, 2024, and custom GPTs were found by entering ophthalmology keywords into the "Explore GPTS" section of the website. General and specific features of cGPTs were recorded, such as knowledge other than GPT-4 training data. The instruction and description sections were analyzed for compatibility using the Likert scale. We analyzed two custom GPTs with the highest Likert score in detail. We attempted to create a convincingly presented yet potentially harmful cGPT to test safety features. Results: We analyzed 22 ophthalmic cGPTs, of which 55% were for general use and the most common subspecialty was glaucoma (18%). Over half (55%) contained knowledge other than GPT-4 training data. The representation of the instructions through the description was between "Moderately representative" and "Very representative" with a median Likert score of 3.5 (IQR 3.0–4.0). The instruction word count was significantly associated with Likert scores (P = 0.03). Tested cGPTs demonstrated potential for specific conversational tone, information, retrieval and combining knowledge from an uploaded source. With these safety settings, creating a malicious GPT was possible. Conclusions: This is the first study to our knowledge to examine the GPT store for a medical field. Our findings suggest that these cGPTs can be immediately implemented in practice and may offer more targeted and effective solutions compared to the standard GPT-4. However, further research is necessary to evaluate their capabilities and limitations comprehensively. The safety features currently appear to be rather limited. It may be helpful for the user to review the instruction section before using a cGPT. [ABSTRACT FROM AUTHOR]

ChatGPT and other LLMs are at the forefront of pedagogical considerations in classrooms across the academy. Many studies have spoken to the technology's capacity to generate one-off texts in a variety of genres. This study complements those by inquiring into its capacity to generate compelling texts at scale. In this study, we quantitatively and qualitatively analyze a small corpus of generated texts in two genres and gauge it against novice and published academic writers along known dimensions of linguistic variation. Theoretically, we position and historicize ChatGPT as a writing technology and consider the ways in which generated text may not be congruent with established trajectories of writing development in higher education. Our study found that generated texts are more informationally dense than authored texts and often read as dialogically closed, "empty," and "fluffy." We close with a discussion of potentially explanatory linguistic features, as well as relevant pedagogical implications. [ABSTRACT FROM AUTHOR]

The potential applications of artificial intelligence (AI) in health care have garnered significant interest in recent years. This study presents the first published exploration of ChatGPT, an AI language model, as a tool for providing postoperative guidance during rhinoplasty recovery. The primary objective was to shed light on the role of ChatGPT in augmenting patient care during the critical postoperative phase. Using the Rhinobase database, standardized questions were formulated to evaluate AI-generated responses addressing pain management, swelling, bruising, and potential asymmetries. Results demonstrated that ChatGPT has the potential to enhance patient education and alleviate emotional distress by providing general information and reassurance during the recovery process. However, the study emphasized that AI should not replace personalized advice from qualified health care professionals. This pioneering investigation offers valuable insights into the integration of AI and human expertise, paving the way for optimized postrhinoplasty recovery care. [ABSTRACT FROM AUTHOR]

Background: The complex nature of rheumatic diseases poses considerable challenges for clinicians when developing individualized treatment plans. Large language models (LLMs) such as ChatGPT could enable treatment decision support. Objective: To compare treatment plans generated by ChatGPT-3.5 and GPT-4 to those of a clinical rheumatology board (RB). Design/methods: Fictional patient vignettes were created and GPT-3.5, GPT-4, and the RB were queried to provide respective first- and second-line treatment plans with underlying justifications. Four rheumatologists from different centers, blinded to the origin of treatment plans, selected the overall preferred treatment concept and assessed treatment plans' safety, EULAR guideline adherence, medical adequacy, overall quality, justification of the treatment plans and their completeness as well as patient vignette difficulty using a 5-point Likert scale. Results: 20 fictional vignettes covering various rheumatic diseases and varying difficulty levels were assembled and a total of 160 ratings were assessed. In 68.8% (110/160) of cases, raters preferred the RB's treatment plans over those generated by GPT-4 (16.3%; 26/160) and GPT-3.5 (15.0%; 24/160). GPT-4's plans were chosen more frequently for first-line treatments compared to GPT-3.5. No significant safety differences were observed between RB and GPT-4's first-line treatment plans. Rheumatologists' plans received significantly higher ratings in guideline adherence, medical appropriateness, completeness and overall quality. Ratings did not correlate with the vignette difficulty. LLM-generated plans were notably longer and more detailed. Conclusion: GPT-4 and GPT-3.5 generated safe, high-quality treatment plans for rheumatic diseases, demonstrating promise in clinical decision support. Future research should investigate detailed standardized prompts and the impact of LLM usage on clinical decisions. [ABSTRACT FROM AUTHOR]

We evaluated the performance of a chatbot compared to the National Comprehensive Cancer Network (NCCN) Guidelines for the management of ovarian cancer. Using NCCN Guidelines, we generated 10 questions and answers regarding management of ovarian cancer at a single point in time. Questions were thematically divided into risk factors, surgical management, medical management, and surveillance. We asked ChatGPT (GPT-4) to provide responses without prompting (unprompted GPT) and with prompt engineering (prompted GPT). Responses were blinded and evaluated for accuracy and completeness by 5 gynecologic oncologists. A score of 0 was defined as inaccurate, 1 as accurate and incomplete, and 2 as accurate and complete. Evaluations were compared among NCCN, unprompted GPT, and prompted GPT answers. Overall, 48% of responses from NCCN, 64% from unprompted GPT, and 66% from prompted GPT were accurate and complete. The percentage of accurate but incomplete responses was higher for NCCN vs GPT-4. The percentage of accurate and complete scores for questions regarding risk factors, surgical management, and surveillance was higher for GPT-4 vs NCCN; however, for questions regarding medical management, the percentage was lower for GPT-4 vs NCCN. Overall, 14% of responses from unprompted GPT, 12% from prompted GPT, and 10% from NCCN were inaccurate. GPT-4 provided accurate and complete responses at a single point in time to a limited set of questions regarding ovarian cancer, with best performance in areas of risk factors, surgical management, and surveillance. Occasional inaccuracies, however, should limit unsupervised use of chatbots at this time. • There is interest in expanding the use of chatbot technology in medicine. • A high percentage of responses from ChatGPT to questions about ovarian cancer were graded as accurate and complete. • ChatGPT had a higher percentage of inaccurate responses compared to the National Comprehensive Cancer Network Guidelines. • ChatGPT received the highest scores in response to questions regarding risk factors, surgical management, and surveillance. [ABSTRACT FROM AUTHOR]

Since 2023, ChatGPT has been leading a research boom in large language models. Research on the applications of large language models in various fields is also being explored. The aim of this study was to explore the use of ChatGPT/GPT-4 for post-surgery patient follow-up after oral surgery. Thirty questions that are the most commonly asked or may be encountered during follow-up and in daily practice were collected to test ChatGPT/GPT-4′s responses. A standard prompt was used for each question. The responses given by ChatGPT/GPT-4 were evaluated by three experienced oral and maxillofacial surgeons to assess the suitability of this technology for clinical follow-up, based on the accuracy of medical knowledge and rationality of the advice in ChatGPT/GPT-4′s responses. ChatGPT/GPT-4 achieved full marks in terms of both the accuracy of its medical knowledge and the rationality of its recommendations. Additionally, ChatGPT/GPT-4 was able to accurately sense patient emotions and provide them with reassurance. In conclusion, ChatGPT/GPT-4 could be used for patient follow-up after oral surgeries, but this should be done with careful consideration of the technology's current limitations and under the guidance of healthcare professionals. [ABSTRACT FROM AUTHOR]

As Natural Language Processing (NLP) continues to advance, driven by the emergence of sophisticated large language models such as ChatGPT, there has been a notable growth in research activity. This rapid uptake reflects increasing interest in the field and induces critical inquiries into ChatGPT's applicability in the NLP domain. This review paper systematically investigates the role of ChatGPT in diverse NLP tasks, including information extraction, Name Entity Recognition (NER), event extraction, relation extraction, Part of Speech (PoS) tagging, text classification, sentiment analysis, emotion recognition and text annotation. The novelty of this work lies in its comprehensive analysis of the existing literature, addressing a critical gap in understanding ChatGPT's adaptability, limitations, and optimal application. In this paper, we employed a systematic stepwise approach following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework to direct our search process and seek relevant studies. Our review reveals ChatGPT's significant potential in enhancing various NLP tasks. Its adaptability in information extraction tasks, sentiment analysis, and text classification showcases its ability to comprehend diverse contexts and extract meaningful details. Additionally, ChatGPT's flexibility in annotation tasks reduces manual efforts and accelerates the annotation process, making it a valuable asset in NLP development and research. Furthermore, GPT-4 and prompt engineering emerge as a complementary mechanism, empowering users to guide the model and enhance overall accuracy. Despite its promising potential, challenges persist. The performance of ChatGPT needs to be tested using more extensive datasets and diverse data structures. Subsequently, its limitations in handling domain-specific language and the need for fine-tuning in specific applications highlight the importance of further investigations to address these issues. [ABSTRACT FROM AUTHOR]

The fast evolution of artificial intelligence frameworks has resulted in the creation of increasingly sophisticated large language models (LLM), ChatGPT being the most famous one. This study paper dives into this LLM with a case study of ChatGPT's architecture and provides a thorough comparative analysis of its numerous versions, tracking its history from its conception to its most recent incarnations. This research intends to give a full knowledge of the model's history by investigating the underlying mechanisms and enhancements provided in each edition. The comparative analysis covers key aspects such as model size, training data, fine-tuning techniques, and performance metrics. Furthermore, this study evaluates the limits of ChatGPT in its many incarnations. These limitations include common sense reasoning difficulties, biased replies, verbosity, sensitivity to input wording, and others. Each constraint is investigated for potential remedies and workarounds. This research article also provides a complete analysis of the ChatGPT architecture and its progress through multiple iterations. It gives vital insights for academics, developers, and users wanting to harness the promise of ChatGPT while managing its restrictions by exploring both the model's strengths and limitations. The distinctiveness of this paper rests in its comprehensive assessment of ChatGPT's architectural development and its practical strategy for resolving the myriad difficulties in producing cohesive and contextually relevant replies. [ABSTRACT FROM AUTHOR]

In part two of this mini-series, we evaluate the range of machine-learning tools now available for application to veterinary clinical text-mining. These tools will be vital to automate extraction of information from large datasets of veterinary clinical narratives curated by projects such as the Small Animal Veterinary Surveillance Network (SAVSNET) and VetCompass, where volumes of millions of records preclude reading records and the complexities of clinical notes limit usefulness of more "traditional" text-mining approaches. We discuss the application of various machine learning techniques ranging from simple models for identifying words and phrases with similar meanings to expand lexicons for keyword searching, to the use of more complex language models. Specifically, we describe the use of language models for record annotation, unsupervised approaches for identifying topics within large datasets, and discuss more recent developments in the area of generative models (such as ChatGPT). As these models become increasingly complex it is pertinent that researchers and clinicians work together to ensure that the outputs of these models are explainable in order to instill confidence in any conclusions drawn from them. [ABSTRACT FROM AUTHOR]

Adverse drug events (ADEs) are one of the major causes of hospital admissions and are associated with increased morbidity and mortality. Post-marketing ADE identification is one of the most important phases of drug safety surveillance. Traditionally, data sources for post-marketing surveillance mainly come from spontaneous reporting system such as the Food and Drug Administration Adverse Event Reporting System (FAERS). Social media data such as posts on X (formerly Twitter) contain rich patient and medication information and could potentially accelerate drug surveillance research. However, ADE information in social media data is usually locked in the text, making it difficult to be employed by traditional statistical approaches. In recent years, large language models (LLMs) have shown promise in many natural language processing tasks. In this study, we developed several LLMs to perform ADE classification on X data. We fine-tuned various LLMs including BERT-base, Bio_ClinicalBERT, RoBERTa, and RoBERTa-large. We also experimented ChatGPT few-shot prompting and ChatGPT fine-tuned on the whole training data. We then evaluated the model performance based on sensitivity, specificity, negative predictive value, positive predictive value, accuracy, F1-measure, and area under the ROC curve. Our results showed that RoBERTa-large achieved the best F1-measure (0.8) among all models followed by ChatGPT fine-tuned model with F1-measure of 0.75. Our feature importance analysis based on 1200 random samples and RoBERTa-Large showed the most important features are as follows: “withdrawals”/”withdrawal”, “dry”, “dealing”, “mouth”, and “paralysis”. The good model performance and clinically relevant features show the potential of LLMs in augmenting ADE detection for post-marketing drug safety surveillance. [ABSTRACT FROM AUTHOR]

Large language models such as ChatGPT have been shown to excel in solving complex math problems. However, they cannot solve basic arithmetic problems such as 758*639 = 484,362. This makes us ponder if LLMs have been trained to solve math and science problems in the right way. When a student learns math at school, she or he starts with arithmetic, then moves to word problems, polynomials, and calculus. Each skill she or he acquires will be used in the next stage to solve more advanced problems. In this paper we propose Scaffolding Learning for LLMs, which imitates how a student learns a subject in a step-by-step manner. For example, we first train an LLM to perform highly specific operations such as multiplication and division, and then apply such "skills" in a more generic task such as solving word problems. This is related to Curriculum Training, which trains a model on tasks following a specific order, such as training on easy tasks first and then gradually increases the difficulty. Our proposed approach goes from specific tasks to generic ones, which can be considered as a special case of Curriculum Training. Our empirical studies show that when an LLM has "mastered" a specific skill, only a small amount of training is required to teach it to apply the skill to a more generic application. [ABSTRACT FROM AUTHOR]

AbstractBackgroundObjectiveMethodsResultsConclusionNuclear Medicine(NM), as an inherently interdisciplinary field, integrates diverse scientific principles and advanced imaging techniques. The advent of ChatGPT, a large language model, opens new avenues for medical educational innovation. With its advanced natural language processing abilities and complex algorithms, ChatGPT holds the potential to substantially enrich medical education, particularly in NM.To investigate the current application of ChatGPT in undergraduate Nuclear Medicine Education(NME).Employing a mixed-methods sequential explanatory design, the research investigates the current status of NME, the use of ChatGPT and the attitude towards ChatGPT among teachers and students in the Second Clinical College of Chongqing Medical University.The investigation yields several salient findings: (1) Students and educators in NM face numerous challenges in the learning process; (2) ChatGPT is found to possess significant applicability and potential benefits in NME; (3) There is a pronounced inclination among respondents to adopt ChatGPT, with a keen interest in its diverse applications within the educational sphere.ChatGPT has been utilized to address the difficulties faced by undergraduates at Chongqing Medical University in NME, and has been applied in various aspects to assist learning. The findings of this survey may offer some insights into how ChatGPT can be integrated into practical medical education. [ABSTRACT FROM AUTHOR]

Sentiment analysis is a vital task in natural language processing (NLP) that aims to identify and extract the emotional states and opinions of text. In this study, we conduct a comprehensive comparison of large language models (LLMs), such as ChatGPT and Google Bard, with conventional methods in sentiment analysis. We employ a rigorous evaluation framework that covers four essential metrics: accuracy, precision, recall, and the F1-score. Our results reveal that TextBlob outperforms other methods, achieving an impressive accuracy of 69% and precision of 83%. On the other hand, Bard shows a relatively poor performance, with only 39% accuracy and 46% precision. This study offers valuable insights into the diverse capabilities of AI models in sentiment analysis. A key finding of this study is the importance of model selection according to the specific requirements of the task. Each model has its own strengths and weaknesses, which are reflected in their performance profiles. Moreover, the context in which these models operate is crucial. For instance, ChatGPT generates varied responses, Bard struggles with multiple sentences, and Robustly Optimized BERT Pretraining Approach (RoBERTa) balances precision and recall. This study also reveals the performance gap between LLMs and state-of-the-art deep learning methods. We believe this work will inspire future research and applications of ChatGPT and similar AI models in sentiment analysis and related tasks. [ABSTRACT FROM AUTHOR]

This study compares how the generative AI (GenAI) large language model (LLM) ChatGPT performs in grading university exams compared to human teachers. Aspects investigated include consistency, large discrepancies and length of answer. Implications for higher education, including the role of teachers and ethics, are also discussed. Three Master's‐level exams were scored using ChatGPT 3.5, and the results were compared with the teachers' scoring and the grading teachers were interviewed. In total, 463 exam responses were graded. With each response being graded at least three times, a total of 1389 gradings were conducted. For the final exam scores, 70% of ChatGPT's gradings were within 10% of the teachers' gradings and 31% within 5%. ChatGPT tended to give marginally higher scores. The agreement on grades is 30%, but 45% of the exams received an adjacent grade. On individual questions, ChatGPT is more inclined to avoid very high or very low scores. ChatGPT struggles to correctly score questions closely related to the course lectures but performs better on more general questions. The AI can generate plausible scores on university exams that, at first glance, look similar to a human grader. There are differences but it is not unlikely that two different human graders could result in similar discrepancies. During the interviews, teachers expressed their surprise at how well ChatGPT's grading matched their own. Increased use of AI can lead to ethical challenges as exams are entrusted to a machine whose decision‐making criteria are not fully understood, especially concerning potential bias in training data. [ABSTRACT FROM AUTHOR]

This letter to the editor discusses the use of artificial intelligence chatbots for providing cancer treatment information. The authors refer to a study that evaluated the responses of a chatbot to common patient queries in dermatology, as well as other studies that assessed the accuracy and reliability of chatbot responses to physician questions and the effectiveness of chatbots in evaluating systemic therapies for plaque psoriasis. The authors emphasize the potential role of large language models in medical education and stress the importance of evaluating the reliability of medical information provided by chatbots. They provide additional resources on retinoids, the natural history of cutaneous propionibacteria, and evidence-based guidelines for acne treatment. [Extracted from the article]

Background: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images. Objective: To assess the ability of multimodal LLMs to interpret pediatric radiological images. Materials and methods: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method. Results: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5–37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7–26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6–68.5%). Conclusion: Multimodal LLMs are not yet capable of interpreting pediatric radiological images. [ABSTRACT FROM AUTHOR]

Recent advances in artificial intelligence (AI) give chatbots, e.g. ChatGPT, more human-like interaction and conversational capability. AI chatbots are becoming more popular for supporting language learning. Most current review research disregards the significance of teachers’ roles in chatbot-assisted language learning. Self-determination theory (SDT) explains the roles by suggesting how teachers satisfy student needs and how chatbots thwart those needs. Therefore, this systemic review aims to: (i) suggest the roles that teachers play in student English learning with AI chatbot; (ii) discuss how those roles satisfy SDT needs of the students; and (iii) discuss the challenges in this learning. This review selected 23 articles published throughout the last ten years (2014–2023). The findings offer (i) four empirical and theoretical contributions: technological pedagogical content knowledge (TPACK) for chatbots, needs satisfaction from teachers, needs thwarting from chatbots, and SDT-based research designs for chatbots; (ii) two practical suggestions: understanding technological knowledge of chatbots and SDT needs support for chatbots; and (iii) six research directions: experimental studies, student language proficiency, more teacher roles, revisiting TPACK, future development of chatbots, and large language model teacher-like chatbots. Overall, the findings enhance our knowledge of TPACK and teacher digital and AI competences for chatbot-assisted language learning. [ABSTRACT FROM AUTHOR]

Background: Uveitis is the ophthalmic subfield dealing with a broad range of intraocular inflammatory diseases. With the raising importance of LLM such as ChatGPT and their potential use in the medical field, this research explores the strengths and weaknesses of its applicability in the subfield of uveitis. Methods: A series of highly clinically relevant questions were asked three consecutive times (attempts 1, 2 and 3) of the LLM regarding current uveitis cases. The answers were classified on whether they were accurate and sufficient, partially accurate and sufficient or inaccurate and insufficient. Statistical analysis included descriptive analysis, normality distribution, non-parametric test and reliability tests. References were checked for their correctness in different medical databases. Results: The data showed non-normal distribution. Data between subgroups (attempts 1, 2 and 3) was comparable (Kruskal-Wallis H test, p-value = 0.7338). There was a moderate agreement between attempt 1 and attempt 2 (Cohen's kappa, ĸ = 0.5172) as well as between attempt 2 and attempt 3 (Cohen's kappa, ĸ = 0.4913). There was a fair agreement between attempt 1 and attempt 3 (Cohen's kappa, ĸ = 0.3647). The average agreement was moderate (Cohen's kappa, ĸ = 0.4577). Between the three attempts together, there was a moderate agreement (Fleiss' kappa, ĸ = 0.4534). A total of 52 references were generated by the LLM. 22 references (42.3%) were found to be accurate and correctly cited. Another 22 references (42.3%) could not be located in any of the searched databases. The remaining 8 references (15.4%) were found to exist, but were either misinterpreted or incorrectly cited by the LLM. Conclusion: Our results demonstrate the significant potential of LLMs in uveitis. However, their implementation requires rigorous training and comprehensive testing for specific medical tasks. We also found out that the references made by ChatGPT 4.o were in most cases incorrect. LLMs are likely to become invaluable tools in shaping the future of ophthalmology, enhancing clinical decision-making and patient care. [ABSTRACT FROM AUTHOR]

Large language models (LLMs) are generating interest in medical settings. For example, LLMs can respond coherently to medical queries by providing plausible differential diagnoses based on clinical notes. However, there are many questions to explore, such as evaluating differences between open- and closed-source LLMs as well as LLM performance on queries from both medical and non-medical users. In this study, we assessed multiple LLMs, including Llama-2-chat, Vicuna, Medllama2, Bard/Gemini, Claude, ChatGPT3.5, and ChatGPT-4, as well as non-LLM approaches (Google search and Phenomizer) regarding their ability to identify genetic conditions from textbook-like clinician questions and their corresponding layperson translations related to 63 genetic conditions. For open-source LLMs, larger models were more accurate than smaller LLMs: 7b, 13b, and larger than 33b parameter models obtained accuracy ranges from 21%–49%, 41%–51%, and 54%–68%, respectively. Closed-source LLMs outperformed open-source LLMs, with ChatGPT-4 performing best (89%–90%). Three of 11 LLMs and Google search had significant performance gaps between clinician and layperson prompts. We also evaluated how in-context prompting and keyword removal affected open-source LLM performance. Models were provided with 2 types of in-context prompts: list-type prompts, which improved LLM performance, and definition-type prompts, which did not. We further analyzed removal of rare terms from descriptions, which decreased accuracy for 5 of 7 evaluated LLMs. Finally, we observed much lower performance with real individuals' descriptions; LLMs answered these questions with a maximum 21% accuracy. Large language models (LLMs) are rapidly changing the biomedical landscape. We compared the ability of multiple LLMs and other search techniques in identifying genetic conditions from text descriptions, including medical and non-medical (colloquial) descriptions. We also investigated the effects of different prompting approaches and the removal of rare terms. [ABSTRACT FROM AUTHOR]

Background Methods Results Conclusion Accurate conflict of interest (COI) information is essential for promoting transparency and trust in research. We aim to assess COI disclosure patterns in monoclonal antibodies (MABs) research for chronic rhinosinusitis with nasal polyposis (CRSwNP) using the Open Payments Database (OPD).Studies on FDA‐approved MABs for CRSwNP (dupilumab, omalizumab, mepolizumab) published between 2019 and 2021 with at least one US author were identified through PubMed. Industry‐reported payments from the manufacturers (Sanofi, Regeneron, Genentech, Novartis, and GlaxoSmithKline) between 2018 and 2021 in OPD's General Payments category were collected. Authors were cross‐checked against OPD metadata using a previously published ChatGPT‐based algorithm. Additionally, this novel algorithm analyzed COI statements for relevant author‒company specific disclosures, identifying disclosed and undisclosed payments made 3‒15 months prior to publication.A total of 214 unique authors from 76 studies were included. Of 30 articles that received at least one relevant payment, 21 (70%) were found to have an undisclosed COI, with a mean total undisclosed payment of $4890 and a median of $10,331. Fifty‐six authors had relevant OPD payments and 40 (71.4%) authors did not declare a potential COI. Interestingly, 158 authors had no relevant payments and 62 (39.2%) declared a potential COI. Author order was not significantly associated with potential under‐ or over‐disclosure.This study characterizes COI disclosure patterns in rhinosinusitis‐relevant MABs research using a novel automated approach. Given the discrepancy between disclosures and industry‐reported payments, our findings suggest a need for improved disclosure education and practices. [ABSTRACT FROM AUTHOR]

Summary Large language models (LLMs) have significantly impacted various fields with their ability to understand and generate human‐like text. This study explores the potential benefits and limitations of integrating LLMs, such as ChatGPT, into haematology practices. Utilizing systematic review methodologies, we analysed studies published after 1 December 2022, from databases like PubMed, Web of Science and Scopus, and assessing each for bias with the QUADAS‐2 tool. We reviewed 10 studies that applied LLMs in various haematology contexts. These models demonstrated proficiency in specific tasks, such as achieving 76% diagnostic accuracy for haemoglobinopathies. However, the research highlighted inconsistencies in performance and reference accuracy, indicating variability in reliability across different uses. Additionally, the limited scope of these studies and constraints on datasets could potentially limit the generalizability of our findings. The findings suggest that, while LLMs provide notable advantages in enhancing diagnostic processes and educational resources within haematology, their integration into clinical practice requires careful consideration. Before implementing them in haematology, rigorous testing and specific adaptation are essential. This involves validating their accuracy and reliability across different scenarios. Given the field's complexity, it is also critical to continuously monitor these models and adapt them responsively. [ABSTRACT FROM AUTHOR]

Background: Large language models (LLMs) such as ChatGPT-4 and Google Gemini show potential for patient health education, but concerns about their accuracy require careful evaluation. This study evaluates the readability and accuracy of ChatGPT-4 and Google Gemini in answering questions about retinal detachment. Methods: Comparative study analyzing responses from ChatGPT-4 and Google Gemini to 13 retinal detachment questions, categorized by difficulty levels (D1, D2, D3). Masked responses were reviewed by ten vitreoretinal specialists and rated on correctness, errors, thematic accuracy, coherence, and overall quality grading. Analysis included Flesch Readability Ease Score, word and sentence counts. Results: Both Artificial Intelligence tools required college-level understanding for all difficulty levels. Google Gemini was easier to understand (p = 0.03), while ChatGPT-4 provided more correct answers for the more difficult questions (p = 0.0005) with fewer serious errors. ChatGPT-4 scored highest on most challenging questions, showing superior thematic accuracy (p = 0.003). ChatGPT-4 outperformed Google Gemini in 8 of 13 questions, with higher overall quality grades in the easiest (p = 0.03) and hardest levels (p = 0.0002), showing a lower grade as question difficulty increased. Conclusions: ChatGPT-4 and Google Gemini effectively address queries about retinal detachment, offering mostly accurate answers with few critical errors, though patients require higher education for comprehension. The implementation of AI tools may contribute to improving medical care by providing accurate and relevant healthcare information quickly. [ABSTRACT FROM AUTHOR]

Large language models like Chat Generative Pre-Trained Transformer (ChatGPT) are increasingly used in academic writing. Faculty may consider use of artificial intelligence (AI)–generated responses a form of cheating. We sought to determine whether general surgery residency faculty could detect AI versus human-written responses to a text prompt; hypothesizing that faculty would not be able to reliably differentiate AI versus human-written responses. Ten essays were generated using a text prompt, "Tell us in 1-2 paragraphs why you are considering the University of Rochester for General Surgery residency" (Current trainees: n = 5, ChatGPT: n = 5). Ten blinded faculty reviewers rated essays (ten-point Likert scale) on the following criteria: desire to interview, relevance to the general surgery residency, overall impression, and AI- or human-generated; with scores and identification error rates compared between the groups. There were no differences between groups for %total points (ChatGPT 66.0 ± 13.5%, human 70.0 ± 23.0%, P = 0.508) or identification error rates (ChatGPT 40.0 ± 35.0%, human 20.0 ± 30.0%, P = 0.175). Except for one, all essays were identified incorrectly by at least two reviewers. Essays identified as human-generated received higher overall impression scores (area under the curve: 0.82 ± 0.04, P < 0.01). Whether use of AI tools for academic purposes should constitute academic dishonesty is controversial. We demonstrate that human and AI-generated essays are similar in quality, but there is bias against presumed AI-generated essays. Faculty are not able to reliably differentiate human from AI-generated essays, thus bias may be misdirected. AI-tools are becoming ubiquitous and their use is not easily detected. Faculty must expect these tools to play increasing roles in medical education. [ABSTRACT FROM AUTHOR]

Without writing a single line of code by a human, an example Monte Carlo simulation-based application for stochastic dependence modeling with copulas is developed through pair programming involving a human partner and a large language model (LLM) fine-tuned for conversations. This process encompasses interacting with ChatGPT using both natural language and mathematical formalism. Under the careful supervision of a human expert, this interaction facilitated the creation of functioning code in MATLAB, Python, and R. The code performs a variety of tasks including sampling from a given copula model, evaluating the model's density, conducting maximum likelihood estimation, optimizing for parallel computing on CPUs and GPUs, and visualizing the computed results. In contrast to other emerging studies that assess the accuracy of LLMs like ChatGPT on tasks from a selected area, this work rather investigates ways how to achieve a successful solution of a standard statistical task in a collaboration of a human expert and artificial intelligence (AI). Particularly, through careful prompt engineering, we separate successful solutions generated by ChatGPT from unsuccessful ones, resulting in a comprehensive list of related pros and cons. It is demonstrated that if the typical pitfalls are avoided, we can substantially benefit from collaborating with an AI partner. For example, we show that if ChatGPT is not able to provide a correct solution due to a lack of or incorrect knowledge, the human-expert can feed it with the correct knowledge, e.g., in the form of mathematical theorems and formulas, and make it to apply the gained knowledge in order to provide a correct solution. Such ability presents an attractive opportunity to achieve a programmed solution even for users with rather limited knowledge of programming techniques. [ABSTRACT FROM AUTHOR]

Computer systems and applications generate large amounts of logs to measure and record information, which is vital to protect the systems from malicious attacks and useful for repairing faults, especially with the rapid development of distributed computing. Among various logs, the anomaly log is beneficial for operations and maintenance (O&M) personnel to locate faults and improve efficiency. In this paper, we utilize a large language model, ChatGPT, for the log parser task. We choose the BERT model, a self-supervised framework for log anomaly detection. BERT, an embedded transformer encoder, with a self-attention mechanism can better handle context-dependent tasks such as anomaly log detection. Meanwhile, it is based on the masked language model task and next sentence prediction task in the pretraining period to capture the normal log sequence pattern. The experimental results on two log datasets show that the BERT model combined with an LLM performed better than other classical models such as Deelog and Loganomaly. [ABSTRACT FROM AUTHOR]

Purpose: This study evaluates the efficacy of two advanced Large Language Models (LLMs), OpenAI's ChatGPT 4 and Google's Gemini Advanced, in providing treatment recommendations for head and neck oncology cases. The aim is to assess their utility in supporting multidisciplinary oncological evaluations and decision-making processes. Methods: This comparative analysis examined the responses of ChatGPT 4 and Gemini Advanced to five hypothetical cases of head and neck cancer, each representing a different anatomical subsite. The responses were evaluated against the latest National Comprehensive Cancer Network (NCCN) guidelines by two blinded panels using the total disagreement score (TDS) and the artificial intelligence performance instrument (AIPI). Statistical assessments were performed using the Wilcoxon signed-rank test and the Friedman test. Results: Both LLMs produced relevant treatment recommendations with ChatGPT 4 generally outperforming Gemini Advanced regarding adherence to guidelines and comprehensive treatment planning. ChatGPT 4 showed higher AIPI scores (median 3 [2–4]) compared to Gemini Advanced (median 2 [2–3]), indicating better overall performance. Notably, inconsistencies were observed in the management of induction chemotherapy and surgical decisions, such as neck dissection. Conclusions: While both LLMs demonstrated the potential to aid in the multidisciplinary management of head and neck oncology, discrepancies in certain critical areas highlight the need for further refinement. The study supports the growing role of AI in enhancing clinical decision-making but also emphasizes the necessity for continuous updates and validation against current clinical standards to integrate AI into healthcare practices fully. [ABSTRACT FROM AUTHOR]

The rapidly increasing adoption of large language models in medicine has drawn attention to potential applications within the field of neurosurgery. This study evaluates the effects of various contextualization methods on ChatGPT's ability to provide expert-consensus aligned recommendations on the diagnosis and management of Chiari Malformation and Syringomyelia. Native GPT4 and GPT4 models contextualized using various strategies were asked questions revised from the 2022 Chiari and Syringomyelia Consortium International Consensus Document. ChatGPT-provided responses were then compared to consensus statements using reviewer assessments of 1) responding to the prompt, 2) agreement of ChatGPT response with consensus statements, 3) recommendation to consult with a medical professional, and 4) presence of supplementary information. Flesch-Kincaid, SMOG, word count, and Gunning-Fog readability scores were calculated for each model using the quanteda package in R. Relative to GPT4, all contextualized GPTs demonstrated increased agreement with consensus statements. PDF+Prompting and Prompting models provided the most elevated agreement scores of 19 of 24 and 23 of 24, respectively, versus 9 of 24 for GPT4 (p=.021, p=.001). A trend toward improved readability was observed when comparing contextualized models at large to ChatGPT4, with significant decreases in average word count (180.7 vs 382.3, p<.001) and Flesch-Kincaid Reading Ease score (11.7 vs 17.2, p=.033). The enhanced performance observed in response to ChatGPT4 contextualization suggests broader applications of large language models in neurosurgery than what the current literature indicates. This study provides proof of concept for the use of contextualized GPT models in neurosurgical contexts and showcases the easy accessibility of improved model performance. [ABSTRACT FROM AUTHOR]

Purpose: The aim of this study was to define the capability of ChatGPT-4 and Google Gemini in analyzing detailed glaucoma case descriptions and suggesting an accurate surgical plan. Methods: Retrospective analysis of 60 medical records of surgical glaucoma was divided into "ordinary" (n = 40) and "challenging" (n = 20) scenarios. Case descriptions were entered into ChatGPT and Bard's interfaces with the question "What kind of surgery would you perform?" and repeated three times to analyze the answers' consistency. After collecting the answers, we assessed the level of agreement with the unified opinion of three glaucoma surgeons. Moreover, we graded the quality of the responses with scores from 1 (poor quality) to 5 (excellent quality), according to the Global Quality Score (GQS) and compared the results. Results: ChatGPT surgical choice was consistent with those of glaucoma specialists in 35/60 cases (58%), compared to 19/60 (32%) of Gemini (p = 0.0001). Gemini was not able to complete the task in 16 cases (27%). Trabeculectomy was the most frequent choice for both chatbots (53% and 50% for ChatGPT and Gemini, respectively). In "challenging" cases, ChatGPT agreed with specialists in 9/20 choices (45%), outperforming Google Gemini performances (4/20, 20%). Overall, GQS scores were 3.5 ± 1.2 and 2.1 ± 1.5 for ChatGPT and Gemini (p = 0.002). This difference was even more marked if focusing only on "challenging" cases (1.5 ± 1.4 vs. 3.0 ± 1.5, p = 0.001). Conclusion: ChatGPT-4 showed a good analysis performance for glaucoma surgical cases, either ordinary or challenging. On the other side, Google Gemini showed strong limitations in this setting, presenting high rates of unprecise or missed answers. [ABSTRACT FROM AUTHOR]

Large Language Models (LLMs) stand on the brink of reshaping the field of aging and dementia care, challenging the one-size-fits-all paradigm with their capacity for precision medicine and individualized treatment strategies. The "Large Pre-Trained Models with a Focus on AD/ADRD and Healthy Aging" symposium, organized by the National Institute on Aging and the Johns Hopkins Artificial Intelligence & Technology Collaboratory for Aging Research, served as a platform for exploring this potential. The symposium brought together diverse experts to discuss the integration of LLMs in aging and dementia care. They highlighted the roles LLMs can play in clinical decision support and predictive analytics, while also addressing critical ethical concerns including bias, privacy, and the responsible use of artificial intelligence (AI). The discussions focused on the need to balance technological advancement with ethical considerations in AI deployment. In conclusion, the symposium projected a future where LLMs not only revolutionize healthcare practices but also pose significant challenges that require careful navigation. [ABSTRACT FROM AUTHOR]

The adoption of large language models (LLMs) in education holds much promise. However, like many technological innovations before them, adoption and access can often be inequitable from the outset, creating more divides than they bridge. In this paper, we explore the magnitude of the country and language divide in the leading open‐source and proprietary LLMs with respect to knowledge of K‐12 taxonomies in a variety of countries and their performance on tagging problem content with the appropriate skill from a taxonomy, an important task for aligning open educational resources and tutoring content with state curricula. We also experiment with approaches to narrowing the performance divide by enhancing LLM skill tagging performance across four countries (the USA, Ireland, South Korea and India–Maharashtra) for more equitable outcomes. We observe considerable performance disparities not only with non‐English languages but with English and non‐US taxonomies. Our findings demonstrate that fine‐tuning GPT‐3.5 with a few labelled examples can improve its proficiency in tagging problems with relevant skills or standards, even for countries and languages that are underrepresented during training. Furthermore, the fine‐tuning results show the potential viability of GPT as a multilingual skill classifier. Using both an open‐source model, Llama2‐13B, and a closed‐source model, GPT‐3.5, we also observe large disparities in tagging performance between the two and find that fine‐tuning and skill information in the prompt improve both, but the closed‐source model improves to a much greater extent. Our study contributes to the first empirical results on mitigating disparities across countries and languages with LLMs in an educational context. Practitioner notesWhat is already known about this topic Recent advances in generative AI have led to increased applications of LLMs in education, offering diverse opportunities.LLMs excel predominantly in English and exhibit a bias towards the US context.Automated content tagging has been studied using English‐language content and taxonomies.What this paper adds Investigates the country and language disparities in LLMs concerning knowledge of educational taxonomies and their performance in tagging content.Presents the first empirical findings on addressing disparities in LLM performance across countries and languages within an educational context.Improves GPT‐3.5's tagging accuracy through fine‐tuning, even for non‐US countries, starting from zero accuracy.Extends automated content tagging to non‐English languages using both open‐source and closed‐source LLMs.Implications for practice and/or policy Underscores the importance of considering the performance generalizability of LLMs to languages other than English.Highlights the potential viability of ChatGPT as a skill tagging classifier across countries. [ABSTRACT FROM AUTHOR]

• Artificial Intelligence (AI) is transforming medical writing, raising ethical and quality concerns. • Med-PaLM 2 achieved 86.5 % on the USMLE but had low scientific consensus about adherence. • While AI tools can undoubtedly enhance literature review, statistical analysis, and translation processes, they faces several criticisms. • Threats include increased research waste, low-quality papers, and fraudulent publications. • Regulation and ethical frameworks are essential for managing AI's impact on science. The debate surrounding the integration of artificial intelligence (AI) into scientific writing has already attracted significant interest in medical and life sciences. While AI can undoubtedly expedite the process of manuscript creation and correction, it raises several criticisms. The crossover between AI and health sciences is relatively recent, but the use of AI tools among physicians and other scientists who work in the life sciences is growing very fast. Within this whirlwind, it is becoming essential to realize where we are heading and what the limits are, including an ethical perspective. Modern conversational AIs exhibit a context awareness that enables them to understand and remember any conversation beyond any predefined script. Even more impressively, they can learn and adapt as they engage with a growing volume of human language input. They all share neural networks as background mathematical models and differ from old chatbots for their use of a specific network architecture called transformer model [ 1 ]. Some of them exceed 100 terabytes (TB) (e.g., Bloom, LaMDA) or even 500 TB (e.g., Megatron-Turing NLG) of text data, the 4.0 version of ChatGPT (GPT-4) was trained with nearly 45 TB, but stays updated by the internet connection and may integrate with different plugins that enhance its functionality, making it multimodal. [ABSTRACT FROM AUTHOR]

ChatGPT, an AI software, has rapidly gained popularity for a wide variety of applications, including educational uses. This research explores the use of ChatGPT to supplement Geographical Information Systems (GIS) coursework creation, to help educators with little experience in GIS. Visualizing sea level rise (SLR) and quantifying the extent of inundated areas under varying SLR scenarios is a common GIS lab activity requiring numerous and sequential software steps. We prompted ChatGPT 3.5 to generate its own version of instructions for a lab assignment to produce two deliverables: 1) a map displaying the extent of inundation under three different SLR scenarios and 2) a data table showing the amount of land inundated in each scenario. The initial ChatGPT instructions were inadequate and following them produced incorrect and incomplete results. However, by refining the prompt and increasing data training, we found that ChatGPT could yield better results for GIS assignment creation applications. [ABSTRACT FROM AUTHOR]

Objective: To evaluate the quality of ChatGPT responses to common issues in obstetrics and assess its ability to provide reliable responses to pregnant individuals. The study aimed to examine the responses based on expert opinions using predetermined criteria, including "accuracy," "completeness," and "safety." Methods: We curated 15 common and potentially clinically significant questions that pregnant women are asking. Two native English‐speaking women were asked to reframe the questions in their own words, and we employed the ChatGPT language model to generate responses to the questions. To evaluate the accuracy, completeness, and safety of the ChatGPT's generated responses, we developed a questionnaire with a scale of 1 to 5 that obstetrics and gynecology experts from different countries were invited to rate accordingly. The ratings were analyzed to evaluate the average level of agreement and percentage of positive ratings (≥4) for each criterion. Results: Of the 42 experts invited, 20 responded to the questionnaire. The combined score for all responses yielded a mean rating of 4, with 75% of responses receiving a positive rating (≥4). While examining specific criteria, the ChatGPT responses were better for the accuracy criterion, with a mean rating of 4.2 and 80% of the questions received a positive rating. The responses scored less for the completeness criterion, with a mean rating of 3.8 and 46.7% of questions received a positive rating. For safety, the mean rating was 3.9 and 53.3% of questions received a positive rating. There was no response with an average negative rating below three. Conclusion: This study demonstrates promising results regarding potential use of ChatGPT's in providing accurate responses to obstetric clinical questions posed by pregnant women. However, it is crucial to exercise caution when addressing inquiries concerning the safety of the fetus or the mother. Synopsis: ChatGPT demonstrated the ability to provide accurate and comprehensive responses to common obstetric questions. [ABSTRACT FROM AUTHOR]

Large Language Models (LLMs) are transformer‐based neural networks with billions of parameters trained on very large text corpora from diverse sources. LLMs have the potential to improve healthcare due to their capability to parse complex concepts and generate context‐based responses. The interest in LLMs has not spared digestive disease academics, who have mainly investigated foundational LLM accuracy, which ranges from 25% to 90% and is influenced by the lack of standardized rules to report methodologies and results for LLM‐oriented research. In addition, a critical issue is the absence of a universally accepted definition of accuracy, varying from binary to scalar interpretations, often tied to grader expertise without reference to clinical guidelines. We address strategies and challenges to increase accuracy. In particular, LLMs can be infused with domain knowledge using Retrieval Augmented Generation (RAG) or Supervised Fine‐Tuning (SFT) with reinforcement learning from human feedback (RLHF). RAG faces challenges with in‐context window limits and accurate information retrieval from the provided context. SFT, a deeper adaptation method, is computationally demanding and requires specialized knowledge. LLMs may increase patient quality of care across the field of digestive diseases, where physicians are often engaged in screening, treatment and surveillance for a broad range of pathologies for which in‐context learning or SFT with RLHF could improve clinical decision‐making and patient outcomes. However, despite their potential, the safe deployment of LLMs in healthcare still needs to overcome hurdles in accuracy, suggesting a need for strategies that integrate human feedback with advanced model training. [ABSTRACT FROM AUTHOR]

Background Society guidelines on colorectal dysplasia screening, surveillance, and endoscopic management in inflammatory bowel disease (IBD) are complex, and physician adherence to them is suboptimal. We aimed to evaluate the use of ChatGPT, a large language model, in generating accurate guideline-based recommendations for colorectal dysplasia screening, surveillance, and endoscopic management in IBD in line with European Crohn's and Colitis Organization (ECCO) guidelines. Methods 30 clinical scenarios in the form of free text were prepared and presented to three separate sessions of ChatGPT and to eight gastroenterologists (four IBD specialists and four non-IBD gastroenterologists). Two additional IBD specialists subsequently assessed all responses provided by ChatGPT and the eight gastroenterologists, judging their accuracy according to ECCO guidelines. Results ChatGPT had a mean correct response rate of 87.8%. Among the eight gastroenterologists, the mean correct response rates were 85.8% for IBD experts and 89.2% for non-IBD experts. No statistically significant differences in accuracy were observed between ChatGPT and all gastroenterologists (P =0.95), or between ChatGPT and the IBD experts and non-IBD expert gastroenterologists, respectively (P =0.82). Conclusions This study highlights the potential of language models in enhancing guideline adherence regarding colorectal dysplasia in IBD. Further investigation of additional resources and prospective evaluation in real-world settings are warranted. [ABSTRACT FROM AUTHOR]

Large language models (LLMs) like ChatGPT and Bard have the potential to greatly benefit healthcare, including the field of uveitis care. LLMs can provide accessible advice for patients, helping them make decisions about when to seek medical attention or escalate their treatment. They can also assist in the diagnosis of uveitis, although more research specific to uveitis is needed. LLMs can improve the consultation experience by engaging with patients and providing empathetic responses. Additionally, they can aid in medication management and streamline administrative processes. LLMs have the potential to accelerate research in uveitis by automating tasks such as data analysis and identifying potential drug targets. However, there are ethical considerations to address, including patient autonomy, bias, transparency, and accountability. Efforts should be made to ensure equity and accessibility in the implementation of LLMs, and regulatory frameworks should be developed to ensure responsible use. The future directions of LLMs in uveitis care include integrating them with visual and auditory processing and training them in real-time. While LLMs show promise, more research is needed to develop them with reliability and replicability. [Extracted from the article]

Research on language and cognition relies extensively on psycholinguistic datasets or "norms". These datasets contain judgments of lexical properties like concreteness and age of acquisition, and can be used to norm experimental stimuli, discover empirical relationships in the lexicon, and stress-test computational models. However, collecting human judgments at scale is both time-consuming and expensive. This issue of scale is compounded for multi-dimensional norms and those incorporating context. The current work asks whether large language models (LLMs) can be leveraged to augment the creation of large, psycholinguistic datasets in English. I use GPT-4 to collect multiple kinds of semantic judgments (e.g., word similarity, contextualized sensorimotor associations, iconicity) for English words and compare these judgments against the human "gold standard". For each dataset, I find that GPT-4's judgments are positively correlated with human judgments, in some cases rivaling or even exceeding the average inter-annotator agreement displayed by humans. I then identify several ways in which LLM-generated norms differ from human-generated norms systematically. I also perform several "substitution analyses", which demonstrate that replacing human-generated norms with LLM-generated norms in a statistical model does not change the sign of parameter estimates (though in select cases, there are significant changes to their magnitude). I conclude by discussing the considerations and limitations associated with LLM-generated norms in general, including concerns of data contamination, the choice of LLM, external validity, construct validity, and data quality. Additionally, all of GPT-4's judgments (over 30,000 in total) are made available online for further analysis. [ABSTRACT FROM AUTHOR]

With the rapid advancement of Internet technology and the increasing volume of police reports, relying solely on extensive human labor and traditional natural language processing methods for key element extraction has become impractical. Applying advanced technologies such as large language models to improve the effectiveness of police report extraction has become an inevitable trend in the field of police data analysis. This study addresses the characteristics of Chinese police reports and the need to extract key elements by employing large language models specific to the public security domain for entity extraction. Several lightweight (6/7b) open-source large language models were tested as base models. To enhance model performance, LoRA fine-tuning was employed, combined with data engineering approaches. A zero-shot data augmentation method based on ChatGPT and prompt engineering techniques tailored for police reports were proposed to further improve model performance. The key police report data from a certain city in 2019 were used as a sample for testing. Compared to the base models, prompt engineering improved the F1 score by approximately 3%, while fine-tuning led to an increase of 10–50% in the F1 score. After fine-tuning and comparing different base models, the Baichuan model demonstrated the best overall performance in extracting key elements from police reports. Using the data augmentation method to double the data size resulted in an additional 4% increase in the F1 score, achieving optimal model performance. Compared to the fine-tuned universal information extraction (UIE) large language model, the police report entity extraction model constructed in this study improved the F1 score for each element by approximately 5%, with a 42% improvement in the F1 score for the "organization" element. Finally, ChatGPT was employed to align the extracted entities, resulting in a high-quality entity extraction outcome. [ABSTRACT FROM AUTHOR]