Search

Backgrounds: The use of large language models (LLMs) in medicine can help physicians improve the quality and effectiveness of health care by increasing the efficiency of medical information management, patient care, medical research, and clinical decision-making. Methods: We collected 34 frequently asked questions about glucocorticoid-induced osteoporosis (GIOP), covering topics related to the disease's clinical manifestations, pathogenesis, diagnosis, treatment, prevention, and risk factors. We also generated 25 questions based on the 2022 American College of Rheumatology Guideline for the Prevention and Treatment of Glucocorticoid-Induced Osteoporosis (2022 ACR-GIOP Guideline). Each question was posed to the LLM (ChatGPT-3.5, ChatGPT-4, and Google Gemini), and three senior orthopedic surgeons independently rated the responses generated by the LLMs. Three senior orthopedic surgeons independently rated the answers based on responses ranging between 1 and 4 points. A total score (TS) > 9 indicated 'good' responses, 6 ≤ TS ≤ 9 indicated 'moderate' responses, and TS < 6 indicated 'poor' responses. Results: In response to the general questions related to GIOP and the 2022 ACR-GIOP Guidelines, Google Gemini provided more concise answers than the other LLMs. In terms of pathogenesis, ChatGPT-4 had significantly higher total scores (TSs) than ChatGPT-3.5. The TSs for answering questions related to the 2022 ACR-GIOP Guideline by ChatGPT-4 were significantly higher than those for Google Gemini. ChatGPT-3.5 and ChatGPT-4 had significantly higher self-corrected TSs than pre-corrected TSs, while Google Gemini self-corrected for responses that were not significantly different than before. Conclusions: Our study showed that Google Gemini provides more concise and intuitive responses than ChatGPT-3.5 and ChatGPT-4. ChatGPT-4 performed significantly better than ChatGPT3.5 and Google Gemini in terms of answering general questions about GIOP and the 2022 ACR-GIOP Guidelines. ChatGPT3.5 and ChatGPT-4 self-corrected better than Google Gemini. [ABSTRACT FROM AUTHOR]

Purpose: ChatGPT is a generative artificial intelligence (AI) technology that can solve multiple complex tasks. ChatGPT-4 can facilitate educational empowerment in China through technology to understand and generate Chinese text. Although ChatGPT's benefits have been widely discussed, its educational capabilities have not been systematically assessed. This study provides evidence of and insights into the educational applications of AI tools in China. Design/Approach/Methods: This study uses various tests to systematically assess the latest iteration of the AI chatbot ChatGPT-4, including the Watson-Glaser Critical Thinking Appraisal (WGCTA), Five Core Competencies Questionnaire, and written test of China's 2022 National Teacher Certificate Examination (NTCE). Findings: The WGCTA results suggest that ChatGPT requires strong critical thinking. Compared with the other four competencies, the tool showed a lower aptitude for creativity. Regarding its educational applications, ChatGPT performed well on the 2022 NTCE written test. As technology enhances, ChatGPT and similar AI tools have potential applications in China for lesson planning, student self-learning, classroom interaction, and checking assignments. Originality/Value: This study systematically tested ChatGPT at a logical level and assessed its core competencies and educational applications. The study innovatively used 2022 NTCE data to test ChatGPT, with results providing support for the application of generative AI in future curricula and instruction in China. [ABSTRACT FROM AUTHOR]

Background: Academic paper writing holds significant importance in the education of medical students, and poses a clear challenge for those whose first language is not English. This study aims to investigate the effectiveness of employing large language models, particularly ChatGPT, in improving the English academic writing skills of these students. Methods: A cohort of 25 third-year medical students from China was recruited. The study consisted of two stages. Firstly, the students were asked to write a mini paper. Secondly, the students were asked to revise the mini paper using ChatGPT within two weeks. The evaluation of the mini papers focused on three key dimensions, including structure, logic, and language. The evaluation method incorporated both manual scoring and AI scoring utilizing the ChatGPT-3.5 and ChatGPT-4 models. Additionally, we employed a questionnaire to gather feedback on students' experience in using ChatGPT. Results: After implementing ChatGPT for writing assistance, there was a notable increase in manual scoring by 4.23 points. Similarly, AI scoring based on the ChatGPT-3.5 model showed an increase of 4.82 points, while the ChatGPT-4 model showed an increase of 3.84 points. These results highlight the potential of large language models in supporting academic writing. Statistical analysis revealed no significant difference between manual scoring and ChatGPT-4 scoring, indicating the potential of ChatGPT-4 to assist teachers in the grading process. Feedback from the questionnaire indicated a generally positive response from students, with 92% acknowledging an improvement in the quality of their writing, 84% noting advancements in their language skills, and 76% recognizing the contribution of ChatGPT in supporting academic research. Conclusion: The study highlighted the efficacy of large language models like ChatGPT in augmenting the English academic writing proficiency of non-native speakers in medical education. Furthermore, it illustrated the potential of these models to make a contribution to the educational evaluation process, particularly in environments where English is not the primary language. [ABSTRACT FROM AUTHOR]

Generative artificial intelligence (GenAI) tools such as ChatGPT with their human‐like intelligence and language processing capabilities are significantly impacting the way we live, work, and communicate with each other. While scholars have increasingly focused on the use of GenAI in higher education since its inception, little is known about how key higher education stakeholders, particularly students, perceive its impact on teaching and learning within the context of academic communication, an area central to students' development of transferable skills and literacy competencies yet heavily influenced by the technology. This empirical study addresses the gap by investigating students' experiences and attitudes toward GenAI tools for English academic communication, focusing on their overall perceptions, perceived benefits, limitations, and challenges. Drawing on data from a questionnaire survey with 475 students and interviews with 12 at two universities in China, our findings indicate that students generally view GenAI positively, considering them useful for learning academic communication skills, particularly in writing, grammar, vocabulary, and reading. However, limitations are recognized in terms of giving feedback on critical thinking, creativity, and speaking skills. In addition, information reliability, ethical issues, and impact on assessment and academic integrity also emerged as important concerns. Our study argues that universities should embrace and capitalize on the affordances of GenAI and address its challenges to better support students' learning of critical academic literacy. [ABSTRACT FROM AUTHOR]

Recently, the prevalence of information technology represented by ChatGPT has aroused extensive discussions. Does the adoption of information technology lead to massive regional unemployment? It is a crucial and unresolved issue for the government, enterprise employees, and researchers. Combining the 2005–2020 panel and the smart city (SC) construction policy in China, this article employs the difference in differences (DID) method to study the impact of informatization construction on labour spatial allocation. We found that informatization construction significantly attracted labour and improved labour spatial allocation. After the SC construction policy's implementation, compared with non-SCs, the average increase of SC pilots in the labour force is about 0.78 million people. Besides, stimulating economic growth, improving the environment, and enhancing public services are the mechanisms of the SC construction policy on labour spatial allocation. Furthermore, this policy effect has heterogeneous industry department and category characteristics. The Tertiary and secondary industry department receives a more significant impact. [ABSTRACT FROM AUTHOR]

Successful applications of artificial intelligence (AI), such as ChatGPT, have been prompting regulators to speed up the related regulation processes. China and the EU have been particularly ambitious in this regard. The EU AI Act has been swiftly progressing through the institutions and is expected to be officially adopted in spring 2024. This article argues that its overall approach is wrong, that it extends EU regulation into policy areas which come under national competences and that it will hurt European AI innovation in particular and society in general. Instead of regulating AI per se, the EU or the member states should regulate the use of AI in specific sectors or, better still, regulate it in technologically independent ways—by specifying what is allowed or prohibited, regardless of the technology used. [ABSTRACT FROM AUTHOR]

Nowadays, the prevalence of ChatGPT and other Large Language Models (LLMs) has posed significant challenges into the education field, particularly in English education. In response, this study aimed to investigate the beliefs of 95 EFL university teachers from Chinese universities regarding the integration of LLMs in language education, as well as the relationships between their beliefs and other factors. The study yielded several findings: (1) According to the quantitative and qualitative results, we revealed several concerns among Chinese EFL university teachers regarding LLMs integration, such as neglection of traditional learning resources, academic integrity, and excessive reliance. (2) Previous experiences with LLMs, frequency of LLMs use, and self-evaluation on stages of LLMs integration all played vital roles in shaping university teachers' beliefs in integrating LLMs in language education. (3) No significant correlation was observed between university teachers' beliefs in integrating LLMs in language education and the availability of IT personnel. (4) No significant correlation was observed between university teachers' beliefs in integrating LLMs in language education their evaluation on IT infrastructure. This research has provided some insights into university teachers' beliefs in ChatGPT and other LLMs to promote effective policies and strategies in the digital era. [ABSTRACT FROM AUTHOR]

The collection of data from websites at great scale - so-called data scraping - is the foundation for ChatGPT and most other Generative AI (GenAI) tools. Much of the previous discussion on the regulation of GenAI has focused on the US and EU and not so much on more technical aspects like data scraping. In response, this article focuses on the regulation of data scraping to build and deploy GenAI in China, and reviews applicable regulation and case law. We find that the sectoral approach to AI regulation in China provides important insights into balancing technological progress and societal values, diverging from the laissez-faire attitude in the US and the horizontal approach with the AI Act in the EU. [ABSTRACT FROM AUTHOR]

Background: Large language models like ChatGPT have revolutionized the field of natural language processing with their capability to comprehend and generate textual content, showing great potential to play a role in medical education. This study aimed to quantitatively evaluate and comprehensively analysis the performance of ChatGPT on three types of national medical examinations in China, including National Medical Licensing Examination (NMLE), National Pharmacist Licensing Examination (NPLE), and National Nurse Licensing Examination (NNLE). Methods: We collected questions from Chinese NMLE, NPLE and NNLE from year 2017 to 2021. In NMLE and NPLE, each exam consists of 4 units, while in NNLE, each exam consists of 2 units. The questions with figures, tables or chemical structure were manually identified and excluded by clinician. We applied direct instruction strategy via multiple prompts to force ChatGPT to generate the clear answer with the capability to distinguish between single-choice and multiple-choice questions. Results: ChatGPT failed to pass the accuracy threshold of 0.6 in any of the three types of examinations over the five years. Specifically, in the NMLE, the highest recorded accuracy was 0.5467, which was attained in both 2018 and 2021. In the NPLE, the highest accuracy was 0.5599 in 2017. In the NNLE, the most impressive result was shown in 2017, with an accuracy of 0.5897, which is also the highest accuracy in our entire evaluation. ChatGPT's performance showed no significant difference in different units, but significant difference in different question types. ChatGPT performed well in a range of subject areas, including clinical epidemiology, human parasitology, and dermatology, as well as in various medical topics such as molecules, health management and prevention, diagnosis and screening. Conclusions: These results indicate ChatGPT failed the NMLE, NPLE and NNLE in China, spanning from year 2017 to 2021. but show great potential of large language models in medical education. In the future high-quality medical data will be required to improve the performance. [ABSTRACT FROM AUTHOR]

The inaccessibility of ChatGPT to Chinese users has fueled public conspiracy theories surrounding the technology. We conducted a critical discourse analysis of 1,576 relevant Weibo posts to identify these conspiracy theories and the sociocultural and political factors at play. Our findings reveal four major themes, including a profound distrust of foreign high technology, nationalist fervor, disconcerting allegations of AI development, and sensational assertions of government manipulation of the fertility rate. More importantly, we observe that the escalation of nationalist sentiments in the online sphere exacerbates the spread of conspiracy theories, reflecting public concerns about the country's technological progress and global reputation. [ABSTRACT FROM AUTHOR]

Since its origin in the 1950s, artificial intelligence (AI) has evolved from technological to educational applications. AI is emerging as an essential tool in education. Its integration into education promises the personalization and the globalization of learning. Despite its potential, it is crucial to consider its ethical challenges and uses. This bibliometric study sought to understand the current state of AI in higher education in order to provide a basis for future research. A bibliometric analysis was conducted between 2017 and 2023, using the Scopus database. The query was performed on 23 October 2023 and focused on titles, keywords, and abstracts. A total of 870 articles were found, and their metadata were analyzed after removing incorrect data. VOSviewer software was used to visualize the similarities, and the publications were studied by country, authors, and collaborations. A steady growth in AI studies in higher education was found, highlighting areas such as computer science and social sciences. China and the United States led in production and citations. Keywords such as "artificial intelligence", "chatgpt", and "machine learning" indicated trends and areas of interest. [ABSTRACT FROM AUTHOR]

While artificial intelligence-based chatbots have demonstrated great potential for writing, little is known about whether and how doctoral students accept the use of ChatGPT in writing. Framed with Technology Acceptance Model, this study investigated doctoral students' acceptance toward ChatGPT in writing and the factors that influence it. The questionnaire survey revealed a high intention to use ChatGPT in writing among doctoral students in China. The findings further indicated that attitude was a significant predictor of behavioural intention to use ChatGPT in writing and mediated the impacts of perceived usefulness and perceived ease of use on it. Perceived ease of ChatGPT use was in turn influenced by students' past ChatGPT use experience. This study provides powerful evidence for the applicability of Technology Acceptance Model in the acceptance of ChatGPT in writing. The results have significant implications for leveraging ChatGPT for writing in higher education. [ABSTRACT FROM AUTHOR]

ChatGPT, a general-purpose intelligent chatbot developed by OpenAI, has introduced numerous opportunities and challenges in the field of language education. With its remarkable ability to generate diverse forms of text, answer questions, and provide translations within minutes, ChatGPT has become an influential tool in the era of advanced AI technology. However, to what extent ChatGPT can be used to assist students in completing language learning tasks remains largely unexplored. Against this background, this study aimed to investigate students' experiences with ChatGPT and their perceptions of its role in language learning through a small-scale qualitative study. The data were collected through semi-structured interviews with five students at a top-tier international university in China. Students' responses revealed that ChatGPT has the potential to serve as a valuable learning partner and aid students in completing language-related tasks. Furthermore, participants exhibited critical judgment in evaluating the quality of ideas and outputs generated by ChatGPT, as well as the ability to modify prompts to maximize learning benefits. Such critical judgment offsets the potential threats to academic integrity posed by ChatGPT. Our findings contribute to the understanding of the potential of ChatGPT in language education by adding empirical evidence from students' perspectives. This study supports the idea that ChatGPT can work as an effective tool for providing students with immediate feedback and personalized learning experiences. Such findings generate implications for future pedagogical practices in the new era by providing students with personalized guidance, designing technology-embedded language support, and developing students' lifelong learning skills (e.g., autonomy and evaluative judgment) with the support of ChatGPT. [ABSTRACT FROM AUTHOR]

ChatGPT, a language model developed by OpenAI, uses a 175 billion parameter Transformer architecture for natural language processing tasks. This study aimed to compare the knowledge and interpretation ability of ChatGPT with those of medical students in China by administering the Chinese National Medical Licensing Examination (NMLE) to both ChatGPT and medical students. We evaluated the performance of ChatGPT in three years' worth of the NMLE, which consists of four units. At the same time, the exam results were compared to those of medical students who had studied for five years at medical colleges. ChatGPT's performance was lower than that of the medical students, and ChatGPT's correct answer rate was related to the year in which the exam questions were released. ChatGPT's knowledge and interpretation ability for the NMLE were not yet comparable to those of medical students in China. It is probable that these abilities will improve through deep learning. [ABSTRACT FROM AUTHOR]

Research Aims. This research project investigates what are the major benefits and risks of Chinese students using ChatGPT for academic activities. Also, the project assesses how, if applicable, should ChatGPT be regulated in Chinese academic settings in order to maintain academic integrity and ethical standards. Methodology. The collection of primary data from relevant newspaper articles serves as the foundation of this research project. Here, the content analysis is used for primary data collection. A combination of keywords ["ChatGPT" AND ("China" OR "Chinese") AND ("students" OR "student")] were typed on the Google news search engine on 12 April 2023. A sum of 40 newspaper articles were deemed eligible for data analysis. Both qualitative and quantitative data were extracted and analyzed. Findings. The opinion of using ChatGPT to fulfil academic responsibilities has been polarized in China. The conservative camps worry that students are using ChatGPT to commit academic cheating. However, some Chinese educators believe AI-powered technologies should be incorporated into academic learning as AI-enabled writing tools can help improve the quality of academic outputs. A major concern that Chinese educators hold, to date, is plagiarism violations by students as an act of academic cheating. Most newspaper articles presented the use of ChatGPT in Chinese academic settings neutrally. Newspaper articles published in March 2023 contained more positive and negative word(s) about the use of ChatGPT in academic learning. Conclusions. Given the benefits ChatGPT can provide and the near-infeasibility of massively ban the use of AI-powered software, more regulations should be set up in Chinese academia. Teachers have to guide students on how to fact-check the details provided by AI and add citations and references accordingly in their coursework. Moreover, teachers should guide students on how to ask AI-powered software questions systematically and creatively, in order to maximize the intellectual outputs generated from ChatGPT. [ABSTRACT FROM AUTHOR]

The advent of transformative technologies, such as OpenAI's ChatGPT, a natural language processing tool, significantly impacts the future of education, society, and the economy. Drawing on the theory of knowledge socialism and public goods, this paper critically examines the potential impact of ChatGPT in the context of China's high-tech development and higher education. We believe these technologies resonate with the principles of knowledge socialism by promoting decentralization, openness, and collective intelligence and are poised to reshape the education sector and beyond. The non-competitive and non-exclusive nature of ChatGPT makes it a public good with significant social benefits. In the educational domain, it has the potential to revolutionize teaching and learning by providing students with convenient, personalized learning experiences, real-time feedback, and multilingual support. However, despite its benefits, educators, and users must work to overcome its limitations, such as limited expertise, language understanding, reasoning skills, and potential risks associated with privacy, security, and misinformation. This article discusses the need to prepare students for a future heavily impacted by AI, which requires a reassessment of digital literacy and competencies. Doing so would require leveraging AI tools and promoting human-centered approaches, developing higher-order thinking skills, emphasizing emotional education, and teaching academic ethics. In addition, introducing AI tools like ChatGPT in higher education requires regulation to prevent plagiarism and improve independent problem-solving. This paper further explores the impact of ChatGPT on China's technological development. The global attention attracted by ChatGPT has stimulated the growth of the AI industry and prompted the emergence of domestic products and applications in countries such as China. It is critical to harmonize the enforcement of competition laws and capitalize on the contributions of sizable enterprises in the realm of technological innovation and research. [ABSTRACT FROM AUTHOR]

In 2023, the capabilities of text communication, language translation, image generation, and code writing demonstrated by ChatGPT and GPT-4 have received widespread attention in and out of China. With years of data and technology accumulation, many Chinese companies and research teams continue to make efforts in the field of large models, and their models cover many industries and have a number of characteristic functions. This paper will introduce the type and development trends of large models, and sort out the methods and characteristics of domestic large models. Finally, we will explain the advantages and disadvantages of the domestic models, and analyze the application prospect and challenges of them in the medical field. [ABSTRACT FROM AUTHOR]

Objective: To investigate the level of understanding and trust of medical students towards ChatGPT-like large language models, as well as their utilization and attitudes towards these models., Methods: Data collection was concentrated from December 2023 to mid-January 2024, utilizing a self-designed questionnaire to assess the use of large language models among undergraduate medical students at Anhui Medical University. The normality of the data was confirmed with Shapiro-Wilk tests. We used Chi-square tests for comparisons of categorical variables, Mann-Whitney U tests for comparisons of ordinal variables and non-normal continuous variables between two groups, Kruskall-Wallis H tests for comparisons of ordinal variables between multiple groups, and Bonferroni tests for post hoc comparisons., Results: A total of 1774 questionnaires were distributed and 1718 valid questionnaires were collected, with an effective rate of 96.84%. Among these students, 34.5% had heard and used large language models. There were statistically significant differences in the understanding of large language models between genders (p < 0.001), grade levels (junior-level students and senior-level students) (p = 0.03), and major (p < 0.001). Male, junior-level students, and public health management had a higher level of understanding of these models. Genders and majors had statistically significant effects on the degree of trust in large language models (p = 0.004; p = 0.02). Male and nursing students exhibited a higher degree of trust in large language models. As for usage, Male and junior-level students showed a significantly higher proportion of using these models for assisted learning (p < 0.001). Neutral sentiments were held by over two-thirds of the students (66.7%) regarding large language models, with only 51(3.0%) expressing pessimism. There were significant gender-based disparities in attitudes towards large language models, and male exhibited a more optimistic attitude towards these models (p < 0.001). Notably, among students with different levels of knowledge and trust in large language models, statistically significant differences were observed in their perceptions of the shortcomings and benefits of these models., Conclusion: Our study identified gender, grade levels, and major as influential factors in students' understanding and utilization of large language models. This also suggested the feasibility of integrating large language models with traditional medical education to further enhance teaching effectiveness in the future., (© 2024. The Author(s).)

Background: With the increasing application of large language models like ChatGPT in various industries, its potential in the medical domain, especially in standardized examinations, has become a focal point of research., Objective: The aim of this study is to assess the clinical performance of ChatGPT, focusing on its accuracy and reliability in the Chinese National Medical Licensing Examination (CNMLE)., Methods: The CNMLE 2022 question set, consisting of 500 single-answer multiple choices questions, were reclassified into 15 medical subspecialties. Each question was tested 8 to 12 times in Chinese on the OpenAI platform from April 24 to May 15, 2023. Three key factors were considered: the version of GPT-3.5 and 4.0, the prompt's designation of system roles tailored to medical subspecialties, and repetition for coherence. A passing accuracy threshold was established as 60%. The χ2 tests and κ values were employed to evaluate the model's accuracy and consistency., Results: GPT-4.0 achieved a passing accuracy of 72.7%, which was significantly higher than that of GPT-3.5 (54%; P<.001). The variability rate of repeated responses from GPT-4.0 was lower than that of GPT-3.5 (9% vs 19.5%; P<.001). However, both models showed relatively good response coherence, with κ values of 0.778 and 0.610, respectively. System roles numerically increased accuracy for both GPT-4.0 (0.3%-3.7%) and GPT-3.5 (1.3%-4.5%), and reduced variability by 1.7% and 1.8%, respectively (P>.05). In subgroup analysis, ChatGPT achieved comparable accuracy among different question types (P>.05). GPT-4.0 surpassed the accuracy threshold in 14 of 15 subspecialties, while GPT-3.5 did so in 7 of 15 on the first response., Conclusions: GPT-4.0 passed the CNMLE and outperformed GPT-3.5 in key areas such as accuracy, consistency, and medical subspecialty expertise. Adding a system role insignificantly enhanced the model's reliability and answer coherence. GPT-4.0 showed promising potential in medical education and clinical practice, meriting further study., (© Shuai Ming, Qingge Guo, Wenjun Cheng, Bo Lei. Originally published in JMIR Medical Education (https://mededu.jmir.org).)

Can ChatGPT-like AI Function as ODR Fourth Party for Handling School-Related Disputes in China?: ChatGPT-like AI is a powerful large language model (LLM) for generating human-like text but cannot replace the third party of online dispute resolution (ODR). A Colombian judge used ChatGPT as an 'assistant' to help in making a court decision, proving that it cannot play the role of the independent third party. However, ChatGPT-like AI can work as the fourth party and is suitable for preventing and resolving school-related disputes in China. The tests of Open AI ChatGPT, ChatSonic and the Chinese version of ChatGPT prove ChatGPT-like AI has the potential to play the role of the fourth party. It is feasible to customize ChatGPT-like AI as the fourth party to handle school-related disputes in China, thereby exploring the new frontier of ODR. In a word, ChatGPT-like AI enhances the role of the fourth party. [ABSTRACT FROM AUTHOR]

The aim of this study was to assess nurses' awareness and use of ChatGPT. The study was conducted in October 2023 with an online questionnaire for 124 nurses in the nursing education programme at West China Hospital. The questionnaire included participants' demographic information, awareness of ChatGPT, and actual experience of using it. A total of 57.3% (71/124) of the nurses completed the survey. Of these, 56.3% (40/71) were aware of ChatGPT and 43.7% (31/71) were not aware of ChatGPT. In terms of use, of the 20 who used ChatGPT, 13 used it for studying, 10 for essay writing, five for research and two for chatting. This study highlights the potential of ChatGPT to improve nurses' professional competence and effectiveness. Further research will focus on how ChatGPT can be used more effectively to support nurses' professional development and growth.

Background: In the era of the internet, individuals have increasingly accustomed themselves to gathering necessary information and expressing their opinions on public web-based platforms. The health care sector is no exception, as these comments, to a certain extent, influence people's health care decisions. During the onset of the COVID-19 pandemic, how the medical experience of Chinese patients and their evaluations of hospitals have changed remains to be studied. Therefore, we plan to collect patient medical visit data from the internet to reflect the current status of medical relationships under specific circumstances., Objective: This study aims to explore the differences in patient comments across various stages (during, before, and after) of the COVID-19 pandemic, as well as among different types of hospitals (children's hospitals, maternity hospitals, and tumor hospitals). Additionally, by leveraging ChatGPT (OpenAI), the study categorizes the elements of negative hospital evaluations. An analysis is conducted on the acquired data, and potential solutions that could improve patient satisfaction are proposed. This study is intended to assist hospital managers in providing a better experience for patients who are seeking care amid an emergent public health crisis., Methods: Selecting the top 50 comprehensive hospitals nationwide and the top specialized hospitals (children's hospitals, tumor hospitals, and maternity hospitals), we collected patient reviews from these hospitals on the Dianping website. Using ChatGPT, we classified the content of negative reviews. Additionally, we conducted statistical analysis using SPSS (IBM Corp) to examine the scoring and composition of negative evaluations., Results: A total of 30,317 pieces of effective comment information were collected from January 1, 2018, to August 15, 2023, including 7696 pieces of negative comment information. Manual inspection results indicated that ChatGPT had an accuracy rate of 92.05%. The F1-score was 0.914. The analysis of this data revealed a significant correlation between the comments and ratings received by hospitals during the pandemic. Overall, there was a significant increase in average comment scores during the outbreak (P<.001). Furthermore, there were notable differences in the composition of negative comments among different types of hospitals (P<.001). Children's hospitals received sensitive feedback regarding waiting times and treatment effectiveness, while patients at maternity hospitals showed a greater concern for the attitude of health care providers. Patients at tumor hospitals expressed a desire for timely examinations and treatments, especially during the pandemic period., Conclusions: The COVID-19 pandemic had some association with patient comment scores. There were variations in the scores and content of comments among different types of specialized hospitals. Using ChatGPT to analyze patient comment content represents an innovative approach for statistically assessing factors contributing to patient dissatisfaction. The findings of this study could provide valuable insights for hospital administrators to foster more harmonious physician-patient relationships and enhance hospital performance during public health emergencies., (©Weixue Huo, Mengwei He, Zhaoxiang Zeng, Xianhao Bao, Ye Lu, Wen Tian, Jiaxuan Feng, Rui Feng. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 02.07.2024.)

This study is aimed at examining the impact of ChatGPT on pediatric endocrine and metabolic conditions, particularly in the areas of screening and diagnosis, in both Chinese and English modes. A 40-question questionnaire covering the four most common pediatric endocrine and metabolic conditions was posed to ChatGPT in both Chinese and English three times each. Six pediatric endocrinologists evaluated the responses. ChatGPT performed better when responding to questions in English, with an unreliable rate of 7.5% compared to 27.5% for Chinese questions, indicating a more consistent response pattern in English. Among the reliable questions, the answers were more comprehensive and satisfactory in the English mode. We also found disparities in ChatGPT's performance when interacting with different target groups and diseases, with improved performance for questions posed by clinicians in English and better performance for questions related to diabetes and overweight/obesity in Chinese for both clinicians and patients. Language comprehension, providing incomprehensive answers, and errors in key data were the main contributors to the low scores, according to reviewer feedback., Conclusion: Despite these limitations, as ChatGPT continues to evolve and expand its network, it has significant potential as a practical and effective tool for clinical diagnosis and treatment., What Is Known: • The deep learning-based large-language model ChatGPT holds great promise for improving clinical practice for both physicians and patients and has the potential to increase the speed and accuracy of disease screening and diagnosis, as well as enhance the overall efficiency of the medical process. However, the reliability and appropriateness of AI model responses in specific field remains unclear. • This study focused on the reliability and appropriateness of AI model responses to straightforward and fundamental questions related to the four most prevalent pediatric endocrine and metabolic disorders, for both healthcare providers and patients, in different language scenarios., What Is New: • The AI model performed better when responding to questions in English, with more consistent, as well as more comprehensive and satisfactory responses. In addition, we also found disparities in ChatGPT's performance when interacting with different target groups and different diseases. • Despite these limitations, as ChatGPT continues to evolve and expand its network, it has significant potential as a practical and effective tool for clinical diagnosis and treatment., (© 2024. The Author(s).)

• People with epilepsy strongly desire to access information related to epilepsy, but lack reliable and easily accessible sources. • ChatGPT provide accurate and consistent answers to common concerns in patients withepilepsy, helping them better understand their condition. • ChatGPT may generate incorrect answers, leading users to adopt incorrect and potentially harmful advice. • ChatGPT is suitable for patient education and should be used as an information channel rather than a basis for decision-making at this stage. • ChatGPT can offer emotional support that may benefit for patients experiencing specific emotional distress. People with epilepsy desire to acquire accurate information about epilepsy and actively engage in its management throughout the long journey of living with seizures. ChatGPT is a large language model and we aimed to assess the accuracy and consistency of ChatGPT in responding to the common concerns of people with epilepsy and to evaluate its ability to provide emotional support. Questions were collected from the International League against Epilepsy and the China Association against Epilepsy. The responses were independently assessed by two board-certified epileptologists from the China Association against Epilepsy, and a third reviewer resolved disagreements. The reviewers assessed its ability to provide emotional support subjectively. A total of 378 questions related to epilepsy and 5 questions related to emotional support were included. ChatGPT provided "correct and comprehensive" answers to 68.4% of the questions. The model provided reproducible answers for 82.3% questions. The model performed poorly in answering prognostic questions, with only 46.8% of the answers rated as comprehensive. When faced with questions requiring emotional support, the model can generate natural and understandable responses. ChatGPT provides accurate and reliable answers to patients with epilepsy and is a valuable source of information. It also provides partial emotional support, potentially assisting those experiencing emotional distress. However, ChatGPT may provide incorrect responses, leading users to inadvertently accept incorrect and potentially dangerous advice. Therefore, the direct use of ChatGPT for medical guidance is not recommended and its primary use at present is in patients education. [ABSTRACT FROM AUTHOR]

Background: There is a dearth of feasibility assessments regarding using large language models (LLMs) for responding to inquiries from autistic patients within a Chinese-language context. Despite Chinese being one of the most widely spoken languages globally, the predominant research focus on applying these models in the medical field has been on English-speaking populations., Objective: This study aims to assess the effectiveness of LLM chatbots, specifically ChatGPT-4 (OpenAI) and ERNIE Bot (version 2.2.3; Baidu, Inc), one of the most advanced LLMs in China, in addressing inquiries from autistic individuals in a Chinese setting., Methods: For this study, we gathered data from DXY-a widely acknowledged, web-based, medical consultation platform in China with a user base of over 100 million individuals. A total of 100 patient consultation samples were rigorously selected from January 2018 to August 2023, amounting to 239 questions extracted from publicly available autism-related documents on the platform. To maintain objectivity, both the original questions and responses were anonymized and randomized. An evaluation team of 3 chief physicians assessed the responses across 4 dimensions: relevance, accuracy, usefulness, and empathy. The team completed 717 evaluations. The team initially identified the best response and then used a Likert scale with 5 response categories to gauge the responses, each representing a distinct level of quality. Finally, we compared the responses collected from different sources., Results: Among the 717 evaluations conducted, 46.86% (95% CI 43.21%-50.51%) of assessors displayed varying preferences for responses from physicians, with 34.87% (95% CI 31.38%-38.36%) of assessors favoring ChatGPT and 18.27% (95% CI 15.44%-21.10%) of assessors favoring ERNIE Bot. The average relevance scores for physicians, ChatGPT, and ERNIE Bot were 3.75 (95% CI 3.69-3.82), 3.69 (95% CI 3.63-3.74), and 3.41 (95% CI 3.35-3.46), respectively. Physicians (3.66, 95% CI 3.60-3.73) and ChatGPT (3.73, 95% CI 3.69-3.77) demonstrated higher accuracy ratings compared to ERNIE Bot (3.52, 95% CI 3.47-3.57). In terms of usefulness scores, physicians (3.54, 95% CI 3.47-3.62) received higher ratings than ChatGPT (3.40, 95% CI 3.34-3.47) and ERNIE Bot (3.05, 95% CI 2.99-3.12). Finally, concerning the empathy dimension, ChatGPT (3.64, 95% CI 3.57-3.71) outperformed physicians (3.13, 95% CI 3.04-3.21) and ERNIE Bot (3.11, 95% CI 3.04-3.18)., Conclusions: In this cross-sectional study, physicians' responses exhibited superiority in the present Chinese-language context. Nonetheless, LLMs can provide valuable medical guidance to autistic patients and may even surpass physicians in demonstrating empathy. However, it is crucial to acknowledge that further optimization and research are imperative prerequisites before the effective integration of LLMs in clinical settings across diverse linguistic environments can be realized., Trial Registration: Chinese Clinical Trial Registry ChiCTR2300074655; https://www.chictr.org.cn/bin/project/edit?pid=199432., (©Wenjie He, Wenyan Zhang, Ya Jin, Qiang Zhou, Huadan Zhang, Qing Xia. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.04.2024.)

Background: The widespread use of electronic health records in the clinical and biomedical fields makes the removal of protected health information (PHI) essential to maintain privacy. However, a significant portion of information is recorded in unstructured textual forms, posing a challenge for deidentification. In multilingual countries, medical records could be written in a mixture of more than one language, referred to as code mixing. Most current clinical natural language processing techniques are designed for monolingual text, and there is a need to address the deidentification of code-mixed text., Objective: The aim of this study was to investigate the effectiveness and underlying mechanism of fine-tuned pretrained language models (PLMs) in identifying PHI in the code-mixed context. Additionally, we aimed to evaluate the potential of prompting large language models (LLMs) for recognizing PHI in a zero-shot manner., Methods: We compiled the first clinical code-mixed deidentification data set consisting of text written in Chinese and English. We explored the effectiveness of fine-tuned PLMs for recognizing PHI in code-mixed content, with a focus on whether PLMs exploit naming regularity and mention coverage to achieve superior performance, by probing the developed models' outputs to examine their decision-making process. Furthermore, we investigated the potential of prompt-based in-context learning of LLMs for recognizing PHI in code-mixed text., Results: The developed methods were evaluated on a code-mixed deidentification corpus of 1700 discharge summaries. We observed that different PHI types had preferences in their occurrences within the different types of language-mixed sentences, and PLMs could effectively recognize PHI by exploiting the learned name regularity. However, the models may exhibit suboptimal results when regularity is weak or mentions contain unknown words that the representations cannot generate well. We also found that the availability of code-mixed training instances is essential for the model's performance. Furthermore, the LLM-based deidentification method was a feasible and appealing approach that can be controlled and enhanced through natural language prompts., Conclusions: The study contributes to understanding the underlying mechanism of PLMs in addressing the deidentification process in the code-mixed context and highlights the significance of incorporating code-mixed training instances into the model training phase. To support the advancement of research, we created a manipulated subset of the resynthesized data set available for research purposes. Based on the compiled data set, we found that the LLM-based deidentification method is a feasible approach, but carefully crafted prompts are essential to avoid unwanted output. However, the use of such methods in the hospital setting requires careful consideration of data security and privacy concerns. Further research could explore the augmentation of PLMs and LLMs with external knowledge to improve their strength in recognizing rare PHI., (©You-Qian Lee, Ching-Tai Chen, Chien-Chang Chen, Chung-Hong Lee, Peitsz Chen, Chi-Shin Wu, Hong-Jie Dai. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 25.01.2024.)

Background: The National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA) has amassed a vast reservoir of genetic data since its inception in 2007. These public data hold immense potential for supporting pathogen surveillance and control. However, the lack of standardized metadata and inconsistent submission practices in SRA may impede the data's utility in public health., Methods: To address this issue, we introduce the Search-based Geographic Metadata Curation (SGMC) pipeline. SGMC utilized Python and web scraping to extract geographic data of sequencing institutions from NCBI SRA in the Cloud and its website. It then harnessed ChatGPT to refine the sequencing institution and location assignments. To illustrate the pipeline's utility, we examined the geographic distribution of the sequencing institutions and their countries relevant to polio eradication and categorized them., Results: SGMC successfully identified 7,649 sequencing institutions and their global locations from a random selection of 2,321,044 SRA accessions. These institutions were distributed across 97 countries, with strong representation in the United States, the United Kingdom and China. However, there was a lack of data from African, Central Asian, and Central American countries, indicating potential disparities in sequencing capabilities. Comparison with manually curated data for U.S. institutions reveals SGMC's accuracy rates of 94.8% for institutions, 93.1% for countries, and 74.5% for geographic coordinates., Conclusion: SGMC may represent a novel approach using a generative AI model to enhance geographic data (country and institution assignments) for large numbers of samples within SRA datasets. This information can be utilized to bolster public health endeavors., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Zhao, Farrell, Mashiku, Abay, Tang, Oberste and Burns.)