Search

ChatGPT has become a prominent tool for fostering personalized and interactive learning with the advancements in AI technology. This study analyzes 212 academic research articles indexed in the Scopus database as of July 2023. It maps the trajectory of educational studies on ChatGPT, identifying primary themes, influential authors, and contributing institutions. By employing bibliometric indicators and network analysis, the study explores collaboration patterns, citation trends, and the evolution of research interests. The findings show the exponential growth of interest in leveraging ChatGPT for educational purposes and provide insights into the specific educational domains and contexts that have garnered the most attention. Furthermore, the study reveals the collaborative dynamics and intellectual foundations shaping the field by examining co-authorship and citation networks. This bibliometric analysis contributes to a comprehensive understanding of the current state of ChatGPT research in education, offering researchers and practitioners valuable insights into evolving trends and potential future directions for this innovative aspect of AI and learning.

While ChatGPT has recently become very popular, AI has a long history and philosophy. This paper intends to explore the promises and pitfalls of the Generative Pre-trained Transformer (GPT) AI and potentially future technologies by adopting a speculative methodology. Speculative future narratives with a specific focus on educational contexts are provided in an attempt to identify emerging themes and discuss their implications for education in the 21st century. Affordances of (using) AI in Education (AIEd) and possible adverse effects are identified and discussed which emerge from the narratives. It is argued that now is the best of times to define human vs AI contribution to education because AI can accomplish more and more educational activities that used to be the prerogative of human educators. Therefore, it is imperative to rethink the respective roles of technology and human educators in education with a future-oriented mindset.

The global higher education sector has been significantly disrupted by the proliferation of generative artificial intelligence tools such as ChatGPT, especially in relation to its implications for assessment. However, few studies to date have explored student perspectives on these tools. This article reports on one of the first large-scale quantitative studies of student views on generative artificial intelligence at an Australian university (n = 1,135). When the survey was conducted, most students had low knowledge, experience, and confidence in using these tools. These results varied across disciplines and across some student sub-groups, such as mature-age students and international students. Confidence appeared to increase with experience, although the data also revealed a portion of students that have never used these tools yet still felt confident in using them. In exploring these results, this article aims to shed new light on this fast-evolving landscape and inform the future direction of supporting students to engage with generative artificial intelligence tools appropriately.

This study offers a critical examination of university policies developed to address recent challenges presented by generative AI (GenAI) to higher education assessment. Drawing on Bacchi's 'What's the problem represented to be' (WPR) framework, we analysed the GenAI policies of 20 world-leading universities to explore what are considered problems in this AI-mediated assessment landscape and how these problems are represented in policies. Although miscellaneous GenAI-related problems were mentioned in these policies (e.g. reliability of AI-generated outputs, equal access to GenAI), the primary problem represented is that students may not submit original work for assessment. In the current framing, GenAI is often viewed as a type of external assistance separate from the student's independent efforts and intellectual contribution, thereby undermining the originality of their work. We argue that such problem representation fails to acknowledge how the rise of GenAI further complicates the process of producing original work and what it means by originality in a time when knowledge production becomes increasingly distributed, collaborative and mediated by technology. Therefore, a critical silence in higher education policies concerns the evolving notion of originality in the digital age and a more inclusive approach to address the originality of students' work is required.

Artificial intelligence (AI) tools, notably ChatGPT, are increasingly recognised for their transformative potential in higher education. This study employs a detailed case study approach complemented by a survey, delving into ChatGPT's impact on pedagogical practices, student engagement, and academic performance. It involved 74 undergraduate and postgraduate students enrolled in data analytics courses in Australia. The quantitative analysis highlights ChatGPT's role in providing personalised and on-demand support, which is highly valued among users for its flexibility and responsiveness, meeting a critical demand in educational settings. Notably, the study identifies a medium effect size ([eta squared] = 0.173) in perceived benefits, indicating that ChatGPT accounts for approximately 17.3% of the variance in improved academic outcomes. However, challenges such as ChatGPT's limited understanding of complex queries and the lack of human interactions are primary concerns, with a medium effect size ([eta squared]=0.289) suggesting significant areas for improvement. Furthermore, statistical analyses reveal a clear relationship between the frequency of ChatGPT usage and the perception of its benefits, underscoring the transformative potential for users who have integrated it into their academic practices. Despite these challenges, the differential impact on users versus non-users highlights the potential for ChatGPT to foster more engaging and effective educational practices. The findings advocate for targeted strategies to epitomise ChatGPT's integration into educational settings, emphasising the need for ongoing research and the development of comprehensive guidelines to navigate its complexities and maximise its educational benefits.

This study is a follow-up to previous research conducted in 2012 on computer-assisted language testing (CALT) that applied a survey approach to investigate the use of technology in sign language testing worldwide. The goal of the current study was to replicate the 2012 study and to obtain updated information on the use of technology in sign language testing. Another goal was to broaden the scope of the previous study by also considering the use of automatic sign language recognition and production (by avatars) in applied testing scenarios through technologies that make use of artificial intelligence (AI). Thirty-two sign language testing professionals participated in the current study. The results of the survey confirm the findings from 2012, but also raise new issues for future research (trends) related to sign language testing, such as the use of automatic sign language recognition for automatic scoring and the problem of acquiring funding for developing and maintaining web-based sign language testing platforms.

As artificial intelligence (AI) continues to evolve, its impact on academic environments, especially in postgraduate research supervision, becomes increasingly significant. This study explored the impact of ChatGPT, an advanced AI conversational model, on five dimensions of research supervision: functional, enculturation, critical thinking, emancipation and relationship development. Using a qualitative approach, we examined the practices and perspectives of 20 postgraduate research students with at least 4 months' experience of using ChatGPT in research activities in Australia. The study revealed several areas of impact, including accelerated research progress, enhanced research quality, improved scholarly development and professional skills, enhanced critical thinking, increased student confidence and autonomy, and a deeper supervisory relationship. The findings suggest a shift in the roles and responsibilities of supervisors and students: the former provides strategic direction and high-level guidance, while the latter transits from apprentices to autonomous researchers due to the independence fostered by ChatGPT. This shift suggests an evolving model of postgraduate research supervision, with educational technology acting as epistemic tools to enhance the supervisory process. The study also considers the ethical implications of AI-enabled support.

Artificial intelligence (AI) technology, such as Chat Generative Pre-trained Transformer (ChatGPT), is evolving quickly and having a significant impact on the higher education sector. Although the impact of ChatGPT on academic integrity processes is a key concern, little is known about whether academics can reliably recognise texts that have been generated by AI. This qualitative study applies Turing's Imitation Game to investigate 16 education academics' perceptions of two pairs of texts written by either ChatGPT or a human. Pairs of texts, written in response to the same task, were used as the stimulus for interviews that probed academics' perceptions of text authorship and the textual features that were important in their decision-making. Results indicated academics were only able to identify AI-generated texts half of the time, highlighting the sophistication of contemporary generative AI technology. Academics perceived the following categories as important for their decision-making: voice, word usage, structure, task achievement and flow. All five categories of decision-making were variously used to rationalise both accurate and inaccurate decisions about text authorship. The implications of these results are discussed with a particular focus on what strategies can be applied to support academics more effectively as they manage the ongoing challenge of AI in higher education.

Chatbots have experienced significant growth over the past decade, with a proliferation of new applications across various domains. Previous studies also demonstrate the trend of new technologies, especially artificial intelligence, being adopted in libraries. The purpose of this study is to determine the current research priorities and findings in the field of chatbots in libraries. A systematic literature review was performed utilising the PRISMA checklist and the databases Scopus and Web of Science, identifying 5734 records. Upon conducting the first screening, abstract screening, full-text assessment, and quality assessments guided by the CASP appraisal checklist, 19 papers were deemed suitable for inclusion in the review. The results of the review indicate that the majority of the existing studies were empirical in nature (primarily adopting qualitative methods) and technology reviews with a focus on reviewing the implementation and maintenance, design, evaluation, characteristics, and application of chatbots. The chatbots of interest were mainly text-based and guided chatbots, with closed-source tools with access portals mostly built on library web pages or integrated with social software. The research findings primarily concerned the development models and necessary tools and technologies, the application of chatbots in libraries. Our systematic review also suggests that studies on chatbots in libraries are still in the early stages. [This paper was presented at the 2023 Libraries in the Digital Age (LIDA) International Conference (Osijek, Croatia, May 24-26, 2023).]

With artificial intelligence (AI) now producing human-quality text in seconds via natural language generation, urgent questions arise about the nature and purpose of the teaching of writing in English. Humans have already been co-composing with digital tools for decades, in the form of spelling and grammar checkers built into word processing software. Yet AI has now advanced such that humans need to have less input in the writing process. This article contrasts these developments with findings from a small study of the teaching of writing in Victoria, Australia. The article proposes conceptual framing for these developments through dialogic encounters with Bill Green's model for compos(IT)ing and Rosi Braidotti's concept of the posthuman, making suggestions for updating thinking and offering practical strategies for future directions in the teaching of writing.

This paper explores analysis of feedback data collected from student consultations at two mathematics support centres at universities in Australia and Ireland. Unstructured text data was collected over six years and includes qualitative data on student queries collected during the consultations from mathematics and statistics related subjects. Topic modelling and clustering algorithms are used to uncover key themes in the data across stages. Common areas of difficulty experienced by undergraduate students at both universities are investigated and a comparison between them is shown. The results suggest that, despite institutional differences, there is considerable overlap in the types of mathematical and statistical difficulties experienced by students in their first and second year of university at these institutions. We discuss how the ability to uncover such common mathematical and statistical themes with the aid of text mining techniques can be used to improve the support provided by mathematics support centres in terms of providing an efficient and effective service. The code for analyses at both institutions is provided in a GitHub repository so other academic support centres may use it. Outcomes of this analysis have implications for mainstream mathematics and statistics instructors who wish to gain further insights into their students' learning.

Research in automated creation of test items for assessment purposes became increasingly important during the recent years. Due to automatic question creation it is possible to support personalized and self-directed learning activities by preparing appropriate and individualized test items quite easily with relatively little effort or even fully automatically. In this paper, which is an extended version of the conference paper of Gutl, Lankmayr and Weinhofer (2010), we present our most recent work on the automated creation of different types of test items. More precisely, we describe the design and the development of the "Enhanced Automatic Question Creator" (EAQC) which extracts most important concepts out of textual learning content and creates single choice, multiple-choice, completion exercises and open ended questions on the basis of these concepts. Our approach combines statistical, structural and semantic methods of natural language processing as well as a rule-based AI solution for concept extraction and test item creation. The prototype is designed in a flexible way to support easy changes or improvements of the above mentioned methods. EAQC is designed to deal with multilingual learning material and in its recent version English and German content is supported. Furthermore, we discuss the usage of the EAGC from the users' viewpoint and also present first results of an evaluation study in which students were asked to evaluate the relevance of the extracted concepts and the quality of the created test items. Results of this study showed that the concepts extracted and questions created by the EAQC were indeed relevant with respect to the learning content. Also the level of the questions and the provided answers were appropriate. Regarding the terminology of the questions and the selection of the distractors, which had been criticized most during the evaluation study, we discuss some aspects that could be considered in the future in order to enhance the automatic generation of questions. Nevertheless the results are promising and suggest that the quality of the automatically extracted concepts and created test items is comparable to human generated ones. (Contains 9 figures and 2 tables.)

Research into the teaching and assessment of student writing shows that many students find academic writing a challenge to learn, with legal writing no exception. Improving the availability and quality of timely formative feedback is an important aim. However, the time-consuming nature of assessing writing makes it impractical for instructors to provide rapid, detailed feedback on hundreds of draft texts which might be improved prior to submission. This paper describes the design of a natural language processing (NLP) tool to provide such support. We report progress in the development of a web application called AWA (Academic Writing Analytics), which has been piloted in a Civil Law degree. We describe: the underlying NLP platform and the participatory design process through which the law academic and analytics team tested and refined an existing rhetorical parser for the discipline; the user interface design and evaluation process; and feedback from students, which was broadly positive, but also identifies important issues to address. We discuss how our approach is positioned in relation to concerns regarding automated essay grading, and ways in which AWA might provide more actionable feedback to students. We conclude by considering how this design process addresses the challenge of making explicit to learners and educators the underlying mode of action in analytic devices such as our rhetorical parser, which we term algorithmic accountability.

Although teamwork has been identified as an essential skill for Computer Science (CS) graduates, these skills are identified as lacking by industry employers, which suggests a need for more proactive measures to teach and assess teamwork. In one CS course, students worked in teams to create a wiki solution to problem-based questions. Through a case-study approach, we test a developed teamwork framework, using manual content analysis and sentiment analysis, to determine if the framework can provide insight into students' teamwork behavior and to determine if the wiki task encouraged students to collaborate, share knowledge, and self-adopt teamwork roles. Analysis revealed the identification of both active and cohesive teams, disengaged students, and particular roles and behaviors that were lacking. Furthermore, sentiment analysis revealed that teams moved through positive and negative emotions over the course of developing their solution, toward satisfaction. The findings demonstrate the value of the detailed analysis of online teamwork. However, we propose the need for automated measures that provide real-time feedback to assist educators in the fair and efficient assessment of teamwork. We present a prototype system and recommendations, based on our analysis, for automated teamwork analysis tools.

In this paper, we present a novel approach for semiautomatic question generation to support academic writing. Our system first extracts key phrases from students' literature review papers. Each key phrase is matched with a Wikipedia article and classified into one of five abstract concept categories: Research Field, Technology, System, Term, and Other. Using the content of the matched Wikipedia article, the system then constructs a conceptual graph structure representation for each key phrase and the questions are then generated based the structure. To evaluate the quality of the computer generated questions, we conducted a version of the Bystander Turing test, which involved 20 research students who had written literature reviews for an IT methods course. The pedagogical values of generated questions were evaluated using a semiautomated process. The results indicate that the students had difficulty distinguishing between computer-generated and supervisor-generated questions. Computer-generated questions were also rated as being as pedagogically useful as supervisor-generated questions, and more useful than generic questions. The findings also suggest that the computer-generated questions were more useful for the first-year students than for second or third-year students. (Contains 12 tables and 4 figures.)

This paper reports the results of a University of Canberra Library survey of student research knowledge, skills, tools and resources. Students are experiencing difficulties interrogating databases, the internet and library catalogues because of the lack of consistency in terminology and various methods of interrogation. This research was an investigation of the need for an online contemporary natural language thesaurus. This report provides evidence that students interviewed in this survey support the idea and it is recommended that further research on a semantic contemporary natural language thesaurus be conducted and focused on higher education in Australia. (Contains 5 tables and 17 notes.)

Background: Osteoporotic fractures signal severely compromised bone strength and are associated with a greatly increased risk of refracture. Despite the availability of effective and safe medications that reduce fracture risk, 70–80% of patients are inadequately investigated or treated for osteoporosis following an initial fracture, constituting a significant 'osteoporosis care gap'. Optimal methods of bridging this gap with primary care at the forefront of secondary fracture prevention remain undetermined. This protocol describes a cluster randomised controlled trial to evaluate the effectiveness of a novel integrated model of secondary fracture prevention and management in primary care. Methods: The cluster randomised controlled trial involves multiple branches of a community-based radiology provider (CRP), a hospital-based secondary fracture prevention program (SFPP) and numerous primary care practices in metropolitan Sydney that refer to either the CRP or SFPP. Using natural language processing tools, patients diagnosed with a potential osteoporotic fracture will be identified by automatically screening radiology reports generated at the CRP or SFPP. The primary care practices that these patients attend will be randomised (1:1) to either the intervention or usual care. The intervention consists of (i) electronic and fax alerts informing the practice/primary care physician that their patient has been diagnosed with a potential osteoporotic fracture; (ii) provision of osteoporosis management guidelines and (iii) follow-up surveys at 4 weeks and 6 months. Practices in the usual care (control) group will receive no alerts and provide usual care. The primary outcome is the proportion of patients undergoing a bone density scan and/or filling a prescription for osteo-protective pharmacotherapy within 3 months of the initial diagnostic imaging report. Secondary outcomes are the proportion of patients: (i) undergoing an osteoporosis-related blood test within 3 months of the initial diagnostic imaging report; (ii) initiated on a chronic disease management plan within 3 months of the diagnostic report, and (iii) filling a second prescription for osteo-protective pharmacotherapy within 9 months post initial diagnostic imaging report. Outcomes will be obtained through de-identified linked data from Medical Benefits Schedule and Pharmaceutical Benefits Scheme held by the Australian Institute of Health and Welfare. Discussion: This is the first randomised trial to integrate case-detection of potential osteoporotic fractures in a hospital and community setting with direct alerts to the patient's primary care provider. This study will determine whether such an intervention is effective in improving investigation and/or treatment rates of osteoporosis in patients with a potential osteoporotic fracture. Trial registration: This study is registered with the Australian New Zealand Clinical Trials Registry (ANZCTR): ACTRN12623000658617p. [ABSTRACT FROM AUTHOR]

BACKGROUND: Malnutrition is a serious health risk facing older people living in residential aged care facilities. Aged care staff record observations and concerns about older people in electronic health records (EHR), including free-text progress notes. These insights are yet to be unleashed. OBJECTIVE: This study explored the risk factors for malnutrition in structured and unstructured electronic health data. METHODS: Data of weight loss and malnutrition were extracted from the de-identified EHR records of a large aged care organization in Australia. A literature review was conducted to identify causative factors for malnutrition. Natural language processing (NLP) techniques were applied to progress notes to extract these causative factors. The NLP performance was evaluated by the parameters of sensitivity, specificity and F1-Score. RESULTS: The NLP methods were highly accurate in extracting the key data, values for 46 causative variables, from the free-text client progress notes. Thirty three percent (1,469 out of 4,405) of the clients were malnourished. The structured, tabulated data only recorded 48% of these malnourished clients, far less than that (82%) identified from the progress notes, suggesting the importance of using NLP technology to uncover the information from nursing notes to fully understand the health status of the vulnerable older people in residential aged care. CONCLUSION: This study identified 33% of older people suffered from malnutrition, lower than those reported in the similar setting in previous studies. Our study demonstrates that NLP technology is important for uncovering the key information about health risks for older people in residential aged care. Future research can apply NLP to predict other health risks for older people in this setting. [ABSTRACT FROM AUTHOR]

Psychotic symptoms are typically measured using clinical ratings, but more objective and sensitive metrics are needed. Hence, we will assess thought disorder using the Research Domain Criteria (RDoC) heuristic for language production, and its recommended paradigm of "linguistic corpus-based analyses of language output". Positive thought disorder (e.g., tangentiality and derailment) can be assessed using word-embedding approaches that assess semantic coherence, whereas negative thought disorder (e.g., concreteness, poverty of speech) can be assessed using part-of-speech (POS) tagging to assess syntactic complexity. We aim to establish convergent validity of automated linguistic metrics with clinical ratings, assess normative demographic variance, determine cognitive and functional correlates, and replicate their predictive power for psychosis transition among at-risk youths. This study will assess language production in 450 English-speaking individuals in Australia and Canada, who have recent onset psychosis, are at clinical high risk (CHR) for psychosis, or who are healthy volunteers, all well-characterized for cognition, function and symptoms. Speech will be elicited using open-ended interviews. Audio files will be transcribed and preprocessed for automated natural language processing (NLP) analyses of coherence and complexity. Data analyses include canonical correlation, multivariate linear regression with regularization, and machine-learning classification of group status and psychosis outcome. This prospective study aims to characterize language disturbance across stages of psychosis using computational approaches, including psychometric properties, normative variance and clinical correlates, important for biomarker development. SPEAK will create a large archive of language data available to other investigators, a rich resource for the field. [ABSTRACT FROM AUTHOR]

Geological maps are powerful models for visualizing the complex distribution of rock types through space and time. However, the descriptive information that forms the basis for a preferred map interpretation is typically stored in geological map databases as unstructured text data that are difficult to use in practice. Herein we apply natural language processing (NLP) to geoscientific text data from Canada, the U.S., and Australia to address that knowledge gap. First, rock descriptions, geological ages, lithostratigraphic and lithodemic information, and other long-form text data are translated to numerical vectors, i.e., a word embedding, using a geoscience language model. Network analysis of word associations, nearest neighbors, and principal component analysis are then used to extract meaningful semantic relationships between rock types. We further demonstrate using simple Naive Bayes classifiers and the area under receiver operating characteristics plots (AUC) how word vectors can be used to: (1) predict the locations of "pegmatitic" (AUC = 0.962) and "alkalic" (AUC = 0.938) rocks; (2) predict mineral potential for Mississippi-Valley-type (AUC = 0.868) and clastic-dominated (AUC = 0.809) Zn-Pb deposits; and (3) search geoscientific text data for analogues of the giant Mount Isa clastic-dominated Zn-Pb deposit using the cosine similarities between word vectors. This form of semantic search is a promising NLP approach for assessing mineral potential with limited training data. Overall, the results highlight how geoscience language models and NLP can be used to extract new knowledge from unstructured text data and reduce the mineral exploration search space for critical raw materials. [ABSTRACT FROM AUTHOR]

Background: Cancer stage at diagnosis is essential for understanding cancer outcomes, guiding cancer control activities and healthcare services, and enabling benchmarking nationally and internationally. Yet, most cancer registries in Australia do not routinely collect this data. This study explored key stakeholders' perceptions of implementing cancer staging utilising Natural Language Processing and Machine Learning algorithms within the Western Australian Cancer Registry. Methods: Perceptions of key breast and colorectal cancer stakeholders, including registry staff, clinicians, consumers, data scientists, biostatisticians, data management, healthcare staff, and health researchers, were collected. Prospective and retrospective qualitative proformas at two-time points of the Western Australian Cancer Staging Project were employed. The Consolidated Framework for Implementation Research was used to guide data collection, analysis and interpretation embedded in a Participatory Action Research approach. Data analysis also incorporated Framework Analysis and an adapted version of grading qualitative data using a visual traffic light labelling system to highlight the levels of positivity, negativity, and implementation concern. Results: Twenty-nine pre-proformas and 18 post-proformas were completed online via REDCap. The grading and visual presentation of barriers and enablers aided interpretation and reviewing predicted intervention outcomes. Of the selected constructs, complexity (the perceived difficulty of the intervention) was the strongest barrier and tension for change (the situation needing change) was the strongest enabler. Implementing cancer staging into the Western Australian Cancer Registry was considered vital. Benefits included improved knowledge and understanding of various outcomes (e.g., treatment received as per Optimum Care Pathways) and benchmarking. Barriers included compatibility issues with current systems/workflows, departmental/higher managerial support, and future sustainment. Conclusions: The findings aid further review of data gaps, additional cancer streams, standardising cancer staging and future improvements. The study offers an adapted version of a rapid qualitative data collection and analytic approach for establishing barriers and enablers. The findings may also assist other population-based cancer registries considering collecting cancer stage at diagnosis. [ABSTRACT FROM AUTHOR]

OBJECTIVES: Mental illness is known to adversely affect the physical health of patients in primary and acute care settings; however, its impact on critically ill patients is less well studied. This study aimed to determine the prevalence, characteristics, and outcomes of patients admitted to the ICU with a preexisting mental health disorder. DESIGN: A multicenter, retrospective cohort study using linked data from electronic ICU clinical progress notes and the Australia and New Zealand Intensive Care Society Adult Patient Database. SETTING/PATIENTS: All patients admitted to eight Australian adult ICUs in the calendar year 2019. Readmissions within the same hospitalization were excluded. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Natural language processing techniques were used to classify preexisting mental health disorders in participants based on clinician documentation in electronic ICU clinical progress notes. Sixteen thousand two hundred twenty-eight patients (58% male) were included in the study, of which 5,044 (31.1%) had a documented preexisting mental health disorder. Affective disorders were the most common subtype occurring in 2,633 patients (16.2%), followed by anxiety disorders, occurring in 1,611 patients (9.9%). Mixed-effects regression modeling found patients with a preexisting mental health disorder stayed in ICU 13% longer than other patients (β-coefficient, 0.12; 95% CI, 0.10–0.15) and were more likely to experience invasive ventilation (odds ratio, 1.42; 95% CI, 1.30–1.56). Severity of illness and ICU mortality rates were similar in both groups. CONCLUSIONS: Patients with preexisting mental health disorders form a significant subgroup within the ICU. The presence of a preexisting mental health disorder is associated with greater ICU length of stay and higher rates of invasive ventilation, suggesting these patients may have a different clinical trajectory to patients with no mental health history. Further research is needed to better understand the reasons for these adverse outcomes and to develop interventions to better support these patients during and after ICU admission. [ABSTRACT FROM AUTHOR]

Objective: Large language models (LLMs) have demonstrated mixed results in their ability to pass various specialist medical examination and their performance within the field of emergency medicine remains unknown. Methods: We explored the performance of three prevalent LLMs (OpenAI's GPT series, Google's Bard, and Microsoft's Bing Chat) on a practice ACEM primary examination. Results: All LLMs achieved a passing score, with scores with GPT 4.0 outperforming the average candidate. Conclusion: Large language models, by passing the ACEM primary examination, show potential as tools for medical education and practice. However, limitations exist and are discussed. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI) has the potential to revolutionize the field of surgery by improving diagnostic accuracy, workflow efficiency, and reducing costs. Large language models (LLMs), such as ChatGPT, have gained attention for their ability to mimic human conversation and respond intelligently to user prompts. LLMs like ChatGPT can streamline tasks such as symptom assessment, diagnostic decision support, and patient education. They also have potential applications in surgical education and research. However, there are concerns about the perpetuation of racism and inequity within AI systems, as well as data privacy and sovereignty issues. The surgical community must ensure the fair and responsible implementation of AI tools. [Extracted from the article]

Background: Incidental durotomy is an intraoperative complication in spine surgery that can lead to postoperative complications, increased length of stay, and higher healthcare costs. Natural language processing (NLP) is an artificial intelligence method that assists in understanding free-text notes that may be useful in the automated surveillance of adverse events in orthopaedic surgery. A previously developed NLP algorithm is highly accurate in the detection of incidental durotomy on internal validation and external validation in an independent cohort from the same country. External validation in a cohort with linguistic differences is required to assess the transportability of the developed algorithm, referred to geographical validation. Ideally, the performance of a prediction model, the NLP algorithm, is constant across geographic regions to ensure reproducibility and model validity.Question/purpose: Can we geographically validate an NLP algorithm for the automated detection of incidental durotomy across three independent cohorts from two continents?Methods: Patients 18 years or older undergoing a primary procedure of (thoraco)lumbar spine surgery were included. In Massachusetts, between January 2000 and June 2018, 1000 patients were included from two academic and three community medical centers. In Maryland, between July 2016 and November 2018, 1279 patients were included from one academic center, and in Australia, between January 2010 and December 2019, 944 patients were included from one academic center. The authors retrospectively studied the free-text operative notes of included patients for the primary outcome that was defined as intraoperative durotomy. Incidental durotomy occurred in 9% (93 of 1000), 8% (108 of 1279), and 6% (58 of 944) of the patients, respectively, in the Massachusetts, Maryland, and Australia cohorts. No missing reports were observed. Three datasets (Massachusetts, Australian, and combined Massachusetts and Australian) were divided into training and holdout test sets in an 80:20 ratio. An extreme gradient boosting (an efficient and flexible tree-based algorithm) NLP algorithm was individually trained on each training set, and the performance of the three NLP algorithms (respectively American, Australian, and combined) was assessed by discrimination via area under the receiver operating characteristic curves (AUC-ROC; this measures the model's ability to distinguish patients who obtained the outcomes from those who did not), calibration metrics (which plot the predicted and the observed probabilities) and Brier score (a composite of discrimination and calibration). In addition, the sensitivity (true positives, recall), specificity (true negatives), positive predictive value (also known as precision), negative predictive value, F1-score (composite of precision and recall), positive likelihood ratio, and negative likelihood ratio were calculated.Results: The combined NLP algorithm (the combined Massachusetts and Australian data) achieved excellent performance on independent testing data from Australia (AUC-ROC 0.97 [95% confidence interval 0.87 to 0.99]), Massachusetts (AUC-ROC 0.99 [95% CI 0.80 to 0.99]) and Maryland (AUC-ROC 0.95 [95% CI 0.93 to 0.97]). The NLP developed based on the Massachusetts cohort had excellent performance in the Maryland cohort (AUC-ROC 0.97 [95% CI 0.95 to 0.99]) but worse performance in the Australian cohort (AUC-ROC 0.74 [95% CI 0.70 to 0.77]).Conclusion: We demonstrated the clinical utility and reproducibility of an NLP algorithm with combined datasets retaining excellent performance in individual countries relative to algorithms developed in the same country alone for detection of incidental durotomy. Further multi-institutional, international collaborations can facilitate the creation of universal NLP algorithms that improve the quality and safety of orthopaedic surgery globally. The combined NLP algorithm has been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/nlp_incidental_durotomy/ . Clinicians and researchers can use the tool to help incorporate the model in evaluating spine registries or quality and safety departments to automate detection of incidental durotomy and optimize prevention efforts.Level Of Evidence: Level III, diagnostic study. [ABSTRACT FROM AUTHOR]

Using a suite of artificial intelligence technologies, the current study sought to determine the prevalence of agitated behaviors in people with dementia in residential aged care facilities (RACFs) in Australia. Computerized natural language processing allowed extraction of agitation instances from the free-text nursing progress notes, a component of electronic health records in RACFs. In total, 59 observable agitated behaviors were found. No difference was found in dementia prevalence between female and male clients (44.1%), across metropolitan and regional facilities (42.1% [SD = 17.9%]), or for agitation prevalence in dementia (76.5% [SD = 18.4%]). The top 10 behaviors were resisting, wandering, speaking in excessively loud voice, pacing, restlessness, pushing, shouting, complaining, frustration, and using profane language. Four to 17 agitated behaviors coexisted in 53% of people with dementia agitation, indicating high caregiver burden in these RACFs. Improving workforce training and redesigning care models are urgent for sustainability of dementia care in RACFs. [Journal of Gerontological Nursing, 48(4), 57–64.] [ABSTRACT FROM AUTHOR]

Automatic deidentification of Electronic Health Records (EHR) is a crucial step in secondary usage for biomedical research. This study introduces evaluation of an intricate hybrid deidentification strategy to enhance patient privacy in secondary usage of EHR. Specifically, this study focuses on assessing automatic deidentification using OpenDeID pipeline across diverse corpora for safeguarding sensitive information within EHR datasets by incorporating diverse corpora. Three distinct corpora were utilized: the OpenDeID v2 corpus containing pathology reports from Australian hospitals, the 2014 i2b2/UTHealth deidentification corpus with clinical narratives from the USA, and the 2016 CEGS N-GRID identification corpus comprising psychiatric notes. The OpenDeID pipeline employs a hybrid approach based on deep learning and contextual rules. Pre-processing steps involved harmonizing and addressing encoding and format issues. Precision, Recall, F-measure metrics were used to assess the performance. The evaluation metrics demonstrated the superior performance of the Discharge Summary BioBERT model. Trained on three corpora with a total of 4,038 reports, the best performing model exhibited robust deidentification capabilities when applied to EHR. It achieved impressive micro-averaged F1-scores of 0.9248 and 0.9692 for strict and relaxed settings, respectively. These results offer valuable insights into the model's efficacy and its potential role in safeguarding patient privacy in secondary usage of EHR.

Background: Cardiometabolic diseases (CMDs) are a group of interrelated conditions, including heart failure and diabetes, that increase the risk of cardiovascular and metabolic complications. The rising number of Australians with CMDs has necessitated new strategies for those managing these conditions, such as digital health interventions. The effectiveness of digital health interventions in supporting people with CMDs is dependent on the extent to which users engage with the tools. Augmenting digital health interventions with conversational agents, technologies that interact with people using natural language, may enhance engagement because of their human-like attributes. To date, no systematic review has compiled evidence on how design features influence the engagement of conversational agent-enabled interventions supporting people with CMDs. This review seeks to address this gap, thereby guiding developers in creating more engaging and effective tools for CMD management., Objective: The aim of this systematic review is to synthesize evidence pertaining to conversational agent-enabled intervention design features and their impacts on the engagement of people managing CMD., Methods: The review is conducted in accordance with the Cochrane Handbook for Systematic Reviews of Interventions and reported in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Searches will be conducted in the Ovid (Medline), Web of Science, and Scopus databases, which will be run again prior to manuscript submission. Inclusion criteria will consist of primary research studies reporting on conversational agent-enabled interventions, including measures of engagement, in adults with CMD. Data extraction will seek to capture the perspectives of people with CMD on the use of conversational agent-enabled interventions. Joanna Briggs Institute critical appraisal tools will be used to evaluate the overall quality of evidence collected., Results: This review was initiated in May 2023 and was registered with the International Prospective Register of Systematic Reviews (PROSPERO) in June 2023, prior to title and abstract screening. Full-text screening of articles was completed in July 2023 and data extraction began August 2023. Final searches were conducted in April 2024 prior to finalizing the review and the manuscript was submitted for peer review in July 2024., Conclusions: This review will synthesize diverse observations pertaining to conversational agent-enabled intervention design features and their impacts on engagement among people with CMDs. These observations can be used to guide the development of more engaging conversational agent-enabled interventions, thereby increasing the likelihood of regular intervention use and improved CMD health outcomes. Additionally, this review will identify gaps in the literature in terms of how engagement is reported, thereby highlighting areas for future exploration and supporting researchers in advancing the understanding of conversational agent-enabled interventions., Trial Registration: PROSPERO CRD42023431579; https://tinyurl.com/55cxkm26., International Registered Report Identifier (irrid): DERR1-10.2196/52973., (©Nick Kashyap, Ann Tresa Sebastian, Chris Lynch, Paul Jansons, Ralph Maddison, Tilman Dingler, Brian Oldenburg. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 07.08.2024.)

Understanding antimicrobial usage patterns and encouraging appropriate antimicrobial usage is a critical component of antimicrobial stewardship. Studies using VetCompass Australia and Natural Language Processing (NLP) have demonstrated antimicrobial usage patterns in companion animal practices across Australia. Doing so has highlighted the many obstacles and barriers to the task of converting raw clinical notes into a format that can be readily queried and analysed. We developed NLP systems using rules‐based algorithms and machine learning to automate the extraction of data describing the key elements to assess appropriate antimicrobial use. These included the clinical indication, antimicrobial agent selection, dose and duration of therapy. Our methods were applied to over 4.4 million companion animal clinical records across Australia on all consultations with antimicrobial use to help us understand what antibiotics are being given and why on a population level. Of these, approximately only 40% recorded the reason why antimicrobials were prescribed, along with the dose and duration of treatment. NLP and deep learning might be able to overcome the difficulties of harvesting free text data from clinical records, but when the essential data are not recorded in the clinical records, then, this becomes an insurmountable obstacle. [ABSTRACT FROM AUTHOR]

Introduction: Over 50% of people affected by cancer report unmet support needs. To address unmet information and psychological needs, non-government organisations such as Cancer Councils (Australia) have developed state-based telephone cancer information and support services. Due to competing demands, evidence of the value of these services is needed to ensure that future investment makes the best use of scarce resources. This research aims to determine the costs and broader economic and social value of a telephone support service, to inform future funding and service provision., Methods and Analysis: A codesigned, evaluative social return on investment analysis (SROI) will be conducted to estimate and compare the costs and monetised benefits of Cancer Council Victoria's (CCV) telephone support line, 13 11 20, over 1-year and 3-year benefit periods. Nine studies will empirically estimate the parameters to inform the SROI and calculate the ratio (economic and social value to value invested): step 1 mapping outcomes (in-depth analysis of CCV's 13 11 20 recorded call data; focus groups and interviews); step 2 providing evidence of outcomes (comparative survey of people affected by cancer who do and do not call CCV's 13 11 20; general public survey); step 3 valuing the outcomes (financial proxies, value games); step 4 establishing the impact (Delphi); step 5 calculating the net benefit and step 6 service improvement (discrete choice experiment (DCE), 'what if' analysis). Qualitative (focus groups, interviews) and quantitative studies (natural language processing, cross-sectional studies, Delphi) and economic techniques (willingness-to-pay, financial proxies, value games, DCE) will be applied., Ethics and Dissemination: Ethics approval for each of the studies will be sought independently as the project progresses. So far, ethics approval has been granted for the first two studies. As each study analysis is completed, results will be disseminated through presentation, conferences, publications and reports to the partner organisations., Competing Interests: Competing interests: NM leads the Economics of Cancer Collaboration between Deakin University and CCV and she holds an honorary position with CCV. JB holds an honorary position with CCV. TH is the Chief Executive Officer, CCV. DSS is the Head of Strategy & Support, CCV. KL was the Head of Cancer Information Support Services, CCV during Steps 1 and 2. SA was CEO of Cancer Council Australia from 2015 to 2020 and currently chairs the Advisory Committee to the Victoria Cancer Registry. All other authors declare that they have no competing interests., (© Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)

Objective: Epilepsy surgery is known to be underutilized. Machine learning-natural language processing (ML-NLP) may be able to assist with identifying patients suitable for referral for epilepsy surgery evaluation., Methods: Data were collected from two tertiary hospitals for patients seen in neurology outpatients for whom the diagnosis of "epilepsy" was mentioned. Individual case note review was undertaken to characterize the nature of the diagnoses discussed in these notes, and whether those with epilepsy fulfilled prespecified criteria for epilepsy surgery workup (namely focal drug refractory epilepsy without contraindications). ML-NLP algorithms were then developed using fivefold cross-validation on the first free-text clinic note for each patient to identify these criteria., Results: There were 457 notes included in the study, of which 250 patients had epilepsy. There were 37 (14.8%) individuals who fulfilled the prespecified criteria for epilepsy surgery referral without described contraindications, 32 (12.8%) of whom were not referred for epilepsy surgical evaluation in the given clinic visit. In the prediction of suitability for epilepsy surgery workup using the prespecified criteria, the tested models performed similarly. For example, the random forest model returned an area under the receiver operator characteristic curve of 0.97 (95% confidence interval 0.93-1.0) for this task, sensitivity of 1.0, and specificity of 0.93., Significance: This study has shown that there are patients in tertiary hospitals in South Australia who fulfill prespecified criteria for epilepsy surgery evaluation who may not have been referred for such evaluation. ML-NLP may assist with the identification of patients suitable for such referral., Plain Language Summary: Epilepsy surgery is a beneficial treatment for selected individuals with drug-resistant epilepsy. However, it is vastly underutilized. One reason for this underutilization is a lack of prompt referral of possible epilepsy surgery candidates to comprehensive epilepsy centers. Natural language processing, coupled with machine learning, may be able to identify possible epilepsy surgery candidates through the analysis of unstructured clinic notes. This study, conducted in two tertiary hospitals in South Australia, demonstrated that there are individuals who fulfill criteria for epilepsy surgery evaluation referral but have not yet been referred. Machine learning-natural language processing demonstrates promising results in assisting with the identification of such suitable candidates in Australia., (© 2024 The Authors. Epilepsia Open published by Wiley Periodicals LLC on behalf of International League Against Epilepsy.)

Automated information extraction might be able to assist with the collection of stroke key performance indicators (KPI). The feasibility of using natural language processing for classification‐based KPI and datetime field extraction was assessed. Using free‐text discharge summaries, random forest models achieved high levels of performance in classification tasks (area under the receiver operator curve 0.95–1.00). The datetime field extraction method was successful in 29 of 43 (67.4%) cases. Further studies are indicated. [ABSTRACT FROM AUTHOR]

To inform the development of automated summarization of clinical conversations, this study sought to estimate the proportion of doctor-patient communication in general practice (GP) consultations used for generating a consultation summary. Two researchers with a medical degree read the transcripts of 44 GP consultations and highlighted the phrases to be used for generating a summary of the consultation. For all consultations, less than 20% of all words in the transcripts were needed for inclusion in the summary. On average, 9.1% of all words in the transcripts, 26.6% of all medical terms, and 27.3% of all speaker turns were highlighted. The results indicate that communication content used for generating a consultation summary makes up a small portion of GP consultations, and automated summarization solutions—such as digital scribes—must focus on identifying the 20% relevant information for automatically generating consultation summaries. [ABSTRACT FROM AUTHOR]

Q fever, caused by the zoonotic bacterium Coxiella burnetii, is a globally distributed emerging infectious disease. Livestock are the most important zoonotic transmission sources, yet infection in people without livestock exposure is common. Identifying potential exposure pathways is necessary to design effective interventions and aid outbreak prevention. We used natural language processing and graphical network methods to provide insights into how Q fever notifications are associated with variation in patient occupations or lifestyles. Using an 18‐year time‐series of Q fever notifications in Queensland, Australia, we used topic models to test whether compositions of patient answers to follow‐up exposure questionnaires varied between demographic groups or across geographical areas. To determine heterogeneity in possible zoonotic exposures, we explored patterns of livestock and game animal co‐exposures using Markov Random Fields models. Finally, to identify possible correlates of Q fever case severity, we modelled patient probabilities of being hospitalized as a function of particular exposures. Different demographic groups consistently reported distinct sets of exposure terms and were concentrated in different areas of the state, suggesting the presence of multiple transmission pathways. Macropod exposure was commonly reported among Q fever cases, even when exposure to cattle, sheep or goats was absent. Males, older patients and those that reported macropod exposure were more likely to be hospitalized due to Q fever infection. Our study indicates that follow‐up surveillance combined with text modelling is useful for unravelling exposure pathways in the battle to reduce Q fever incidence and associated morbidity. [ABSTRACT FROM AUTHOR]

This paper examines tax justice advocacy (e.g., around multinational corporate tax avoidance) in the UK and Australia between 2010 and 2018 to understand how digital advocacy organizations relate to one another as well as different groups in a shared discursive space. Data from public Facebook pages is analyzed using topic modeling and network analysis, supplemented by interviews with organizational actors. The transnational organizational links between two high-profile digital advocacy organizations – 38 Degrees and GetUp – are not reflected by proximity in the discourse network, which instead displays dominant nationally specific differences around the concepts and corporate targets associated with tax justice advocacy. [ABSTRACT FROM AUTHOR]

Antimicrobial Resistance is a global crisis that veterinarians contribute to through their use of antimicrobials in animals. Antimicrobial stewardship has been shown to be an effective means to reduce antimicrobial resistance in hospital environments. Effective monitoring of antimicrobial usage patterns is an essential part of antimicrobial stewardship and is critical in reducing the development of antimicrobial resistance. The aim of this study is to describe how frequently antimicrobials were used in veterinary consultations and identify the most frequently used antimicrobials. Using VetCompass Australia, Natural Language Processing techniques, and the Australian Strategic Technical Advisory Group's (ASTAG) Rating system to classify the importance of antimicrobials, descriptive analysis was performed on the antimicrobials prescribed in consultations from 137 companion animal veterinary clinics in Australia between 2013 and 2017 (inclusive). Of the 4,400,519 consultations downloaded there were 595,089 consultations where antimicrobials were prescribed to dogs or cats. Antimicrobials were dispensed in 145 of every 1000 canine consultations; and 38 per 1000 consultations involved high importance rated antimicrobials. Similarly with cats, 108 per 1000 consultations had antimicrobials dispensed, and in 47 per 1000 consultations an antimicrobial of high importance rating was administered. The most common antimicrobials given to cats and dogs were cefovecin and amoxycillin clavulanate, respectively. The most common topical antimicrobial and high-rated topical antimicrobial given to dogs and cats was polymyxin B. This study provides a descriptive analysis of the antimicrobial usage patterns in Australia using methods that can be automated to inform antimicrobial use surveillance programs and promote antimicrobial stewardship. [ABSTRACT FROM AUTHOR]

Background: Social media has the potential to be of great value in understanding patterns in public health using large-scale analysis approaches (eg, data science and natural language processing [NLP]), 2 of which have been used in public health: sentiment analysis and topic modeling; however, their use in the area of food security and public health nutrition is limited., Objective: This study aims to explore the potential use of NLP tools to gather insights from real-world social media data on the public health issue of food security., Methods: A search strategy for obtaining tweets was developed using food security terms. Tweets were collected using the Twitter application programming interface from January 1, 2019, to December 31, 2021, filtered for Australia-based users only. Sentiment analysis of the tweets was performed using the Valence Aware Dictionary and Sentiment Reasoner. Topic modeling exploring the content of tweets was conducted using latent Dirichlet allocation with BigML (BigML, Inc). Sentiment, topic, and engagement (the sum of likes, retweets, quotations, and replies) were compared across years., Results: In total, 38,070 tweets were collected from 14,880 Twitter users. Overall, the sentiment when discussing food security was positive, although this varied across the 3 years. Positive sentiment remained higher during the COVID-19 lockdown periods in Australia. The topic model contained 10 topics (in order from highest to lowest probability in the data set): "Global production," "Food insecurity and health," "Use of food banks," "Giving to food banks," "Family poverty," "Food relief provision," "Global food insecurity," "Climate change," "Australian food insecurity," and "Human rights." The topic "Giving to food banks," which focused on support and donation, had the highest proportion of positive sentiment, and "Global food insecurity," which covered food insecurity prevalence worldwide, had the highest proportion of negative sentiment. When compared with news, there were some events, such as COVID-19 support payment introduction and bushfires across Australia, that were associated with high periods of positive or negative sentiment. Topics related to food insecurity prevalence, poverty, and food relief in Australia were not consistently more prominent during the COVID-19 pandemic than before the pandemic. Negative tweets received substantially higher engagement across 2019 and 2020. There was no clear relationship between topics that were more likely to be positive or negative and have higher or lower engagement, indicating that the identified topics are discrete issues., Conclusions: In this study, we demonstrated the potential use of sentiment analysis and topic modeling to explore evolution in conversations on food security using social media data. Future use of NLP in food security requires the context of and interpretation by public health experts and the use of broader data sets, with the potential to track dimensions or events related to food security to inform evidence-based decision-making in this area., (©Annika Molenaar, Dickson Lukose, Linda Brennan, Eva L Jenkins, Tracy A McCaffrey. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 21.03.2024.)

Natural Language Processing (NLP) is a powerful technique for extracting valuable information from unstructured electronic health records (EHRs). However, a prerequisite for NLP is the availability of high-quality annotated datasets. To date, there is a lack of effective methods to guide the research effort of manually annotating unstructured datasets, which can hinder NLP performance. Therefore, this study develops a five-step workflow for manually annotating unstructured datasets, including (1) annotator training and familiarising with the text corpus, (2) vocabulary identification, (3) annotation schema development, (4) annotation execution, and (5) result validation. This framework was then applied to annotate agitation symptoms from the unstructured EHRs of 40 Australian residential aged care facilities. The annotated corpus achieved an accuracy rate of 96%. This suggests that our proposed annotation workflow can be used in manual data processing to develop annotated training corpus for developing NLP algorithms.

Artificial Intelligence (AI) has great potential to improve healthcare, but implementation into routine practice remains a challenge. This study scoped the extent to which AI and Natural Language Processing (NLP) is being implemented into routine practice in Australian healthcare organisations. An environmental scan of publicly available data was undertaken to identify AI applications. Publicly available data consisted of news posts from Australian public healthcare organisations and conference proceedings from key research organisations. Two researchers reviewed and analysed posts related to AI applications to create a list of potential implementation case studies. The final list of AI applications was reviewed by a governance committee in order to identify any missing applications. One application was identified by the governance committee and subsequently added. The environmental scan identified eighteen AI applications, of which eleven met all eligibility criteria. Only one application included NLP. Twelve applications were included when the application identified by the governance committee was added to the list. Implementation of AI applications is spread across four broad categories of use: 1) Decision Support, 2) Monitoring Treatment Effectiveness, 3) Personalised Care and 4) Risk Prediction.

Nursing staff record observations about older people under their care in free-text nursing notes. These notes contain older people's care needs, disease symptoms, frequency of symptom occurrence, nursing actions, etc. Therefore, it is vital to develop a technique to uncover important data from these notes. This study developed and evaluated a deep learning and transfer learning-based named entity recognition (NER) model for extracting symptoms of agitation in dementia from the nursing notes. We employed a Clinical BioBERT model for word embedding. Then we applied bidirectional long-short-term memory (BiLSTM) and conditional random field (CRF) models for NER on nursing notes from Australian residential aged care facilities. The proposed NER model achieves satisfactory performance in extracting symptoms of agitation in dementia with a 75% F1 score and 78% accuracy. We will further develop machine learning models to recommend the optimal nursing actions to manage agitation.

There is a wealth of information about the characteristics (traits) of organisms within collections of taxonomic descriptions of plants and animals called a 'Flora' or 'Fauna' of a region. However, such knowledge is usually encoded as text paragraphs, and is thus unavailable for immediate analysis. In order to make use of the knowledge embedded in taxon descriptions, text must be organised into standardised, queryable datasets. Despite the recent development of natural language processing (NPL) tools to analyse taxonomic descriptions to extract trait values, the complexity and specificity of these methods currently limits broad application. Accessible and flexible methods for extracting traits across large numbers of taxonomic descriptions are therefore needed. Here we present such an R-based workflow, which can be adapted for use on any organismal group using a language familiar to researchers in the biological sciences. We document a way to (1) assemble tens of thousands of taxonomic descriptions into a standardised format, (2) split the taxon descriptions into different topics, (3) extract trait values as defined by the user, and (4) assign traits described at the genus and family level to lower level taxa to maximise trait coverage. As a case study, we apply the workflow to a collection of taxonomic descriptions drawn from Australia's state and national floras and describe useful techniques for creating workflows and thereby research-grade trait datasets. Using this method, we were able to extract 615,812 trait values from 38 different plant traits. Trait data collated using this method are freely available as part of the AusTraits trait database and have already contributed to analyses in several scientific publications. [ABSTRACT FROM AUTHOR]