Search

With the growing use of artificial intelligence, search engine providers are increasingly pushing advertisers to use automated bidding strategies based on machine learning. Such automated decision‐making systems leave advertisers in the dark about the data being used and how they can influence the outcome of the decision‐making process. Previous literature on artificial intelligence lacks an understanding of the dangers related to artificially intelligent systems and their lack of transparency. In response, our paper addresses the inherent risks of the automated optimization of advertisers' bidding strategies in search engine advertising. The selected empirical case of a service company therefore demonstrates how data availability can trigger a long‐term decline in advertising performance and how search engine advertising performance metrics develop before and after an event of data scarcity. Based on data collected for 525 days, difference‐in‐differences analysis shows that the algorithmic approach has a considerable and lasting negative impact on advertising performance. Furthermore, the empirical case indicates that self‐regulated learning can initialize a downward spiral that gradually impairs advertising performance. Thus, the aim of this study is to increase awareness regarding automated decision‐making dangers in search engine advertising and help advertisers take preventive measures to reduce the risks of algorithm missteps. [ABSTRACT FROM AUTHOR]

Background: Teachers often rely on the use of open‐ended questions to assess students' conceptual understanding of assigned content. Particularly in the context of mathematics; teachers use these types of questions to gain insight into the processes and strategies adopted by students in solving mathematical problems beyond what is possible through more close‐ended problem types. While these types of problems are valuable to teachers, the variation in student responses to these questions makes it difficult, and time‐consuming, to evaluate and provide directed feedback. It is a well‐studied concept that feedback, both in terms of a numeric score but more importantly in the form of teacher‐authored comments, can help guide students as to how to improve, leading to increased learning. It is for this reason that teachers need better support not only for assessing students' work but also in providing meaningful and directed feedback to students. Objectives: In this paper, we seek to develop, evaluate, and examine machine learning models that support automated open response assessment and feedback. Methods: We build upon the prior research in the automatic assessment of student responses to open‐ended problems and introduce a novel approach that leverages student log data combined with machine learning and natural language processing methods. Utilizing sentence‐level semantic representations of student responses to open‐ended questions, we propose a collaborative filtering‐based approach to both predict student scores as well as recommend appropriate feedback messages for teachers to send to their students. Results and Conclusion: We find that our method outperforms previously published benchmarks across three different metrics for the task of predicting student performance. Through an error analysis, we identify several areas where future works may be able to improve upon our approach. Lay Description: What is already known about this topic: Open‐ended questions are used by teachers in the domain of mathematics to assess their students understanding but automated support for these types of questions are limited in online learning platforms. Recent advancements in areas of machine learning and natural language processing have led to promising results in a range of domains and applications. What this paper adds: Emulating how teachers identify similar student answers can be used to build tools that support teachers in assessing and providing feedback to student open‐ended work. Implications for practice: Developing better automated supports can increase the amount of direct feedback students receive to guide their learning. [ABSTRACT FROM AUTHOR]

Background: Formative assessments are needed to enable monitoring how student knowledge develops throughout a unit. Constructed response items which require learners to formulate their own free‐text responses are well suited for testing their active knowledge. However, assessing such constructed responses in an automated fashion is a complex task and requires the application of natural language processing methodology. In this article, we implement and evaluate multiple machine learning models for coding energy knowledge in free‐text responses of German K‐12 students to items in formative science assessments which were conducted during synchronous online learning sessions. Dataset: The dataset we collected for this purpose consists of German constructed responses from 38 different items dealing with aspects of energy such as manifestation and transformation. The units and items were implemented with the help of project‐based pedagogy and evidence‐centered design, and the responses were coded for seven core ideas concerning the manifestation and transformation of energy. The data was collected from students in seventh, eighth and ninth grade. Methodology: We train various transformer‐ and feature‐based models and compare their ability to recognize the respective ideas in students' writing. Moreover, as domain knowledge and its development can be formally modeled through knowledge networks, we evaluate how well the detection of the ideas within responses translated into accurate co‐occurrence‐based knowledge networks. Finally, in terms of the descriptive accuracy of our models, we inspect what features played a role for which prediction outcome and if the models pick up on undesired shortcuts. In addition to this, we analyze how much the models match human coders in what evidence within responses they consider important for their coding decisions. Results: A model based on a modified GBERT‐large can achieve the overall most promising results, although descriptive accuracy varies much more than predictive accuracy for the different ideas assessed. For reasons of comparability, we also evaluate the same machine learning architecture using the SciEntsBank 3‐Way benchmark with an English RoBERTa‐large model, where it achieves state‐of‐the‐art results in two out of three evaluation categories. Lay Description: What is already known about this topic?: Formative assessments are needed to test and monitor the development of learners' knowledge throughout a unit to provide them with appropriate automated feedback.Constructed response items which require learners to formulate their own free‐text responses are well suited for testing their active knowledge.Assessing constructed responses in an automated fashion is a widely researched topic, but the problem is far from solved and most of the work focuses on predicting holistic scores or grades.To allow for a more fine‐grained and analytic assessment of learners' knowledge, systems which go beyond predicting simple grades are required.To guarantee that models are stable and make their predictions for the correct reasons, methods for explaining the models are required. What this papers adds?: A core topic in physics education is the concept of energy.We implement and evaluate multiple systems based on natural language processing technology for assessing learners' conceptual knowledge about energy physics using transformer language models as well as feature‐based approaches.The systems assess students' knowledge about various forms of energy, indicators for the same and the transformation of energy from form into another.As our systems are based on machine learning methodology, we introduce a novel German short answer dataset for training them to detect the respective knowledge elements within students' free‐text responses.We evaluate the performance of these systems using this dataset as well as the well‐established SciEntsBand‐3‐Way dataset and manage to achieve, to our best knowledge, new state‐of‐the‐art results for the latter.Moreover, we apply methodology for explaining model predictions to assess whether predictions are carried out for the correct reasons. Implications for practice and/or policy: It is indeed possible to assess constructed responses for the demonstrated knowledge about energy physics in an analytic fashion using natural language processing.Transformer language models can outperform more specialized feature‐based approaches for this task in terms of predictive and descriptive accuracy.Co‐occurrences of different concepts within the same responses can lead models to learn undesired shortcuts which make them unstable. [ABSTRACT FROM AUTHOR]

Background: Assignments that involve writing based on several texts are challenging to many learners. Formative feedback supporting learners in these tasks should be informed by the characteristics of evolving written product and by the characteristics of learning processes learners enacted while developing the product. However, formative feedback in writing tasks based on multiple texts has almost exclusively focused on essay product and rarely included SRL processes. Objectives: We explored the viability of using product and process features to develop machine learning classifiers that identify low‐ and high‐performing essays in a multi‐text writing task. Methods: We examined learning processes and essay submissions of 163 graduate students working on an authentic multi‐text writing assignment. We utilised learners' trace data to obtain process features and state‐of‐the‐art natural language processing methods to obtain product features for our classifiers. Results and Conclusions: Of four popular classifiers examined in this study, Random Forest achieved the best performance (accuracy = 0.80 and recall = 0.77). The analysis of important features identified in the Random Forest classification model revealed one product (coverage of reading topics) and three process (elaboration/organisation, re‐reading and planning) features as important predictors of writing quality. Major Takeaways: The classifier can be used as a part of a future automated writing evaluation system that will support at scale formative assessment in writing tasks based on multiple texts in different courses. Based on important predictors of essay performance, a guidance can be tailored to learners at the outset of a multi‐text writing task to help them do well in the task. Lay Description: What is already known about this topic?: Both product and process features should be used to inform formative feedback on writing.Providing product‐ and process‐oriented feedback to learners is challenging.Automatic writing evaluation systems have mainly relied upon product features.Automated analysis of learners' trace data and their essay drafts is a promising venue. What this paper adds?: An accurate machine learning classifier that identifies low‐ and high‐scoring essays.The classifier utilized both product and process features.We obtained process features from learners' trace data in digital learning environment.We computed product features using state‐of‐the‐art text analytical methods. Implications for practice and/or policy: The classifier can be used as a part of a future automated writing evaluation system.We revealed learning processes and essay characteristics that influence performance.Based on important predictors of performance, formative feedback can be given to learners. [ABSTRACT FROM AUTHOR]

Background: Auditory–perceptual assessment of voice is a subjective procedure. Artificial intelligence with deep learning (DL) may improve the consistency and accessibility of this task. It is unclear how a DL model performs on different acoustic features. Aims: To develop a generalizable DL framework for identifying dysphonia using a multidimensional acoustic feature. Methods & Procedures: Recordings of sustained phonations of /a/ and /i/ were retrospectively collected from a clinical database. Subjects contained 238 dysphonic and 223 vocally healthy speakers of Chinese Mandarin. All audio clips were split into multiple 1.5‐s segments and normalized to the same loudness level. Mel frequency cepstral coefficients and mel‐spectrogram were extracted from these standardized segments. Each set of features was used in a convolutional neural network (CNN) to perform a binary classification task. The best feature was obtained through a five‐fold cross‐validation on a random selection of 80% data. The resultant DL framework was tested on the remaining 20% data and a public German voice database. The performance of the DL framework was compared with those of two baseline machine‐learning models. Outcomes & Results: The mel‐spectrogram yielded the best model performance, with a mean area under the receiver operating characteristic curve of 0.972 and an accuracy of 92% in classifying audio segments. The resultant DL framework significantly outperformed both baseline models in detecting dysphonic subjects on both test sets. The best outcomes were achieved when classifications were made based on all segments of both vowels, with 95% accuracy, 92% recall, 98% precision and 98% specificity on the Chinese test set, and 92%, 95%, 90% and 89%, respectively, on the German set. Conclusions & Implications: This study demonstrates the feasibility of DL for automatic detection of dysphonia. The mel‐spectrogram is a preferred acoustic feature for the task. This framework may be used for vocal health screening and facilitate automatic perceptual evaluation of voice in the era of big data. WHAT THIS PAPER ADDS: What is already known on this subject: Auditory–perceptual assessment is the current gold standard in clinical evaluation of voice quality, but its value may be limited by the rater's reliability and accessibility. DL is a new method of artificial intelligence that can overcome these disadvantages and promote automatic voice assessment. This study explored the feasibility of a DL approach for automatic detection of dysphonia, along with a quantitative comparison of two common sets of acoustic features. What this study adds to existing knowledge: A CNN model is excellent at decoding multidimensional acoustic features, outperforming the baseline parameter‐based models in identifying dysphonic voices. The first 13 mel‐frequency cepstral coefficients (MFCCs) are sufficient for this task. The mel‐spectrogram results in greater performance, indicating the acoustic features are presented in a more favourable way than the MFCCs to the CNN model. What are the potential or actual clinical implications of this work?: DL is a feasible method for the detection of dysphonia. The current DL framework may be used for remote vocal health screening or documenting voice recovery after treatment. In future, DL models may potentially be used to perform auditory–perceptual tasks in an automatic, efficient, reliable and low‐cost manner. [ABSTRACT FROM AUTHOR]

Background: Different types of assessments influence learning and learning behaviour. Multiple‐choice questions (MCQs) reward partial knowledge and encourage surface learning, while open‐ended questions (OEQs) promote deeper learning. Currently, MCQs is part of team‐based learning (TBL) curriculum, and it is challenging to implement OEQs as immediate feedback is necessary. Objectives: We asked if MCQ and OEQs affect student achievement, student learning and student engagement differently in a TBL classroom. Methods: MCQs and OEQs test scores of N = 66 students were automatically captured in Learning Activity Management System (LAMS) and were compared using a switching replications quasi‐experimental design with pre‐ and post‐tests to answer the research questions. Student learning approaches and engagement in the team activities were assessed using the study process questionnaire and the structure of observed learning outcomes taxonomy respectively. Results and Conclusions: Students get significantly higher MCQ scores than OEQs for the same set of questions, but the reverse is true for application exercises (AEs), which focus on higher‐level application. Most students significantly deepened their learning approaches before OEQs, while poorly prepared students were less engaged during OEQ discussions. Interestingly students subjected to OEQs took less time and scored higher in AE discussions, suggesting better focus on higher‐level thinking. Implications: This project is significant as it bridges our understanding of the value of OEQs and TBL. Our approach is transferable to other courses, and thus it can improve the quality of teaching and learning in tertiary education. Lay Description: What is already known about this topic?: Multiple‐choice questions (MCQs) do not promote deep learning as much as open‐ended questions (OEQ) do.Team‐based learning (TBL) requires MCQs because of the need for immediate feedback.Automated marking can make it more efficient to provide immediate feedback on OEQs as required by the TBL approach. What this paper adds?: OEQs changed student‐learning approaches in the TBL context.OEQs changed the quality of team discussions during the team‐readiness assessment.Test (tRAT) and application exercise phases.Students were concerned about the validity of automated grading. Implications for practitioners: Consider using OEQs in TBL context for deepening student learning.Consider how to defuse students' scepticism of new technology.Consider forming teams with a mix of deep and surface learners. [ABSTRACT FROM AUTHOR]

Developing knowledge‐transforming skills in writing may help students increase learning by actively building knowledge, regardless of the domain. However, many undergraduate students struggle to transform knowledge when drafting essays based on multiple sources. Writing analytics can be used to scaffold knowledge transforming as writers bring evidence to bear in supporting claims. We investigated how to automatically identify sentences representing knowledge transformation in argumentative essays. A synthesis of cognitive theories of writing and Bloom's typology identified 22 linguistic features to model processes of knowledge transforming in a corpus of 38 undergraduates' essays. Findings indicate undergraduates mostly paraphrase or copy information from multiple sources rather than engage deeply with sources' content. Eight linguistic features were important for discriminating evidential sentences as telling versus transforming source knowledge. We trained a machine learning algorithm that accurately classified nearly three of four evidential sentences as knowledge‐telling or knowledge‐transforming, offering potential for use in future research. Lay Description: What is already known about this topic: Engagement in knowledge transforming in multi‐source writing benefits learning.Post‐secondary writers rarely succeed in knowledge transforming.Computer‐generated formative feedback may promote knowledge‐transforming processes.Computational tools so far developed focused on a composition's rhetorical features only. What this paper adds: We proposed a novel methodology to identify knowledge transforming in argumentative essays.Computational approach involved both rhetorical and content characteristics of evidential sentences.Machine learning algorithm was developed to classify text as knowledge‐transforming versus telling.Eight linguistic features predicted evidential sentences as telling or transforming source knowledge. Implications for practice: There is a clear need to teach post‐secondary students tactics for knowledge transforming.The results can inform the development of writing analytics tool to scaffold knowledge‐transforming revisions. [ABSTRACT FROM AUTHOR]