Search

This study aims to compare three methods for translating ancient texts with sparse corpora: (1) the traditional statistical translation method of phrase alignment, (2) in-context LLM learning, and (3) proposed inter methodological approach - statistical machine translation method using sentence piece tokens derived from unified set of source-target corpus. The performance of the proposed approach in this study is 36.71 in BLEU score, surpassing the scores of SOLAR-10.7B context learning and the best existing Seq2Seq model. Further analysis and discussion are presented., Comment: ACL2024 submitted

Large Language Models (LLMs) have a great potential to serve as readily available and cost-efficient Conversational Intelligent Tutoring Systems (CITS) for teaching L2 learners of English. Existing CITS, however, are designed to teach only simple concepts or lack the pedagogical depth necessary to address diverse learning strategies. To develop a more pedagogically informed CITS capable of teaching complex concepts, we construct a BIlingual PEDagogically-informed Tutoring Dataset (BIPED) of one-on-one, human-to-human English tutoring interactions. Through post-hoc analysis of the tutoring interactions, we come up with a lexicon of dialogue acts (34 tutor acts and 9 student acts), which we use to further annotate the collected dataset. Based on a two-step framework of first predicting the appropriate tutor act then generating the corresponding response, we implemented two CITS models using GPT-4 and SOLAR-KO, respectively. We experimentally demonstrate that the implemented models not only replicate the style of human teachers but also employ diverse and contextually appropriate pedagogical strategies., Comment: ACL 2024

As the recent Large Language Models(LLM's) become increasingly competent in zero-shot and few-shot reasoning across various domains, educators are showing a growing interest in leveraging these LLM's in conversation-based tutoring systems. However, building a conversation-based personalized tutoring system poses considerable challenges in accurately assessing the student and strategically incorporating the assessment into teaching within the conversation. In this paper, we discuss design considerations for a personalized tutoring system that involves the following two key components: (1) a student modeling with diagnostic components, and (2) a conversation-based tutor utilizing LLM with prompt engineering that incorporates student assessment outcomes and various instructional strategies. Based on these design considerations, we created a proof-of-concept tutoring system focused on personalization and tested it with 20 participants. The results substantiate that our system's framework facilitates personalization, with particular emphasis on the elements constituting student modeling. A web demo of our system is available at http://rlearning-its.com., Comment: Accepted to ACM CHI 2024 LBW

In Korean ancient documents, there is no spacing or punctuation, and they are written in classical Chinese characters. This makes it challenging for modern individuals and translation models to accurately interpret and translate them. While China has models predicting punctuation and spacing, applying them directly to Korean texts is problematic due to data differences. Therefore, we developed the first models which predict punctuation and spacing for Korean historical texts and evaluated their performance. Our punctuation restoration model achieved an F1 score of 0.84, and Spacing model achieved a score of 0.96. It has the advantage of enabling inference on low-performance GPUs with less VRAM while maintaining quite high accuracy., Comment: 5 Pages, 2 Figures

Computerized Adaptive Testing (CAT) is a widely used, efficient test mode that adapts to the examinee's proficiency level in the test domain. CAT requires pre-trained item profiles, for CAT iteratively assesses the student real-time based on the registered items' profiles, and selects the next item to administer using candidate items' profiles. However, obtaining such item profiles is a costly process that involves gathering a large, dense item-response data, then training a diagnostic model on the collected data. In this paper, we explore the possibility of leveraging response data collected in the CAT service. We first show that this poses a unique challenge due to the inherent selection bias introduced by CAT, i.e., more proficient students will receive harder questions. Indeed, when naively training the diagnostic model using CAT response data, we observe that item profiles deviate significantly from the ground-truth. To tackle the selection bias issue, we propose the user-wise aggregate influence function method. Our intuition is to filter out users whose response data is heavily biased in an aggregate manner, as judged by how much perturbation the added data will introduce during parameter estimation. This way, we may enhance the performance of CAT while introducing minimal bias to the item profiles. We provide extensive experiments to demonstrate the superiority of our proposed method based on the three public datasets and one dataset that contains real-world CAT response data., Comment: CIKM 2023

A named entity recognition and classification plays the first and foremost important role in capturing semantics in data and anchoring in translation as well as downstream study for history. However, NER in historical text has faced challenges such as scarcity of annotated corpus, multilanguage variety, various noise, and different convention far different from the contemporary language model. This paper introduces Korean historical corpus (Diary of Royal secretary which is named SeungJeongWon) recorded over several centuries and recently added with named entity information as well as phrase markers which historians carefully annotated. We fined-tuned the language model on history corpus, conducted extensive comparative experiments using our language model and pretrained muti-language models. We set up the hypothesis of combination of time and annotation information and tested it based on statistical t test. Our finding shows that phrase markers clearly improve the performance of NER model in predicting unseen entity in documents written far different time period. It also shows that each of phrase marker and corpus-specific trained model does not improve the performance. We discuss the future research directions and practical strategies to decipher the history document., Comment: 7 pages, 9 figures

The paper develops a methodology to enable microscopic models of transportation systems to be accessible for a statistical study of traffic accidents. Our approach is intended to permit an understanding not only of historical losses, but also of incidents that may occur in altered, potential future systems. Through such a counterfactual analysis, it is possible, from an insurance, but also from an engineering perspective, to assess the impact of changes in the design of vehicles and transport systems in terms of their impact on road safety and functionality. Structurally, we characterize the total loss distribution approximatively as a mean-variance mixture. This also yields valuation procedures that can be used instead of Monte Carlo simulation. Specifically, we construct an implementation based on the open-source traffic simulator SUMO and illustrate the potential of the approach in counterfactual case studies.

We extend the scope of risk measures for which backtesting models are available by proposing a multinomial backtesting method for general distortion risk measures. The method relies on a stratification and randomization of risk levels. We illustrate the performance of our methods in numerical case studies.

Uncertainty requires suitable techniques for risk assessment. Combining stochastic approximation and stochastic average approximation, we propose an efficient algorithm to compute the worst case average value at risk in the face of tail uncertainty. Dependence is modelled by the distorted mix method that flexibly assigns different copulas to different regions of multivariate distributions. We illustrate the application of our approach in the context of financial markets and cyber risk.

The paper derives saddlepoint expansions for conditional expectations in the form of $\mathsf{E}[\overline{X} | \overline{\mathbf Y} = {\mathbf a}]$ and $\mathsf{E}[\overline{X} | \overline{\mathbf Y} \geq {\mathbf a}]$ for the sample mean of a continuous random vector $(X, {\mathbf Y}^\top)$ whose joint moment generating function is available. Theses conditional expectations frequently appear in various applications, particularly in quantitative finance and risk management. Using the newly developed saddlepoint expansions, we propose fast and accurate methods to compute the sensitivities of risk measures such as value-at-risk and conditional value-at-risk, and the sensitivities of financial options with respect to a market parameter. Numerical studies are provided for the accuracy verification of the new approximations., Comment: 36 pages, 5 figures