Search

A novel course concept to enable master's degree students to acquire research skills is described as an experience report. A hybrid, inquiry-based format was developed to simulate each phase of writing and presenting a research paper at a scientific conference. Students self-organized into groups to conduct each phase of the research process to conduct a research project for a scientific topic, bionic computation. Weekly group meetings with the instructor were conducted via video conference. Students learned to formulate research questions, write abstracts, acquire experimental data and evaluate open-source libraries and tools to build their own experimental models. An in-house, mini-conference was held in person, in which each group held a presentation of their work. At the end of the semester, a retrospective was held to reflect on students' experiences and summarize the lessons they have learned. After the course, slight majority of the students stated that they could imagine writing a paper about their master's theses and taking part in a real conference. [For the full proceedings, see ED636095.]

Worked examples, which present an explained code for solving typical programming problems are among the most popular types of learning content in programming classes. Most approaches and tools for presenting these examples to students are based on line-by-line explanations of the example code. However, instructors rarely have time to provide explanations for many examples typically used in a programming class. In this paper, we assess the feasibility of using LLMs to generate code explanations for passive and active example exploration systems. To achieve this goal, we compare the code explanations generated by chatGPT with the explanations generated by both experts and students. [This paper was published in: "Proceedings of Machine Learning Research" (2024).]

Teaching robotics courses online is challenging due to the complexity of the interdisciplinary topics involved. One of the most challenging topics is 3D coordinate transformations. Students often struggle to grasp the concept of 3D coordinate transformations and their relevance to real-world robotic applications. This paper applies the Scholarship of Teaching and Learning methodology to address this challenge and shares the self-evaluation assignments given to students to gradually enhance their ability to solve a real-world robotic navigation problem -- a crucial skill required in almost all robotic applications. Each assignment includes an informative description that explains the purpose of the task and its connection to the next assignment. Manageable MATLAB resources are provided in each assignment, allowing students to study fundamental Matlab scripts and use MATLAB Grader for self-evaluation before submission. The instructor provides feedback on incorrect answers through MATLAB Grader. The assignments focus on problem-solving and can be automatically graded, building upon prior work done by the instructor. The paper will also detail strategies for motivating students to engage with these challenging assignments and how the instructor assists distracted students in catching up with any missing assignments. [For the full proceedings, see ED656038.]

This paper discusses the steps involve in project development for developing the mobile application, namely Blood Bank Application and developing the convertor for software testing. The project development is important for Computer Science students for them to learn the important steps in developing the application and testing the reliability of the application. The first step involved is the development of the mobile application. Then the convertor is developed to convert the mobile application in ".apk" format into Java program in ".java" format. The java program is then tested under Eclipse environment using JUnit. Finally, the Java program is tested for its capability of generating test cases using SenaTLSParser. The comparison of the results of the time taken to produce test cases is presented using Junit and SenaTLSParser. Based on the results, SenaTLSParser is more reliable compared with JUnit since its response time is less than JUnit. The whole steps involved in this project development are discussed in this paper. [For the complete proceedings, see ED655360.]

Knowledge components (KCs) have many applications. In computing education, knowing the demonstration of specific KCs has been challenging. This paper introduces an entirely data-driven approach for: (1) discovering KCs; and (2) demonstrating KCs, using students' actual code submissions. Our system is based on two expected properties of KCs: (1) generate learning curves following the power law of practice; and (2) are predictive of response correctness. We train a neural architecture (named KC-Finder) that classifies the correctness of student code submissions and captures problem-KC relationships. Our evaluation on data from 351 students in an introductory Java course shows that the learned KCs can generate reasonable learning curves and predict code submission correctness. At the same time, some KCs can be interpreted to identify programming skills. We compare the learning curves described by our model to four baselines, showing that: (1) identifying KCs with naive methods is a difficult task; and (2) our learning curves exhibit a substantially better curve fit. Our work represents a first step in solving the data-driven KC discovery problem in computing education. [For the complete proceedings, see ED630829.]

Physical programming (PP) is defined as the design of interactive physical systems between the real and virtual worlds using a combination of software and hardware. A physical programming platform (PPP) defines a structure that includes hardware and software where such applications can be made. The aim of this study is to introduce the MIDEP computer card, which can be used at all levels of education, does not require a computer and can be used as a computer, and to evaluate its usability in physical programming training. In the study, which was designed with a one-group pretest-posttest experimental model, the evaluation process was carried out in two stages with two different study groups. The study group of the application consisted of teacher candidates in the first stage and K12 level students in the second stage. In data collection, academic achievement tests prepared separately for pre-service teachers and K12 level students and focus group interviews with students were utilised. The findings show that the training given by using MIDEP card and MIDEP workbook increased the academic achievement of the students in physical programming at a statistically significant level. In addition, as a result of the findings, it can be said that the projects and activities included in the training exhibited a balanced distribution in terms of the gains related to the sub-dimensions of hardware, software and Internet of Things, and that the tools and materials used allowed students without any experience to learn gradually from easy to difficult. [This paper was published in: "EJER Congress 2023 International Eurasian Educational Research Congress Conference Proceedings," Ani Publishing, 2023, pp. 570-590.]

The paper explores current learning approaches. The authors present the Trigger-Based Discussion-Oriented Continuous learning model (TbDoC) that focuses on creating a continuous learning experience over the online and off-line (on-campus) learning environments. The model aims to create a more engaging learning environment that encourages discussion and increases the ability of students to be independent learners. The proposed learning model utilizes an adapted version of the Hook model, the simplification aspect of the Fogg behavior model, and a socio-educational approach to learning. The model starts with the lecturer sending the students an email on the night prior to the lecture. The email contains a set of questions related to the lecture material. The students are prompted to search for answers to the questions and in doing so they are preparing for the lecture. On the next day during the lecture, the lecturer begins to direct the student-to-lecturer discussion. The lecture material is fully uncovered by the students with the help of the lecturer through the discussion. The qualitative behaviour observations showed positive impact of the model on the students and the learning environment. It was observed that students became more independent learners and were more motivated to engage in the discussions. [For the full proceedings, see ED636095.]

Due the wideness of the term Physical Computing, there is a need for a better structure of this topic. This paper is about an approach of structuring this field by finding attributes of different physical computing devices which can be used in class. Those attributes are meant to enforce teachers as well as researchers to analyze different devices from a pedagogical perspective in order to use those devices in class and use their full educational potential. [For the full proceedings, see ED636095.]

A typical first computer science course (CS1) introduces the student to coding conventions, variables, methods, control structures, conditionals, and the semantics of classes and objects. Advanced concepts of inheritance, polymorphism, abstract classes, interfaces, and their use in the design process, are covered in a second-level course (CS2). CS2 concepts are abstract, requiring reinforcement through considerable practice. It has been observed that traditional CS2 projects fail to capture the imagination and enthusiasm of students and are seldom useful past the end of the semester, yet interesting projects drawn from the natural sciences may be either too complex or too algorithmic to facilitate the required design experience. Game programming, in contrast, is purpose-driven and has great appeal. Unfortunately, popular game engines hide the engine's complexity and provide too much built-in functionality, relegating the user to writing glue logic in a scripting language. What is needed instead is a challenge that will provide transferable skills for solving generic problems using a statically typed language. The authors of this paper describe a Java game engine and lesson plan they developed for one semester of object-oriented instruction for students who have completed CS2. Early anecdotal results demonstrate that students find the approach challenging, informative, and incentivizing. [For the full proceedings, see ED656038.]

Career development is an important aspect of every young person's life as it helps them identify their skills, interests and goals and find the right career path for themselves. Career skills are a broad range of competencies and abilities that are necessary for success in the workplace. Developing them through education, training and experience can help students identify them and achieve their career goals. Design thinking is a powerful approach for this purpose. With its components-problem solving, innovation and creativity, collaboration and teamwork, iterative approaches, and empathy-design thinking can help students develop the skills they need to succeed in a wide range of professional fields. The specificity of the hackathon as a form of organization further contributes to building these essential skills that are directly related to students' career development. This paper explores the possibilities of forming career skills in students through design thinking hackathons. [For the complete proceedings, see ED655360.]

The use of technology as a facilitator in learning environments has become increasingly prevalent with the global pandemic caused by COVID-19. As such, computer-supported collaborative learning (CSCL) gains a wider adoption in contrast to traditional learning methods. At the same time, the need for automated tools capable of assessing and stimulating collaboration between participants has become more stringent, as human monitoring of the increasing volume of conversations becomes overwhelming. This paper introduces a method grounded in dialogism for evaluating students' involvement in chat conversations based on semantic chains computed using language models. These semantic chains reflect emergent voices from dialogism that span and interact throughout the conversation. Our integrated method uses contextual information captured by BERT transformer models to identify links in a chain that connects semantically related concepts from a voice uttered by one or more participants. Two types of visualizations were generated to depict the longitudinal propagation and the transversal inter-animation of voices within the conversation. In addition, a list of handcrafted features derived from the constructed chains and computed for each participant is introduced. Several machine learning algorithms were tested using these features to evaluate the extent to which semantic chains are predictive of student involvement in chat conversations. [This paper was published in: "Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education, Proceedings of the 6th International Conference on Smart Learning Ecosystems and Regional Development," edited by Ó. Mealha et al., Springer Nature Singapore Pte Ltd., 2022, pp. 67-78.]

In this paper, we propose a novel approach to combine domain modelling and student modelling techniques in a single, automated pipeline which does not require expert knowledge and can be used to predict future student performance. Domain modelling techniques map questions to concepts and student modelling techniques generate a mastery score for a concept. We conducted an evaluation using six large datasets from a Python programming course, evaluating the performance of different domain and student modelling techniques. The results showed that it is possible to develop a successful and fully automated pipeline which learns from raw data. The best results were achieved using alternating least squares on hill-climbing Q-matrices as domain modelling and exponential moving average as student modelling. This method outperformed all baselines in terms of accuracy and showed excellent run time. [For the full proceedings, see ED623995.]

Collaborative learning is a complex process during which two or more learners exchange opinions, construct shared knowledge, and solve problems together. While engaging in this interactive process, learners' satisfaction toward their partners plays a crucial role in defining the success of the collaboration. If intelligent systems could predict peer satisfaction early during collaboration, they could intervene with adaptive support. However, while extensive studies have associated peer satisfaction with factors such as social presence, communication, and trustworthiness, there is no research on automatically predicting learners' satisfaction toward their partners. To fill this gap, this paper investigates the automatic prediction of peer satisfaction by analyzing 44 middle school learners' interactions during collaborative coding tasks. We extracted three types of features from dialogues: (1) linguistic features indicating semantics; (2) acoustic-prosodic features including energy and pitch; and (3) visual features including eye gaze, head pose, facial behaviors, and body pose. We then trained several regression models to predict the peer satisfaction scores that learners received from their partners. The results revealed that head position and body location were significant indicators of peer satisfaction: lower head and body distances between partners were associated with more positive peer satisfaction. This work is the first to investigate the multimodal prediction of peer satisfaction during collaborative problem solving, and represents a step toward the development of real-time intelligent systems that support collaborative learning. [For the full proceedings, see ED623995.]

In an Intelligent Tutoring System (ITS), problem (or question) difficulty is one of the most critical parameters, directly impacting problem design, test paper organization, result analysis, and even the fairness guarantee. However, it is very difficult to evaluate the problem difficulty by organized pre-tests or by expertise, because these solutions are labor-intensive, time-consuming, leakage-prone, or subjective in some way. Thus, it is of importance to automatically evaluate problem difficulty via information technology. To this end, we propose a novel difficulty prediction framework, named SQL-DP, for Structured Query Language (SQL) programming problems, mastering which plays a vital role in learning the database technology. In SQL-DP, semantic features of problem stems and structure features of problem answers in the form of SQL codes are both computed at first, using the NLP and the neural network techniques. Then, these features are used as the input to train a difficulty prediction model with the statistic error rates in tests as the training labels, where the whole modeling does not introduce any experts, some as knowledge labeling. Finally, with the trained model, we can automatically predict the difficulty of each SQL programming problem. Moreover, SQL programming problem answering log data of hundreds of undergraduates from Guangxi University of China are collected, and the experiments conducted on the collected log data demonstrate the propped SQL-DP framework outperforms the state-of-the-art solutions apparently. In particular, SQL-DP decreases the RMSE of difficulty prediction by at most 7.23%, compared with the best-related framework. [For the full proceedings, see ED623995.]

This work explores how population-based engagement prediction can address cold-start at scale in large learning resource collections. The paper introduces: (1) VLE, a novel dataset that consists of content and video based features extracted from publicly available scientific video lectures coupled with implicit and explicit signals related to learner engagement; (2) two standard tasks related to predicting and ranking context-agnostic engagement in video lectures with preliminary baselines; and (3) a set of experiments that validate the usefulness of the proposed dataset. Our experimental results indicate that the newly proposed VLE dataset leads to building context-agnostic engagement prediction models that are significantly performant than ones based on previous datasets, mainly attributing to the increase of training examples. VLE dataset's suitability in building models towards Computer Science/ Artificial Intelligence education focused on e-learning/MOOC use-cases is also evidenced. Further experiments in combining the built model with a personalising algorithm show promising improvements in addressing the cold-start problem encountered in educational recommenders. This is the largest and most diverse publicly available dataset to our knowledge that deals with learner engagement prediction tasks. The dataset, helper tools, descriptive statistics and example code snippets are available publicly. [For the full proceedings, see ED623995.]

The recent trend of embedding source code for machine learning applications also enables new opportunities in learning analytics in programming education, but which code embedding approach is most suitable for learning analytics remains an open question. A common approach to embedding source code lies in extracting syntactic information from a program's syntax tree and learning to merge these into continuous distributed vectors (e.g., CODE2VEC). CODE2VEC has been predominantly investigated in the context of professional programming languages, but learning analytics are particularly important in the context of educational programming languages such as SCRATCH. In this paper, we therefore instantiate the popular embedding approach CODE2VEC for SCRATCH programs, create three different classification tasks with corresponding datasets, and empirically evaluate CODE2VEC on them. Our experiments demonstrate that a transfer of CODE2VEC to the educational environment of SCRATCH is feasible. Our findings serve as a basis to apply code embeddings to further educational tasks such as automated detection of misconceptions of programming concepts in SCRATCH programs. [For the full proceedings, see ED623995.]

The paper shares the results of the process of forming a cross-disciplinary collaborative team and using a user-centered design framework to co-create a self-directed, self-paced, personalized, flexible learning environment as an alternative approach to preparing PK-8 preservice teachers to teach computational thinking. It is part of a larger project that attempts to respond to the need to create a computer science curriculum within a credentialing learning environment.

This study aims to explore an educational learning environment that supports students to learn conceptual modelling with the unified modelling language (UML). In this study, we call the describing models "UML programming." In this paper, we show an educational UML programming environment for science, technology, engineering, art, and mathematics (STEAM) related subjects (especially for Technology or Engineering) in schools, which are able to apply from elementary school to university. At first, we explain why, what, and how doing the UML programming. In this study, we use a state machine diagram for UML programming. To draw this model, the students just put some states in rectangular shape and transitions in arrow shape. Two types of educational model notations in state machine diagram are introduced. Then, some advanced functions of the SRPS are described. They are an educational model editor, and management of users, learning tasks and models submitted by students. Next, two case studies with the SRPS are shown. One case study is adopted to the summer camp for 5th- and 6th-grade students. The participants were 20 students and were engaged in 4 hours workshop. We used a car-type robot with two DC motors, a one-touch sensor, and one infrared sensor connected to a micro:bit. The other case study is a formal technology class at one Japanese public junior high school. One teacher and five classes worked on UML programming for traffic lights. Each class had 20 9th-grade students. One student at this school was given a traffic light robot with three Light-emitting diode (LED) lights, a one-touch sensor, and one infrared sensor connected to a micro:bit. Finally, on the basis of these two case studies using our environment, we discuss the potential for innovative STEAM education with UML programming.

This paper proposes the design of an ontology of multiple programming languages and give three examples to show the methodology. Our ontology aims to summarize the core of computational thinking logic by elaborating the concepts of three object-oriented programming languages in the industry: Python, Java, and C#. Therefore, the construction of the ontology lies not only on these three programming languages but also on their common concepts. This kind of ontology design facilitates the ontology extension and merging when concepts of other programming languages are added in the future. This ontology could be used to not only provide a guideline for any research work focusing on computational thinking education but also describe the common concept of visual programming tools and existing programming languages. Based on this ontology, an adaptive tutoring system intended to provide learners with personalized programming exercises, is under development.

As we talked with school teachers about the use of physical computing systems in class, one major drawback of these systems became obvious: almost nobody--next to enthusiastic autodidacts--has been able to tell us on an abstract basis for which educational purposes a certain physical computing system can be used, i.e. in a goal-oriented way. This insight lead to the approach to structure the field of physical computing systems with the overall goal to make it easier for teachers to improve their lectures due integrating physical computing systems into computer science education. The aim of this paper is to structure the field, to find categories and to show how existing devices can be integrated into the founded structure.

The automatic code generation is the process of generating source code snippets from a program, i.e., code for generating code. Its importance lies in facilitating software development, particularly important is helping in the implementation of software designs such as engineering diagrams, in such a case, automatic code generation copes with the problem of how to obtain code from a graphic representation, for instance an UML diagram or a Relational Diagram. Some advantages of automatic code generation are: a) to obtain the source code more quickly and to do it with lower margins of error; b) it is promising to be applied in teaching contexts, whilst provide instructors with a tool to teach, the expected results of assignments can be assessed by comparing the results of students and the automatic generated code. Furthermore, one of the most frequently tasks in classrooms when teaching relational databases is the design of Entity-Relationship Diagrams which eventually become SQL code. The manual transition from an Entity-Relationship Diagram to SQL code is a time-consuming process and requires of a skilled eye to be successfully performed. In this paper, we present "DerSql," an extension of the DIA Diagrammer, a well-known free software engineering tool, to automatically generate SQL code from an Entity-Relationship Diagrams. The results are tested for the case of 1 -- 1 and 1 -- n arities relationships. We consider that "DerSql" represents a remarkable tool for teaching while it is a promising advance in developing DIA as a 4th Generation software engineering application. [For the full proceedings, see ED638044.]

This paper systematically explores how Large Language Models (LLMs) generate explanations of code examples of the type used in intro-to-programming courses. As we show, the nature of code explanations generated by LLMs varies considerably based on the wording of the prompt, the target code examples being explained, the programming language, the temperature parameter, and the version of the LLM. Nevertheless, they are consistent in two major respects for Java and Python: the readability level, which hovers around 7-8 grade, and lexical density, i.e., the relative size of the meaninful words with respect to the total explanation size. Furthermore, the explanations score very high in correctness but less on three other metrics: completeness, conciseness, and contextualization. [This paper is in: Proceedings of the workshop on Generative AI for Education(GAIED): Advances, Opportunities, and Challenges, 2003.]

Is there a pattern in how students solve Parsons puzzles? Is there a difference between the puzzle-solving strategies of C++ and Java students? We used Markov transition matrix to answer these questions. We analyzed the solutions of introductory programming students solving Parsons puzzles involving if-else statements and while loops in C++ and Java from fall 2016 to fall 2020. We present the results of our analysis qualitatively as heat maps and quantitatively using descriptive statistics. We found that most students solved the puzzles in the order in which lines appeared in the correct solution. Counter-intuitively, we found this pattern even in the solutions of the puzzles involving nested if-else statements, multiple while loops and nested while loops. Students who solved the puzzles with the fewest actions acted upon fewer lines out of order, i.e., not in the order in which they appear in the final solution. Whenever we found a statistically significant difference between C++ and Java solutions, C++ solutions involved fewer out-of-order and redundant actions than Java solutions. We discuss the implications of these results for the use of Parsons puzzles as a tool for teaching introductory programming. [For the complete proceedings, see ED630829.]

In MOOCs for programming, Automated Testing and Feedback (ATF) systems are frequently integrated, providing learners with immediate feedback on code assignments. The analysis of the large amounts of trace data collected by these systems may provide insights into learners' patterns of utilizing the automated feedback, which is crucial for the design of effective tools and maximizing their potential to promote learning. However, data-driven research on the impact of ATF on learning is scarce, especially in the context of MOOCs. In the current study, we combine a theoretical framework of feedback with educational data mining methods to investigate the effect of feedback characteristics on learning behavior in a MOOC. Sequence pattern analysis is implemented to explore and visualize the actions taken by learners in response to feedback which composed of cognitive, meta-cognitive, and motivational elements. We applied our research approach in an empirical design which consists of five cohorts (total over 2200 learners) utilizing different versions of ATF. The findings suggest that learners tend to adopt learning strategies in response to feedback and exhibit a preference for utilizing example solutions, while still coping with the challenge of solving the assignments independently. The impact of feedback function, content and structure is discussed in light of a detailed view of the differences as well as common trends in learning paths. Allowing for fine-grained insights, we found our research approach contributes to a more comprehensive understanding of the effect of automated feedback characteristics in MOOCs for programming. [For the complete proceedings, see ED630829.]

Improving competence requires practicing, e.g. by solving tasks. The Self-Assessment task type is a new form of scalable online task providing immediate feedback, sample solution and iterative improvement within the newly developed SAFRAN plugin. Effective learning not only requires suitable tasks but also their meaningful usage within the student's learning process. So far, learning processes of students working on such Self-Assessment tasks have not been studied. Thus, SAFRAN was extended with activity logging allowing process mining. SAFRAN was used in a first-year computer science university course. Students' behavior was clustered and analyzed using log data. 3 task completion behavior patterns were identified indicating positive, neutral or negative impact on task processing. Differences in the use of feedback and sample solutions were also identified. The results are particularly relevant for instructors who can tailor adaptive feedback content better to its target group. The analytics approach described may be useful for researchers who want to implement and study adaptive and personalized task processing support. [For the complete proceedings, see ED630829.]

Automatically identifying struggling students learning to program can assist teachers in providing timely and focused help. This work presents a new deep-learning language model for predicting "bug-fix-time", the expected duration between when a software bug occurs and the time it will be fixed by the student. Such information can guide teachers' attention to students most in need. The input to the model includes snapshots of the student's evolving software code and additional meta-features. The model combines a transformerbased neural architecture for embedding students' code in programming language space with a time-aware LSTM for representing the evolving code snapshots. We evaluate our approach with data obtained from two Java development environments created for beginner programmers. We focused on common programming errors which differ in their difficulty and whether they can be uniquely identified during compilation. Our deep language model was able to outperform several baseline models that use an alternative embedding method or do not consider how the programmer's code changes over time. Our results demonstrate the added value of utilizing multiple code snapshots to predict bug-fix-time using deep language models for programming. [For the complete proceedings, see ED630829.]

Problem decomposition into sub-problems or subgoals and recomposition of the solutions to the subgoals into one complete solution is a common strategy to reduce difficulties in structured problem solving. In this study, we use a datadriven graph-mining-based method to decompose historical student solutions of logic-proof problems into Chunks. We design a new problem type where we present these chunks in a Parsons Problem fashion and asked students to reconstruct the complete solution from the chunks. We incorporated these problems within an intelligent logic tutor and called them Chunky Parsons Problems (CPP). These problems demonstrate the process of problem decomposition to students and require them to pay attention to the decomposed solution while they reconstruct the complete solution. The aim of introducing CPP was to improve students' problem-solving skills and performance by improving their decomposition-recomposition skills without significantly increasing training difficulty. Our analysis showed that CPPs could be as easy as Worked Examples (WE). And, students who received CPP with simple explanations attached to the chunks had marginally higher scores than those who received CPPs without explanation or did not receive them. Also, the normalized learning gain of these students shifted more towards the positive side than other students. Finally, as we looked into their proof-construction traces in posttest problems, we observed them to form identifiable chunks aligned with those found in historical solutions with higher efficiency. [For the complete proceedings, see ED630829.]

Large language models (LLMs), such as Codex, hold great promise in enhancing programming education by automatically generating feedback for students. We investigate using LLMs to generate feedback for fixing syntax errors in Python programs, a key scenario in introductory programming. More concretely, given a student's buggy program, our goal is to generate feedback comprising a fixed program along with a natural language explanation describing the errors/fixes, inspired by how a human tutor would give feedback. While using LLMs is promising, the critical challenge is to ensure high precision in the generated feedback, which is imperative before deploying such technology in classrooms. The main research question we study is: "Can we develop LLMs-based feedback generation techniques with a tunable precision parameter, giving educators quality control over the feedback that students receive?" To this end, we introduce PyFiXV, our technique to generate high-precision feedback powered by Codex. The key idea behind PyFiXV is to use a novel run-time validation mechanism to decide whether the generated feedback is suitable for sharing with the student; notably, this validation mechanism also provides a precision knob to educators. We perform an extensive evaluation using two real-world datasets of Python programs with syntax errors and show the efficacy of PyFiXV in generating high-precision feedback. [For the complete proceedings, see ED630829.]

Visualizations have become more complex in recent years so that users need to learn how to use these systems. Onboarding systems can help users to learn how to interact with a visualization efficiently. Simple tutorials are often not sufficient. There are still open questions on how to design such onboarding systems. So far, a few usability studies have been conducted to clarify this issue. We decided to adopt an approach using sketching. We conducted a study with 29 participants. Possible users of onboarding systems were asked to develop prototypes of an onboarding system based on sketches. The process of developing these prototypes and the results were analyzed in a qualitative study. We can show that onboarding systems should follow a step-by-step approach and be integrated into the visualization. In addition, it seems that questions, tasks, and examples should be part of an onboarding system, so that users can try out how to use the system on their own. [For the full proceedings, see ED636095.]