Assessment of Ethics and Social Justice Aspects in Data Science and Artificial Intelligence.

This work aims to develop a set of materials and tools, both quantitative and qualitative, for two purposes: First, for the assessment of ethical and social justice (ESJ) considerations in research projects, and second, as a pedagogical toolkit that allows users to improve their understanding of these aspects of data ethics. Below we describe three existing assessment methodologies for evaluating ESJ in data science research projects: a scoring rubric, a questionnaire, and a canvas sheet (i.e., a user-friendly template and tool that captures data), and we propose one additional method, a predictive machine learning model. This document describes an evaluation of the feedback from 124 students in two different classes who used the questionnaire and canvas sheet to assess their team projects. This data set is also being used to test a proof of concept for the machine learning model. Our emphasis at this stage is to improve the instruments, with a quantitative analysis of the numerical and scale-based responses, and a qualitative evaluation of the text-based suggestions from participants. The primary insights from this first round of evaluations indicate that students showed no strong preference between the questionnaire and the canvas sheet, with slight advantages on "Perspective" and "Further Research" for the canvas sheet, and a similar advantage for "Group Discussion" for the questionnaire. [ABSTRACT FROM AUTHOR]