Back to Search
Start Over
Setting Standards With Multiple-Choice Tests: A Preliminary Intended-User Evaluation of SmartStandardSet
- Source :
- Frontiers in Education, Vol 6 (2021)
- Publication Year :
- 2021
- Publisher :
- Frontiers Media S.A., 2021.
-
Abstract
- Software that easily helps higher education instructors to remove poor quality items and set appropriate grade boundaries is generally lacking. To address these challenges, the SmartStandardSet system provides a graphical-user interface for removing defective items, weighting student scores using a two-parameter model IRT score analysis, and a mechanism for standard-setting. We evaluated the system through a series of six interviews with teachers and six focus groups involving 19 students to understand how key stakeholders would view the use of the tool in practice. Generally, both groups of participants reported high levels of feasibility, accuracy, and utility in SmartStandardSet’s statistical scoring of items and score calculation for test-takers. Teachers indicated the data displays would help them improve future test items; students indicated the system would be fairer and would motivate greater effort on more difficult test items. However, both groups had concerns about implementing the system without institutional policy endorsement. Students specifically were concerned that academics may set grade boundaries on arbitrary and invalid grounds. Our results provide useful insights into the perceived benefits of using the tool for standard setting, and suggest concrete next steps for gaining wider acceptance that will be the focus of future work.
- Subjects :
- Focus (computing)
Higher education
Computer science
business.industry
pedagogical issues
Interface (computing)
Applied psychology
Education (General)
Focus group
Education
Test (assessment)
Weighting
human-computer interface
teaching/learning strategies
data science applications in education
post-secondary education
L7-991
business
Set (psychology)
Multiple choice
Subjects
Details
- Language :
- English
- Volume :
- 6
- Database :
- OpenAIRE
- Journal :
- Frontiers in Education
- Accession number :
- edsair.doi.dedup.....5a309db342b3567d175764ecac861583
- Full Text :
- https://doi.org/10.3389/feduc.2021.735088/full