Optimizing CAT-ASVAB item selection using form assembly techniques

The Armed Services Vocational Aptitude Battery (ASVAB) is a test that approximately 700,000 students in 12,000 high schools take each year to determine military occupation placement. Form Assembly for the ASVAB refers to the selection of 20-35 questions, known as items, from an item pool of approximately 300 items to create a paper and pencil test in one of its ten topics. Previous research formulates form assembly as an Integer Linear Program (ILP). The current ASVAB mostly uses a Computer Adaptive Test (CAT), which estimates an examinee's ability after the examinee answers each item and selects the next item based on prior performance. The current CAT-ASVAB implementation does not control the number of items selected from each subject (taxonomy group) for a test. This thesis introduces ILPs, previously used for form assembly, that impose taxonomy restrictions and applies them to the CAT-ASVAB. We create four ILP variations and test them against the current method of item selection, by simulating 3,500 examinees (500 examinees each for seven given ability levels). The results show that all of the ILPs have acceptable solution times for CAT use, and taxonomy restrictions can be imposed while also having more even exposure rates (the number of times an item is administered divided by the number of examinees) than the current implementation of the CAT-ASVAB. A variation that relaxes most of the binary variables and constrains the difficulty of each item to be within a predetermined magnitude of the current ability estimate, performs the best in terms of item exposure (for both under and over-utilized items) and error between an examinee's estimated ability level and actual ability level. http://archive.org/details/optimizingcatasv109452777 Defense Manpower Data Center author (civilian). Approved for public release; distribution is unlimited.