Search

Competence‐based test development is a recent and innovative method for the construction of tests that are as informative as possible about the competence state (the set of skills an individual has available) underlying the observed item responses. It finds application in different contexts, including the development of tests from scratch, and the improvement or shortening of existing tests. Given a fixed collection of competence states existing in a population of individuals and a fixed collection of competencies (each of which being the subset of skills that allow for solving an item), the competency deletion procedure results in tests that differ from each other in the competencies but are all equally informative about individuals' competence states. This work introduces a streamlined version of the competency deletion procedure that considers information necessary for test construction only, illustrates a straightforward way to incorporate test developer preferences about competencies into the test construction process, and evaluates the performance of the resulting tests in uncovering the competence states from the observed item responses. [ABSTRACT FROM AUTHOR]

An intelligent tutoring system for learning basic statistics, called Stat-Knowlab, is presented and analyzed. The algorithms implemented in the system are based on the competence-based knowledge space theory, a mathematical theory developed for the formative assessment of knowledge and learning. The system's architecture consists of the two assessment and learning modules that interact with each other in a continuous exchange of information about the current knowledge state of a student. This allows the system to personalize the student's learning, providing only with the learning objects that she is ready to learn. During the browsing of the system, several types of navigation data are recorded. In this work, we analyzed data from two studies that were aimed at examining the learning processes induced by the navigation of the system. The results of both studies highlighted that the system is useful for monitoring the student learning processes during a university course of basic statistics.

The Gain-Loss model is a probabilistic skill multimap model for assessing learning processes. In practical applications, more than one skill multimap could be plausible, while none corresponds to the true one. The article investigates whether constraining the error probabilities is a way of uncovering the best skill assignment among a number of alternatives. A simulation study shows that this approach allows the detection of the models that are closest to the correct one. An empirical application shows that it allows the detection of models that are entirely derived from plausible assumptions about the skills required for solving the problems. (Contains 5 tables, 5 figures, and 2 footnotes.)

Within the theoretical framework of knowledge space theory, a probabilistic skill multimap model for assessing learning processes is proposed. The learning process of a student is modeled as a function of the student's knowledge and of an educational intervention on the attainment of specific skills required to solve problems in a knowledge domain. Model parameters are initial probabilities of the skills, effects of learning objects on gaining and losing the skills, and careless error and lucky guess probabilities of the problems. An empirical application shows that the model is effective in assessing knowledge and effectiveness of educational intervention at both classroom and student levels. Practical implications for teaching and learning are discussed.

In the Basic Local Independence Model (BLIM) of Doignon and Falmagne ("Knowledge Spaces," Springer, Berlin, 1999), the probabilistic relationship between the latent knowledge states and the observable response patterns is established by the introduction of a pair of parameters for each of the problems: a lucky guess probability and a careless error probability. In estimating the parameters of the BLIM with an empirical data set, it is desirable that such probabilities remain reasonably small. A special case of the BLIM is proposed where the parameter space of such probabilities is constrained. A simulation study shows that the constrained BLIM is more effective than the unconstrained one, in recovering a probabilistic knowledge structure.

Recent literature has pointed out that the basic local independence model (BLIM) when applied to some specific instances of knowledge structures presents identifiability issues. Furthermore, it has been shown that for such instances the model presents a stronger form of unidentifiability named empirical indistinguishability, which leads to the fact that the existence of certain knowledge states in such structures cannot be empirically tested. In this article the notion of indistinguishability is extended to skill maps and, more generally, to the competence‐based knowledge space theory. Theoretical results are provided showing that skill maps can be empirically indistinguishable from one another. The most relevant consequence of this is that for some skills there is no empirical evidence to establish their existence. This result is strictly related to the type of probabilistic model investigated, which is essentially the BLIM. Alternative models may exist or can be developed in knowledge space theory for which this indistinguishability problem disappears. [ABSTRACT FROM AUTHOR]