Search

Abstract. A significant problem that assessments of scientific reasoning face at the level of higher education is the question of domain generality, that is, whether a test will produce biased results for students from different domains. This study applied three recently developed methods of analyzing differential item functioning (DIF) to evaluate the domain generality assumption of a common scientific reasoning test. Additionally, we evaluated the usefulness of these new, tree- and lasso-based, methods to analyze DIF and compared them with methods based on classical test theory. We gave the scientific reasoning test to 507 university students majoring in physics, biology, or medicine. All three DIF analysis methods indicated a domain bias present in about one-third of the items, mostly benefiting biology students. We did not find this bias by using methods based on classical test theory. Those methods indicated instead that all items were easier for physics students compared to biology students. Thus, the tree- and lasso-based methods provide a clear added value to test evaluation. Taken together, our analyses indicate that the scientific reasoning test is neither entirely domain-general, nor entirely domain-specific. We advise against using it in high-stakes situations involving domain comparisons.

Suboptimal vitamin B status might affect cognitive performance in early childhood. We tested the hypothesis that short-term supplementation with folic acid and selected B vitamins improves cognitive function in healthy children in a population with relatively low folate status. We screened 1,002 kindergarten children for suboptimal folate status by assessing the total urinary para-aminobenzoylglutamate excretion. Two hundred and fifty low ranking subjects were recruited into a double blind, randomized, controlled trial to receive daily a sachet containing 220 μg folic acid, 1.1 mg vitamin B2, 0.73 mg B6, 1.2 μg B12 and 130 mg calcium, or calcium only for 3 months. Primary outcomes were changes in verbal IQ, short-term memory and processing speed between baseline and study end. Secondary outcomes were urinary markers of folate and vitamin B12 status, acetyl-para-aminobenzoylglutamate and methylmalonic acid, respectively, and, in a subgroup of 120 participants, blood folate and plasma homocysteine. Pre- and post-intervention cognitive measurements were completed by 115 children in the intervention and 122 in the control group. Compared to control, median blood folate increased by about 50 % (P for difference, P

Findings of studies on the unique effects of reasoning and working memory regarding complex problem solving are inconsistent. To find out if these inconsistencies are due to a lack of symmetry between the studies, we reconsidered the findings of three published studies on this issue, which resulted in conflicting conclusions regarding the inter-relations between reasoning, working memory, and complex problem solving. This was achieved by analysing so far unpublished problem solving data from the study of Bühner, Krumm, Ziegler, and Plücken (2006) (N= 124). One of the three published studies indicated unique effects of working memory and reasoning on complex problem solving using aggregated scores, a second study found no unique contribution of working memory using only figural scores, and a third study reported a unique influence only for reasoning using only numerical scores. Our data featured an evaluation of differences across content facets and levels of aggregation of the working memory scores. Path models showed that the results of the first study could not be replicated using content aggregated scores; the results of the second study could be replicated if only figural scores were used, and the results of the third study could be obtained by using only numerical scores. For verbal content, none of the published results could be replicated. This leads to the assumption that not only symmetry is an issue when correlating non-symmetrical data, but that content also has to be taken into account when comparing different studies on the same topic.

The Heidelberger Drogenbogen (HDB) is a German language assessment of substance-specific knowledge and consumption patterns of the illicit psychoactive substances cannabis, MDMA (ecstasy), amphetamines, cocaine, and hallucinogens. The behavior modules for each of these five drugs/drug groups allow for a diagnostic evaluation of the extent of harmful consumption behavior. Each of the five modules represents a single standardized test.This paper outlines several statistical parameters, Cronbach's alpha, retest reliabilities, as well as numerous validity and cut-off-criteria of the behavioral modules.Participants (N = 4,794) were recruited at schools, universities, in subcultural contexts, and in institutions of substance abuse treatment.Internal consistencies range from = .68 to .79 while test-retest reliabilities between .87 and .94 were found. The behavior modules of the HDB can discriminate between populations with and without clinical levels of substance use. Furthermore, this measure has incremental validity and higher diagnostic accuracy over competing measures.The behavior modules of the HDB are reliable and valid measures of substance use and misuse.

Publication bias remains a controversial issue in psychological science. The tendency of psychological science to avoid publishing null results produces a situation that limits the replicability assumption of science, as replication cannot be meaningful without the potential acknowledgment of failed replications. We argue that the field often constructs arguments to block the publication and interpretation of null results and that null results may be further extinguished through questionable researcher practices. Given that science is dependent on the process of falsification, we argue that these problems reduce psychological science’s capability to have a proper mechanism for theory falsification, thus resulting in the promulgation of numerous “undead” theories that are ideologically popular but have little basis in fact.

Since the implementation of the Bologna process, the quality of engineering study courses should be measured according to what extent academic courses of education facilitate and support the acquisition of engineering competences. So far it is not possible to assess this in a reliable and valid way. The foundation of all competence diagnostics is a model of the competences to be developed as well as the subsequent development and application of test instruments. This article theoretically derives a competence model as the basis for an assessment of learning results in engineering mechanics and places it within the current state of research of empirical educational research. Hypotheses on the competence structure will be derived and, for part of the model, statics, tested on the basis of data from a pilot study.