Reliability of Essay Type Questions — effect of structuring

The present study was designed to test the reliability of traditional essay type questions and to see the effect of ‘structuring’ on the reliability of those questions. Sixty‐two final MBBS students were divided into two groups of 31 each. Group A was administered a 2‐hour test containing five traditional essay questions picked up from previous university papers, while Group B was administered the same questions in a structured format. The answer sheets were xeroxed and evaluated independently by seven examiners. The dispersion of marks was significantly greater in Group A, as was the variance between marks awarded by the seven examiners to each student. Correlation of individual marks awarded was also poor for Group A scripts. The internal consistency for Group A was 0.31 (p > 0.05) while that for Group B was 0.69 (p < 0.05). Inter‐examiner agreement on ranks awarded was better for Group B. These findings suggest that reliability of traditional essay questions can be improved by structuring them.