Background: Logbooks have been introduced in the competency-based undergraduate medical curriculum by the National Medical Commission in India. Assessments of competencies marked in logbooks serve as a useful guide to students for skill acquired and progress. Each student should get accurate and objective feedback.
Aims and Objectives: This study was planned to evaluate the effect of examiner variability on the grades awarded to learners in the logbook.
Materials and Methods: This cross-sectional study involved phase 1 MBBS students. As part of their routine practical exercise, students made thin peripheral blood smears, which were graded by 3 different examiners as exceeding (“E”), meeting (“M”), or below (“B”) depending on whether a particular smear E, M, or B was the expectancy of a particular examiner. Students in Grade B were supposed to repeat the exercise. Data for grading were recorded in Excel and analyzed using statistical software R for agreement between examiners.
Results: 248 phase 1 MBBS students participated in the study. The level of agreement between the three examiners was poor, meaning all examiners graded the student differently. Examiners 1, 2, and 3 were biased and inclined toward grades M, E, and B, respectively. Cramer’s V statistics confirmed the strong association (0.23, P < 0.01) of the examiner with a particular grade. A high level of agreement (0.407, P < 0.01) and strong association (0.26, P < 0.01) was found between the examiner and the grading of revision required (B) or not (E or M).
Conclusion: Thus, the grading system involves examiner bias, which may be reduced by defining more objective, observable criteria for grading and reducing the number of grade categories to reduce borderline cases.
Key words: Competency-based Medical Education; Logbook; Grading; Agreement; Examiners
|
