Assessment of individual competence
A sequential mixed model 1
Aims: the assessment of individual competence in medical education is about finding a balance between having sufficient resources to make valid and reliable decisions and not using more resources than necessary. Sequential assessment, in which more resources are used for borderline performing candidates than for poorly or clearly satisfactorily performing candidates, can be used to achieve that balance. Although sequential assessment is commonly associated with larger groups of candidates to be assessed, in many practical settings numbers of candidates may be small.
Objective: this article presents a single case design with a statistical model for the assessment of individual competence that can be used regardless of the number of candidates.
Method: a worked example of a solution that can be used for an individual candidate, using simulated data in the zero-cost Open Source statistical program R version 4.0.5., is provided.
Results: the aforementioned solution provides statistics that can be used to make pass/fail decisions at the level of the individual candidate as well as to make decisions regarding the length and timing of an exam (or parts thereof) for the individual candidate.
Conclusion: the solution provided can help to reduce resources needed for assessment to a considerable extent while maximizing resources for borderline candidates. This facilitates both decision making and cost reduction in assessment.
Mancuso G, Strachan S, Capey S. Sequential testing in high stakes OSCE: a stratified cross-validation approach [Internet]. MedEdPublish. 2019;1-20. https://doi.org/10.15694/mep.2019.000132.1
Leppink J. The art of modelling the learning process: uniting educational research and practice. Cham: Springer; 2020. https://doi.org/10.1007/978-3-030-43082-5
Maric M, Van der Werff V. Single-case experimental designs in clinical intervention research. In: Van de Schoot R, Milocević (M Editors). Small sample size solutions: A guide for applied researchers and practitioners. New York: Routlege; 2020 [cited 2021 Sep 06]. p. 102-11. Available from: https://library.oapen.org/bitstream/handle/20.500.12657/22385/9780367221898_text%20(1).pdf?sequence=1#page=116
Pinheiro J, Bates D, DebRoy S, Sarkar D, Team RC. nlme: Linear and nonlinear mixed effects models. R Package Ver. 2013;3:111.
R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing (version 4.0.5); [cited 2021 Sep 06]. Available from: https://www.r-project.org
Copyright (c) 2021 Scientia Medica
This work is licensed under a Creative Commons Attribution 4.0 International License.
The submission of originals to Scientia Medica implies the transfer by the authors of the right for publication. Authors retain copyright and grant the journal right of first publication. If the authors wish to include the same data into another publication, they must cite Scientia Medica as the site of original publication.
Creative Commons License
Except where otherwise specified, material published in this journal is licensed under a Creative Commons Attribution 4.0 International license, which allows unrestricted use, distribution and reproduction in any medium, provided the original publication is correctly cited.