Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

The Medical Students' Non-Technical Skills (Medi-StuNTS) is a behavioural marker scheme (BMS) designed to assess non-technical skills (NTS) in medical students in emergency simulations. This study aimed to assess the evidence for validity and usability of Medi-StuNTS by naive, near-peer educators. Nine doctors assessed four students in simulations of common medical emergencies. The scores were used to assess inter-rater reliability, inter-class correlation, and observability. Students and assessors completed questionnaires that assessed the tool's usability and consequence. Inter-rater agreement across all skill elements was "high" with rWG scores >0.8. An inter-class correlation was "good" with ICC3K kappa scores of 0.86 and 0.89 overall, when measured per simulation and per skills element respectively. Overall skill observability was high (>80%) except for coping with stress. Assessors found the tool "difficult to use" but "useful for feeding back in a constructive way". Students appreciated the comprehensiveness of the feedback as well as knowing what to expect during debriefs. This study has shown that the Medi-StuNTS BMS has good usability and evidence of validity in naive assessors and near-peer educators. It shows the particularly good internal structure and overall beneficial consequences. Further study will be necessary to understand how best to deploy it in formative and summative contexts.

Original publication




Journal article



Publication Date





human factors, medical school education, non-technical skills, skills and simulation training, validation study