Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

OBJECTIVE: To evaluate a new tool to assess emergency physicians' non-technical skills. METHODS: This was a multicentre observational study using data collected at four emergency departments in England. A proportion of observations used paired observers to obtain data for inter-rater reliability. Data were also collected for test-retest reliability, observability of skills, mean ratings and dispersion of ratings for each skill, as well as a comparison of skill level between hospitals. Qualitative data described the range of non-technical skills exhibited by trainees and identified sources of rater error. RESULTS: 96 assessments of 43 senior trainees were completed. At a scale level, intra-class coefficients were 0.575, 0.532 and 0.419 and using mean scores were 0.824, 0.702 and 0.519. Spearman's ρ for calculating test-retest reliability was 0.70 using mean scores. All skills were observed more than 60% of the time. The skill Maintenance of Standards received the lowest mean rating (4.8 on a nine-point scale) and the highest mean was calculated for Team Building (6.0). Two skills, Supervision & Feedback and Situational Awareness-Gathering Information, had significantly different distributions of ratings across the four hospitals (p<0.04 and 0.007, respectively), and this appeared to be related to the leadership roles of trainees. CONCLUSION: This study shows the performance of the assessment tool is acceptable and provides valuable information to structure the assessment and training of non-technical skills, especially in relation to leadership. The framework of skills may be used to identify areas for development in individual trainees, as well as guide other patient safety interventions.

Original publication




Journal article


Emerg Med J

Publication Date





437 - 443


Emergency medicine, assessment, leadership, non-technical skills, safety, training, Clinical Competence, Decision Making, Efficiency, Organizational, Emergency Medicine, England, Feedback, Humans, Interdisciplinary Communication, Leadership, Patient Care Team, Physicians, Psychometrics, Qualitative Research, Quality of Health Care, Reproducibility of Results, Social Behavior, Surveys and Questionnaires