Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

STUDY OBJECTIVE: Nontechnical skills are "the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance." Our research team developed and evaluated the task of developing and validating a behavioral marker system for the observational assessment of emergency physicians' nontechnical skills. METHODS: The development of the tool was divided into 3 phases and used triangulation of data from a number of sources. During phase 1, a provisional assessment tool was developed according to published literature and curricula. Phase 2 used analysis of staff interviews and field observations to determine whether the skill list contained any significant omissions. These studies were also used to identify behavioral markers linked to nontechnical skills in the context of the emergency department (ED) and establish whether skills included in the tool were observable. Phase 3 involved evaluating the content validity index of exemplar behaviors, using a survey of experts. RESULTS: A behavioral marker system was developed that comprised 12 emergency medicine-specific nontechnical skills, grouped into 4 categories. Content validity was assessed with a survey of 148 emergency medicine staff, and 75% of items achieved the recommended content validity index greater than 0.75. Data from the survey enabled further refinement of the behavioral markers to produce a final version of the tool. CONCLUSION: Although further evaluative studies are needed, this behavioral marker system provides a structured approach to the assessment and training of nontechnical skills in the ED.

Original publication




Journal article


Ann Emerg Med

Publication Date





376 - 385.e4


Clinical Competence, Communication, Emergency Medicine, Emergency Service, Hospital, Humans, Interviews as Topic, Patient Care Team, Physicians, Surveys and Questionnaires, United Kingdom