Performance evaluation of expert team members

Abstract

Abstract : We found that CWS can do a reasonable job of capturing expertise in arithmetic compared to indices that incorporate correct answers. We also found (in conjunction with Rick Thomas and Alex Kirlik) similar results for a performance domain, putting a golf ball. In addition, we initiated the development of designs that allow the extraction of individual performance from team scores. We used the CTEAM simulation to study the development of expertise of teams of simulated air traffic controllers. CWS served as the primary index of performance, although traditional measures were examined as well. As a performance measure, CWS (based on time through sector) was observed to be more sensitive to improvement over practice than operational errors. Experts make too few errors to provide differential accuracy information, while CWS attends to aspects of performance that can always be observed, whether errors are made or not.