Automating Trajectory Prediction Performance Analyses for the FAA Traffic Management Advisor

Abstract

This paper presents a new approach for analyzing the trajectory prediction performance of the FAA’s Traffic Management Advisor (TMA). TMA is a deployed system that generates scheduled time-of-arrival constraints for en-route air traffic controllers in the US. The new automated analysis provides a repeatable evaluation of the current trajectory performance metrics, for new releases of TMA, in different traffic and airspace environments, and for current traffic situations. Using a wider set of data, it provides also a higher level of understanding on the causes of possible degradation of the trajectory prediction performance of TMA. The bulk of the work consisted of the development of the ability to filter flights not impacted by controller intervention. Identifying interrupted flights from recorded data is challenging but necessary for a fair and accurate performance test. Currently, no method for identification of flights exists other than a manual review of voice communications. The automated approach was tested with two data sets, from 2006 and 2013. The 2013 data consisted of 24 hours of traffic arriving into Dallas Forth Worth Airport. The results of the testing on this data set showed that the approach selected a statistically significant number of flights to validate the TMA trajectory predictor’s performance against the system requirements. New metrics for the evaluation of the TMA trajectory predictor’s performance are introduced and compared with the current set of metrics used by the FAA.