Deep Speech Pattern Analysis of Controller-Pilot Voice Communications for Enhancing Future Aviation Systems Safety

View Video Presentation: https://doi.org/10.2514/6.2023-4410.vidIn this study, air traffic controller-pilot speech attributes and transcribed texts of voice messages are analyzed to gain insights into air traffic controller-pilot miscommunications. A Deep Speech Pattern (DSP) analysis is used to convert the speech audio into a spectrogram which is broken down into six sound features: loudness, articulation, tempo, rhythm, melody, and timbre. An initial analysis of these six basic features did not reveal enough information about sound attributes and classification of messages. Therefore, a deeper analysis of the audio messages was needed, which resulted in defining and analyzing additional characteristics of basic features such as: spectral flatness, spectral centroid, onset count, onset rate, onset strength, and beat count. Two methods of calculating tempo (default prior and uniform prior) were defined and analyzed. However, some audio files had heavy background noise that restrained the analysis. Therefore, a comparative study was conducted between the original audio files and noise-reduced audio files. The noise-reduced audio files were found to better perform in the analysis, enabling a second set of speech feature vectors to be extracted and utilized from noise-reduced historical audio files for clear audio communication scenarios. A Principal Component Analysis (PCA) was then used to identify significant speech attributes and to classify historical controller-pilot communication messages, allowing for a deeper understanding of the potential speech related reasons for miscommunications. Such analyses can be integrated with situation assessment modules to identify anomalies in flight trajectories, and then added into NASA’s In-Time Aviation Safety Management System (IASMS) aiming to improve the safety of en route and terminal airspace operations.