Contextual Semi-Supervised Learning: An Approach to Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems

Contextual Semi-Supervised Learning: An Approach to Leverage Air-Surveillance and Untranscribed ATC Data in ASR Systems

Paper in proceedings (conference paper)

Air traffic management and specifically air-traffic control (ATC)rely mostly on voice communications between Air Traffic Controllers(ATCos) and pilots. In most cases, these voice communicationsfollow a well-defined grammar that could be leveragedin Automatic Speech Recognition (ASR) technologies. Thecallsign used to address an airplane is an essential part of allATCo-pilot communications. We propose a two-step approachto add contextual knowledge during semi-supervised training toreduce the ASR system error rates at recognizing the part of theutterance that contains the callsign. Initially, we represent in aWFST the contextual knowledge (i.e. air-surveillance data) ofan ATCo-pilot communication. Then, during Semi-SupervisedLearning (SSL) the contextual knowledge is added by secondpassdecoding (i.e. lattice re-scoring). Results show that unseendomains (e.g. data from airports not present in the supervisedtraining data) are further aided by contextual SSL whencompared to standalone SSL. For this task, we introduce theCallsign Word Error Rate (CA-WER) as an evaluation metric,which only assesses ASR performance of the spoken callsignin an utterance. We obtained a 32.1% CA-WER relative improvementapplying SSL with an additional 17.5% CA-WERimprovement by adding contextual knowledge during SSL on achallenging ATC-based test set gathered from LiveATC.

Air traffic management and specifically air-traffic control (ATC)rely mostly on voice communications between Air Traffic Controllers(ATCos) and pilots. In most cases, these voice communicationsfollow a well-defined grammar that could be leveragedin Automatic Speech Recognition (ASR) technologies. Thecallsign used to address an airplane is an essential part of allATCo-pilot communications. We propose a two-step approachto add contextual knowledge during semi-supervised training toreduce the ASR system error rates at recognizing the part of theutterance that contains the callsign. Initially, we represent in aWFST the contextual knowledge (i.e. air-surveillance data) ofan ATCo-pilot communication. Then, during Semi-SupervisedLearning (SSL) the contextual knowledge is added by secondpassdecoding (i.e. lattice re-scoring). Results show that unseendomains (e.g. data from airports not present in the supervisedtraining data) are further aided by contextual SSL whencompared to standalone SSL. For this task, we introduce theCallsign Word Error Rate (CA-WER) as an evaluation metric,which only assesses ASR performance of the spoken callsignin an utterance. We obtained a 32.1% CA-WER relative improvementapplying SSL with an additional 17.5% CA-WERimprovement by adding contextual knowledge during SSL on achallenging ATC-based test set gathered from LiveATC.

automatic speech recognition, contextual semisupervisedlearning, air traffic control, air-surveillance data,callsign detection.

automatic speech recognition, contextual semisupervisedlearning, air traffic control, air-surveillance data,callsign detection.

ZULUAGA-GOMEZ, J.; NIGMATULINA, I.; PRASAD, A.; MOTLÍČEK, P.; VESELÝ, K.; KOCOUR, M.; SZŐKE, I.

2022

30.08.2021

International Speech Communication Association

Brno

Proceedings Interspeech 2021

1990-9772

Proceedings of Interspeech

2021

8

French Republic

3296

3300

5

https://www.isca-speech.org/archive/interspeech_2021/zuluagagomez21_interspeech.html

http://hdl.handle.net/

zuluagagomez21_interspeech