Efficient Data Selection for Domain Adaptation of ASR Using Pseudo-Labels and Multi-Stage Filtering

Efficient Data Selection for Domain Adaptation of ASR Using Pseudo-Labels and Multi-Stage Filtering

Stať ve sborníku v databázi WoS či Scopus

Fine-tuning pretrained ASR models for specific domains is challenging for small organizations with limited labeled data and computational resources. Here we explore different data selection pipelines and propose a robust approach that improves ASR adaptation by filtering pseudo-labels generated using Whisper (encoder-decoder) and Zipformer (transducer) models. Our approach integrates multiple selection strategies-including word error rate (WER) prediction, named entity recognition (NER), and character error rate (CER) analysis-to extract high-quality training segments. We evaluate our method on Whisper and Zipformer using a 7500-hour baseline, comparing it to a CER-based approach relying on hypotheses from three ASR systems. Fine-tuning on 7500 hours of pseudo-labeled call center data achieves 12.3% WER, while our filtering reduces the dataset to 100 hours (1.4%) with similar performance; a similar trend is observed on Fisher English.

Fine-tuning pretrained ASR models for specific domains is challenging for small organizations with limited labeled data and computational resources. Here we explore different data selection pipelines and propose a robust approach that improves ASR adaptation by filtering pseudo-labels generated using Whisper (encoder-decoder) and Zipformer (transducer) models. Our approach integrates multiple selection strategies-including word error rate (WER) prediction, named entity recognition (NER), and character error rate (CER) analysis-to extract high-quality training segments. We evaluate our method on Whisper and Zipformer using a 7500-hour baseline, comparing it to a CER-based approach relying on hypotheses from three ASR systems. Fine-tuning on 7500 hours of pseudo-labeled call center data achieves 12.3% WER, while our filtering reduces the dataset to 100 hours (1.4%) with similar performance; a similar trend is observed on Fisher English.

speech recognition, data selection, whisper, zip-formers

speech recognition, data selection, whisper, zip-formers

RANGAPPA, P.; CAROFILIS, A.; PRAKASH, J.; KUMAR, S.; BURDISSO, S.; MADIKERI, S.; VILLATORO-TELLO, E.; SHARMA, B.; MOTLÍČEK, P.; HACIOGLU, K.; VENKATESAN, S.; VYAS, S.; STOLCKE, A.

2026

17.08.2025

Isca-Int Speech Communication Assoc

Rotterdam, The Netherlands

Interspeech

Interspeech

Francouzská republika

4928

4932

5

https://www.fit.vut.cz/research/group/speech/public/publi/2025/rangappa_INTERSPEECH_2025_co-author_Motlicek.pdf

rangappa_INTERSPEECH_2025_co-author_Motlicek