Publication result detail

Dealing with Unknowns in Continual Learning for End-to-end Automatic Speech Recognition

ŠŮSTEK, M.; SADHU, S.; HEŘMANSKÝ, H.

Original Title

Dealing with Unknowns in Continual Learning for End-to-end Automatic Speech Recognition

English Title

Dealing with Unknowns in Continual Learning for End-to-end Automatic Speech Recognition

Type

Paper in proceedings (conference paper)

Original Abstract

Learning continually from data is a task executed effortlessly by humans but remains to be of significant challenge for machines. Moreover, when encountering unknown test scenarios machines fail to generalize. We propose a mathematically motivated dynamically expanding end-to-end model of independent sequence-to-sequence components trained on different data sets that avoid catastrophically forgetting knowledge acquired from previously seen data while seamlessly integrating knowledge from new data. During inference, the likelihoods of the unknown test scenario are computed using internal model activation distributions. The inference made by each independent component is weighted by the normalized likelihood values to obtain the final decision.

English abstract

Learning continually from data is a task executed effortlessly by humans but remains to be of significant challenge for machines. Moreover, when encountering unknown test scenarios machines fail to generalize. We propose a mathematically motivated dynamically expanding end-to-end model of independent sequence-to-sequence components trained on different data sets that avoid catastrophically forgetting knowledge acquired from previously seen data while seamlessly integrating knowledge from new data. During inference, the likelihoods of the unknown test scenario are computed using internal model activation distributions. The inference made by each independent component is weighted by the normalized likelihood values to obtain the final decision.

Keywords

continual learning, multistream speech recognition, speech recognition

Key words in English

continual learning, multistream speech recognition, speech recognition

Authors

ŠŮSTEK, M.; SADHU, S.; HEŘMANSKÝ, H.

RIV year

2023

Released

01.09.2022

Publisher

International Speech Communication Association

Location

Incheon

Book

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ISBN

1990-9772

Periodical

Proceedings of Interspeech

Volume

2022

Number

9

State

French Republic

Pages from

1046

Pages to

1050

Pages count

5

URL

https://www.isca-speech.org/archive/pdfs/interspeech_2022/sustek22_interspeech.pdf

BibTex

@inproceedings{BUT182527,
  author="ŠŮSTEK, M. and SADHU, S. and HEŘMANSKÝ, H.",
  title="Dealing with Unknowns in Continual Learning for End-to-end Automatic Speech Recognition",
  booktitle="Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",
  year="2022",
  journal="Proceedings of Interspeech",
  volume="2022",
  number="9",
  pages="1046--1050",
  publisher="International Speech Communication Association",
  address="Incheon",
  doi="10.21437/Interspeech.2022-11139",
  issn="1990-9772",
  url="https://www.isca-speech.org/archive/pdfs/interspeech_2022/sustek22_interspeech.pdf"
}