Speaker embeddings by modeling channel-wise correlations

Speaker embeddings by modeling channel-wise correlations

Paper in proceedings (conference paper)

Speaker embeddings extracted with deep 2D convolutional neuralnetworks are typically modeled as projections of first andsecond order statistics of channel-frequency pairs onto a linearlayer, using either average or attentive pooling along the timeaxis. In this paper we examine an alternative pooling method,where pairwise correlations between channels for given frequenciesare used as statistics. The method is inspired bystyle-transfer methods in computer vision, where the style ofan image, modeled by the matrix of channel-wise correlations,is transferred to another image, in order to produce a new imagehaving the style of the first and the content of the second.By drawing analogies between image style and speaker characteristics,and between image content and phonetic sequence,we explore the use of such channel-wise correlations featuresto train a ResNet architecture in an end-to-end fashion. Ourexperiments on VoxCeleb demonstrate the effectiveness of theproposed pooling method in speaker recognition.

Speaker embeddings extracted with deep 2D convolutional neuralnetworks are typically modeled as projections of first andsecond order statistics of channel-frequency pairs onto a linearlayer, using either average or attentive pooling along the timeaxis. In this paper we examine an alternative pooling method,where pairwise correlations between channels for given frequenciesare used as statistics. The method is inspired bystyle-transfer methods in computer vision, where the style ofan image, modeled by the matrix of channel-wise correlations,is transferred to another image, in order to produce a new imagehaving the style of the first and the content of the second.By drawing analogies between image style and speaker characteristics,and between image content and phonetic sequence,we explore the use of such channel-wise correlations featuresto train a ResNet architecture in an end-to-end fashion. Ourexperiments on VoxCeleb demonstrate the effectiveness of theproposed pooling method in speaker recognition.

speaker recognition, style-transfer, deep learning

speaker recognition, style-transfer, deep learning

STAFYLAKIS, T.; ROHDIN, J.; BURGET, L.

2022

30.08.2021

International Speech Communication Association

Brno

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

1990-9772

Proceedings of Interspeech

2021

8

French Republic

501

505

5

https://www.isca-speech.org/archive/interspeech_2021/stafylakis21_interspeech.html

stafylakis21_interspeech