Gaussian meta-embeddings for efficient scoring of a heavy-tailed PLDA model

Gaussian meta-embeddings for efficient scoring of a heavy-tailed PLDA model

Stať ve sborníku v databázi WoS či Scopus

Embeddings in machine learning are low-dimensional representationsof complex input patterns, with the property that simplegeometric operations like Euclidean distances and dot productscan be used for classification and comparison tasks. Weintroduce meta-embeddings, which live in more general innerproduct spaces and which are designed to better propagate uncertaintythrough the embedding bottleneck. Traditional embeddingsare trained to maximize between-class and minimizewithin-class distances. Meta-embeddings are trained to maximizerelevant information throughput. As a proof of conceptin speaker recognition, we derive an extractor from the familiargenerative Gaussian PLDA model (GPLDA). We show thatGPLDA likelihood ratio scores are given by Hilbert space innerproducts between Gaussian likelihood functions, which weterm Gaussian meta-embeddings (GMEs). Meta-embedding extractorscan be generatively or discriminatively trained. GMEsextracted by GPLDA have fixed precisions and do not propagateuncertainty. We show that a generalization to heavy-tailedPLDA gives GMEs with variable precisions, which do propagateuncertainty. Experiments on NIST SRE 2010 and 2016show that the proposed method applied to i-vectors withoutlength normalization is up to 20% more accurate than GPLDAapplied to length-normalized i-vectors.

Embeddings in machine learning are low-dimensional representationsof complex input patterns, with the property that simplegeometric operations like Euclidean distances and dot productscan be used for classification and comparison tasks. Weintroduce meta-embeddings, which live in more general innerproduct spaces and which are designed to better propagate uncertaintythrough the embedding bottleneck. Traditional embeddingsare trained to maximize between-class and minimizewithin-class distances. Meta-embeddings are trained to maximizerelevant information throughput. As a proof of conceptin speaker recognition, we derive an extractor from the familiargenerative Gaussian PLDA model (GPLDA). We show thatGPLDA likelihood ratio scores are given by Hilbert space innerproducts between Gaussian likelihood functions, which weterm Gaussian meta-embeddings (GMEs). Meta-embedding extractorscan be generatively or discriminatively trained. GMEsextracted by GPLDA have fixed precisions and do not propagateuncertainty. We show that a generalization to heavy-tailedPLDA gives GMEs with variable precisions, which do propagateuncertainty. Experiments on NIST SRE 2010 and 2016show that the proposed method applied to i-vectors withoutlength normalization is up to 20% more accurate than GPLDAapplied to length-normalized i-vectors.

embeddings, machine learning, speaker recognition

embeddings, machine learning, speaker recognition

BRUMMER, J.; SILNOVA, A.; BURGET, L.; STAFYLAKIS, T.

2019

26.06.2018

International Speech Communication Association

Les Sables d'Olonne

Proceedings of Odyssey 2018

2312-2846

Proceedings of Odyssey: The Speaker and Language Recognition Workshop Odyssey 2014, Joensuu, Finland

2018

6

Finská republika

349

356

8

https://www.fit.vut.cz/research/publication/11790/

http://hdl.handle.net/

brummer_odyssey2018_51