Integration of Variational Autoencoder and Spatial Clustering for Adaptive Multi-Channel Neural Speech Separation

Integration of Variational Autoencoder and Spatial Clustering for Adaptive Multi-Channel Neural Speech Separation

Paper in proceedings (conference paper)

In this paper, we propose a method combining variational autoencodermodel of speech with a spatial clustering approach for multichannelspeech separation. The advantage of integrating spatial clusteringwith a spectral model was shown in several works. As thespectral model, previous works used either factorial generative modelsof the mixed speech or discriminative neural networks. In ourwork, we combine the strengths of both approaches, by building afactorial model based on a generative neural network, a variationalautoencoder. By doing so, we can exploit the modeling power ofneural networks, but at the same time, keep a structured model. Sucha model can be advantageous when adapting to new noise conditionsas only the noise part of the model needs to be modified. We showexperimentally, that our model significantly outperforms previousfactorial model based on Gaussian mixture model (DOLPHIN), performscomparably to integration of permutation invariant trainingwith spatial clustering, and enables us to easily adapt to new noiseconditions.

In this paper, we propose a method combining variational autoencodermodel of speech with a spatial clustering approach for multichannelspeech separation. The advantage of integrating spatial clusteringwith a spectral model was shown in several works. As thespectral model, previous works used either factorial generative modelsof the mixed speech or discriminative neural networks. In ourwork, we combine the strengths of both approaches, by building afactorial model based on a generative neural network, a variationalautoencoder. By doing so, we can exploit the modeling power ofneural networks, but at the same time, keep a structured model. Sucha model can be advantageous when adapting to new noise conditionsas only the noise part of the model needs to be modified. We showexperimentally, that our model significantly outperforms previousfactorial model based on Gaussian mixture model (DOLPHIN), performscomparably to integration of permutation invariant trainingwith spatial clustering, and enables us to easily adapt to new noiseconditions.

Multi-channel speech separation, variational autoencoder,spatial clustering, DOLPHIN

Multi-channel speech separation, variational autoencoder,spatial clustering, DOLPHIN

ŽMOLÍKOVÁ, K.; DELCROIX, M.; BURGET, L.; NAKATANI, T.; ČERNOCKÝ, J.

2022

19.01.2021

IEEE Signal Processing Society

Shenzhen - virtual

978-1-7281-7066-4

2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings

889

896

8

https://ieeexplore.ieee.org/document/9383612

zmolikova_slt2021