Topological analysis of unaligned audio and text data


Views: 78 / PDF downloads: 129

Authors

  • Zh. Kozhirbayev National Laboratory Astana
  • Zh. Yessenbayev National Laboratory Astana

DOI:

https://doi.org/10.32523/2616-7263-2022-141-4-116-126

Keywords:

unsupervised speech processing, variational autoencoders, word embeddings, topological data analysis, persistent homology and diagrams

Abstract

We have performed preliminary work on topological analysis of audio and text data for unsupervised speech processing. The work assumes that phoneme frequencies and contextual relationships are similar in the acoustic and text domains for the same language. Accordingly, this allowed the creation of a mapping between these spaces that considers their geometric structure. As a first step, generative methods based on variational autoencoders were chosen to map audio and text data into two latent vector spaces. In the next stage, persistent homology methods are used to analyze the topological structure of two spaces. Although the results obtained support the idea of the similarity of the two spaces, further research is needed to correctly map acoustic and text spaces, as well as to evaluate the real effect of including topological information in the autoencoder training process.

Published

2022-12-30

How to Cite

Kozhirbayev Ж., & Yessenbayev Ж. (2022). Topological analysis of unaligned audio and text data. Bulletin of L.N. Gumilyov Eurasian National University Technical Science and Technology Series, 141(4), 116–126. https://doi.org/10.32523/2616-7263-2022-141-4-116-126

Issue

Section

Article