Code-switching automatic speech recognition using modified ESPNet

Abstract

Recently, a drastically increased focus has been observed in multilingual Automatic Speech Recognition (ASR). To cater to multiple low resource languages, a speech recognition system is used. This is performed by taking advantage of low amounts of labeled corpora in multiple languages has. The prosperity of low-resource multilingual and code-switching ASR often depends on the variety of languages in terms of linguistic characteristics as well as the amount of data available. This work focuses on modifying the multilingual and code-switching ASR system through two different subtasks including a total of seven Indian languages. To counter this the model has been provided with several hours of transcribed speech data, comprising of train and test sets, in these languages including two code-switched language pairs, Hindi-English and Bengali-English. In this work, a modified ESPNet architecture is proposed to perform multilingual ASR which improved the performance of the baseline system resulting in accuracy of Word Error Rate (WER) is 27.69%. © 2023 Author(s).