Language Detection in Overlapping Multilingual Speech: A Focus on Indian Languages

Abstract

The growing demand for technology capable of recognizing spoken languages and extracting information from real-world audio, especially in scenarios with overlapping speech, has become a significant focus of research due to its essential role in improving global connectivity and accessibility. In our paper, we focus on identifying languages present in audio files that consist of overlapping speech. We have focused our research particularly on Indian languages, as there is limited research on identifying low-resource languages in overlapping speech. In this paper, we have synthesized a custom dataset from the VoxLingua107 dataset due to the lack of overlapping Indian speech data. Further, we have developed a novel solution that first separates the overlapped audio using a speaker separation model and then uses a language recognition model to detect the languages present in the separated audio. We have compared the results obtained through our method with the current state-of-the-art model, Whisper, and concluded that our solution significantly outperforms the Whisper model. The results highlight the potential for significant improvements in multilingual communication systems and speech processing applications, paving the way for more inclusive and accurate language recognition technologies. © 2025 IEEE.