Conference Papers

Permanent URI for this collectionhttps://idr.nitk.ac.in/handle/123456789/28506

Browse

Author: search.filters.author.Mohan, B.R.Subject: search.filters.subject.LSTM

Search Results

Now showing 1 - 2 of 2
  • No Thumbnail Available
    Item
    A Comparative Study of Deep Learning Models for Word-Sense Disambiguation
    (Springer Science and Business Media Deutschland GmbH, 2022) Jadiya, A.; Dondemadahalli Manjunath, T.; Mohan, B.R.
    Word-sense disambiguation (WSD) has been a persistent issue since its introduction to the community of natural language processing (NLP). It has a wide range of applications in different areas like information retrieval (IR), sentiment analysis, knowledge graph construction, machine translation, lexicography, text mining, information extraction, and so on. Analysis of the performance of deep learning algorithms with different word embeddings is required to be done since various deep learning models are deployed for the task of disambiguation of word sense. In this paper, comparison of several deep learning models like CNN, LSTM, bidirectional LSTM, and CNN + LSTM is done with trainable as well as pretrained GloVe embeddings with common preprocessing methods. Performance evaluation of temporal convolutional network (TCN) model is done along with the comparison of the same with the formerly mentioned models. This paper shows that using GloVe embeddings may not result in better accuracy in the case of word-sense disambiguation, i.e., trainable embeddings perform better. It also includes a framework for evaluating deep learning models for WSD and analysis of the usage of embeddings for the same. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
  • No Thumbnail Available
    Item
    Comparative Study of Pruning Techniques in Recurrent Neural Networks
    (Springer Science and Business Media Deutschland GmbH, 2023) Choudhury, S.; Rout, A.K.; Pragnesh, T.; Mohan, B.R.
    In recent years, there has been a drastic development in the field of neural networks. They have evolved from simple feed-forward neural networks to more complex neural networks such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). CNNs are used for tasks such as image recognition where the sequence is not essential, while RNNs are useful when order is important such as machine translation. By increasing the number of layers in the network, we can improve the performance of the neural network (Alford et al. in Pruned and structurally sparse neural networks, 2018 [1]). However, this will also increase the complexity of the network, and also training will require more power and time. By introducing sparsity in the architecture of the neural network, we can tackle this problem. Pruning is one of the processes through which a neural network can be made sparse (Zhu and Gupta in To prune, or not to prune: exploring the efficacy of pruning for model compression, 2017 [2]). Sparse RNNs can be easily implemented on mobile devices and resource-constraint servers (Wen et al. in Learning intrinsic sparse structures within long short-term memory, 2017 [3]). We investigate the following methods to induce sparsity in RNNs: RNN pruning and automated gradual pruning. We also investigate how the pruning techniques impact the model’s performance and provide a detailed comparison between the two techniques. We also experiment by pruning input-to-hidden and hidden-to-hidden weights. Based on the results of pruning experiments, we conclude that it is possible to reduce the complexity of RNNs by more than 80%. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.