Conference Papers
Permanent URI for this collectionhttps://idr.nitk.ac.in/handle/123456789/28506
Browse
2 results
Search Results
Item Multimodal Meme Troll and Domain Classification Using Contrastive Learning(Institute of Electrical and Electronics Engineers Inc., 2024) Phadatare, A.; Jayanth, P.; Anand Kumar, M.A.This paper presents a holistic approach to meme trolling detection and domain classification, focusing on Telugu and Kannada languages. Leveraging a spectrum of methodologies ranging from basic machine learning models such as Support Vector Machines (SVM), Random Forest, Naive Bayes, to image-based models like Convolutional Neural Networks (CNN), ResNet-50, and state-of-the-art models such as CLIP, multilingual BERT, XLM-BERT, and Vision Transformers, we explore diverse modalities including image classification, extracted text classification, and combined text-caption classification. Our system integrates multiple models to achieve two primary goals: accurately detecting trolling behavior and classifying memes into thematic domains like politics, movies, sports.. By training on multilingual data and considering linguistic diversity, our approach ensures robust performance across different linguistic contexts, providing valuable insights into meme culture and trolling behavior in Telugu and Kannada-speaking communities. © 2024 IEEE.Item Disaster Classification Using Multimodality Techniques by Integrating Images and Text(Springer Science and Business Media Deutschland GmbH, 2025) Medapati, B.M.R.; Pais, S.M.; Bhattacharjee, S.In today’s digital era, the abundance of information shared through social media platforms in times of disaster has emerged as a crucial asset for enhancing disaster response operations. This research initiative was specifically dedicated to enhance disaster categorization by integrating image and tweet text data. The devised model comprises two distinct modules aimed at optimizing the classification process. The primary module focuses on extracting insights from text and images independently using VGG-16 for images, and Bidirectional Long Short-Term Memory and Convolutional Neural Network for texts, subsequently executing the classification task. Conversely, the secondary module is designed to learn the interconnectedness between textual content and images using Contrastive Learning Image Pretraining (CLIP). After this late fusion is used to combine the outcomes of these modules and later softmax classification is used for the classification of the incident into one of seven humanitarian categories thereby enhancing the precision and efficacy of disaster classification. The developed model gives an accuracy of 75% with no data and image augmentations and the result was improved to 93% with different combinations of augmentations. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.