Faculty Publications
Permanent URI for this communityhttps://idr.nitk.ac.in/handle/123456789/18736
Publications by NITK Faculty
Browse
4 results
Search Results
Item Diagnostic Performance Evaluation of Deep Learning-Based Medical Text Modelling to Predict Pulmonary Diseases from Unstructured Radiology Free-Text Reports(Prague University of Economics and Business, 2023) Shetty, S.; Ananthanarayana, V.S.; Mahale, A.The third most common cause of death worldwide is attributed to pulmonary diseases, making it imperative to diagnose them promptly. Radiology is a medical discipline that utilizes medical imaging to guide treatment. Radiologists prepare reports interpreting details and findings analysed from medical images. Radiology free-text reports are a rich source of textual information that can be exploited to enhance the efficacy of medical prognosis, treatment and research. Radiology reports exist in an unstructured format as are not suitable by themselves for mathematical computation or machine learning operations. Therefore, natural language processing (NLP) strategies are employed to convert unstructured natural language text into a structured format that can be fed into machine learning (ML) or deep learning (DL) models for information extraction. We propose a DL-based medical text modelling framework incorporating a knowledge base to predict pulmonary diseases from unstructured radiology free-text reports. We make detailed diagnostic performance evaluations of our proposed technique by comparing it with state-of-the-art NLP techniques on radiology free-text reports extracted from two medical institutions. The comprehensive analysis shows that the proposed model achieves superior results compared to existing state-of-the-art text modelling techniques. © 2023 Prague University of Economics and Business. All Rights Reserved.Item Cross-modal Deep Learning-based Clinical Recommendation System for Radiology Report Generation from Chest X-rays(Materials and Energy Research Center, 2023) Shetty, S.; Ananthanarayana, V.S.; Mahale, A.Radiology report generation is a critical task for radiologists, and automating the process can significantly simplify their workload. However, creating accurate and reliable radiology reports requires radiologists to have sufficient experience and time to review medical images. Unfortunately, many radiology reports end with ambiguous conclusions, resulting in additional testing and diagnostic procedures for patients. To address this, we proposed an encoder-decoder-based deep learning framework that utilizes chest X-ray images to produce diagnostic radiology reports. In our study, we have introduced a novel text modelling and visual feature extraction strategy as part of our proposed encoder-decoder-based deep learning framework. Our approach aims to extract essential visual and textual information from chest X-ray images to generate more accurate and reliable radiology reports. Additionally, we have developed a dynamic web portal that accepts chest X-rays as input and generates a radiology report as output. We conducted an extensive analysis of our model and compared its performance with other state-of-the-art deep learning approaches. Our findings indicate significant improvement achieved by our proposed model compared to existing models, as evidenced by the higher BLEU scores (BLEU1 = 0.588, BLEU2 = 0.4325, BLEU3 = 0.4017, BLEU4 = 0.3860) attained on the Indiana University Dataset. These results underscore the potential of our deep learning framework to enhance the accuracy and reliability of radiology reports, leading to more efficient and effective medical treatment. © 2023 Materials and Energy Research Center. All rights reserved.Item Multimodal medical tensor fusion network-based DL framework for abnormality prediction from the radiology CXRs and clinical text reports(Springer, 2023) Shetty, S.; S, A.V.; Mahale, A.Pulmonary disease is a commonly occurring abnormality throughout this world. The pulmonary diseases include Tuberculosis, Pneumothorax, Cardiomegaly, Pulmonary atelectasis, Pneumonia, etc. A timely prognosis of pulmonary disease is essential. Increasing progress in Deep Learning (DL) techniques has significantly impacted and contributed to the medical domain, specifically in leveraging medical imaging for analysis, prognosis, and therapeutic decisions for clinicians. Many contemporary DL strategies for radiology focus on a single modality of data utilizing imaging features without considering the clinical context that provides more valuable complementary information for clinically consistent prognostic decisions. Also, the selection of the best data fusion strategy is crucial when performing Machine Learning (ML) or DL operation on multimodal heterogeneous data. We investigated multimodal medical fusion strategies leveraging DL techniques to predict pulmonary abnormality from the heterogeneous radiology Chest X-Rays (CXRs) and clinical text reports. In this research, we have proposed two effective unimodal and multimodal subnetworks to predict pulmonary abnormality from the CXR and clinical reports. We have conducted a comprehensive analysis and compared the performance of unimodal and multimodal models. The proposed models were applied to standard augmented data and the synthetic data generated to check the model’s ability to predict from the new and unseen data. The proposed models were thoroughly assessed and examined against the publicly available Indiana university dataset and the data collected from the private medical hospital. The proposed multimodal models have given superior results compared to the unimodal models. © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.Item SMC-CNN: Stacked Multi-Channel Convolution Neural Network for Predicting Acute Brain Infarct From Magnetic Resonance Imaging Sequences(Institute of Electrical and Electronics Engineers Inc., 2024) Shetty, S.; Ananthanarayana, V.S.; Mahale, A.; Devi, S.A.Acute brain infarct is a major cause of stroke and the second most common cause of fatality worldwide. It's characterized by abrupt symptoms persisting over 24 hours or leading to death due to blood vessel blockage. There is a need for a fast and automated way to diagnose and predict the outcome of this condition. Medical image analysis has witnessed promising outcomes with the application of deep learning (DL) techniques. To address this problem, we propose two Stacked Multi-Channel Convolutional Neural Networks (SMC-CNNs) for predicting acute infarct using individual and multiple Magnetic Resonance Imaging (MRI) sequences, including Diffusion-Weighted Imaging (DWI), Apparent Diffusion Coefficient (ADC), T2-weighted Fluid-Attenuated Inversion Recovery (T2-FLAIR), and Susceptibility-Weighted Imaging (SWI). We collected de-identified and de-linked MRI sequences from KMC Hospital (Mangalore, India) and compared the efficacy of our models with eight baseline state-of-the-art DL models in an extensive benchmarking study. The collected dataset was pre-processed using a proposed contour-based brain segmentation technique to isolate brain contours from the MRI sequences. These contours were ingested into the two proposed models: Stacked Multi-Channel Convolutional Neural Network for Individual sequences (SMC-CNN-I) and Stacked Multi-Channel Convolutional Neural Network for Multiple sequences (SMC-CNN-M), to predict acute infarct. We conducted experimental evaluations on individual MRI sequences to assess the effectiveness of the models for each sequence and found that the DWI and T2-FLAIR imaging sequences contained more discriminative features for acute infarct prediction than the other sequences. We performed an ablation study by varying and fusing different MRI sequences and observed that the proposed model achieved superior results when all four MRI sequences were used as inputs. We also tested the proposed models on synthetic MRI data generated using a Deep Convolutional Generative Adversarial Network (DCGAN) architecture. We found that the models produced improved results, demonstrating their ability to perform well on real-world data. We conducted a quantitative analysis followed by a qualitative analysis by visualizing infarcts using the Gradient-weighted Class Activation Mapping (Grad-CAM) technique. This demonstrated the model's ability to detect the precise location of abnormalities. © 2013 IEEE.