GAN-Based Encoder-Decoder Model for Multi-Label Diagnostic Scan Classification and Automated Radiology Report Generation

Abstract

X-ray imaging is one of the most popular diagnostic imaging techniques and plays a critical role in the diagnosis and treatment process. Given the huge volume of patients and scans performed in most hospitals each day, the current practice of manual analysis of such scan images by experienced radiologists is a time-consuming and often error-prone process, worsened by the cognitive burden experienced by the radiologists. Conventional diagnostic reports written by radiologists after radiological image capture contain radiography-specific keywords (tags), observations of different body parts in the image (findings), and the actual diagnosis (impression). Automated multi-label classification of X-ray scans for disease prediction, and generation of an associated textual diagnostic scan report can ease the burden for radiologists, while also enabling fast, localized, and explanatory analysis. In this work, GAN-MLC, a CNN-LSTM description generator model trained in the adversarial setup, is proposed for the multi-label classification of X-ray images and improved feature learning for capturing disease-specific findings. Experiments performed on the NIH Chest X-ray Dataset revealed that the proposed GAN-MLC outperformed CNN-based models by a significant margin of more than seven percent. For the text diagnostic report generation task, the GAN-MLC achieved promising BLEU scores and was more robust against overfitting issues. © 2024 selection and editorial matter, Bhanu Chander, Koppala Guravaiah, B. Anoop, and G. Kumaravelan; individual chapters, the contributors.