Generating Clinically Relevant Radiology Reports using Multimodal Deep Learning Models

Abstract

Generating radiology reports is an important component of the diagnostic process, facilitating the interpretation of medical images and subsequent clinical decision making. Recent advancements have opened new pathways to automate the generation of radiology reports, a task traditionally performed by radiologists through meticulous analysis of medical images. This study presents a framework that uses deep learning and NLP to automate the generation of radiology reports from chest X-ray images. Our approach utilizes a pre-trained RAD-DINO model to extract features from radio-graphic images and an encoder-decoder model to create coherent textual descriptions. The proposed models produced coherent reports outperforming several existing state-of-the-art methods when evaluated using standard metrics like BLEU. © 2024 Copyright held by the owner/author(s).