MSDNet: a deep neural ensemble model for abnormality detection and classification of plain radiographs

Abstract

Modern medical diagnostic techniques facilitate accurate diagnosis and treatment recommendations in healthcare. Such diagnostics procedures are performed daily in large numbers, thus, the clinical interpretation workload of radiologists is very high. Identification of abnormalities is a predominantly manual task that is performed by radiologists before the medical scans are available to the patient’s referring doctor for further recommendations. On the other hand, for a radiologist to delineate the imaging study’s findings/observations as a textual report is also a tedious task. Automated methods for radiographic image examination for identifying abnormalities and generating reliable radiology report are thus a fundamental requirement in clinical workflow management applications. In this work, we present an automated approach for abnormality classification, localization and diagnostic report retrieval for identified abnormalities. We propose MSDNet, an ensemble of Convolutional Neural models for abnormality classification, which combines the features of multiple CNN models to enhance abnormality classification performance. The proposed model also is designed to localize and visualize the detected abnormality on the radiograph image, based on an abnormal region detection algorithm to further optimize the diagnosis quality. Furthermore, the extracted features generated by MSDNet are used to automatically generate the diagnosis text report using an automatic content-based report retrieval algorithm. The upper extremity musculo-skeletal images from the MURA dataset and chest X-ray images from Indiana dataset were used for the experimental evaluation of the proposed approach. The proposed model achieved promising results, with an accuracy of 82.69%, showing its significant impact on alleviating radiologists’ cognitive load, thus improving the overall efficiency. © 2022, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.