Automated Summarization of Gastrointestinal Endoscopy Video

Abstract

Gastrointestinal (GI) endoscopy enables many minimally invasive procedures for diagnosing diseases such as esophagitis, ulcer, polyps and cancers. Guided by the endoscope’s video sequence, a physician can diagnose the diseases and administer the treatment. Unfortunately, due to the huge amount of data generated, physicians are currently discarding procedural video and rely on a small number of carefully chosen images to record a procedure. In addition, when a patient seeks a second opinion, the assessment of lesions in a huge video stream necessitates a thorough examination, which is a time-consuming process that demands much attention. To reduce the length of the video stream, an automated method to generate the summary of endoscopy video recordings consisting only of abnormal frames by using deep convolutional neural networks trained to classify normal, abnormal and uninformative frames is proposed. Results show that our method can efficiently detect abnormal frames and is robust to the variations in the frames. The proposed CNN architecture outperforms the other classification models with an accuracy of 0.9698 with less number of parameters. © IFIP International Federation for Information Processing 2023.