Depth image super-resolution with local medians and bilateral filtering

Abstract

In this paper, we propose an approach for depth image super-resolution (SR). Given a noisy low resolution (LR) depth image and its corresponding registered high resolution (HR) colour image, our approach improves the resolution of the LR image while suppressing noise. We use the segmentation of HR colour images as a cue for depth image super-resolution. Our method begins with a highly over-segmented color image (using well-known segmentation approaches such as mean shift (MS) or simple linear iterative clustering (SLIC), and an interpolated LR depth image. We then use a combination of the local medians in the depth image (corresponding to the colour segments) and bicubic interpolation, followed by bilateral filtering to compute the SR depth image. We performed experiments for higher magnification factors 4, 8 using the Middlebury depth image dataset and evaluate the SR performance using the PSNR and SSIM metrics. The experimental results show that proposed method (including some variants), while being relatively simplistic, shows an average improvement of 1.2dB and 1.7dB on noiseless and noisy data respectively, over the popular method of guided image filtering (GIF) for upsampling factor 8. © 2016 IEEE.