Xinyi Zhang* Hang Dong* Zhe Hu Wei-Sheng Lai Fei Wang Ming-Hsuan Yang
Accept by IJCV on 13 January 2020.
Abstract
Single image super resolution aims to enhance image quality with respect to spatial content, which is a fundamental task in computer vision. In this work, we address the task of single frame super resolution with the presence of image degradation, e.g., blur, haze, or rain streaks. Due to the limitations of frame capturing and formation processes, image degradation is inevitable, and the artifacts would be exacerbated by super resolution methods. To address this problem, we propose a dual-branch convolutional neural network to extract base features and recovered features separately. The base features contain local and global information of the input image. On the other hand, the recovered features focus on the degraded regions and are used to remove the degradation. Those features are then fused through a recursive gate module to obtain sharp features for super resolution. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process by avoiding learning the mixed degradation all-in-one and thus enhance the final high-resolution prediction results. We evaluate the proposed method in three degradation scenarios. Experiments on these scenarios demonstrate that the proposed method performs more efficiently and favorably against the state-of-the-art approaches on benchmark datasets.
Technical Papers and Codes
Performance Versus Inference Time and Model Parameters
Performance of Deblurring and Super-Resolution.
Visual Results on Degraded Images
- Visual Results on Deblurring.
- Visual Results on Dehazing.
- Visual Results on Deraining.
Application on Detection
- We test different joint dehazing and super-resolution methods, and use YOLOv3 as the detection algorithm.
Detection results on the KITTI detection dataset with haze degradation.
- We test different joint deblurring and super-resolution methods, and use YOLOv3 as the detection algorithm.
Detection results on the KITTI detection dataset with non-uniform motion blur.
