Real-world blind super-resolution via feature matching with implicit high-resolution priors

Chen, Chaofeng, Shi, Xinyu, Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126, Li, Xiaoming, Yang, Tao, Han, Xiaoguang and Guo, Shihui 2022. Real-world blind super-resolution via feature matching with implicit high-resolution priors. Presented at: the 30th ACM International Conference on Multimedia (ACMMM 2022), Lisbon, Portugal, 10 - 14 October 2022. Proceedings of the 30th ACM International Conference on Multimedia (ACMMM 2022). ACM, pp. 1329-1338. 10.1145/3503161.3547833

Preview

PDF - Accepted Post-Print Version Download (22MB) | Preview

Abstract

A key challenge of real-world image super-resolution (SR) is to recover the missing details in low-resolution (LR) images with complex unknown degradations (\eg, downsampling, noise and compression). Most previous works restore such missing details in the image space. To cope with the high diversity of natural images, they either rely on the unstable GANs that are difficult to train and prone to artifacts, or resort to explicit references from high-resolution (HR) images that are usually unavailable. In this work, we propose Feature Matching SR (FeMaSR), which restores realistic HR images in a much more compact feature space. Unlike image-space methods, our FeMaSR restores HR images by matching distorted LR image features to their distortion-free HR counterparts in our pretrained HR priors, and decoding the matched features to obtain realistic HR images. Specifically, our HR priors contain a discrete feature codebook and its associated decoder, which are pretrained on HR images with a Vector Quantized Generative Adversarial Network (VQGAN). Notably, we incorporate a novel semantic regularization in VQGAN to improve the quality of reconstructed images. For the feature matching, we first extract LR features with an LR encoder consisting of several Swin Transformer blocks and then follow a simple nearest neighbour strategy to match them with the pretrained codebook. In particular, we equip the LR encoder with residual shortcut connections to the decoder, which is critical to the optimization of feature matching loss and also helps to complement the possible feature matching errors.Experimental results show that our approach produces more realistic HR images than previous methods. Code will be made publicly available.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	ACM
ISBN:	9781450392037
Date of First Compliant Deposit:	28 July 2022
Date of Acceptance:	29 June 2022
Last Modified:	03 Jul 2025 14:45
URI:	https://orca.cardiff.ac.uk/id/eprint/151095

Citation Data

Cited 14 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item