Visual Artifact Detection and Correction for Digital Images Via Deep Neural Networks (Real-ESRGAN)

Abstract

This work pursues to the improvement of digital images quality and the correction of visual artifacts, in particular noise, by means of deep neural networks, through an application-oriented investigation based on pre-trained models which are ready to be used in the execution on a resource-limited platform. The study is inspired by a typical problem in processing digital images, on one hand traditional image enhancement algorithms do not work well on real world low-quality images because they cannot well trade off between over smoothing noise and preserving visual details and image structural information, on the other hand, high quality ground truth images corresponding to low quality real images are not available. The experimental part of this work was performed on a real noisy digital image from an open-source platform. The image was processed using a specialized processing pipeline developed in Python. The PyTorch library was used to run the DL model and some dedicated libraries are also included such as Real-ESRGAN as the default model to enhance the image quality, BasicSR as the generic framework to manage the process workflow, and the installation of GFPGAN and FaceXLib to facilitate optional facial restoration. Prior to all other testing were done on the CPU, these experiments were conducted using only the Central Processing Unit(DCPU) and not any high end Graphical Processing Units(GPUs) The processing approach was to use the Real-ESRGAN model only in the inference stage without any training process or modification on the model architecture. The discussion of results was grounded on qualitative visual inspection aided with a descriptive quantitative analysis of result indicators that can be obtained from the image itself before and after processing, such as the image size, the total number of pixels, and the spatial upscaling factor. The result indicates a 16 times enlargement in pixel number after processing (under the upscaling factor of ×4), as well as an intuitive improvement on clearness of details and the visual noise decreasing, which means the enhancement on perceptual quality of the image. Results show that exploiting pre-trained DNN models is a realistic and time-efficient strategy for improving quality of noisy images of the real world, even if computational power is scarce at the time of acquisition of images. In addition, the work demonstrates the feasibility of leveraging open source software in the domain of digital imaging and thus paves the way for potential future investigation incorporating larger datasets or employing standardized quantitative evaluation metrics when appropriate evaluation conditions are obtainable.