Parameter-Efficient Harmonic Networks for JPEG Compression Artifact Removal

Many modern learning algorithms try to solve JPEG compression artifact removal (CAR) problem in pixel domain by mapping low-quality compressed image to high-quality image. Although JPEG artifacts arise from quantizing DCT coefficients of non-overlapped image blocks, researchers utilize transform domain as auxiliary information at most. On the other hand, it is well known and approved that extracting image blocks with overlap improves decompression performance. Inspired by these observations, we propose novel and fully transform domain convolutional neural networks (CNNs) for the problem. We choose DCT and DST, another effective DCT-like transform in terms of energy compactness, as the transform to be utilized and refer them as harmonic transforms. We perform harmonic transform on small overlapping blocks of compressed image in the first layer of the proposed networks, and we create spectral feature maps by properly ordering their harmonic transform coefficients with the same frequency. After a series of convolution blocks, we take inverse harmonic transform of the corresponding image blocks at the end of the network and put the resulting decompressed blocks back in their place. We show that forward and inverse transform layers of our harmonic networks are efficiently implemented with fast convolution and deconvolution layers by using 2D harmonic basis images as convolution kernels with a mathematical justification. Experimental study indicates that although our harmonic networks have a simple network topology and much fewer parameters than compared state-of-the-art deep networks, they are effective and efficient to suppress compression artifacts and give comparable results.