A comparison of patch-based models in video denoising

Pablo Arias, Gabriele Facciolo, Jean-Michel Morel
CMLA, ENS Paris-Saclay


Overview


Several state-of-the-art patch-based methods for video denoising rely on grouping similar patches and jointly denoising them. Different denoising algorithms have been proposed that are based on different models for the groups of patches. These models differ on their complexity. In general, more complex models achieve better results but are also more costly to compute.

But the modeling of the groups of patches is not the only difference between the algorithms. Other differences can be the type of patches, the search strategies used for determining the groups of similar patches and the weights used in the aggregation. This makes it difficult to determine the actual impact of the patch model on the results.

In this work we compare two of the models that have produced better results in equal conditions: models based on a fixed transform like BM3D, and adaptive transforms, which are more costly but have produced better results. In addition we propose a third simple model which can be interpreted as a non-local version of the classical DCT denoising and add it to the comparison. This allows to have a clearer picture of the performance trade-offs between the complexity of the models and the denoising performance.


Publications


P. Arias, G. Facciolo, J.-M. Morel, A comparison of patch-based models in video denoising. IVMSP, 2018.

P.Arias, J.-M. Morel Video denoising via empirical Bayesian estimation of space-time patches. JMIV 60(1):70-93, 2017.


Results on grayscale sequences



Results on the pedestrian area test sequence (960x540 grayscale).


In the table bellow, we show results obtained for seven 960x540 grayscale test sequences. The sequences were taken from Derf's video databased. The originals are RGB sequences of resolution 1920x1080. We converted them to grayscale by averaging the channels and downscaled them.

The sequence names in the table header and some of the PSNR values in the table are links to the corresponding videos. The videos are uncompressed.

We compared the following algorithms:

SPTWO:
Method of [Buades, Lisani, Miladinović, 2016]
V-BM4D-mp:
author's implementation of [Maggioni et al. 2012] setting the parameters according to the modified profile (best parameter profile available)
V-BM3D-np:
author's implementation of [Dabov, Foi, Egiazarian, 2007] setting the parameters according to the normal profile (best parameter profile available)
VNLB:
Method of [Arias, Morel, 2017], using the hard-thresholding estimator for the a priori variances (VNLB-H in the paper), with a 3D patch size of 10x10x2.
VNLDCT:
Version of the block DCT denoising using a Wiener filter estimated from similar patches. Patch size: 10x10x2. See the article for more details.
BM4D-OF:
Slightly modified version of the BM4D method [Maggioni et al. 2013] using 3D patches 10x10x2. See the article for more details.



PSNR (dB) results on 960x540 grayscale test sequences.
Click on the PSNR values to download a zip containing the corresponding result.
The highest PSNR for each sequence and noise level is written in boldface.

σ Method crowd park joy pedestrians station sunflower touchdown tractor Average
10 Noisy 28.13 28.13 28.13 28.13 28.13 28.13 28.13 28.13
SPTWO 36.57 35.87 41.02 41.24 42.84 40.45 38.92 39.56
V-BM3D-np 35.76 35.00 40.90 39.14 40.13 39.25 37.51 38.24
V-BM4D-mp 36.05 35.31 40.61 40.85 41.88 39.79 37.73 38.88
VNLB 37.24 36.48 42.23 42.14 43.70 41.23 40.20 40.57
VNLDCT 35.96 35.23 41.37 41.12 42.43 40.37 39.00 39.35
BM4D-OF 35.86 35.15 41.55 41.75 42.76 40.62 39.09 39.54
20 Noisy 22.11 22.11 22.11 22.11 22.11 22.11 22.11 22.11
SPTWO 32.94 32.35 37.01 38.09 38.83 37.55 35.15 35.99
V-BM3D-np 32.34 31.50 37.06 35.91 36.25 36.17 33.53 34.68
V-BM4D-mp 32.40 31.60 36.72 36.84 37.78 36.44 33.95 35.10
VNLB 33.49 32.80 38.61 38.78 39.82 37.47 36.67 36.81
VNLDCT 32.62 31.94 37.88 37.88 38.92 37.15 35.58 36.00
BM4D-OF 32.52 31.79 38.14 38.22 39.38 37.51 35.76 36.19
40 Noisy 16.09 16.09 16.09 16.09 16.09 16.09 16.09 16.09
SPTWO 29.02 28.79 31.32 32.37 32.61 31.80 30.61 30.93
V-BM3D-np 28.73 27.93 33.00 32.57 32.39 33.38 29.80 31.11
V-BM4D-mp 28.72 27.99 32.62 32.93 33.66 33.68 30.20 31.40
VNLB 29.88 29.28 34.68 34.65 35.44 34.18 32.58 32.95
VNLDCT 29.32 28.73 34.14 34.23 34.92 33.76 31.95 32.43
BM4D-OF 29.39 28.70 34.57 34.46 35.78 34.67 32.09 32.81


Institutional acknowledgements


Work supported by IDEX Paris-Saclay IDI 2016, ANR-11-IDEX-0003-02, ONR grant N00014-17-1-2552, CNES MISS project, DGA Astrid ANR-17-ASTR-0013-01, DGA ANR-16-DEFA-0004-01, and MENRT.