Wavelet Based Video Denoising using Probabilistic Models

Wavelet based image processing techniques do not strictly follow the conventional probabilistic models that are unrealistic for real world images. However, the key features of joint probability distributions of wavelet coefficients are well captured by HMT (Hidden Markov Tree) model. This paper presents the HMT model based technique consisting of Wavelet based Multiresolution analysis to enhance the results in image processing applications such as compression, classification and denoising. The proposed technique is applied to colored video sequences by implementing the algorithm on each video frame independently. A 2D (Two Dimensional) DWT (Discrete Wavelet Transform) is used which is implemented on popular HMT model used in the framework of Expectation-Maximization algorithm. The proposed technique can properly exploit the temporal dependencies of wavelet coefficients and their non-Gaussian performance as opposed to existing wavelet based denoising techniques which consider the wavelet coefficients to be jointly Gaussian or independent. Denoised frames are obtained by processing the wavelet coefficients inversely. Comparison of proposed method with the existing techniques based on CPSNR (Coloured Peak Signal to Noise Ratio), PCC (Pearson’s Correlation Coefficient) and MSSIM (Mean Structural Similarity Index) has been carried out in detail . The proposed denoising method reveals improved results in terms of quantitative and qualitative analysis for both additive and multiplicative noise and retains nearly all the structural contents of a video frame.

are limited in their scope as these methods do not take temporal correlation between frames into account [1][2][3].
Wiener filter is an example of a spatial filter that removes spatial noise from images and succeeds in achieving high gain. However, this filter cannot restore edges especially in less noisy areas [4].
Temporal domain methods consider the inter-frame correlation between frames and perform well for still videos without motion [5]. In the case of videos having motion, temporal domain methods do not provide significant results. Ozkan et. al. [6] proposed a temporal filter for denoising of frames that provide considerably good results in the noise removal process and produce less blocking artifacts but it causes blurring effect.
Liu and Luo [7] introduced a method based on TV (Total Variation) and temporal filtering for image denoising. The temporal filter maintains structure and edges well but it cannot reduce noise. The TV algorithm is applied to a noisy frame to reduce noise but it could not restore structure information.
Spatio-temporal methods consider both spatial and temporal correlations between different frames in a video sequence and provide efficient results.
Maggioni et. al. [8] have addressed the problem of denoising in video sequences that are corrupted by random and fixed pattern noise. In this method, the data is sparsified in 3D spatiotemporal transform domain by leveraging the spatial and temporal correlations within each volume. 3D threshold array is used to shrink the coefficients of the 3D volume spectrum.
Wang et. al. [9] proposed a spatial-temporal depth filter by jointly using the depth and texture information in the spatial-temporal domain that performs filtering in three steps. First, a pixel to be filtered and its reference pixels are selected based on the similar pixel vectors. Second, the most correlated pixels are recognized among reference pixels. Finally, the to-be-filtered pixel is obtained by using a median filter among the reference pixels.
Transform domain methods exploit the sparsity of data and have good localization properties and multiresolution characteristics in either temporal domain or frequency domain. These properties make it more useful to separate a useful signal from noise. Hence, wavelet has gained popularity for image denoising [10][11]. Wavelet transform can be 2D or 3D. 3D transform domain methods do not perform well for denoising purpose because of long delay and inability to adapt to fast motions in a video sequence [12].
Zhigang et. al. [13] proposed wavelet based threshold function to overcome the discontinuity of the hard threshold method and the soft threshold method at the threshold value.
Ho and Hwang [15] proposed an image noise reduction method through the wavelet domain Bayesian threshold criterion coefficient of shrinkage method.
Techniques like VBM3D and E-RF3D have been the most efficient ones in denoising as they exploit DCT (Discrete Cosine Transform) in their framework [16].
Non-local means technique proposed in [17] performing efficient denoising and have become popular in recent years. Some of the denoising methods are based on motion estimation and compensation process as the removal of noise and visual quality is mainly dependent on the amount of motion occured in a video sequence.
Zuo et. al. [18] proposed a video noise removal method to exploit spatial-temporal correlations between different frames. First of all, the motion is estimated between current noisy and previously denoised frames and then Kalmanbilateral filtering is applied to the current noisy frame.
Aydin and Foroosh [19] proposed wavelet based METF (Motion Estimation Temporal Filtering) that applies ME directly on wavelet coefficients.

Wavelet Based Video Denoising using Probabilistic Models
Hong-Zhi et. al. [20] introduced a spatiotemporal method to minimize noise in video frames by discriminating the still regions from moving regions. In this technique, Kalman bilateral filtering is applied to still regions that do not show any motion and spatial bilateral filtering is performed on moving regions.
Although there are a number of denoising techniques but there is always a room for improvement. The proposed method is an extension of HMT based framework used in [14,21].
In [14], HMT based statistical signal processing technique is proposed for compression of signals in wavelet domain.
Whereas in [21], denoising of gray scale images using HMT is presented. Using the above modeling framework, a combined spatial and temporal filtering technique is proposed in this paper that can remove Gaussian as well as speckle noise from color image and video sequences considerably. By exploiting the dependencies among wavelet coefficients, better performance has been achieved. The proposed method deals with non-Gaussian behavior of wavelet coefficients that are often encountered in practice and gives efficient results for de-speckling of images as well. The results show that the proposed method dose not remove noise only but also retains almost all the structural information of a video frame.
The remaining part of this paper is organized as follows.
In Section 2, we discuss about the wavelet-based denoising model. In Section 3, modeling of wavelet coefficients by using HMT model is elaborated. Section 4 defines the proposed denoising model. In Section 5 the experimental results and discussion of the proposed model are given. Finally, we conclude this paper in the last section.

WAVELET-BASED VIDEO DENOISING
A WT leads to a sparse and efficient representation of an image as it hybrid the spatial and transform domain. 2D-DWT has been used in this paper.
DWT decompose the image into one low-frequency subband and several high-frequency sub-bands in such a way that most of the important information is concentrated in LL sub-band of the highest level also known as DWT approximation as shown in Fig. 1(a-b).

Capturing Non-Gaussian Densities
The non-Gaussian density of wavelet coefficients can be captured efficiently by GMM (Gaussian Mixture Model) and a multidimensional GMM is referred to as HMT. HMT models the wavelet coefficients as random variables having probability density function as a mixture of zero mean Gaussian distributions by means of a hidden state to designate small and large coefficient.
The pdf of the wavelet coefficient C is defined as: where p s (n) is pmf (probability mass function), S is the hidden state variable which is invisible and it controls the magnitude of wavelet coefficient.
is the conditional pmf given by the following Equation (4): where  n and  n are the mean and variance respectively.

Capturing Dependencies
For The state transition matrix shows parent  children state to state links between the hidden states that is given as: shows the child coefficient is in state u given parent coefficient is in state w [23]. Also, represent the probability of a wavelet coefficient to be small or large given its parent is small or large. All the parameters of the HMT model are grouped together in the form of vector θ given in Equation (6).
It is to be noted here that each wavelet coefficient has different variances and state transition probabilities which lead to greater complexity in the HMT model. We can reduce this computational complexity by a method of tying within scale [11]. According to this method, the wavelet coefficients have the same density within a scale.

DENOISING TECHNIQUE
HMT based denoising technique in the perspective of

Noisy wavelet Coefficients
Let Q be a natural clean image with NxN dimension and Q' be its noisy version such that Q' = E where E is zero mean white Gaussian noise. By performing wavelet decomposition on Q' the wavelet coefficient q'is obtained. Due to the linearity of the wavelet transform, we have: where q and e are the wavelet coefficients of Q and E respectively. We need to estimate the q given q'

Model Parameters Determination
HMT model is used to find a set of parameters  q ,. Initially, a two-state GMM is used to characterize each wavelet coefficient and a noisy observation is used to initiate the HMT model. Then the interscale dependencies are captured by the Markov-tree and EM algorithm is used to obtain  q' .
According to [10], the added noise in a signal only increases its variance by leaving the other parameters unchanged.
Hence, the noise free observation  q can be extracted by fitting the HMT to the noisy observation and then subtracting the noise variance from it.

Clean Coefficients
To determine the noise free coefficients, the EM algorithm is used for training the model. There is a need to determine the noise free coefficients q from q'.
The conditional pmf of hidden states S and its maximization is given by following expressions: Once  q is determined [10] and state probability is given through HMT, we can get   q q q E  ,   by using Bayes estimator to get the clean coefficients:  (12) where j,k, denote the m-th coefficient in scale j and subband k.

Reconstructed Frames
At the end, the inverse wavelet transform is applied to the obtained clean coefficients to get the reconstructed frames of a video sequence. The algorithm for the proposed denoising technique can be summarized as follows:

Denoising Algorithm
Add AWGN noise to the video sequence Apply inverse wavelet transform to get the reconstructed frames.  FIG. 3(a). ORIGINAL IMAGE FIG. 3(b). NOISY IMAGE WITH SIGMA=15   FIG. 3(c).

CONCLUSIONS
This paper presents a video denoising technique that is based on the HMT model for color video denoising. It is used in the framework of 2D-GMM and 2D-DWT scales and location. DWT has a good performance in a task like a video compression and denoising.The excellence of this method is in parent-children correlation within and across the scales.
Experimental results reveal that the proposed method outperforms the existing state-of-the-art techniques for color video sequences both in terms of qualitative and quantitative analysis. This method is capable of noise reduction and edge preservation.
The computational complexity of the proposed algorithm is less in terms of its execution time as compared with NLMC, VBM3D and CIFIC. In future, this technique will be extended to high-resolution images.