A Global Sampling Based Image Matting Using Non-Negative Matrix Factorization

Image matting is a technique in which a foreground is separated from the background of a given image along with the pixel wise opacity. This foreground can then be seamlessly composited in a different background to obtain a novel scene. This paper presents a global non-parametric sampling algorithm over image patches and utilizes a dimension reduction technique known as NMF (Non-Negative Matrix Factorization). Although some existing non-parametric approaches use large nearby foreground and background regions to sample patches but these approaches fail to take the whole image to sample patches. It is because of the high memory and computational requirements. The use of NMF in the proposed algorithm allows the dimension reduction which reduces the computational cost and memory requirement. The use of NMF also allow the proposed approach to use the whole foreground and background region in the image and reduces the patch complexity and help in efficient patch sampling. The use of patches not only allows the incorporation of the pixel colour but also the local image structure. The use of local structures in the image is important to estimate a high-quality alpha matte especially in the images which have regions containing high texture. The proposed algorithm is evaluated on the standard data set and obtained results are comparable to the state-of-the-art matting techniques.


INTRODUCTION
background and α=1 for definite foreground object while 0 << 1 represents the semi-transparent foreground pixels.
The problem is inherently ill posed because Equation (1) consists of more unknown terms F,B and than the known term I. The problem is alleviated by manually aiding the information of the definite foreground, background and semitransparent regions in the form of a tri-level map as illustrated in Fig. 1. The information from the definite regions is used to solve Equation (1) for the semitransparent foreground pixels.
Equation (1) shows that an image I is the combination of two layers F, B and alpha, weighted by the blending proportion. The pixels which have α=0 refer asdefinite Approaches like [2][3][4] have defined different affinities among the neighboring pixels because there exist a strong correlation among the local pixels that could be used for smoothness assumption. Sun et. al. [2] applied Poisson formulation to Equation (1), under the assumption that the intensity changes are locally smooth. Grady et. al. [3] proposed a method, similar to [2], that is based on Random Walks and define the affinity matrix in term of Gaussian function. Levin et. al. [4] formulated a novel quadratic cost function in α to obtained the alpha matte. Since matting information is propagated from the known regions to the unknown region pixels these approaches are prone to error propagation across the matte.
Some approaches like [5][6][7] fit statistical models on the pixels of the local known regions to estimate the colour and alpha value of each unknown region pixels. Recent methods [8][9][10][11][12] estimates alpha values by sampling local known regions pixels non-parametrically. These approaches fail where the local samples are not representative of the true colours of the unknown region pixels. In [8] authors have used Non-negative matrix factorization but it is limited to the foreground and background patch spatially closest to the unknown pixel and does not colour information in patch sampling. Like other approaches [8] is also limited to the local regions.
This work presents a global non-parametric sampling based approach to overcome the aforementioned problem present in the local non-parametric modeling. We sampled image patches as in [11] instead of pixel to incorporate the local image structure along with colour information.
NMF [13] is used to efficiently determine the similar patches of the known regions for each unknown region pixels. Only high confidence sample pairs are chosen using linear analysis to estimate the alpha matte.

RELATED WORK
The first reported work on non-parametric sampling was Corel Knockout [14]. Foreground colour is estimated by using weighted average of the local samples at the unknown foreground boundary. Background colour is also estimated first by taking weighted average and then refined.
Wang and Cohen [9] introduced the colour fitness criteria for the spatially close sample pixels. Higher confidence sample pairs are used to estimate alpha. Rhemann et. al.
[10] used a similar approach [9] but the selection of the candidate samples are based on geodesic distance instead of Euclidean. Gastal and Oliveira [12] sample pixels that lies along rays shooting from the unknown pixels and samples are further shared among neighbouring pixels.

NFM FACTORIZATION
A dimension reduction technique NMF [13] is used to find the linear appropriate representation of non-negative data. If a M dimensional data with N samples is denoted by a matrix X MXN , then the linear approximation is given as:

FIG. 1. A NATURAL IMAGE ALONG WITH A MANUALLY DEFINED TRIMAP
where, W is a basic matrix with columns representing basic vector, H is a coefficient matrix with columns as coefficient vector and 0 is the order of decomposition.
The optimal W and H for X can be obtained by iteratively minimizing the error function: In order to minimizing the cost function Equation (3), many algorithms area available [15]. In this paper, Alternate least square optimization method is used to minimize Equation (3) due to its fast convergence rate and consistency.

IMAGE MATTING USING NMF
Proposed algorithm follows as (1)

Collecting Candidate Samples
Initially an image is divided into foreground, background and unknown regions. A global foreground patch space S f is constructed by localising a patch of dimension mxn at each pixel in the known region. Since NMF algorithm requires matrix to be two dimensional so foreground model M f is formed by arranging each ψ i ∈S f , of size mxn, as a column of: Since we are using RGB (Red, Green and Blue) colour space so the dimension of M=mxnx3 and N represents the total number of patches in foreground F. To find the similar samples for a patch φ p in the unknown region, foreground model M f can be factorized as: A patch φ p is the unknown region can be approximated by projecting coefficient matrix H p on the basis matrix W f as: The most similar patch ψ q , is the foreground model V f , to φ p is obtained by minimizing the L2 norm of H p and H i as: Colour of the center pixel of the patch ψ q as taken as the foreground colour to the pixel p is the unknown region as shown in Fig. 2. If the foreground contains textures then using only the most similar patch to estimate foreground colour could lead to segmentation inaccuracies hence N most similar patches are used to find the optimum foreground colour. Similar approach is used to determine most similar patches in the background region.

Selecting Best Candidate Samples
After collecting multiple candidate samples, the best foreground and background pair of [F p ,B p ], for a pixel p is obtained as: The alpha values used for optimization ranges in [0 1] with an interval of 0.01. The optimization is performed for all unknown region pixels. The resulted alpha matte might contain discontinuities where optimum pair is not representative of the true colour of unknown region pixels.
Matte optimization is used to improve the initially estimated alpha matte as shown in Fig. 3.

Matte Optimization
A matte optimization approach similar to shared matting [12] and a cost function is defined as: Matting Laplacian L is taken as in [4] while λ and g are the weighting parameters. The matrix D whose diagonal elements are 1 for known and 0 for unknown pixels while matrix Γ having diagonal element 0 for known pixels and a confidence C for each unknown pixel which is computed as: Where σ is a scaling parameter. This optimization process provides an alpha α propagation from pixels have high confidence value to the pixels with low confidence.

RESULTS AND EVALUATION
The

Dataset
For Evaluation a benchmark image matting data set from [16] is used. The dataset contains different images having different level of complexity regarding matte estimation.
The proposed work is evaluated on images containing long, thin and intricate boundaries to show the effectiveness of the method. The images also contain holes within the foreground object which pose a strong problem for matting algorithms. The dataset contain images having an approximate spatial resolution of 800x640.

Qualitative Evaluation
For qualitative evaluation two images with different level of difficulty are considered in Fig. 4. The first image has

Quantitative Evaluation
Ground truth along with the alpha mattes obtained using different algorithms are shown in Fig. 5

CONCLUSION
A global non-parametric sample based natural image matting using NMF is presented which not only consider the colour but also the local image structure to provide better alpha mattes. The comparison of proposed algorithm with simple SSD shows that the reduction in the dimension in search space using NMF results in efficient comparison with no visible lose to matte quality.
The qualitative and quantitative comparison shows that the proposed method performs better than the state-ofthe-art matting approaches especially for images that contain texture, intricate boundaries like hair and large transparent regions. The mean square error values suggest that the proposed approach estimates alpha matte with significant reduction in the overall error. In future, the use of weights in matrix factorization would be investigated to improve matte estimation. This would allow the algorithm to give different importance to different patches. The weights could be assigned based upon spatial distance and colour similarity of the patches.

ACKNOWLEDGEMENT
Authors are thankful to the Department of Computer