Coherent Spatial and Colour Blended Exemplar Inpainting

In an image processing field the digital image recovery is termed as inpainting. Efficient retrieval of an image, especially having large objects with high curvature and complex texture is an immensely challenging problem for image inpainting researchers and practitioner. This enthused researchers and emerge various inpainting algorithms and many are in progress. Generally inpainting techniques approaches the available area source of given image(s) to restore the unavailable area target by the information available at the target edge. This paper represents a novel approach BSDD (Blended Spatial and Dimensional Distances) by sampling patches at each pixel of the source region. From the given sample, selection of local edge patch is gradient based without priority computation overhead as previous techniques. These local patches are searched globally by linear distance in which both spatial and dimensional distances are considered with regularization factor. The main motive of this method consists in achieving the efficiency, curvature and textural challenges of inpainting without compromising the quality of inpainted image. We have tested the proposed method in real as well as synthetic images with high curvature and complex textures in all cases results are comparable with other well-known techniques. In view of quality and optical the proposed algorithm exhibits better results.


INTRODUCTION
direction of the largest spatial change and this vector is rotated to 90 degrees. Local variation in colour smoothness is estimated using 2D (Two Dimensional) laplacian and such variation is propagated along the isophotes directions. For preserving boundaries across the inpainted region anisotropic diffusions is utilized, on few iteration impressive result are produced but requires several minutes for relatively small area soon after Chen and Shan [2] proposed TV (Total Variation) diffusion based algorithm on contrast isophotes using the Eularlagrange equation inside the mask region this was helpful for noise removing but unable to connect broken edges this results W ell studied problem image inpainting aims to restore the effected portion of the image in plausible way as shown in Fig. 1. Digital inpainting is very popular in present era of digital media typically literature defines the variational and exemplar model to approach inpainting problems. Diffusion based approaches (Bertalmio et. al. [1]) pioneered a digital image inpainting algorithm based on PDEs (Partial Differential Equations). These algorithms worked on mask specify portions of the input image to be retouched. The discredited gradient of the mask region along level lines (Isphotes) is computed at each pixel which gives the the extension as CDD (Curvature Driven Diffusion) [3] that preserves the geometric information while defining strength of isophotes this have overcome broken edge issue but introduces blur in resultant inpainted image later computational cost of PDEs alleviate by introducing FM (Fast Marching) [4] it smoothes image by determining weighted average of target pixel from its known neighbours, It inpaints the near neighbours pixel first and preserve the narrow band pixels that separates on pixels and off pixels and also highlight next pixel to be inpaint. These algorithms utilizes the first order and second order differential equation which are complex and computationally expensive later Gillette and Bertozzi [5] introduced binary image inpainting with fourth order PDE using modified Cahn-Hilliard equation Bertozzi model was relatively simple model that has desired properties of curvature driven inpainting model like smooth connection of broken edges.
Later Tai [6] and Tai and Paul [7] utilizes decomposition method DDM (Digital Development Management) for solving PDEs i.e. for solving the original image problem that divides the problem constrained into sequential subspaces and solve in parallel respectively. For small structure propagation PDEs are quite good but fails when large objects background of natural image needs to recuperate, somehow two dimensional textural synthesis managed large objects background recovery by searching globally the texture pattern of target edge in the source region, the most compatible texture pattern of source region is partially replicate to the target region [8][9][10][11][12][13][14][15][16] while processing large natural textured image these algorithms are not satisfactory in fill front. The (Bertalmio et. al. [17]) utilizes both texture and structure information for propagating source to target region by splitting the image in two functions, first function decomposes bounded variation and second deals with texture and noise, this algorithm is limited to small hole. For large natural background construction exemplar approaches are favourable, Freeman and Egon [18] emphasis on the texture of surrounding known pixels along the target edge, but have probability of suffering as surrounding noise sometime effects or diminish linear structure. Harrison [14] represent a Non-hierarchical model for reconstructing an image using two dimensional mask with the same texture for localizing closed neighbor pixels, Morel [19] has proposed restoration model by hierarchical approach, the first level of hierarchy label the textured map and second, segment composite texture into sub texture. Criminisi and Jeremy [9], Criminisi [20] combined both textural and structural properties for reconstruction of missing signal of an image, this model defines priorities to every patch at contour of the target region centralized at each pixels on the basis of confidence and data terms, Won [21] defined weights to the global similarity samples to known source region of an image and these weighted samples are utilized to recuperate effected region of image that produces blur in resultant inpainted image. Sangeetha et. al. [22] and Muthu and Vallilka [23] has extended Criminisi [20] algorithm in two ways first by defining additive weights to priority terms instead of multiplicative confidence and data term priorities and second by restricting search area on an assumption that the most resembled patch lie very close to the target patch, at run time a rectangle is introduced for limiting search area. Ankur et. al. [24] sample gradient base image on the assumption that colour based linear structure interpolation is not sufficient. The proposed novel algorithm BSDD uses linear distances for finding similarity between the two patches refers section 2. Unlike existing exemplar techniques, the criteria for target patch selection is gradient based. The proposed algorithm revoked priorities computation on the basis of confidence and data term, as priorities based target patch selection can increase the probability of unwanted interpolation and may lead to a wrong inpainted result. Fundamentals of algorithm are in section 2.1, detail of algorithm is under section 2.2, for comparative results refer section 3.

BLENDED SPATIAL AND DIMENSIONALDISTANCES
In general exemplar based inpainting algorithms [22][23][24] followed Criminisi and Jeremy [9] and Criminisi [20] and uses dimensional distances for similarity measures. Considering colours distances alone may lead to unwanted inpainting result as the only colour similarity cannot assure continuous stationary texture hence proposed BSDD is good for finding similarity between two patches. Formally, Where D is the blended spatial and dimensional distance. D c is the Euclidean distance in colour space, D s is the euclidean distance of spatially closet patch cantered at pixel p, measured using 2D binary mask, as shown in Fig.   2, and α α α α α is a regularization factor lie between [0 1].

Proposed Inpainting Methodology
In an aberrant Image I, region of interest (target/missing signal) is indicated by Ω, known source region is  .
The algorithm initiates from target boundary, update and propagates inward in eachiteration. A single iteration of algorithm is defined by selecting a patch ψ p such that its centre pixel p is at target boundary and ψ p ∈(Ω∩ψ Ω∩ψ Ω∩ψ Ω∩ψ Ω∩ψ) where ψ p ∈Ω Ω Ω Ω Ω needs to recuperate by the available dimensional and spatial information of ψ p ∈ the most similar patch ψ q in source region is find using BSDD, refer section II-B, the target portion of ψ ψ ψ ψ ψ p is filled by the correspond portion of obtained closet patch ψ ψ ψ ψ ψ q as shown in Fig. 2. The process continues till the whole target region gets restored.

Best Patch Search
Algorithm starts by creating patch space S, by localizing a patch of size axb at all the pixel in an Image I, such that each patch is used to form the column of matrix V MxN . The dimension of V S is MxN where M=3xaxb for RGB colour

FIG. 2. ANOMALOUS IMAGE TREATED WITH BSDD
image and is the number of patches in space S as shown in Fig. 3.
The search space V s MxN is obtained by separating the target and partial patches from , the separated portion is transformed into V t MxN this separation is necessary as these patches are used for finding the best exemplar for fill front using BSDD Equation (2) as: To find the best exemplar ψ q first the spatial distance D qs are measured with the help of binary mask considering Euclidean distance and the available colour information of ψ p is compared in V s MxN using L2 norm as: Which gives the minimum colour dimensional distance D qc . The combination of D qc and D qs is the minimum optimal distance D q of desired source patch ψ q localized at the centre pixel q. After obtaining the most similar patch ψ q , the missing region of p is filled in by the corresponding signal of q.

RESULTS AND COMPARISON
The natural and synthetic images having high curvature and complex texture are recuperated by BSDD and compared with Criminisi and Jeremy [9].The algorithm is tested on fifteen images out of which seven images having different level of difficulties in terms of texture and structure are presented in the paper, similar to the evaluation found in other articles on inpainting. Visual and quantitative statistics shows that our algorithm can create better background with texture pattern without leaving blurring and unpleasant effect to a normal eye.

Visual Comparison
We tested BSDD on multiple images some are presented here, visual comparison shows our algorithm performs well for reconstructing background and is more acceptable to normal eye as shown in Fig. 4.