A Detailed Survey on Various Image Inpainting Techniques

A Detailed Survey on Various Image Inpainting Techniques

V. Janarthanan and G. Jananii

Abstract

Inpainting, the technique of transform an image in an imperceptible form, is as past as art itself. The main objective of inpainting is from the reinstallation of damaged paintings and photographs to the elimination of chosen objects. Image Inpainting is used to filling the misplaced or smashed region in an image make use of spatial information of its neighbouring region. Inpainting algorithm have numerous applications. It is attentively used for restoration of older films and object removal in digital photographs. It is also useful to red-eye correction, compression etc. The objective of the Inpainting is to change the damaged region in an image in which the inpainted region is invisible to the common observers who are not familiar with the original image. There have been quite a few approaches are proposed for the image inpainting techniques. This proposed work presents a brief survey of different image inpainting techniques and relative study of these techniques. In this paper provide an analysis of different techniques used for image Inpainting. Finally a best inpainting technique is suggested in this paper.

Keywords-- Inpainting, Filling Region, Object Removal, Patch Propagation, Texture Synthesis

I. INTRODUCTION

THE filling of lost information is essential in image processing, with applications as well as image coding and wireless image transmission, special effects and image restoration. The basic idea at the back of the algorithms that have been proposed in the literature is to fill-in these regions with available information from their environment. The alteration of images in a way that is non-detectable for an observer who do not be acquainted with the original image is a practice as old as inventive creation itself. Medieval artwork started to be restored as early as the rebirth, the motives being frequently as much to bring medieval pictures “up to date” as to fill in any gaps [1, 2]. This practice is called inpainting.

Inpainting is conceded out by specialized artist and generally its very time consuming process because it is the manual process. The major objective of this procedure is to rebuild damaged parts or missing parts of image. In addition to this process the reconstructs image in such a way that the inpainted area won’t be identified by a casual observer. Inpainting technique has set up a extensive use in many applications such as restoration of old films, object removal in

^[1]digital photos, red eye correction, super declaration, compression, image coding and communication. Image Inpainting restructure the damaged region or mislaid parts in an image utilizing spatial information of neighbouring region. Image Inpainting could also be called as alteration and exploitation of an image. In image inpainting would like to create original image but it is absolutely not viable without the prior knowledge about the image. In case of digital images, only the available image is taken for the experiment and thus filling in a hole that encompasses a whole object. It is not possible to replace that whole object based on the required current information. By considering this as a plan of the inpainting algorithm and it is not simply rebuild what would be used in that hole. But instead of creating a visually pleasant continuance of the data nearer to the hole in such a way that it is not detected by a normal observer.

In this paper, different types of image inpainting techniques presented are discussed. Section 2 discuss about the survey on image inpainting techniques. Inferences of the existing works are discussed in section 3. Section four discuss about the conclusion.

II. SURVEY ON IMAGE INPAINTING TECHNIQUES

Diffusion based Inpainting was the first digital Inpainting technique in which missing region is filled by diffusing the image data on or after the identified region into the missing region at the pixel point. Fundamentally these algorithms are based on the variational method and Partial Differential equation (PDE). The diffusion based Inpainting algorithm which may produces accurate results or filling the nontextured regions or comparatively lesser mislaid region. The drawbacks of the diffusion process are it initiates a number of blur, which becomes obvious when filling larger regions. Every PDE based inpainting models are more appropriate for carrying out small, non-textured target region [3].

Kokaram et al. [4] use movement estimation and autoregressive models to exclaim losses in films from adjacent frames. The fundamental idea is to copy into the breach the right pixels from neighboring frames. The system cannot be applied to still images or to films where the regions to be inpainted span many frames.

Hirani and Totsuka [5] unite frequency and spatial domain information in turn to fill up a specified region with a selected texture. This is a very easy technique that produces unbelievable high-quality results. Alternatively, the algorithm mostly deals with texture synthesis and not with prearranged background, and requires the user to select the texture to be copied into the region to be inpainted. For images anywhere

the region is to be replaced covers more than a few different structures, the user would require to go through the great work of segmenting them and searching through the corresponding replacements throughout the picture. Though a part of this search can be finished without human intervention, this is extremely time consuming and requires the non-trivial selection of many critical parameters [6]. Other texture synthesis algorithms are presented in [6, 7], can be used as well to restructure a pre-selected texture to fill-in a square region to be inpainted. In the collection of disocclusion algorithms, a pioneering works. A technique is presented for removing occlusions with the target of image segmentation. The essential idea is to link the T-junctions at the similar graylevel with elastically reducing the curves. The method was mostly introduced for simple images, with only a few objects with stable gray-levels, and will not be appropriate for the examples with natural images presented. Masnou and Morel introduced a technique in recent times which enlarge these ideas, by an extremely inspiring general variational formulation for disocclusion and a practical algorithm not completely based on PDE’s implementing some of the ideas in this formulation. The algorithm performs inpainting by amalgamation with geodesic curves the points of the isophotes lines of equal gray values incoming at the boundary of the region to be inpainted. Since reported by the authors, the regions to be inpainted are deficient to having effortless topology, e.g., holes are not allowed. Additionally, the angle with which the level lines reach your destination at the boundary of the inpainted region is not potted well the algorithm uses straight lines to join equal gray value pixels.

Semi-automatic image inpainting necessitates user assistance over there. The required user assistance is in the form of guide lines to help in structure completion has found favor with researchers. Jian et.al [8] proposed a method of inpainting with Structure propagation. This technique follows a two-step process in which the first step is a user physically specifies significant missing information in the hole by drawing object borders from the known to the unknown region and after that a patch based texture synthesis is used to make the texture. The misplaced image patches are produced all along the user specified curves by preparing the problem as a worldwide optimization problem under a variety of structural and constancy constraints. Effortless dynamic programming can be used to obtain the optimal reply if only a single curve is in attendance. Intended for multiple objects, the optimization is immense deal with more difficult and proposes approximated answer by means of certainty propagation. Each and every method discussed on top take minutes to hours to end depending on the size of the Inpainting area and therefore making it as an insupportable for interactive user applications. To speed up the conventional technique of image Inpainting algorithms, new classes of speedy Inpainting techniques are being developed.
I. A. Ismail et al. [9] have planned an integrated or an indulged technique for the recognition and removal of cracks on digitized images. By means of Steepest Descent Algorithm (SDA), to begin with the cracks have been acknowledged. Afterward, the identified crack has been purged by means of both a gradient Function (GRF) and processed data or a
semiautomatic method based on region increasing. Finally, crack is filled and it has been carried out using the steepest descent method. The technique has been implemented using Matlab, Surfer and Visual Fortran programming. Experimental results have shown that their technique has performed efficiently on digitized images agony from cracks.

Gunamani Jena [10] has presented an inpainting algorithm, which is used to fill the damaged region with remarkable results. Many algorithms that are presented before generally required several minutes for inpainting the smaller areas on existing personal computers, it a time consuming. That time is not acceptable for interactive sessions and provoked us to intend a simpler and faster algorithm able to producing comparable results within a few seconds. The experimental results produced by the algorithm are two to three in the orders of magnitude faster than the existing one.

The contourlet transform is a new addition or an extension to the wavelet transform in two dimensions by means of nonseparable filter blanks and directional filter banks. Due to its multiscale and directional properties, it can efficiently take into custody the edges of the image along with one dimensional contour with a small number of coefficients. The author examines image modeling in the contourlet transform area and its applications. Start on with a detailed aspect study of the information of the contourlet coefficients, which make their non-Gaussian trivial statistics and strong dependencies. Hardened on neighboring coefficient magnitudes, contourlet coefficients are established to be just about Gaussian. Experiment is conducted using this model in image denoising and texture retrieval. In denoising process, contourlet HMT is more efficient than wavelet HMT and additional conventional methods in terms of both visual quality and peak signal-tonoise ratio (PSNR). In texture recovery, it shows enhancement in performance greater than wavelet methods for various oriented textures [11].

The author introduced a total varation diffusion to control and restore the missing or damaged regions in the Nonsubsampled Contourlet transform (NSCT) domain, in its place of the pixel domain in which conventional inpainting problems are definite. The new techniques can be used to preserve image edges efficiently. Experiment results show that the NSCT algorithm can also considerably restore geometric features efficiently, even with fairly large number of lost coefficients [12].

III. INFERENCE FROM THE EXISTING WORK

The various transforms are wavelet transform, contourlet transform, Nonsubsampled transform and techniques are presented here. The main drawback of wavelet transform is that there is a problem of filling missing data will occur and it has poor directional specificity of the images. In contourlet transform the image improvement cannot capture the geometric information of images and be liable to amplify noises when they are applied to noisy images also that they cannot distinguish noises from weak edges. The entire drawback is overcome by the Nonsubsampled Contourlet transform. In order to determine the effectiveness of the techniques, experiments were carried using two UCI machine

learning dataset and evaluated based on the PSNR value and MSE.

IV. CONCLUSION

In this paper a variety of image Inpainting techniques such as texture synthesis based Inpainting, PDE based Inpainting, Exemplar based Inpainting and semi-automatic and fast Inpainting techniques are studied. For each technique a detailed explanation of the techniques can be given which are used for filling the missing region building use of image. From this survey, a number of shortcomings and limitations were highlighted in each and every technique. It is experiential that the PDE based Inpainting algorithms cannot fill the large missing region and it cannot renovate the texture pattern. The analysis proved that the exemplar based image Inpainting will create better results for Inpainting the huge missing region also that these algorithms can inpaint both the formation and textured image efficiently. But it will work well only if the missing region contains only simple structure and texture. The Nonsubsampled Contourlet transform is a recent developed technique which can be more accurate and it overcomes all the problem of the other existing techniques. Advance study includes growth of efficient algorithm to reduce computational cost and to decrease the time required for Inpainting.

REFERENCES

[1] S. Walden. The Ravished Image. St. Martin’s Press, New York, 1985.
[2] G. Emile-Male. The Restorer’s Handbook of Easel Painting. Van Nostrand Reinhold, New York, 1976.
[3] Komal s Mahajan, M.B. Vaidya, “Image in Painting Techniques: A survey”, IOSR Journal of Computer Engineering (IOSRJCE) ISSN: 2278-0661, ISBN: 2278-8727 Vol. 5, No. 4 (Sep-Oct. 2012), Pp 45-49.
[4] A.C. Kokaram, R.D. Morris, W.J. Fitzgerald, P.J.W. Rayner. Interpolation of missing data in image sequences. IEEE Transactions on Image Processing 11(4), Pp.1509-1519, 1995.
[5] A. Hirani and T. Totsuka. Combining Frequency and spatial domain information for fast interactive image noise removal. Computer Graphics, Pp. 269-276. SIGGRAPH 96, 1996.
[6] A. Efros and T. Leung, “Texture synthesis by non-parametric sampling,” Proc. IEEE International Conference Computer Vision, Pp. 1033-1038, Corfu, Greece, September 1999.
[7] D. Heeger and J. Bergen. Pyramid based texture analysis/synthesis. Computer Graphics, Pp. 229-238, SI GGRAPH 95, 1995.
[8] Z. Xu and S. Jian, “Image inpainting by patch propagation using patch sparsity,” IEEE Transactions on Image Processing, Vol. 19, Pp. 1153$1165,2010$.
[9] I.A. Ismail, E.A. Rakh, S.I. Zaki, M.A. Ashabrawy, M.K. Shaat, “Crack detection and filling, using steepest descent method”, International Journal of Computer and Electrical Engineering, Vol. 1, No. 4, October, 2009.
[10] Gunamani Jena, “Restoration of Still Images using Inpainting techniques”, International Journal of Computer Science & Communication, Vol. 1, No. 2, , Pp. 71-74, July- December 2010.
[11] Po, Duncan D K, “Directional multiscale modeling of images using the contourlet transform”, Statistical Signal Processing, 2003 IEEE Workshop on 28 Sept.-1 Oct. 2003.
[12] Guo, Jing, “Image inpainting based on nonsubsampled contourlet transform and total varation”, This paper appears in:Information Science and Engineering (ICISE), 2010 2nd International Conference on 4-6 Dec. 2010.