Texture optimization for example-based synthesis

ACM Transactions on Graphics, 2009

The essence of a 3D shape can often be well captured by its salient feature curves. In this paper, we explore the use of salient curves in synthesizing intuitive, shape-revealing textures on surfaces. Our texture synthesis is guided by two principles: matching the direction of the texture patterns to those of the salient curves, and aligning the prominent feature lines in the texture to the salient curves exactly. We have observed that textures synthesized by these principles not only fit naturally to the surface geometry, but also visually reveal, even reinforce, the shape's essential characteristics. We call these feature-aligned shape texturing . Our technique is fully automatic, and introduces two novel technical components in vector-field-guided texture synthesis: an algorithm that orients the salient curves on a surface for constrained vector field generation, and a feature-to-feature texture optimization.

Integrating Materials and Manufacturing Innovation, 2014

A pervasive method for reconstructing microstructures from two-dimensional microstructures imaged on orthogonal planes is presented. The algorithm reconstructs 3D images through matching of 3D slices at different voxels to the representative 2D micrographs and an optimization procedure that ensures patches from the 2D micrographs meshed together seamlessly in the 3D image. We show that the method effectively models the three-dimensional features in the microstructure using three cases (i) disperse spheres, (ii) anisotropic lamellar microstructure, and (iii) a polycrystalline microstructure. The method is validated by comparing the point probability functions of the reconstructed images to the original 2D image, as well as by comparing the elastic properties of reconstructed image to the experimental data.

APSIPA Transactions on Signal and Information Processing

An explainable, efficient, and lightweight method for texture generation, called TGHop (an acronym of Texture Generation PixelHop), is proposed in this work. Although synthesis of visually pleasant texture can be achieved by deep neural networks, the associated models are large in size, difficult to explain in theory, and computationally expensive in training. In contrast, TGHop is small in its model size, mathematically transparent, efficient in training and inference, and able to generate high-quality texture. Given an exemplary texture, TGHop first crops many sample patches out of it to form a collection of sample patches called the source. Then, it analyzes pixel statistics of samples from the source and obtains a sequence of fine-to-coarse subspaces for these patches by using the PixelHop++ framework. To generate texture patches with TGHop, we begin with the coarsest subspace, which is called the core, and attempt to generate samples in each subspace by following the distributi...

The Visual Computer

This paper presents an automatic image synthesis method to transfer the style of an example image to a content image. When standard neural style transfer approaches are used, the textures and colours in different semantic regions of the style image are often applied inappropriately to the content image, ignoring its semantic layout and ruining the transfer result. In order to reduce or avoid such effects, we propose a novel method based on automatically segmenting the objects and extracting their soft semantic masks from the style and content images, in order to preserve the structure of the content image while having the style transferred. Each soft mask of the style image represents a specific part of the style image, corresponding to the soft mask of the content image with the same semantics. Both the soft masks and source images are provided as multichannel input to an augmented deep CNN framework for style transfer which incorporates a generative Markov random field model. The results on various images show that our method outperforms the most recent techniques. Keywords Deep neural networks • Style transfer • Soft mask • Semantic segmentation 1 Introduction Style transfer is a process of migrating a style from a "style image" to a "content image". The goal is to be able to generate different renditions of the same scene according to different style images. Image style transfer has become a popular problem in computer vision and graphics and can generate impressive results covering a wide variety of styles for both images [16] and videos [35]. It has also been widely employed to solve problems such as texture synthe

Petroleum Science

Modeling reservoir permeability is one of the crucial tasks in reservoir simulation studies. Traditionally, it is done by kriging-based methods. More rigorous modeling of the permeability results in more reliable outputs of the reservoir models. Recently, a new category of geostatistical methods has been used for this purpose, namely multiple point statistics (MPS). By this new category of permeability modeling methods, one is able to predict the heterogeneity of the reservoir permeability as a continuous variable. These methods consider the direction of property variation in addition to the distances of known locations of the property. In this study, the reservoir performance of a modified version of the SPE 10 solution project as a pioneer case is used for investigating the efficiency of these methods and paralleling them with the kriging-based one. In this way, the permeability texture concept is introduced by applying some MPS methods. This study is accomplished in the condition...

The Visual Computer, 2018

We introduce a method to stylize photographs with auxiliary textures, by means of the Laplacian pyramid. Laplacian pyramid coefficients from a synthetic texture are combined with the coefficients from the original image by means of a smooth maximum function. The final result is a stylized image which maintains the structural characteristics from the input, including edges, color, and existing texture, while enhancing the image with additional fine-scale details. Further, we extend patch-based texture synthesis to include a guidance channel so that texture structures are aligned with an orientation field, obtained through the image structure tensor.

Proceedings of the 28th ACM International Conference on Multimedia, 2020

Origin Corrupted Without Text Text-guided (a) Text-guided (b) Text-guided (c)

Applied Sciences, 2022

Face photographs taken on a bright sunny day or in floodlight contain unnecessary shadows of objects on the face. Most previous works deal with removing shadow from scene images and struggle with doing so for facial images. Faces have a complex semantic structure, due to which shadow removal is challenging. The aim of this research is to remove the shadow of an object in facial images. We propose a novel generative adversarial network (GAN) based image-to-image translation approach for shadow removal in face images. The first stage of our model automatically produces a binary segmentation mask for the shadow region. Then, the second stage, which is a GAN-based network, removes the object shadow and synthesizes the effected region. The generator network of our GAN has two parallel encoders—one is standard convolution path and the other is a partial convolution. We find that this combination in the generator results not only in learning an incorporated semantic structure but also in d...

Computer Vision – ECCV 2020, 2020

Existing image inpainting methods often produce artifacts when dealing with large holes in real applications. To address this challenge, we propose an iterative inpainting method with a feedback mechanism. Specifically, we introduce a deep generative model which not only outputs an inpainting result but also a corresponding confidence map. Using this map as feedback, it progressively fills the hole by trusting only high-confidence pixels inside the hole at each iteration and focuses on the remaining pixels in the next iteration. As it reuses partial predictions from the previous iterations as known pixels, this process gradually improves the result. In addition, we propose a guided upsampling network to enable generation of high-resolution inpainting results. We achieve this by extending the Contextual Attention module [34] to borrow highresolution feature patches in the input image. Furthermore, to mimic real object removal scenarios, we collect a large object mask dataset and synthesize more realistic training data that better simulates user inputs. Experiments show that our method significantly outperforms existing methods in both quantitative and qualitative evaluations. More results and Web APP are available at https://zengxianyu.github.io/iic.

ACM Transactions on Graphics, 2008

Example-based texture synthesis algorithms have gained widespread popularity for their ability to take a single input image and create a perceptually similar non-periodic texture. However, previous methods rely on single input exemplars that can capture only a limited band of spatial scales. For example, synthesizing a continent-like appearance at a variety of zoom levels would require an impractically high input resolution. In this paper, we develop a multiscale texture synthesis algorithm. We propose a novel example-based representation, which we call an exemplar graph, that simply requires a few low-resolution input exemplars at different scales. Moreover, by allowing loops in the graph, we can create infinite zooms and infinitely detailed textures that are impossible with current example-based methods. We also introduce a technique that ameliorates inconsistencies in the user's input, and show that the application of this method yields improved interscale coherence and highe...

Computer Graphics Forum, 2015

Figure 1: Our example-based structure synthesis method can be used to generate structures with discrete elements (left), continuous geometries (middle), and their mixtures (right); on different domains such as surfaces, bounding volumes, or curves.

International Journal of Scientific Research in Science and Technology

In recent years, after the study ‘A Neural Algorithm of Artistic Style’ published by Gatys et al. in 2016b, research on style transfer boomed drastically. Style transfer is the process of copying an art style from a ‘style image’ to the contents of the ‘content image’ and producing a ‘draft image’ that is on par with respect to quality expectations. This paper explores different techniques of achieving style transformations namely Style Fusion and Convolutional Neural Networks (CNNs). Although CNNs are the state-of-the-art architecture to tackle cognitive visual tasks, and that they clearly perform much better than most conventional algorithms, the image processing-based style fusion method comes close to the CNN in terms of image output quality and supersedes in terms of time and computation and resources complexity. The procedure of both of these methods has been discussed in detail in this paper and it was concluded that CNNs have a lot more room for improvement that can be facil...

Computer Graphics Forum

Figure 1: Texturing a fluid animation with eroded deformable lapped textures. Deformable patches are colored on the left and the resulting lapped texture is shown on the right.

Multimedia Tools and Applications, 2019

We propose a novel automatic photographic painting style technique with a single example image by using Convolutional Neural Networks (CNN). The photographic painting style is a challenging problem in the research community. Even though, researchers have been trying to obtain good results on painting style, but not much has been done on photographic stylization. Portrait painting techniques are mainly designed for the graphite style and/or are based on image analogies; an example painting as well as its original unpainted version are required. This preceding issue is a motivation of our proposed methods. As a result, our method extends the limits of their domain of applicability. We present a novel multi-convolutional-learning technique that is developed for both images (NPR/PR) labeling, style transmission and elevating a particular unified CNN model per weight sharing. A new painting technique is generated that follows the example style in the example image and maintains the integrity of facial structures. We believe this novel interpretation connects these two important research fields and could enlighten future researches. Moreover, our proposed technique is not restricted to headshot images or specific styles as our method can also change the photographic painting style in the wild.