Interactive Segmentation

Bayesian image analysis has played a large role over the last 40+ years in solving problems in image noise-reduction, de-blurring, feature enhancement, and object detection. However, these problems can be complex and lead to computational difficulties, due to the modeled interdependence between spatial locations. The Bayesian image analysis in Fourier space (BIFS) approach proposed here reformulates the conventional Bayesian image analysis paradigm for continuous valued images as a large set of independent (but heterogeneous) processes over Fourier space. The original high-dimensional estimation problem in image space is thereby broken down into (trivially parallelizable) independent one-dimensional problems in Fourier space. The BIFS approach leads to easy model specification with fast and direct computation, a wide range of possible prior characteristics, easy modeling of isotropy into the prior, and models that are effectively invariant to changes in image resolution.

The recruitment of chromatin-modifying coregulator complexes by transcription factors to specific sites of the genome constitutes an important step in many eukaryotic transcriptional regulatory pathways. The histone deacetylase-associated Sin3 corepressor complex is recruited by a large and diverse array of transcription factors through direct interactions with the N-terminal PAH domains of Sin3. Here we describe the solution structures of the mSin3A PAH1 domain in the apo-form and when bound to SAP25, a component of the corepressor complex. Unlike the apo-mSin3A PAH2 domain, the apo-PAH1 domain is conformationally pure and is largely, but not completely, folded. Portions of the interacting segments of both mSin3A PAH1 and SAP25 undergo folding upon complex formation. SAP25 binds through an amphipathic helix to a predominantly hydrophobic cleft on the surface of PAH1. Remarkably, the orientation of the helix is reversed compared to that adopted by NRSF, a transcription factor unrelated to SAP25, upon binding to the mSin3B PAH1 domain. The reversal in helical orientations is correlated with a reversal in the underlying PAH1-interaction motifs, echoing a theme previously described for the mSin3A PAH2 domain. The definition of these so-called type I and type II PAH1-interaction motifs has allowed us to predict the precise location of these motifs within previously experimentally-characterized PAH1 binders. Finally, we explore the specificity determinants of protein-protein interactions involving the PAH1 and PAH2 domains. These studies reveal that even conservative replacements of PAH2 residues with equivalent PAH1 residues are sufficient to dramatically alter the affinity and specificity of these protein-protein interactions.

We present a color segmentation approach based on a two-dimensional color map derived from the input image. Pathologists stain tissue biopsies with various colored dyes to see the expression of biomarkers. In these images, because of color variation due to inconsistencies in experimental procedures and lighting conditions, the segmentation used to analyze biological features is usually ad-hoc. Many algorithms like K-means use a single metric to segment the image into different color classes and rarely provide users with powerful color control. Our 2D color map interactive segmentation technique based on human color perception information and the color distribution of the input image, enables user control without noticeable delay. Our methodology works for different staining types and different types of cancer tissue images. Our proposed method's results show good accuracy with low response and computational time making it a feasible method for user interactive applications involving segmentation of histological images.

The problem of segmenting an image into separate regions and tracking them over time is one of the most significant problems in vision. Tenopoulos et al have proposed an approach to detect the contour regions of complex shapes, assuming a user selected initial contour not very far from the desired solution. We propose to further explore the information provided by the user's selected points and apply-an optimal method to detect contours which allows a segmentation of the image. The method is based on dynamic programming @P), and applies to a wide variety of shapes. It is exact and not iterative. We also consider a multiscale approach capable of speeding up the algorithm by a factor of 20, although at the expense of losing the guaranteed optimality characteristic. The problem of tracking and matching these contours is addressed. For tracking, the final contour obtained at one frame is sampled and used as initial points for the next frame. Then, the same DP process is applied. For matching, a novel strategy is proposed where the solution is a smooth displacement field in which unmatched regions are allowed while cross vectors are not. The algorithm is again based on DP and the optimal solution is guaranteed. We have demonstrated the algorithms on natural objects in a large spectrum of applications, including interactive segmentation and automatic tracking of the regions of interest in medical images.

Robust methods for the segmentation and analysis of cells in 3D time sequences (3D+t) are critical for quantitative cell biology. While many automated methods for segmentation perform very well, few generalize reliably to diverse datasets. Such automated methods could significantly benefit from at least minimal user guidance. Identification and correction of segmentation errors in time-series data is of prime importance for proper validation of the subsequent analysis. The primary contribution of this work is a novel method for interactive segmentation and analysis of microscopy data, which learns from and guides user interactions to improve overall segmentation. We introduce an interactive cell analysis application, called CellECT, for 3D+t microscopy datasets. The core segmentation tool is watershed-based and allows the user to add, remove or modify existing segments by means of manipulating guidance markers. A confidence metric learns from the user interaction and highlights regi...

Colorectal cancer (CRC) is the third most common type of cancer with the liver being the most common site for cancer spread. A precise understanding of patient liver anatomy and pathology, as well as surgical planning based on that, plays a critical role in the treatment process. In some cases, surgeons request a 3D reconstruction, which requires a thorough analysis of the available images to be converted into 3D models of relevant objects through a segmentation process. Liver vessel segmentation is challenging due to the large variations in size and directions of the vessel structures as well as difficult contrasting conditions. In recent years, deep learning-based methods had been outperforming the conventional image analysis methods in the field of medical imaging. Though Convolutional Neural Networks (CNN) have been proved to be efficient for the task of medical image segmentation, the way of handling the image data and the preprocessing techniques play an important role in segm...

This paper explores a novel approach to interactive user-guided image segmentation, using eyegaze information as an input. The method includes three steps: 1) eyegaze tracking for providing user input, such as setting object and background seed pixel selection; 2) an optimization method for image labeling that is constrained or affected by user input; and 3) linking the two previous steps via a graphical user interface for displaying the images and other controls to the user and for providing real-time visual feedback of eyegaze and seed locations, thus enabling the interactive segmentation procedure. We developed a new graphical user interface supported by an eyegaze tracking monitor to capture the user's eyegaze movement and fixations (as opposed to traditional mouse moving and clicking). The user simply looks at different parts of the screen to select which image to segment, to perform foreground and background seed placement and to set optional segmentation parameters. There is an eyegaze-controlled "zoom" feature for difficult images containing objects with narrow parts, holes or weak boundaries. The image is then segmented using the random walker image segmentation method. We performed a pilot study with 7 subjects who segmented synthetic, natural and real medical images. Our results show that getting used the new interface takes about only 5 minutes. Compared with traditional mouse-based control, the new eyegaze approach provided a 18.6% speed improvement for more than 90% of images with high object-background contrast. However, for low contrast and more difficult images it took longer to place seeds using the eyegaze-based "zoom" to relax the required eyegaze accuracy of seed placement.

Recent advances in scanning technology provide high resolution EM (Electron Microscopy) datasets that allow neuroscientists to reconstruct complex neural connections in a nervous system. However, due to the enormous size and complexity of the resulting data, segmentation and visualization of neural processes in EM data is usually a difficult and very time-consuming task. In this paper, we present NeuroTrace, a novel EM volume segmentation and visualization system that consists of two parts: a semi-automatic multiphase level set segmentation with 3D tracking for reconstruction of neural processes, and a specialized volume rendering approach for visualization of EM volumes. It employs view-dependent on-demand filtering and evaluation of a local histogram edge metric, as well as on-the-fly interpolation and ray-casting of implicit surfaces for segmented neural structures. Both methods are implemented on the GPU for interactive performance. NeuroTrace is designed to be scalable to large datasets and data-parallel hardware architectures. A comparison of NeuroTrace with a commonly used manual EM segmentation tool shows that our interactive workflow is faster and easier to use for the reconstruction of complex neural processes.

Image processing techniques are now widely spread out over a large quantity of domains: like medical imaging, movies post-production, games... Automatic detection and extraction of regions of interest inside an image, a volume or a video is challenging problem since it is a starting point for many applications in image processing. However many techniques were developed during the last years and the state of the art methods suffer from some drawbacks: The Level Sets method only provides a local minimum while the Graph Cuts method comes from Combinatorial Community and could take advantage of the specificity of image processing problems. In this thesis, we propose two extensions of the previously cited methods in order to soften or remove these drawbacks. We first discuss the existing methods and show how they are related to the segmentation problem through an energy formulation. Then we introduce stochastic perturbations to the Level Sets method and we build a more generic framework: the Stochastic Level Sets (SLS). Later we provide a direct application of the SLS to image segmentation that provides a better minimization of energies. Basically, it allows the contours to escape from local minimum. Then we propose a new formulation of an existing algorithm of Graph Cuts in order to introduce some interesting concept for image processing community: like initialization of the algorithm for speed improvement. We also provide a new approach for layer extraction from video sequence that retrieves both visible and hidden layers in it. 7.1 Dynamic segmentation of a video sequence. The Active cuts algorithm (yellow) runs 2-6 times faster than the state-of-theart max-flow algorithm in [18] (red). In each new frame initial cut for our algorithm is set to an optimal cut/segmentation from the previous frame. The speed of our algorithm almost linearly proportional to the magnitude of motion shown by the plot of the Hausdorff distance between the segments in consecutive frames (blue). Note that active cuts can be further accelerated in dynamic applications by "recycling" flow computed in the previous frame [

We describe a novel algorithm for local approximation of scattered surface points, implementing a 2D finite element interpolation algorithm in combination with approximation of coordinates using quadrics for conversion of noisy data to sufficiently smooth data sets. The applied methods and time performance of the developed algorithm are discussed. Experimental results are included to demonstrate the functionality of our approximation technique.

Computer assisted teaching plays an increasing role in surgical education. The presented paper describes the development of virtual reality (VR) and 3D visualizations for educational purposes concerning aortocoronary bypass grafting and their prototypical implementation into a database-driven and internet-based educational system in heart surgery. A multimedia storyboard has been written and digital video has been encoded. Understanding of these videos was not always satisfying; therefore, additional 3D and VR visualizations have been modelled as VRML, QuickTime, QuickTime Virtual Reality and MPEG-1 applications. An authoring process in terms of integration and orchestration of different multimedia components to educational units has been started. A virtual model of the heart has been designed. It is highly interactive and the user is able to rotate it, move it, zoom in for details or even fly through. It can be explored during the cardiac cycle and a transparency mode demonstrates ...

We propose a complete still image based 2D-3D mobile conversion system for touch screen use. The system consists of interactive segmentation followed by 3D rendering. The interactive segmentation is conducted dynamically by color Gaussian mixture model updates and dynamic-iterative graph-cut. A coloring gesture is used to guide the way and entertain the user during the process. Output of the image segmentation is then fed to the 3D rendering stage of the system. For rendering stage, two novel improvements are proposed to handle holes resulting from depth image based rendering process. These improvements are also expected to enhance the 3D perception. These two methods are subjectively tested and their results are presented.

We develop a novel interactive segmentation and 3-D visualization system for cone-beam computed-tomography (CBCT) data in a personal computer environment. Its design includes both a new user interface to ease the interactive manual segmentation of the ...

In the last decades, a large diversity of automatic, semi-automatic and manual approaches for video segmentation and knowledge extraction from video-data has been proposed. Due to the high complexity in both the spatial and temporal domain, it continues to be a challenging research area. In order to develop, train, and evaluate new algorithms, ground truth of video-data is crucial. Pixel-wise annotation of ground truth is usually time-consuming, does not contain semantic relations between objects and uses only simple geometric primitives. We provide a brief review of related tools for video annotation, and introduce our novel interactive and semi-automatic segmentation tool iSeg. Extending an earlier implementation, we improved iSeg with a semantic time line, multithreading and the use of ORB features. A performance evaluation of iSeg on four data sets is presented. Finally, we discuss possible opportunities and applications of semantic polygon-shaped video annotation, such as 3D reconstruction and video inpainting.

In interactive segmentation, the most common way to model object appearance is by GMM or histogram, while MRFs are used to encourage spatial coherence among the object labels. This makes the strong assumption that pixels within each object are i.i.d. when in fact most objects have multiple distinct appearances and exhibit strong spatial correlation among their pixels. At the very least, this calls for an MRF-based appearance model within each object itself and yet, to the best of our knowledge, such a "two-level MRF" has never been proposed. We propose a novel segmentation energy that can model complex appearance. We represent the appearance of each object by a set of distinct spatially coherent models. This results in a two-level MRF with "super-labels" at the top level that are partitioned into "sub-labels" at the bottom. We introduce the hierarchical Potts (hPotts) prior to govern spatial coherence within each level. Finally, we introduce a novel alg...

In recent years, interactive methods for segmentation are increasing in popularity due to their success in different domains such as medical image processing, photo editing, etc. We present an interactive segmentation algorithm that can segment an object of interest from its background with minimum guidance from the user, who just has to select a single seed pixel inside the object of interest. Due to minimal requirements from the user, we call our algorithm semiautomatic. To obtain a reliable and robust segmentation with such low user guidance, we have to make several assumptions. Our main assumption is that the object to be segmented is of compact shape, or can be approximated by several connected roughly collinear compact pieces. We base our work on the powerful graph cut segmentation algorithm of Boykov and Jolly, which allows straightforward incorporation of the compact shape constraint. In order to make the graph cut approach suitable for our semiautomatic framework, we address several well-known issues of graph cut segmentation technique. In particular, we counteract the bias towards shorter segmentation boundaries and develop a method for automatic selection of parameters. We demonstrate the effectiveness of our approach on the challenging industrial application of transistor gate segmentation in images of integrated chips. Our approach produces highly accurate results in real-time.

In the last decade, graph-cut optimization has been popular for a variety of labeling problems. Typically graph-cut methods are used to incorporate smoothness constraints on a labeling, encouraging most nearby pixels to have equal or similar labels. In addition to smoothness, ordering constraints on labels are also useful. For example, in object segmentation, a pixel with a "car wheel" label may be prohibited above a pixel with a "car roof" label. We observe that the commonly used graphcut α-expansion move algorithm is more likely to get stuck in a local minimum when ordering constraints are used. For a certain model with ordering constraints, we develop new graphcut moves which we call order-preserving. The advantage of orderpreserving moves is that they act on all labels simultaneously, unlike α-expansion. More importantly, for most labels α, the set of α-expansion moves is strictly smaller than the set of order-preserving moves. This helps to explain why in practice optimization with order-preserving moves performs significantly better than α-expansion in presence of ordering constraints. We evaluate order-preserving moves for the geometric class scene labeling (introduced by Hoiem et al.) where the goal is to assign each pixel a label such as "sky", "ground", etc., so ordering constraints arise naturally. In addition, we use order-preserving moves for certain simple shape priors in graph-cut segmentation, which is a novel contribution in itself.

Among the most exciting advances in early vision has been the development of efficient energy minimization algorithms for pixel-labeling tasks such as depth or texture computation. It has been known for decades that such problems can be elegantly expressed as Markov random fields, yet the resulting energy minimization problems have been widely viewed as intractable. Recently, algorithms such as graph cuts and loopy belief propagation (LBP) have proven to be very powerful: For example, such methods form the basis for almost all the top-performing stereo methods. However, the trade-offs among different energy minimization algorithms are still not well understood. In this paper, we describe a set of energy minimization benchmarks and use them to compare the solution quality and runtime of several common energy minimization algorithms. We investigate three promising recent methods-graph cuts, LBP, and tree-reweighted message passing-in addition to the well-known older iterated conditional mode (ICM) algorithm. Our benchmark problems are drawn from published energy functions used for stereo, image stitching, interactive segmentation, and denoising. We also provide a general-purpose software interface that allows vision researchers to easily switch between optimization methods. The benchmarks, code, images, and results are available at http://vision.middlebury.edu/MRF/.

In this paper we present a comparative evaluation of four popular interactive segmentation algorithms. The evaluation was carried out as a series of user-experiments, in which participants were tasked with extracting one hundred objects from a common dataset: twenty-five with each algorithm, constrained within a time limit of two minutes for each object. To facilitate the experiments, a "scribble-driven" segmentation tool was developed to enable interactive image segmentation by simply marking areas of foreground and background with the mouse. As the participants refined and improved their respective segmentations, the corresponding updated segmentation mask was stored along with the elapsed time. We then collected and evaluated each recorded mask against a manually segmented ground-truth, thus allowing us to gauge segmentation accuracy over time. Two benchmarks were used for the evaluation: the well-known Jaccard index for measuring object accuracy, and a new fuzzy metric, proposed in this paper, designed for measuring boundary accuracy. Analysis of the experimental results demonstrates the effectiveness of the suggested measures and provides valuable insights into the performance and characteristics of the evaluated algorithms.

This paper provides the algorithm that applies concept of continuous time quantum walks to image segmentation problem. The work, inspired by results from its classical counterpart [9], presents and compares two versions of the solution regarding calculation of pixel-segment association: the version using limiting distribution of the walk and the version using last step distribution. The obtained results vary in terms of accuracy and possibilities to be ported to a real quantum device. The described results were obtained by simulation on classical computer, but the algorithms were designed in a way that will allow to use a real quantum computer, when ready.

Digital matting consists in extracting a foreground element from a background image. Besides the image, usual matting methods need to be initialized with two disjoint regions : the set of foreground only pixels and the set of background only pixels. Pixels belonging to none of these two regions are considered as an undetermined blending between the foreground and the background. Here, one has to estimate the opacity (alpha channel) and the original foreground and background colors that have been blended. Initialization is a crucial step for these methods, and usually one has to specify accurate initial regions, leaving undetermined as few pixels as possible. This is especially true for recent methods that use local models. This paper proposes an unsupervised segmentation scheme that initializes any matting method by extracting the foreground and background regions from just a small subset of them. Standard statistical models are used for the foreground and background regions, while ...

Heritage preservation initiatives have started to use laser scanners to efficiently sample large architectural structures. However, these require many viewpoints to correctly determine point positions. Errors can be introduced in the sampling process leading to regions that are overly dense, undersampled or absent.

Deep convolutional neural networks are now mainstream for click-based interactive image segmentation. Most frameworks refine false negatives and false positive regions via a succession of positive and negative clicks placed centrally in these regions. We propose a simple yet intuitive two-in-one refinement strategy placing clicks on the boundary of the object of interest. As boundary clicks are a strong cue for extracting the object of interest and we find that they are much more effective in correcting wrong segmentation masks. In addition, we propose a boundary-aware loss that encourages segmentation masks to respect instance boundaries. We place our new refinement scheme and loss formulation within a task-specialized segmentation framework and achieve state-of-theart performance on the standard datasets Berkeley, Pascal VOC 2012, DAVIS, and MS COCO. We exceed competing methods by 6.5%,9.4%,10.5% and 2.5%, respectively.

Vascular segmentation plays an important role in the assessment of peripheral arterial disease. The segmentation is very challenging especially for arteries with severe stenosis or complete occlusion. We present a cascading algorithm for vascular centerline tree detection specializing in detecting centerlines in diseased peripheral arteries. It takes a three-dimensional computed tomography angiography (CTA) volume and returns a vascular centerline tree, which can be used for accelerating and facilitating the vascular segmentation. The algorithm consists of four levels, two of which detect healthy arteries of varying sizes and two that specialize in different types of vascular pathology: severe calcification and occlusion. We perform four main steps at each level: appropriate parameters for each level are selected automatically, a set of centrally located voxels is detected, these voxels are connected together based on the connection criteria, and the resulting centerline tree is corrected from spurious branches. The proposed method was tested on 25 CTA scans of the lower limbs, achieving an average overlap rate of 89% and an average detection rate of 82%. The average execution time using four CPU cores was 70 s, and the technique was successful also in detecting very distal artery branches, e.g., in the foot.

Heritage preservation initiatives have started to use laser scanners to efficiently sample large architectural structures. However, these require many viewpoints to correctly determine point positions. Errors can be introduced in the sampling process leading to regions that are overly dense, undersampled or absent.

An automatic multi-organ segmentation pipeline is presented. The segmentation starts with stripping the body of skin and subcutaneous fat using threshold-based level-set methods. After registering the image to be processed against a standard subject picked from the training datasets, a series of model-based level set segmentation operations is carried out guided by hierarchical shape priors. The hierarchical shape priors are organized according to the anatomical hierarchy of the human body, starting with ventral cavity, and then divided into thoracic cavity and abdominopelvic cavity. The third level contains the individual organs such as liver, spleen and kidneys. The segmentation is performed in a top-down fashion, where major structures are segmented first, and their location information is then passed down to the lower level to initialize the segmentation, while boundary information from higher-level structures also constrains the segmentation of the lower-level structures. In our preliminary experiments, the proposed method yielded a Dice coefficient around 90% for most major thoracic and abdominal organs in both contrastenhanced CT and non-enhanced datasets, while the average running time for segmenting ten organs was about 10 minutes.

We propose a new paradigm for interactive volume visualization which allows enhanced exploration of datasets through the use of real-time volume processing (e.g., segmentation) tightly coupled with rendering. We show examples of how this paradigm supports a new way of examining a dataset by providing interactive visual feedback to segmentation parameter adjustment. We have developed a hardware co-processor, called PAVLOV, based on a SIMD mesh architecture. It supports our paradigm by providing volume processing at over 15Hz while also rendering of the results at 30Hz. We show how PAVLOV's programmability is used for volume processing applications, such as interactive segmentation and feature extraction. We further compare 3D segmentation on mesh architectures to known efficient methods for 2D segmentation.

The recruitment of chromatin-modifying coregulator complexes by transcription factors to specific sites of the genome constitutes an important step in many eukaryotic transcriptional regulatory pathways. The histone deacetylase-associated Sin3 corepressor complex is recruited by a large and diverse array of transcription factors through direct interactions with the N-terminal PAH domains of Sin3. Here we describe the solution structures of the mSin3A PAH1 domain in the apo-form and when bound to SAP25, a component of the corepressor complex. Unlike the apo-mSin3A PAH2 domain, the apo-PAH1 domain is conformationally pure and is largely, but not completely, folded. Portions of the interacting segments of both mSin3A PAH1 and SAP25 undergo folding upon complex formation. SAP25 binds through an amphipathic helix to a predominantly hydrophobic cleft on the surface of PAH1. Remarkably, the orientation of the helix is reversed compared to that adopted by NRSF, a transcription factor unrelated to SAP25, upon binding to the mSin3B PAH1 domain. The reversal in helical orientations is correlated with a reversal in the underlying PAH1-interaction motifs, echoing a theme previously described for the mSin3A PAH2 domain. The definition of these so-called type I and type II PAH1-interaction motifs has allowed us to predict the precise location of these motifs within previously experimentally-characterized PAH1 binders. Finally, we explore the specificity determinants of protein-protein interactions involving the PAH1 and PAH2 domains. These studies reveal that even conservative replacements of PAH2 residues with equivalent PAH1 residues are sufficient to dramatically alter the affinity and specificity of these protein-protein interactions.

The split-step solution to the parabolic wave equation describing beam propagation permits examination of the signal degradation in a Doppler weather radar caused by wind turbines blockage under general atmospheric conditions and at arbitrary transmitter and receiver configurations. At radar wavelengths, an understanding of turbine and terrain obscuration effects is essential for deciding the reliability of radar measurements affected by blockage.

Neurons are cellular compartments possessing branching morphologies, with information processing functionality, and the ability to communicate with each other via synaptic junctions (e.g. neurons come within less than a nano-meter of each other in a specialized way). A collection of neurons in each part of the brain form a dense forest of such branching structures, with myriad intertwined branches, inter-neuron synaptic connections, and a packing density that leaves only 5%-10% volume fraction of exterior-cellular space. Small-scale variations in branching morphology of neurons and inter-neuron spacing can exert dramatically different electrical effects that are overlooked by models that treat dendrites as cylindrical compartments in one dimension with lumped parameters. In this paper, we address the problems of generating topologically accurate and spatially realistic boundary element meshes of a forest of neuronal membranes for analyzing their collective electrodynamic properties through simulation. We provide a robust multi-surface reconstruction and quality meshing solution for the forest of densely packed multiple branched structures starting from a stack of segmented 2D serial sections from electron microscopy imaging. The entire 3D domain is about 8 cubic microns, with inter-neuron spacing down to sub-nanometers, adding additional complexity to the robust reconstruction and meshing problem.

We propose a novel method for smoothing partition of unity (PU) implicit surfaces consisting of sets of nonconforming linear functions with spherical supports. We derive new discrete differential operators and Laplacian smoothing using a spherical covering of PU as a grid-like data structure. These new differential operators are applied to the smoothing of PU implicit surfaces. First, Laplacian smoothing is performed for the vector field defined by the gradient of the PU implicit surface, which is then updated to reflect the smoothing of the gradient field. This process achieves a method for noise robust surface reconstruction from scattered points.

The split-step solution to the parabolic wave equation describing beam propagation permits examination of the signal degradation in a Doppler weather radar caused by wind turbines blockage under general atmospheric conditions and at arbitrary transmitter and receiver configurations. At radar wavelengths, an understanding of turbine and terrain obscuration effects is essential for deciding the reliability of radar measurements affected by blockage.

Using the A* algorithm, we propose a new heuristic search strategy to find object boundaries. We show that a cost function with a Gaussian curvature is more appropriate to develop only the best paths. An application to the detection of cells boundaries is presented.

In this paper, we propose a method for automatic foreground extraction in video frames by analyzing the spatiotemporal aspect. We divide our contribution to tree steps: Automatic seeds detection, formulating the energy function, and using the random walk algorithm to minimize this function. First, we detect seeds by extracting a sparse of good features to track in the current frame and compute the difference between those pixels and its adjacent in the previous frame, the difference of pixels is treated in HSV color space to make the result more accurate, we thresholds this difference, and we classify moving and stationary pixels. Secondly, we formulate our foreground extraction as a graph based problem, then we define an energy function to evaluate spatiotemporal smoothness. Finally, we applied the random walk algorithm with seeds detected in the first step to minimize the energy function problem, the solution leads to evaluate the potential that every pixel in the video sequences is marked in motion or a stationary pixel. We suggest that our unsupervised method has the potential to be used for many kinds of motion detection and real-time video.

Colorectal cancer (CRC) is the third most common type of cancer with the liver being the most common site for cancer spread. A precise understanding of patient liver anatomy and pathology, as well as surgical planning based on that, plays a critical role in the treatment process. In some cases, surgeons request a 3D reconstruction, which requires a thorough analysis of the available images to be converted into 3D models of relevant objects through a segmentation process. Liver vessel segmentation is challenging due to the large variations in size and directions of the vessel structures as well as difficult contrasting conditions. In recent years, deep learning-based methods had been outperforming the conventional image analysis methods in the field of medical imaging. Though Convolutional Neural Networks (CNN) have been proved to be efficient for the task of medical image segmentation, the way of handling the image data and the preprocessing techniques play an important role in segm...

The problem of interactive foreground/background segmentation in still images is of great practical importance in image editing. The state of the art in interactive segmentation is probably represented by the graph cut algorithm of Boykov and Jolly (ICCV 2001). Its underlying model uses both colour and contrast information, together with a strong prior for region coherence. Estimation is performed by solving a graph cut problem for which very efficient algorithms have recently been developed. However the model depends on parameters which must be set by hand and the aim of this work is for those constants to be learned from image data. First, a generative, probabilistic formulation of the model is set out in terms of a "Gaussian Mixture Markov Random Field" (GMMRF). Secondly, a pseudolikelihood algorithm is derived which jointly learns the colour mixture and coherence parameters for foreground and background respectively. Error rates for GMMRF segmentation are calculated throughout using a new image database, available on the web, with ground truth provided by a human segmenter. The graph cut algorithm, using the learned parameters, generates good object-segmentations with little interaction. However, pseudolikelihood learning proves to be frail, which limits the complexity of usable models, and hence also the achievable error rate.

Implicit surfaces are often created by summing a collection of radial basis functions. Recently, researchers have begun to create implicit surfaces that exactly interpolate a given set of points by solving a simple linear system to assign weights to each basis function. Due to their ability to interpolate, these implicit surfaces are more easily controllable than traditional "blobby" implicits. There are several additional forms of control over these surfaces that make them attractive for a variety of applications. Surface normals may be directly specified at any location over the surface, and this allows the modeller to pivot the normal while still having the surface pass through the constraints. The degree of smoothness of the surface can be controlled by changing the shape of the basis functions, allowing the surface to be pinched or smooth. On a point-by-point basis the modeller may decide whether a constraint point should be exactly interpolated or approximated. Applications of these implicits include shape transformation, creating surfaces from computer vision data, creation of an implicit surface from a polygonal model, and medical surface reconstruction.

In interactive medical image segmentation, anatomical structures are extracted from reconstructed volumetric images. The first iterations of user interaction traditionally consist of drawing pictorial hints as an initial estimate of the object to extract. Only after this time consuming first phase, the efficient selective refinement of current segmentation results begins. Erroneously labeled seeds, especially near the border of the object, are challenging to detect and replace for a human and may substantially impact the overall segmentation quality. We propose an automatic seeding pipeline as well as a configuration based on saliency recognition, in order to skip the time-consuming initial interaction phase during segmentation. A median Dice score of 68.22% is reached before the first user interaction on the test data set with an error rate in seeding of only 0.088%.

Interactive image segmentation bears the advantage of correctional updates to the current segmentation mask when compared to fully automated systems. Especially in the field of inter-operative medical image processing of a single patient, where a high accuracy is an uncompromisable necessity, a human operator guiding a system towards an optimal segmentation result is a time-efficient constellation benefiting the patient. There are recent categories of neural networks which can incorporate human-computer interaction (HCI) data as additional input for segmentation. In this work, we simulate this HCI data during training with state-of-the-art user models, also called robot users, which aim to act similar to real users given interactive image segmentation tasks. We analyze the influence of chosen robot users, which mimic different types of users and scribble patterns, on the segmentation quality. We conclude that networks trained with robot users with the most spread out seeding pattern...

For complex segmentation tasks, fully automatic systems are inherently limited in their achievable accuracy for extracting relevant objects. Especially in cases where only few data sets need to be processed for a highly accurate result, semi-automatic segmentation techniques exhibit a clear benefit for the user. One area of application is medical image processing during an intervention for a single patient. We propose a learning-based cooperative segmentation approach which includes the computing entity as well as the user into the task. Our system builds upon a state-of-the-art fully convolutional artificial neural network (FCN) as well as an active user model for training. During the segmentation process, a user of the trained system can iteratively add additional hints in form of pictorial scribbles as seed points into the FCN system to achieve an interactive and precise segmentation result. The segmentation quality of interactive FCNs is evaluated. Iterative FCN approaches can y...

This paper presents a system for an efficient image segmentation on mobile phones using multi-level graph cut. As the computational capacity of mobile devices is often limited, a fluent and smooth image segmentation is a challenging task with existing segmentation algorithms, increased in difficulty by mobile phone cameras continually upgraded to take photos of higher resolution. Our solution is to carefully tweak the classic graph cut algorithm for an interactive image cutout, enhancing the performance without compromising the quality of the segmentation. This is achieved by down-sampling the original high-resolution image and selecting a rough cutout region on this low-resolution image with a superpixel based pre-segmentation. The segmented foreground is then mapped back to the full-size image and the image undergoes an adaptive boundary refinement. This second segmentation performs the optimization locally and can be accomplished within milliseconds. We test our system on an Appl...

This paper presents a toolbox for interactive image segmentation based on a series of nested partitions of increasing coarseness. The user can navigate among the different resolution levels and select regions with simple mouse clicks. The whole family of partitions can be created during a single morphological flooding of the image, leading to a very fast algorithm. The information is stored in a minimum spanning tree.

We present an out-of-core mesh decimation algorithm that is able to handle input and output meshes of arbitrary size. The algorithm reads the input from a data stream in a single pass and writes the output to another stream while using only a fixed-sized in-core buffer. By applying randomized multiple choice optimization, we are able to use incremental mesh decimation based on edge collapses and the quadric error metric. The quality of our results is comparable to state-of-the-art high-quality mesh decimation schemes (which are slower than our algorithm) and the decimation performance matches the performance of the most efficient out-of-core techniques (which generate meshes of inferior quality).

The random walker (RW) algorithm is used for both image segmentation and registration, and possesses several useful properties that make it popular in medical imaging, such as being globally optimizable, allowing user interaction, and providing uncertainty information. The RW algorithm defines a weighted graph over an image and uses the graph's Laplacian matrix to regularize its solutions. This regularization reduces to solving a large system of equations, which may be excessively time consuming in some applications, such as when interacting with a human user. Techniques have been developed that precompute eigenvectors of a Laplacian offline, after image acquisition but before any analysis, in order speed up the RW algorithm online, when segmentation or registration is being performed. However, precomputation requires certain algorithm parameters be fixed offline, limiting their flexibility. In this paper, we develop techniques to update the precomputed data online when RW param...

We present Spotlight, an automated user guidance technique for improving quality and efficiency of interactive segmentation tasks. Spotlight augments interactive segmentation algorithms by automatically highlighting areas in need of attention to the user during the interaction phase. We employ a 3D Livewire algorithm as our base segmentation method where the user quickly provides a minimal initial contour seeding. The quality of the initial segmentation is then evaluated based on three different metrics that probe the contour edge strength, contour stability and object connectivity. The result of this evaluation is fed into a novel algorithm that autonomously suggests regions that require user intervention. Essentially, Spotlight flags potentially problematic image regions in a prioritized fashion based on an optimization process for improving the final 3D segmentation. We present a variety of qualitative and quantitative examples demonstrating Spotlight's intuitive use and proven utility in reducing user input by increasing automation.

Colorectal cancer (CRC) is the third most common type of cancer with the liver being the most common site for cancer spread. A precise understanding of patient liver anatomy and pathology, as well as surgical planning based on that, plays a critical role in the treatment process. In some cases, surgeons request a 3D reconstruction, which requires a thorough analysis of the available images to be converted into 3D models of relevant objects through a segmentation process. Liver vessel segmentation is challenging due to the large variations in size and directions of the vessel structures as well as difficult contrasting conditions. In recent years, deep learning-based methods had been outperforming the conventional image analysis methods in the field of medical imaging. Though Convolutional Neural Networks (CNN) have been proved to be efficient for the task of medical image segmentation, the way of handling the image data and the preprocessing techniques play an important role in segm...

This tutorial review paper consolidates the existing applications of the power watershed (PW) optimization framework in the context of image processing. In the literature, it is known that PW framework when applied to some well-known graph-based image segmentation and filtering algorithms such as random walker, isoperimetric partitioning, ratio-cut clustering, multi-cut and shortest path filters yield faster yet consistent solutions. In this paper, the intuition behind the working of PW framework, i.e. exploitation of contrast invariance on image data is explained. The intuitions are illustrated with toy images and experiments on simulated astronomical images. This article is primarily aimed at researchers working on image segmentation and filtering problems in application areas such as astronomy where images typically have huge number of pixels. Classic graph-based cost minimization methods provide good results on images with small number of pixels but do not scale well for images with large number of pixels. The ideas from the article can be adapted to a large class of graph-based cost minimization methods to obtain scalable segmentation and filtering algorithms.