Virtual Training for Multi-View Object Class Recognition

2003

ABSTRACT This paper presents a new view based 3D object recognition system. The system relies on the sequentiality of a set of object views. These views are represented by short feature vectors extracted from the object shape curvature. Since the acquisition of all possible views of a real object is problematic, the system can be trained a priori with databases of virtual objects that are expected to be observed. Nevertheless, our system is capable of learning real objects in an unsupervised way.

We propose techniques for designing and training of pose-invariant object recognition systems using realistic 3d computer graphics models. We look at the relation between the size of the training set and the classification accuracy for a basic recognition task and provide a method for estimating the degree of difficulty of detecting an object. We show how to sample, align, and cluster images of objects on the view sphere. We address the problem of training on large, highly redundant data and propose a novel active learning method which generates compact training sets and compact classifiers.

Lecture Notes in Computer Science, 2003

In the past decades most object recognition systems were based on passive approaches. But in the last few years a lot of research was done in the field of active object recognition. In this context there are several unique problems to be solved, like the fusion of several views and the selection of the best next viewpoint. In this paper we present an approach to solve the problem of choosing optimal views (viewpoint selection) and the fusion of these for an optimal 3D object recognition (viewpoint fusion). We formally define the selection of additional views as an optimization problem and we show how to use reinforcement learning for viewpoint training and selection in continuous state spaces without user interaction. We also present an approach for the fusion of multiple views based on recursive density propagation. The experimental results show that our viewpoint selection is able to select a minimal number of views and perform an optimal object recognition with respect to the classification.

Proceedings of the British Machine Vision Conference 2014, 2014

In this paper we present a top-down, scene-driven, approach to classify the viewpoint of elongated objects. Viewpoint classification is achieved based on the correspondence between the image evidence acquired by a detector on one hand, and bounding boxes of object proposals generated from scene consistent cues on the other hand. We explore several methods to generate scene-driven proposals: first, by generating object proposals in the 3D scene and, using the ground plane, projecting them to the image space, and second, by sampling objects from a history set of objects previously seen with the same camera setup in the scene. Experiments on the challenging KITTI dataset show the performance of the proposed method for viewpoint classification for controlled-camera applications such as autonomous navigation.

arXiv (Cornell University), 2024

Human decision-making often relies on visual information from multiple perspectives or views. In contrast, machine learning-based object recognition utilizes information from a single image of the object. However, the information conveyed by a single image may not be sufficient for accurate decision-making, particularly in complex recognition problems. The utilization of multi-view 3D representations for object recognition has thus far demonstrated the most promising results for achieving state-of-the-art performance. This review paper comprehensively covers recent progress in multi-view 3D object recognition methods for 3D classification and retrieval tasks. Specifically, we focus on deep learning-based and 1

In this paper we present a top-down, scene-driven, approach to classify the viewpoint of elongated objects. Viewpoint classification is achieved based on the correspondence between the image evidence acquired by a detector on one hand, and bounding boxes of object proposals generated from scene consistent cues on the other hand. We explore several methods to generate scene-driven proposals: first, by generating object proposals in the 3D scene and, using the ground plane, projecting them to the image space, and second, by sampling objects from a history set of objects previously seen with the same camera setup in the scene. Experiments on the challenging KITTI dataset show the performance of the proposed method for viewpoint classification for controlled-camera applications such as autonomous navigation.

This thesis addresses the problem of learning object classification using multi-view range data. Class membership is determined by shared characteristics, which can be visual, structural or functional. The major steps in object recognition and object classification are: segmentation, feature extraction, object representation and learning.

2004

We present a new approach to appearance-based object recognition, which captures the relationships between multiple model views and exploits them to improve recognition performance.

2004

Given a collection of sets of 2-D views of 3-D objects and a similarity measure between them, we present a method for summarizing the sets using a small subset called a bounded canonical set (BCS), whose members best represent the members of the original set. This means that members of the BCS are as dissimilar from each other as possible, while at the same time being as similar as possible to the non-BCS members. This paper will extend our earlier work on computing canonical sets in several ways: by omitting the need for a multi-objective optimization, by allowing the imposition of cardinality constraints, and by introducing a total similarity function. We evaluate the applicability of BCS to view selection in a view-based object recognition environment.

1994

This paper describes the main features of a view-based model of object recognition. The model tries to capture general properties to be expected in a biological architecture for object recognition. The basic module is a regularization network in which each of the hidden units is broadly tuned to a speci c view of the object to be recognized. The network output, which m a y be largely view independent, is rst described in terms of some simple simulations. The following re nements and details of the basic module are then discussed: (1) some of the units may represent only components of views of the object { the optimal stimulus for the unit, its \center", is e ectively a complex feature (2) the units' properties are consistent with the usual description of cortical neurons as tuned to multidimensional optimal stimuli (3) in learning to recognize new objects, preexisting centers may be used and modi ed, but also new centers may be created incrementally so as to provide maximal invariance (4) modules are part of a hierarchical structure: the output of a network may be used as one of the inputs to another, in this way synthesizing increasingly complex features and templates (5) in several recognition tasks, in particular at the basic level, a single center using view-invariant features may be su cient. Modules of this type can deal with recognition of speci c objects, for instance a speci c face under various transformations such as those due to viewpoint and illumination, provided that a su cient n umber of example views of the speci c object are available. An architecture for 3D object recognition, however, must cope { to some extent { e v en when only a single model view is given. The main contribution of this paper is an outline of a recognition architecture that deals with objects of a nice class undergoing a broad spectrum of transformations { due to illumination, pose, expression and so on { by exploiting prototypical examples. A nice class of objects is a set of objects with su ciently similar transformation properties under speci c transformations, such as viewpoint transformations. For nice object classes, we discuss two possibilities: (a) class-speci c transformations are to be applied to a single model image to generate additional virtual example views, thus allowing some degree of generalization beyond what a single model view could otherwise provide (b) class speci c, view-invariant features are learned from examples of the class and used with the novel model image, without an explicit generation of virtual examples.