Object Recognition with 3D Models.

2003

ABSTRACT This paper presents a new view based 3D object recognition system. The system relies on the sequentiality of a set of object views. These views are represented by short feature vectors extracted from the object shape curvature. Since the acquisition of all possible views of a real object is problematic, the system can be trained a priori with databases of virtual objects that are expected to be observed. Nevertheless, our system is capable of learning real objects in an unsupervised way.

arXiv (Cornell University), 2023

To alleviate the high annotation cost in LiDAR-based 3D object detection, active learning is a promising solution that learns to select only a small portion of unlabeled data to annotate, without compromising model performance. Our empirical study, however, suggests that mainstream uncertainty-based and diversitybased active learning policies are not effective when applied in the 3D detection task, as they fail to balance the trade-off between point cloud informativeness and box-level annotation costs. To overcome this limitation, we jointly investigate three novel criteria in our framework CRB for point cloud acquisitionlabel conciseness, feature representativeness and geometric balance, which hierarchically filters out the point clouds of redundant 3D bounding box labels, latent features and geometric characteristics (e.g., point cloud density) from the unlabeled sample pool and greedily selects informative ones with fewer objects to annotate. Our theoretical analysis demonstrates that the proposed criteria aligns the marginal distributions of the selected subset and the prior distributions of the unseen test set, and minimizes the upper bound of the generalization error. To validate the effectiveness and applicability of CRB, we conduct extensive experiments on the two benchmark 3D object detection datasets of KITTI and Waymo and examine both one-stage (i.e., SECOND) and two-stage 3D detectors (i.e., PV-RCNN). Experiments evidence that the proposed approach outperforms existing active learning strategies and achieves fully supervised performance requiring 1% and 8% annotations of bounding boxes and point clouds, respectively. Source code: https://github.com/Luoyadan/CRB-active-3Ddet.

2016

Object recognition is one of the fundamental tasks of computer vision. Recent advances in the field enable reliable 2D detections from a single cluttered image. However, many challenges still remain. Object detection needs timely response for real world applications. Moreover, we are genuinely interested in estimating the 3D pose and shape of an object or human for the sake of robotic manipulation and human-robot interaction. In this thesis, a suite of solutions to these challenges is presented. First, Active Deformable Part Models (ADPM) is proposed for fast part-based object detection. ADPM dramatically accelerates the detection by dynamically scheduling the part evaluations and efficiently pruning the image locations. Second, we unleash the power of marrying discriminative 2D parts with an explicit 3D geometric representation. Several methods of such scheme are proposed for recovering rich 3D information of both rigid and non-rigid objects from monocular RGB images. (1) The accur...

IEEE Access

Three-dimensional point clouds produced by 3D scanners are often noisy and contain outliers. Such data inaccuracies can significantly affect current deep learning-based methods and reduce their ability to classify objects. Most deep neural networks-based object classification methods were targeted to achieve high classification accuracy without considering classification robustness. Thus, despite their great success, they still fail to achieve good classification accuracy with low levels of noise and outliers. This work is carried out to develop a robust network structure that can solidly identify objects. The proposed method uses patches of planar segments, which can robustly capture object appearance. The planar segments information are then fed into a deep neural network for classification. We base our approach on the PointNet deep learning architecture. Our method was tested against several kinds of data inaccuracies such as scattered outliers, clustered outliers, noise and missing points. The proposed method shows excellent performance in the presence of these inaccuracies compared to state-of-the-art techniques. By decomposing objects into planes, the suggested method is simple, fast, provides good classification accuracy and can handle different kinds of point cloud data inaccuracies. The code can be found at https://github.com/AymanMukh/Pl-Net3D.

This thesis addresses the problem of learning object classification using multi-view range data. Class membership is determined by shared characteristics, which can be visual, structural or functional. The major steps in object recognition and object classification are: segmentation, feature extraction, object representation and learning.

Image and Vision Computing, 2001

The recognition of free-form 3D objects using 3D models under different viewing conditions based on the geometric hashing algorithm and global veri®cation is presented. The matching stage of the algorithm uses the hash-table prepared in the off-line stage. Given a scene of feature points, one tries to match the measurements taken at scene points to those memorised in the hash-table. The technique used for feature recovery is the generalisation of the CSS method (IEEE Trans. Pattern Anal. Mach. Intell., 14 (1992) 789±805), which is a powerful shape descriptor expected to be an MPEG-7 standard. Smoothing is used to remove noise and reduce the number of feature points to add to the ef®ciency and robustness of the system. The local maxima of Gaussian and mean curvatures are selected as feature points. Furthermore, the torsion maxima of the zero-crossing contours of Gaussian and mean curvatures are also selected as feature points. Recognition results are demonstrated for rotated and scaled as well as partially occluded objects. In order to verify match, 3D translation, rotation and scaling parameters are used for veri®cation and results indicate that our technique is invariant to those transformations. Our technique for smoothing and feature extraction is more suitable than level set methods or volumetric diffusion for object recognition applications since it is applicable to incomplete surface data that arise during occlusion. It is also more ef®cient and allows for accurate estimation of curvature values. q

Pattern Recognition, 2000. …, 2000

Frontiers in Robotics and AI, 2023

There are a large number of publicly available datasets of 3D data, they generally suffer from some drawbacks, such as small number of data samples, and class imbalance. Data augmentation is a set of techniques that aim to increase the size of datasets and solve such defects, and hence to overcome the problem of overfitting when training a classifier. In this paper, we propose a method to create new synthesized data by converting complete meshes into occluded 3D point clouds similar to those in real-world datasets. The proposed method involves two main steps, the first one is hidden surface removal (HSR), where the occluded parts of objects surfaces from the viewpoint of a camera are deleted. A low-complexity method has been proposed to implement HSR based on occupancy grids. The second step is a random sampling of the detected visible surfaces. The proposed two-step method is applied to a subset of ModelNet40 dataset to create a new dataset, which is then used to train and test three different deep-learning classifiers (VoxNet, PointNet, and 3DmFV). We studied classifiers performance as a function of the camera elevation angle. We also conducted another experiment to show how the newly generated data samples can improve the classification performance when they are combined with the original data during training process. Simulation results show that the proposed method enables us to create a large number of new data samples with a small size needed for storage. Results also show that the performance of classifiers is highly dependent on the elevation angle of the camera. In addition, there may exist some angles where performance degrades significantly. Furthermore, data augmentation using our created data improves the performance of classifiers not only when they are tested on the original data, but also on real data.

Symmetry, 2019

Small series production with a high level of variability is not suitable for full automation. So, a manual assembly process must be used, which can be improved by cooperative robots and assisted by augmented reality devices. The assisted assembly process needs reliable object recognition implementation. Currently used technologies with markers do not work reliably with objects without distinctive texture, for example, screws, nuts, and washers (single colored parts). The methodology presented in the paper introduces a new approach to object detection using deep learning networks trained remotely by 3D virtual models. Remote web application generates training input datasets from virtual 3D models. This new approach was evaluated by two different neural network models (Faster RCNN Inception v2 with SSD, MobileNet V2 with SSD). The main advantage of this approach is the very fast preparation of the 2D sample training dataset from virtual 3D models. The whole process can run in Cloud. T...

This paper synthesizes insights from recent research on 3D object detection and recognition in digital images. The surveyed literature showcases advancements in deep learning architectures, sensor fusion techniques, real-time processing, robustness to occlusion, and domain adaptation. Notably, the integration of point cloud data in deep learning models enhances accuracy, while sensor fusion improves reliability in diverse lighting conditions. Optimized real-time processing, multi-view systems, and domain adaptation methods address specific challenges, contributing to the field's progress. Standard metrics and benchmark evaluations validate the effectiveness of proposed methodologies, highlighting their potential for real-world applications.