Image based localization in urban environments

Computer Vision – ECCV 2010, 2010

Finding an image's exact GPS location is a challenging computer vision problem that has many real-world applications. In this paper, we address the problem of finding the GPS location of images with an accuracy which is comparable to hand-held GPS devices.We leverage a structured data set of about 100,000 images build from Google Maps Street View as the reference images. We propose a localization method in which the SIFT descriptors of the detected SIFT interest points in the reference images are indexed using a tree. In order to localize a query image, the tree is queried using the detected SIFT descriptors in the query image. A novel GPS-tag-based pruning method removes the less reliable descriptors. Then, a smoothing step with an associated voting scheme is utilized; this allows each query descriptor to vote for the location its nearest neighbor belongs to, in order to accurately localize the query image. A parameter called Confidence of Localization which is based on the Kurtosis of the distribution of votes is defined to determine how reliable the localization of a particular image is. In addition, we propose a novel approach to localize groups of images accurately in a hierarchical manner. First, each image is localized individually; then, the rest of the images in the group are matched against images in the neighboring area of the found first match. The final location is determined based on the Confidence of Localization parameter. The proposed image group localization method can deal with very unclear queries which are not capable of being geolocated individually.

This paper proposed a new vehicle geo-localization method in urban environment integrating a new source of information that is a virtual 3D city model. This 3D model provides a realistic representation of the navigation environment of the vehicle. To optimize the performance of vehicle geo-localization system, several sources of information are integrated for their complementarity and redundancy: a GPS receiver, proprioceptive sensors (odometers and gyrometer), a video camera and a virtual 3D city model. The pose estimation algorithm used to fuse the different sensors data is an IMM-UKF (Interacting Multiple Model -Unscented Kalman Filter). The proprioceptive sensors allow to continuously estimating the dead-reckoning position and orientation of the vehicle. This dead-reckoning estimation of the pose is corrected by GPS measurements. Moreover, a 3D model/camera based observation of the vehicle pose is constructed to compensate the drift of the dead-reckoning localization when GPS measurements are unavailable for a long time. This pose observation is based on the matching between the virtual image extracted from the 3D city model and the real image acquired by the camera. The observation construction is composed of two major parts. The first part consists in detecting and matching the feature points of the real and virtual images. Three features are compared: Harris corner, SIFT (Scale Invariant Feature Transform) and SURF (Speed Up Robust Features). The second part is the pose computation using POSIT algorithm and the previously matched features set. The developed approach has been tested on a real sequence and the obtained results proved the feasibility and robustness of the approach.

Lecture Notes in Computer Science, 2015

In this paper we propose a simple and lightweight solution to estimate the geospatial trajectory of a moving vehicle from images captured by a cellphone exploiting the map and the imagery provided by Google Streetview. Images are intelligently compared against the streetview data, and a recursive Bayesian framework is applied to perform continuous localization of the vehicle inside the discrete structure of the streetview graph. Experiments run on a dataset 10.7 km long, show that the system is able to infer its position and orientation despite the low resolution and limited field of view offered by an off the shelf consumer device. Our method shows to be robust with respect to significant changes in appearance and structure of the environment across the images, obtaining an average accuracy of 13 meters in position, and 16 • in orientation.

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016

Vision based localization is widely investigated for the autonomous navigation and robotics. One of the basic steps of vision based localization is the extraction of interest points in images that are captured by the embedded camera. In this paper, SIFT and SURF extractors were chosen to evaluate their performance in localization. Four street view image sequences captured by a mobile mapping system, were used for the evaluation and both SIFT and SURF were tested on different image scales. Besides, the impact of the interest point distribution was also studied. We evaluated the performances from for aspects: repeatability, precision, accuracy and runtime. The local bundle adjustment method was applied to refine the pose parameters and the 3D coordinates of tie points. According to the results of our experiments, SIFT was more reliable than SURF. Apart from this, both the accuracy and the efficiency of localization can be improved if the distribution of feature points are well constrained for SIFT.

Proceedings of the 19th ACM international conference on Multimedia - MM '11, 2011

We study the feasibility of solving the challenging problem of geolocalizing ground level images in urban areas with respect to a database of images captured from the air such as satellite and oblique aerial images. We observe that comprehensive aerial image databases are widely available while complete coverage of urban areas from the ground is at best spotty. As a result, localization of ground level imagery with respect to aerial collections is a technically important and practically significant problem. We exploit two key insights: (1) satellite image to oblique aerial image correspondences are used to extract building facades, and (2) building facades are matched between oblique aerial and ground images for geo-localization. Key contributions include: (1) A novel method for extracting building facades using building outlines; (2) Correspondence of building facades between oblique aerial and ground images without direct matching; and (3) Position and orientation estimation of ground images. We show results of ground image localization in a dense urban area.

2002

We have previously developed a mobile robot system which uses scale invariant visual landmarks to localize and simultaneously build a 3D map of the environment In this paper, we look at global localization, also known as the kidnapped robot problem, where the robot localizes itself globally, without any prior location estimate. This is achieved by matching distinctive landmarks in the current frame to a database map. A Hough Transform approach and a RANSAC approach for global localization are compared, showing that RANSAC is much more efficient. Moreover, robust global localization can be achieved by matching a small sub-map of the local region built from multiple frames. 0-7803-7398-7/02/$17.00 ©2002 IEEE

Isprs Journal of Photogrammetry and Remote Sensing, 2018

A landmark based localization with uncertainty analysis based on cameras and geo-referenced landmarks is presented in this paper. The system is developed to adapt different camera configurations for six degree-of-freedom pose estimation. Local bundle adjustment is applied for optimization and the georeferenced landmarks are integrated to reduce the drift. In particular, the uncertainty analysis is taken into account. On the one hand, we estimate the uncertainties of poses to predict the precision of localization. On the other hand, uncertainty propagation is considered for matching, tracking and landmark registering. The proposed method is evaluated on both KITTI benchmark and the data acquired by a mobile mapping system. In our experiments, decimeter level accuracy can be reached.

2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), 2016

With the fast development of Geographic Information Systems, visual global localization has gained a lot of attention due to the low price of a camera and the practical implications. In this paper, we leverage Google Street View and a monocular camera to develop a refined and continuous positioning in urban environments: namely a topological visual place recognition and then a 6 DoF pose estimation by local bundle adjustment. In order to avoid discrete localization problems, augmented Street View data are virtually synthesized to render a smooth and metric localization. We also demonstrate that this approach significantly improves the sub-meter accuracy and the robustness to important viewpoint changes, illumination and occlusion.

In this paper, we address the problem of cross-view image geo-localization. Specifically, we aim to estimate the GPS location of a query street view image by finding the matching images in a reference database of geo-tagged bird's eye view images, or vice versa. To this end, we present a new framework for cross-view image geo-localization by taking advantage of the tremendous success of deep convolutional neural networks (CNNs) in image classification and object detection. First, we employ the Faster R-CNN [16] to detect buildings in the query and reference images. Next, for each building in the query image , we retrieve the k nearest neighbors from the reference buildings using a Siamese network trained on both positive matching image pairs and negative pairs. To find the correct NN for each query building, we develop an efficient multiple nearest neighbors matching method based on dominant sets. We evaluate the proposed framework on a new dataset that consists of pairs of street view and bird's eye view images. Experimental results show that the proposed method achieves better geo-localization accuracy than other approaches and is able to generalize to images at unseen locations .

ArXiv, 2015

An efficient method is proposed for refining GPS-acquired location coordinates in urban areas using camera images, Google Street View (GSV) and sensor parameters. The main goal is to compensate for GPS location imprecision in dense area of cities due to proximity to walls and buildings. Avail-able methods for better localization often use visual information by using query images acquired with camera-equipped mobile devices and applying image retrieval techniques to find the closest match in a GPS-referenced image data set. The search areas required for reliable search are about 1-2 sq. Km and the accuracy is typically 25-100 meters. Here we describe a method based on image retrieval where a reliable search can be confined to areas of 0.01 sq. Km and the accuracy in our experiments is less than 10 meters. To test our procedure we created a database by acquiring all Google Street View images close to what is seen by a pedestrian in a large region of downtown Chicago and saved all coor...