Lukas Polok

Brno University of Technology, Faculty of Information Technology, Faculty Member

Followers

Following

Co-authors

Public Views

Interests

Uploads

Papers by Lukas Polok

Accelerated Sparse Matrix Operations in Nonlinear Least Squares Solvers

This thesis focuses on data structures for sparse block matrices and the associated algorithms fo... more This thesis focuses on data structures for sparse block matrices and the associated algorithms for performing linear algebra operations that I have developed. Sparse block matrices occur naturally in many key problems, such as Nonlinear LEast Squares (NLS) on graphical models. NLS are used by e.g. Simultaneous Localization and Mapping (SLAM) in robotics, Bundle Adjustment (BA) or Structure from Motion (SfM) in computer vision. Sparse block matrices also occur when solving Finite Element Methods (FEMs) or Partial Differential Equations (PDEs) in physics simulations. The majority of the existing state of the art sparse linear algebra implementations use elementwise sparse matrices and only a small fraction of them support sparse block matrices. This is perhaps due to the complexity of sparse block formats which reduces computational efficiency, unless the blocks are very large. Some of the more specialized solvers in robotics and computer vision use sparse block matrices internally to reduce sparse matrix assembly costs, but finally end up converting such representation to an elementwise sparse matrix for the linear solver. Most of the existing sparse block matrix implementations focus only on a single operation, such as the matrix-vector product. The solution proposed in this thesis covers a broad range of functions: it includes efficient sparse block matrix assembly, matrix-vector and matrix-matrix products as well as triangular solving and Cholesky factorization. These operations can be used to construct both direct and iterative solvers as well as to compute eigenvalues. Highly efficient algorithms for both Central Processing Units (CPUs) and Graphics Processing Units (GPUs) are provided. The proposed solution is integrated in SLAM++, a nonlinear least squares solver focused on robotics and computer vision. It is evaluated on standard datasets where it proves to significantly outperform other similar state of the art implementations, without sacrificing generality or accuracy in any way.

“Local Rank Differences” Image Feature Implemented on GPU

A currently popular trend in object detection and pattern recognition is usage of statistical cla... more A currently popular trend in object detection and pattern recognition is usage of statistical classifiers, namely AdaBoost and its modifications. The speed performance of these classifiers largely depends on the low level image features they are using: both on the amount of information the feature provides and the executional time of its evaluation. Local Rank Differences is an image feature that is alternative to commonly used haar wavelets. It is suitable for implementation in programmable (FPGA) or specialized (ASIC) hardware, butas this paper shows-it performs very well on graphics hardware (GPU) as well. The paper discusses the LRD features and their properties, describes an experimental implementation of LRD in graphics hardware, presents its empirical performance measures compared to alternative approaches and suggests several notes on practical usage of LRD and proposes directions for future work.

Download

Fast sparse matrix multiplication on GPU

ABSTRACT

Low-Level Image Features for Real-Time Object Detection

Pattern Recognition Recent Advances, 2010

Download

Fast covariance recovery in incremental nonlinear least square solvers

2015 IEEE International Conference on Robotics and Automation (ICRA), 2015

ABSTRACT

Cache efficient implementation for block matrix operations

Proceedings of the High Performance Computing Symposium, Apr 7, 2013

ABSTRACT Efficiently manipulating and operating on block matrices can be beneficial in many appli... more ABSTRACT Efficiently manipulating and operating on block matrices can be beneficial in many applications, among others those involving iteratively solving nonlinear systems. These types of problems consist of repeatedly assembling and solving sparse linear systems. In the case of very large systems, without a careful manipulation of the corresponding matrices, solving can become very time consuming. This paper proposes a memory storage scheme convenient for both, numeric and structural matrix modification and, at the same time, allowing efficient arithmetic operation. This scheme was used in the implementation of a simple BLAS-like library. The advantage of the new scheme is demonstrated through exhaustive tests on the popular University of Florida Sparse Matrix Collection. Furthermore, this library was used in solving several nonlinear graph optimization problems.

Efficient implementation for block matrix operations for nonlinear least squares problems in robotic applications

2013 IEEE International Conference on Robotics and Automation, 2013

ABSTRACT A large number of robotic, computer vision and computer graphics applications rely on ef... more ABSTRACT A large number of robotic, computer vision and computer graphics applications rely on efficiently solving the associated sparse linear systems. Simultaneous localization and mapping (SLAM), structure from motion (SfM), non-rigid shape recovery, and elastodynamic simulations are only few examples in this direction. In general, these problems are nonlinear and the solution can be approximated by incrementally solving a series of linearized problems. In some applications, the size of the system considerably affects the performance, especially when the sparsity is low. This paper exploits the block structure of such problems and offers very efficient solutions to manipulate block matrices within iterative nonlinear solvers. The resulting method considerably speeds-up the execution of the implementation of the nonlinear optimization problem. In this work, in particular, we focus our effort on testing the method on SLAM applications, but the applicability of the technique remains general. Our implementation outperforms the state of the art SLAM implementations on all tested datasets. In incremental mode, where a larger portion of time is spent in updating the system, our implementation is on average two times faster than the others.

Quality assurance in large collections of video sequences

2015 IEEE International Conference on Image Processing (ICIP), 2015

In the modern digital cinema production, extremely large volumes (in order of 10s of TB) of foota... more In the modern digital cinema production, extremely large volumes (in order of 10s of TB) of footage data are captured every day. The process of cataloging and reviewing such footage is nowadays largely manual and time consuming process. In our work, we aim at technical quality aspects, such as correct exposure, color compatibility of adjacent shots, and focusing. The main goal is to assist the reviewing process by providing shot quality meta-data and possibly ordering or even culling significant portion of the data from the review, as better quality shot of the same scene exists. However, in order to meaningfully compare technical quality, temporal shot synchronization needs to be performed first. We propose a fast and robust method for time synchronization of video sequences, capturing similar scenes, which arise naturally in digital cinema production. The method is tested on an extensive library of sequences and its performance is evaluated. We further present a preview of application of the proposed method to detecting focusing errors.

Fast Incremental Bundle Adjustment with Covariance Recovery

Efficient algorithms exist to obtain a sparse 3D representation of the environment. Bundle adjust... more Efficient algorithms exist to obtain a sparse 3D representation of the environment. Bundle adjustment (BA) and structure from motion (SFM) are techniques used to estimate both the camera poses and the set of sparse points in the environment. Many applications require such reconstruction to be performed online, while acquiring the data, and produce an updated result every step. Furthermore, using active feedback about the quality of the reconstruction can help selecting the best views to increase the accuracy as well as to maintain a reasonable size of the collected data. This paper provides novel and efficient solutions to solving the associated NLS incrementally, and to compute not only the optimal solution, but also the associated uncertainty. The proposed technique highly increases the efficiency of the incremental BA solver for long camera trajectory applications , and provides extremely fast covariance recovery.

Download

Pivoting Strategy for Fast LU decomposition of Sparse Block Matrices

Solving large linear systems is a fundamental task in many interesting problems, including finite... more Solving large linear systems is a fundamental task in many interesting problems, including finite element methods (FEM) or (non-)linear least squares (NLS), among others. Furthermore, the problems of interest here are sparse: not all the vertices in a typical FEM mesh are connected, or similarly not all vertices in a graphical inference model are linked by observations, as is the case in e.g. simultaneous localization and mapping (SLAM) in robotics or bundle adjustment (BA) in computer vision. The two places where most of the time is spent in solving such problems are usually the sparse matrix assembly and solving the underlying linearized system. An interesting property of the above-mentioned problems is their block structure. It is given by the variables existing in a multi-dimensional space such as 2D, 3D or even se(3) and hence their respective derivatives being dense blocks of the corresponding dimension. In our previous work (Polok, Ila, and Smrž 2013), we demonstrated the benefits of explicitly representing those blocks in the sparse matrix, namely reduced assembly time and increased efficiency of arithmetic operations. In this paper, we propose a novel implementation of sparse block LU decomposition and demonstrate its benefits on standard datasets. While not difficult to implement, the enabling feature is the pivoting strategy that makes the method numerically stable. The proposed algorithm is on average three times faster (over 50× faster in the best case), causes less fill-in and produces decompositions of comparable and often better precision than the conventional methods.

Download

SLAM++-A highly efficient and temporally scalable incremental SLAM framework

by Lukas Polok and Pavel Svoboda

The most common way to deal with the uncertainty present in noisy sensorial perception and action... more The most common way to deal with the uncertainty present in noisy sensorial perception and action is to model the problem with a probabilistic framework. Maximum likelihood estimation is a well-known estimation method used in many robotic and computer vision applications. Under Gaussian assumption, the maximum likelihood estimation converts to a nonlinear least squares problem. Efficient solutions to nonlinear least squares exist and they are based on iteratively solving sparse linear systems until convergence. In general, the existing solutions provide only an estimation of the mean state vector, the resulting covariance being computationally too expensive to recover. Nevertheless, in many simultaneous localization and mapping (SLAM) applications, knowing only the mean vector is not enough. Data association, obtaining reduced state representations, active decisions and next best view are only a few of the applications that require fast state covariance recovery. Furthermore, computer vision and robotic applications are in general performed online. In this case, the state is updated and recomputed every step and its size is continuously growing, therefore, the estimation process may become highly computationally demanding. This paper introduces a general framework for incremental maximum likelihood estimation called SLAM++, which fully benefits from the incremental nature of the online applications, and provides efficient estimation of both the mean and the covariance of the estimate. Based on that, we propose a strategy for maintaining a sparse and scalable state representation for large scale mapping, which uses information theory measures to integrate only informative and non-redundant contributions to the state representation. SLAM++ differs from existing implementations by performing all the matrix operations by blocks. This led to extremely fast matrix manipulation and arithmetic operations used in nonlinear least squares. Even though this paper tests SLAM++ efficiency on SLAM problems, its applicability remains general.

Download

Fast sparse matrix multiplication on GPU

by Viorela Ila and Lukas Polok

ABSTRACT

Jigsaw

by Viorela Ila, Simon Pabst, and Lukas Polok

ACM SIGGRAPH 2015 Posters on - SIGGRAPH '15, 2015

IMPART: BIG MEDIA DATA PROCESSING AND ANALYSIS FOR FILM PRODUCTION

A typical high-end film production generates several terabytes of data per day, either as footage... more A typical high-end film production generates several terabytes of data per day, either as footage from multiple cameras or as background information regarding the set (laser scans, spherical captures, etc). The EU project IMPART (impart.upf.edu) has been researching solutions that improve the integration and understanding of the quality of the multiple data sources to support creative decisions onset or near it, and an enhanced post-production as well. The main results covered in this paper are: a public multisource production dataset made available for research purposes, monitoring and quality assurance of multicamera setups , multisource registration, anthropocentric visual analysis for semantic content annotation, acceleration of 3D reconstruction, and integrated 2D-3D web visualization tools. Index Terms— Multi-modal data processing, big media data analysis, web 3D visualization

Download

Implementing Random Indexing on GPU

Vector space models have received a significant attention in recent years. They have been applied... more Vector space models have received a significant attention in recent years. They have been applied in a wide spectrum of areas including information filtering, information retrieval, document indexing and relevancy ranking. Random indexing is one of the methods employing distributional statistics of term co-occurrences to generate vector space models from a set of documents. If the size of the document collection is large, a significant computational power is required to compute the results. This paper presents an efficient implementation of the random indexing method on GPU which allows efficient training on large datasets. It is only limited by the amount of memory available on the GPU. Various ways to overcome the dependence on the GPU memory are discussed. Speedups in magnitude of tens are achieved for training from random seed vectors, and even much higher figures for retraining. The implementation scales well with both the term vector dimension and the seed length.

Download

Fast and Accurate Refinement Method for 3D Reconstruction from Stereo Spherical Images

by Lukas Polok and M. Solony

Realistic 3D models of the environment are beneficial in many fields, from natural or man-made st... more Realistic 3D models of the environment are beneficial in many fields, from natural or man-made structure inspection and volumetric analysis, to movie-making, in particular, special effects integration to natural scenes. Spherical cameras are becoming popular in environment modelling because they capture the full surrounding scene visible from the camera location as a consistent seamless image at once. In this paper, we propose a novel pipeline to obtain fast and accurate 3D reconstructions from spherical images. In order to have a better estimation of the structure, the system integrates a joint camera pose and structure refinement step. This strategy proves to be much faster, yet equally accurate, when compared to the conventional method, registration of a dense point cloud via iterative closest point (ICP). Both methods require an initial estimate for successful convergence. The initial positions of the 3D points are obtained from stereo processing of pair of spherical images with known baseline. The initial positions of the cameras are obtained from a robust wide-baseline matching procedure. The performance and accuracy of the 3D reconstruction pipeline is analysed through extensive tests on several indoor and outdoor datasets.

Download

3D Reconstruction Quality Analysis and Its Acceleration on GPU Clusters

—3D reconstruction has a wide variety of applications in computer graphics, robotics or digital c... more —3D reconstruction has a wide variety of applications in computer graphics, robotics or digital cinema production, among others. With the rapid increase in computing power, it has become more feasible for the reconstruction algorithms to run online, even on mobile devices. Maximum likelihood estimation (MLE) is the adopted technique to deal with the sensor uncertainty. Most of the existing 3D reconstruction frameworks only recover the mean of the reconstructed geometry. Recovering also the variance is highly computationally intensive and is seldom performed. However, variance is the natural choice of estimate quality indicator. In this paper, the associated costs are analyzed and efficient but exact solutions to calculating partial matrix inverses are proposed, which apply to any general problem with many mutually independent variables. Speedups exceeding an order of magnitude are reported.

Download

The Effect of Different Parameterisations in Incremental Structure from Motion

Accurate online estimation of the structure of the environment together with the pose of the robo... more Accurate online estimation of the structure of the environment together with the pose of the robot is an important component to enable autonomous robotic applications. This paper analyses the different parameterisations used in structure from motion (SFM) problem in the context of accuracy and efficiency of the on-line solutions. Three point parameterisations are compared: Euclidean, inverse depth and inverse distance. At the same time two representations , global and local point coordinates are tested. Different metrics are used to compare the results, camera localisation errors, re-projection errors, execution time as well as a complete analysis on how different parameter-isations affect the convergence, system's condition number and the incremental solving are provided. The paper shows that, with the correct parameterisation, efficient globally consistent SFM is possible, which under the assumption of small, bounded number of correspondences performs in constant time in open loop.

Download

Fast Linear Algebra on GPU

—GPUs have been successfully used for acceleration of many mathematical functions and libraries. ... more —GPUs have been successfully used for acceleration of many mathematical functions and libraries. A common limitation of those libraries is the minimal size of primitives being handled, in order to achieve a significant speedup compared to their CPU versions. The minimal size requirement can prove prohibitive for many applications. It can be loosened by batching operations in order to have sufficient amount of data to perform the calculation maximally efficiently on the GPU. A fast OpenCL implementation of two basic vector functions – vector reduction and vector scaling – is described in this paper. Its performance is analyzed by running benchmarks on two of the most common GPUs in use – Tesla and Fermi GPUs from NVIDIA. Reported experimental results show that our implementation significantly outperforms the current state-of-the-art GPU-based basic linear algebra library CUBLAS.

Download

TERRAIN RENDERING ALGORITHM PERFORMANCE ANALYSIS

Nowadays, the flight guidance equipment supplies practically all the information, required for ai... more Nowadays, the flight guidance equipment supplies practically all the information, required for aircraft navigation. Pseudo-realistic terrain visualization is undoubtedly an important part of this information. Although modern graphics processors are able to render realistic terrain at interactive frame rates, in some applications, it is beneficial to use low-power graphics hardware, perhaps from weight or power supply capacity restrictions. These low-power graphics processors typically manifest much lower computational power than conventional hardware, severely limiting the capability of terrain visualization. A novel method for caching terrain tiles is presented in this paper, enabling faster and more detailed terrain rendering, using lighter devices that consume less power. The main focus was on memory and time efficiency on common low-power graphics hardware. The terrain rendering algorithm being employed in our implementation is derived from the seamless patches algorithm. Different aspects of terrain tile swapping were examined in order to devise a simple hardware metric. An efficient tile caching approach was developed, based on this hardware metric, and its performance was evaluated.

Download

Lukas Polok

Uploads

Papers by Lukas Polok

Log In