Publications
2013
Falko Schindler and Wolfgang Förstner, "DijkstraFPS: Graph Partitioning in Geometry and Image Processing", Photogrammetrie, Fernerkundung, Geoinformation (PFG). Vol. 4, pp. 285-296. 2013.
Data partitioning is a common problem in the field of point cloud and image processing applicable to segmentation and clustering. The general principle is to have high similarity of two data points, e.g.pixels or 3D points, within one region and low similarity among regions. This pair-wise similarity between data points can be represented in an attributed graph. In this article we propose a novel graph partitioning algorithm. It integrates a sampling strategy known as farthest point sampling with Dijkstra's algorithm for deriving a distance transform on a general graph, which does not need to be embedded in some space. According to the pair-wise attributes a Voronoi diagram on the graph is generated yielding the desired segmentation. We demonstrate our approach on various applications such as surface triangulation, surface segmentation, clustering and image segmentation.
@article{Schindler2013DijkstraFPS,
author = {Schindler, Falko and Förstner, Wolfgang},
title = {DijkstraFPS: Graph Partitioning in Geometry and Image Processing},
journal = {Photogrammetrie, Fernerkundung, Geoinformation (PFG)},
year = {2013},
volume = {4},
pages = {285--296},
doi = {10.1127/1432-8364/2013/0177}
}
Falko Schindler, "Man-Made Surface Structures from Triangulated Point-Clouds". Thesis at: Department of Photogrammetry, University of Bonn. 2013.
Photogrammetry aims at reconstructing shape and dimensions of objects captured with cameras, 3D laser scanners or other spatial acquisition systems. While many acquisition techniques deliver triangulated point clouds with millions of vertices within seconds, the interpretation is usually left to the user. Especially when reconstructing man-made objects, one is interested in the underlying surface structure, which is not inherently present in the data. This includes the geometric shape of the object, e.g. cubical or cylindrical, as well as corresponding surface parameters, e.g. width, height and radius. Applications are manifold and range from industrial production control to architectural on-site measurements to large-scale city models.
The goal of this thesis is to automatically derive such surface structures from triangulated 3D point clouds of man-made objects. They are defined as a compound of planar or curved geometric primitives. Model knowledge about typical primitives and relations between adjacent pairs of them should affect the reconstruction positively.
After formulating a parametrized model for man-made surface structures, we develop a reconstruction framework with three processing steps: During a fast pre-segmentation exploiting local surface properties we divide the given surface mesh into planar regions. Making use of a model selection scheme based on minimizing the description length, this surface segmentation is free of control parameters and automatically yields an optimal number of segments. A subsequent refinement introduces a set of planar or curved geometric primitives and hierarchically merges adjacent regions based on their joint description length. A global classification and constraint parameter estimation combines the data-driven segmentation with high-level model knowledge. Therefore, we represent the surface structure with a graphical model and formulate factors based on likelihood as well as prior knowledge about parameter distributions and class probabilities. We infer the most probable setting of surface and relation classes with belief propagation and estimate an optimal surface parametrization with constraints induced by inter-regional relations. The process is specifically designed to work on noisy data with outliers and a few exceptional freeform regions not describable with geometric primitives. It yields full 3D surface structures with watertightly connected surface primitives of different types.
The performance of the proposed framework is experimentally evaluated on various data sets. On small synthetically generated meshes we analyze the accuracy of the estimated surface parameters, the sensitivity w.r.t. various properties of the input data and w.r.t. model assumptions as well as the computational complexity. Additionally we demonstrate the flexibility w.r.t. different acquisition techniques on real data sets. The proposed method turns out to be accurate, reasonably fast and little sensitive to defects in the data or imprecise model assumptions.
@phdthesis{Schindler2013Man-Made,
author = {Schindler, Falko},
title = {Man-Made Surface Structures from Triangulated Point-Clouds},
school = {Department of Photogrammetry, University of Bonn},
year = {2013},
url = {http://hss.ulb.uni-bonn.de/2013/3435/3435.htm}
}
2012
Ribana Roscher and Jan Siegemund and Falko Schindler and Wolfgang Förstner, "Object Tracking by Segmentation Using Incremental Import Vector Machines" 2012.
We propose a framework for object tracking in image sequences, following the concept of tracking-by-segmentation. The separation of object and background is achieved by a consecutive semantic superpixel segmentation of the images, yielding tight object boundaries. I.e., in the first image a model of the object's characteristics is learned from an initial, incomplete annotation. This model is used to classify the superpixels of subsequent images to object and background employing graph-cut. We assume the object boundaries to be tight-fitting and the object motion within the image to be affine. To adapt the model to radiometric and geometric changes we utilize an incremental learner in a co-training scheme. We evaluate our tracking framework qualitatively and quantitatively on several image sequences.
@techreport{Roscher2012Object,
author = {Ribana Roscher and Jan Siegemund and Falko Schindler and Wolfgang Förstner},
title = {Object Tracking by Segmentation Using Incremental Import Vector Machines},
year = {2012}
}
Falko Schindler and Wolfgang Förstner, "Real-time Camera Guidance for 3d Scene Reconstruction", In ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. I-3 2012.
We propose a framework for multi-view stereo reconstruction exploiting the possibility to interactively guiding the operator during the image acquisition process.
Multi-view stereo is a commonly used method to reconstruct both camera trajectory and 3D object shape. After determining an initial solution, a globally optimal reconstruction is usually obtained by executing a bundle adjustment involving all images. Acquiring suitable images, however, still requires an experienced operator to ensure accuracy and completeness of the final solution.
We propose an interactive framework for guiding unexperienced users or possibly an autonomous robot. Using approximate camera orientations and object points we estimate point uncertainties within a sliding bundle adjustment and suggest appropriate camera movements. A visual feedback system communicates the decisions to the user in an intuitive way.
We demonstrate the suitability of our system with a virtual image acquisition simulation as well as in real-world scenarios. We show that following the camera movements suggested by our system the final scene reconstruction with the automatically extracted key frames is both more complete and more accurate.
Possible applications are non-professional 3D acquisition systems on low-cost platforms like mobile phones, autonomously navigating robots as well as online flight planning of unmanned aerial vehicles.
@inproceedings{Schindler2012Real,
author = {Falko Schindler and Wolfgang Förstner},
title = {Real-time Camera Guidance for 3d Scene Reconstruction},
booktitle = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
year = {2012},
volume = {I-3}
}
Johannes Schneider and Falko Schindler and Thomas Läbe and Wolfgang Förstner, "Bundle Adjustment for Multi-camera Systems with Points at Infinity", In ISPRS Annals of Photogrammetry, Remote Sensing and the Spatial Information Sciences; Proc. of 22nd Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS). 2012.
We present a novel approach for a rigorous bundle adjustment for omnidirectional and multi-view cameras, which enables an efficient maximum-likelihood estimation with image and scene points at infinity. Multi-camera systems are used to increase the resolution, to combine cameras with different spectral sensitivities (Z/I DMC, Vexcel Ultracam) or - like omnidirectional cameras - to augment the effective aperture angle (Blom Pictometry, Rollei Panoscan Mark III). Additionally multi-camera systems gain in importance for the acquisition of complex 3D structures. For stabilizing camera orientations - especially rotations - one should generally use points at the horizon over long periods of time within the bundle adjustment that classical bundle adjustment programs are not capable of. We use a minimal representation of homogeneous coordinates for image and scene points. Instead of eliminating the scale factor of the homogeneous vectors by Euclidean normalization, we normalize the homogeneous coordinates spherically. This way we can use images of omnidirectional cameras with single-view point like fisheye cameras and scene points, which are far away or at infinity. We demonstrate the feasibility and the potential of our approach on real data taken with a single camera, the stereo camera FinePix Real 3D W3 from Fujifilm and the multi-camera system Ladybug3 from Point Grey.
@inproceedings{Schneider2012Bundle,
author = {Johannes Schneider and Falko Schindler and Thomas Läbe and Wolfgang Förstner},
title = {Bundle Adjustment for Multi-camera Systems with Points at Infinity},
booktitle = {ISPRS Annals of Photogrammetry, Remote Sensing and the Spatial Information Sciences; Proc. of 22nd Congress of the International Society for Photogrammetry and Remote Sensing (ISPRS)},
year = {2012},
note = {to appear}
}
2011
Timo Dickscheid and Falko Schindler and Wolfgang Förstner, "Coding Images with Local Features", International Journal of Computer Vision. Vol. 94, pp. 154-174. 2011.
We present a scheme for measuring completeness of local feature extraction in terms of image coding. Completeness is here considered as good coverage of relevant image information by the features. As each feature requires a certain number of bits which are representative for a certain subregion of the image, we interpret the coverage as a sparse coding scheme. The measure is therefore based on a comparison of two densities over the image domain: An entropy density p_H(x) based on local image statistics, and a feature coding density p_c(x) which is directly computed from each particular set of local features. Motivated by the coding scheme in JPEG, the entropy distribution is derived from the power spectrum of local patches around each pixel position in a statistically sound manner. As the total number of bits for coding the image and for representing it with local features may be different, we measure incompleteness by the Hellinger distance between p_H(x) and p_c(x). We will derive a procedure for measuring incompleteness of possibly mixed sets of local features and show results on standard datasets using some of the most popular region and keypoint detectors, including Lowe, MSER and the recently published SFOP detectors. Furthermore, we will draw some interesting conclusions about the complementarity of detectors.
@article{Dickscheid2011Coding,
author = {Timo Dickscheid and Falko Schindler and Wolfgang Förstner},
title = {Coding Images with Local Features},
journal = {International Journal of Computer Vision},
year = {2011},
volume = {94},
pages = {154--174},
url = {http://www.ipb.uni-bonn.de/completeness/},
doi = {10.1007/s11263-010-0340-z}
}
Ribana Roscher and Falko Schindler and Wolfgang Förstner, "What would you look like in Springfield? Linear Transformations between High-Dimensional Spaces" 2011.
High-dimensional data structures occur in many fields of computer vision and machine learning. Transformation between two high-dimensional spaces usually involves the determination of a large amount of parameters and requires much labeled data to be given. There is much interest in reducing dimensionality if a lower-dimensional structure is underlying the data points. We present a procedure to enable the determination of a low-dimensional, projective transformation between two data sets, making use of state-of-the-art dimensional reduction algorithms. We evaluate multiple algorithms during several experiments with different objectives. We demonstrate the use of this procedure for applications like classification and assignments between two given data sets. Our procedure is semi-supervised due to the fact that all labeled and unlabeled points are used for the dimensionality reduction, but only few them have to be labeled. Using test data we evaluate the quantitative and qualitative performance of different algorithms with respect to the classification and assignment task. We show that with these algorithms and our transformation approach high-dimensional data sets can be related to each other. Finally we can use this procedure to match real world facial images with cartoon images from Springfield, home town of the famous Simpsons.
@techreport{Roscher2011What,
author = {Ribana Roscher and Falko Schindler and Wolfgang Förstner},
title = {What would you look like in Springfield? Linear Transformations between High-Dimensional Spaces},
year = {2011}
}
Falko Schindler and Wolfgang Förstner and Jan-Michael Frahm, "Classification and Reconstruction of Surfaces from Point Clouds of Man-made Objects", In IEEE International Conference on Computer Vision, Workshop on Computer Vision for Remote Sensing of the Environment. 2011.
We present a novel surface model and reconstruction method for man-made environments that take prior knowledge about topology and geometry into account. The model favors but is not limited to horizontal and vertical planes that are pairwise orthogonal. The reconstruction method does not require one particular class of sensors, as long as a triangulated point cloud is available. It delivers a complete 3D segmentation, parametrization and classification for both surface regions and inter-plane relations. By working on a pre-segmentation we reduce the computational cost and increase robustness to noise and outliers. All reasoning is statistically motivated, based on a few decision variables with meaningful interpretation in measurement space. We demonstrate our reconstruction method for visual reconstructions and laser range data.
@inproceedings{Schindler2011Classification,
author = {Falko Schindler and Wolfgang Förstner and Jan-Michael Frahm},
title = {Classification and Reconstruction of Surfaces from Point Clouds of Man-made Objects},
booktitle = {IEEE International Conference on Computer Vision, Workshop on Computer Vision for Remote Sensing of the Environment},
year = {2011},
doi = {10.1109/ICCVW.2011.6130251}
}
Falko Schindler and Wolfgang Förstner, "Fast Marching for Robust Surface Segmentation", In Lecture Notes in Computer Science, Photogrammetric Image Analysis. 2011. (Best Paper)
We propose a surface segmentation method based on Fast Marching Farthest Point Sampling designed for noisy, visually reconstructed point clouds or laser range data. Adjusting the distance metric between neighboring vertices we obtain robust, edge-preserving segmentations based on local curvature. We formulate a cost function given a segmentation in terms of a description length to be minimized. An incremental-decremental segmentation procedure approximates a global optimum of the cost function and prevents from under- as well as strong over-segmentation. We demonstrate the proposed method on various synthetic and real-world data sets.
@inproceedings{Schindler2011Fast,
author = {Falko Schindler and Wolfgang Förstner},
title = {Fast Marching for Robust Surface Segmentation},
booktitle = {Lecture Notes in Computer Science, Photogrammetric Image Analysis},
year = {2011},
url = {http://www.ipb.uni-bonn.de/projects/fastfps/},
doi = {10.1007/978-3-642-24393-6_13}
}
Johannes Schneider and Falko Schindler and Wolfgang Förstner, "Bündelausgleichung für Multikamerasysteme", In Proceedings of the 31th DGPF Conference. 2011.
Wir stellen einen Ansatz für eine strenge Bündelausgleichung für Multikamerasysteme vor. Hierzu verwenden wir eine minimale Repräsentation von homogenen Koordinatenvektoren für eine Maximum-Likelihood-Schätzung. Statt den Skalierungsfaktor von homogenen Vektoren durch Verwendung von euklidischen Grössen zu eliminieren, werden die homogenen Koordinaten sphärisch normiert, so dass Bild- und Objektpunkte im Unendlichen repräsentierbar bleiben. Dies ermöglicht auch Bilder omnidirektionaler Kameras mit Einzelblickpunkt, wie Fisheyekameras, und weit entfernte bzw. unendlich ferne Punkte zu behandeln. Speziell Punkte am Horizont können über lange Zeiträume beobachtet werden und liefern somit eine stabile Richtungsinformation. Wir demonstrieren die praktische Umsetzung des Ansatzes anhand einer Bildfolge mit dem Multikamerasystem "Ladybug3" von Point Grey, welches mit sechs Kameras 80~% der gesamten Sphäre abbildet.
@inproceedings{Schneider2011Bundelausgleichung,
author = {Johannes Schneider and Falko Schindler and Wolfgang Förstner},
title = {Bündelausgleichung für Multikamerasysteme},
booktitle = {Proceedings of the 31th DGPF Conference},
year = {2011}
}
2010
Ribana Roscher and Falko Schindler and Wolfgang Förstner, "High Dimensional Correspondences from Low Dimensional Manifolds -- An Empirical Comparison of Graph-based Dimensionality Reduction Algorithms", In The 3rd International Workshop on Subspace Methods, in conjunction with ACCV2010. 2010.
We discuss the utility of dimensionality reduction algorithms to put data points in high dimensional spaces into correspondence by learning a transformation between assigned data points on a lower dimensional structure. We assume that similar high dimensional feature spaces are characterized by a similar underlying low dimensional structure. To enable the determination of an affine transformation between two data sets we make use of well-known dimensional reduction algorithms. We demonstrate this procedure for applications like classification and assignments between two given data sets and evaluate six well-known algorithms during several experiments with different objectives. We show that with these algorithms and our transformation approach high dimensional data sets can be related to each other. We also show that linear methods turn out to be more suitable for assignment tasks, whereas graph-based methods appear to be superior for classification tasks.
@inproceedings{Roscher2010High,
author = {Ribana Roscher and Falko Schindler and Wolfgang Förstner},
title = {High Dimensional Correspondences from Low Dimensional Manifolds -- An Empirical Comparison of Graph-based Dimensionality Reduction Algorithms},
booktitle = {The 3rd International Workshop on Subspace Methods, in conjunction with ACCV2010},
year = {2010},
doi = {10.1007/978-3-642-22819-3_34}
}
2009
Wolfgang Förstner and Timo Dickscheid Falko Schindler, "On the Completeness of Coding with Image Features", In 20th British Machine Vision Conference. 2009.
We present a scheme for measuring completeness of local feature extraction in terms of image coding. Completeness is here considered as good coverage of relevant image information by the features. As each feature requires a certain number of bits which are representative for a certain subregion of the image, we interpret the coverage as a sparse coding scheme. The measure is therefore based on a comparison of two densities over the image domain: An entropy density pH(x) based on local image statistics, and a feature coding density pc(x) which is directly computed from each particular set of local features. Motivated by the coding scheme in JPEG, the entropy distribution is derived from the power spectrum of local patches around each pixel position in a statistically sound manner. As the total number of bits for coding the image and for representing it with local features may be different, we measure incompleteness by the Hellinger distance between p_H(x) and p_c(x). We will derive a procedure for measuring incompleteness of possibly mixed sets of local features and show results on standard datasets using some of the most popular region and keypoint detectors, including Lowe, MSER and the recently published SFOP detectors. Furthermore, we will draw some interesting conclusions about the complementarity of detectors.
@inproceedings{Forstner2009Completeness,
author = {Wolfgang Förstner and Timo Dickscheid Falko Schindler},
title = {On the Completeness of Coding with Image Features},
booktitle = {20th British Machine Vision Conference},
year = {2009}
}
Wolfgang Förstner and Timo Dickscheid and Falko Schindler, "Detecting Interpretable and Accurate Scale-Invariant Keypoints", In 12th IEEE International Conference on Computer Vision. 2009.
This paper presents a novel method for detecting scale invariant keypoints. It fills a gap in the set of available methods, as it proposes a scale-selection mechanism for junction-type features. The method is a scale-space extension of the detector proposed by Förstner (1994) and uses the general spiral feature model of Bigün (1990) to unify different types of features within the same framework. By locally optimising the consistency of image regions with respect to the spiral model, we are able to detect and classify image structures with complementary properties over scalespace, especially star and circular shapes as interpretable and identifiable subclasses. Our motivation comes from calibrating images of structured scenes with poor texture, where blob detectors alone cannot find sufficiently many keypoints, while existing corner detectors fail due to the lack of scale invariance. The procedure can be controlled by semantically clear parameters. One obtains a set of keypoints with position, scale, type and consistency measure. We characterise the detector and show results on common benchmarks. It competes in repeatability with the Lowe detector, but finds more stable keypoints in poorly textured areas, and shows comparable or higher accuracy than other recent detectors. This makes it useful for both object recognition and camera calibration.
@inproceedings{Forstner2009Detecting,
author = {Wolfgang Förstner and Timo Dickscheid and Falko Schindler},
title = {Detecting Interpretable and Accurate Scale-Invariant Keypoints},
booktitle = {12th IEEE International Conference on Computer Vision},
year = {2009},
url = {http://www.ipb.uni-bonn.de/sfop/},
doi = {10.1109/ICCV.2009.5459458}
}
