Component Identification and Interpretation A Perspective on Tower of Knowledge

This chapter addresses the problem of identifying and interpreting the components (e.g., balconies and windows) of the 3D model of a building. First, a voting scheme is presented for solving the problem of component identification in the 3D model. It is intuitive that interferences, such as occlusions, rarely happen at the same place nor at different times, when a person looks at a scene from different directions. In the spirit of this intuition, the voting scheme combines the information from various multiple view images to identify and segment the components of a building. For the component identification task, we use (from 3 to 11 views per building) multiple view images with short baselines in our experiments. Here, a priori 3D building model with a set of perpendicular and rectangular planes is set up for the identification task. The experimental results show the effectiveness of our scheme in identifying the components of 3D models of several buildings. With the identified components, we can proceed to the interpreting stage using the proposed tower of knowledge (ToK) approach, which automatically labels 3D components of buildings. Specifically, ToK is designed for discovering and encoding the logic rules (such as functionalities) for labeling components of the 3D model of a building. Then, we show how to make decisions on labeling components using ToK and utility theory. In order to deal with the case of lacking training data for making such decisions, we introduce a recursive version of ToK. Finally, a prototype of labeling components of building scenes is employed for validating the proposed ToK approach.