An evaluation framework for benchmarking indoor modelling methods

Despite recent progress in the development of methods for automated reconstruction of indoor models, a comparative performance evaluation of these methods is not available due to the lack of publicly available benchmark datasets and a common evaluation framework. The ISPRS Benchmark on Indoor Modelling is an effort to enable comparison and benchmarking of indoor modelling methods by providing a benchmark dataset and a comprehensive evaluation framework. In this paper, we propose a framework for the evaluation of indoor modelling methods, and discuss various quality aspects of the reconstruction methods as well as the reconstructed models. We discuss the challenges in quantitative quality evaluation of indoor models through comparison with a reference model, and propose suitable measures and methods for comparing an automatically reconstructed indoor model with a reference.