Extracting spatial relations among objects for failure detection

A cognitive robot may face failures during the execution of its actions. These failures are mostly due to the gap between the physical world and the constructed symbolic plans, some internal problems that may occur in its embodiment or unexpected external events. In this paper, we propose a visual scene interpretation system for extracting spatial relations among objects in a scene and using these relations to detect failures during the plan execution. Our system uses LINE-MOD and HS histograms in order to recognize textureless objects with different shapes and colors. Then, it analyzes the scene to specify the world state after each action execution. Our focus in this research is on particularly the following spatial relations: on, on table, clear and unstable. In the experiments, we test the performance of our system on recognizing objects, determining pairwise spatial relations among them, and detecting failures using these relations. Our preliminary results reveal that our system can be successfully used to extract spatial relations in a scene, and to determine failures during plan execution by using this information.