Learning interactions among objects through spatio-temporal reasoning

In this study, we propose a method for learning interactions among different types of objects to devise new plans using these objects. Learning is accomplished by observing a given sequence of events with their timestamps and using spatial information on the initial state of the objects in the environment. We assume that no intermediate state information is available about the states of objects. We have used the Incredible Machine game as a suitable domain for analyzing and learning object interactions. When a knowledge base about relations among objects is provided, interactions to devise new plans are learned to a desired extent. Moreover, using spatial information of objects or temporal information of events makes it feasible to learn the conditional effects of objects on each other. Our analyses show that, integrating spatial and temporal data in a spatio-temporal learning approach gives closer results to that of the knowledge-based approach by providing applicable event models for planning. This is promising because gathering spatio-temporal information does not require great amount of knowledge.