Object-based video indexing for virtual studio productions

This paper introduces an object-based approach for temporal video partitioning and content-based indexing, where the basic indexing unit is `lifespan of a video object,' rather than a `camera shot' or `story unit.' We propose a system to extract content-based features of video objects (VOs), based on a compact 2D triangular mesh representation of them. An adaptive mesh-based video object tracking scheme is then employed to compute the motion trajectories of all node points. A set of `key snapshots' which constitute a visual summary of the lifespan of the object are automatically selected using motion and shape information. The system provides direct access to the VOs and gives the functionalities such as object-based search, manipulation, animation, and tracking.