On Single-Sequence and Multi-Sequence Factorizations

Abstract

Subspace based factorization methods are commonly used for a variety of applications, such as 3D reconstruction, multi-body segmentation and optical flow estimation. These are usually applied to a single video sequence. In this paper we present an analysis of the multi-sequence case and place it under a single framework with the single sequence case. In particular, we start by analyzing the characteristics of subspace based spatial and temporal segmentation. We show that in many cases objects moving with different 3D motions will be captured as a single object using multi-body (spatial) factorization approaches. Similarly, frames viewing different shapes might be grouped as displaying the same shape in the temporal factorization framework. Temporal factorization provides temporal grouping of frames by employing a subspace based approach to capture non-rigid shape changes (Zelnik-Manor and Irani, 2004). We analyze what causes these degeneracies and show that in the case of multiple sequences these can be made useful and provide information for both temporal synchronization of sequences and spatial matching of points across sequences.