The paper presents a new analysis method for effectively recognizing story structures of video programs. First

video stream are decomposed into sequences of video pages

and key frames are also extracted to represent video pages. Techniques and formulations are then proposed to match and cluster video pages of similar visual contents

taking into account the visual features and temporal arrangement of clustering elements. In addition

we use the Sequence Structure Graph representation to show the story development clue—story line extracted from video. The proposed analyses lead to automatic segmentation of story units and the building of a compact representation of video contents. Hence we are able to decompose video into compact representations that reflect the flow of stories. This offers an efficient mean for browsing and non-linear access of video. Experimental results demonstrate the effectiveness of the new analysis method.