Highlight detection and indexing in broadcast sports video by collaborative processing of text, audio, and image.

In this paper we propose a highlight detection method and an indexing method for broadcast sports video using the collaborative processing of text, audio, and image. In the proposed method, the appearance pattern of words in the closed caption text stream is analyzed, and candidate intervals for highlights are detected. Next, these intervals are checked based on their audio levels, and those which seem to be erroneous are rejected. Finally, the resulting highlight intervals are segmented into shots, and the shots are indexed by identifying the highlight shots based on audio levels and dominant color information. In the results of using this method on a real football broadcast, highlight intervals were effectively detected with a recall rate of 77% and a precision rate of 84%. Moreover, for the intervals in which highlights were correctly detected, shot indexing was performed accurately 75% for only the first candidate, and 97% for up to the second candidate. We verified experimentally that efficient processing can be achieved by stepwise analysis of the text, audio, and image. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(12): 22–31, 2003; Published online in Wiley InterScience (<URL>www.interscience.wiley.com</URL>). DOI 10.1002/scj.10493 [ABSTRACT FROM AUTHOR]