1. Extraction of Speech Shots Focusing on Visual and Audio Features within and between Shots
- Author
-
KUMAGAI, Shogo, DOMAN, Keisuke, TAKAHASHI, Tomokazu, DEGUCHI, Daisuke, IDE, Ichiro, and MURASE, Hiroshi
- Subjects
audio−visual features ,映像検索 ,ニュース映像 ,スピーチショット抽出 ,Speech shot extraction ,news video ,video retrieval ,画像・音声特徴 - Abstract
本報告では,ショット内及びショット間の特徴に基づく被写体と話者の異同判定によるニュース映像からのスピーチショット抽出手法を提案する.スピーチショットはマルチメディア情報を豊富に含み,資料的価値が高い.そこで我々はこれまで,被写体の口唇動作と話者の声から得られる複数の音声特徴と画像特徴の相関に基づく被写体と話者の異同判定手法を提案してきた.この手法は,音声ノイズの少ないショットに対しては高精度な異同判定が可能であるが,多量の音声ノイズを含むショットに対しての異同判定は困難であった.そこで本報告では,2段階の処理による被写体と話者の異同判定手法を提案する.まず第1段階で,これまでに提案した手法により異同判定を行う.その後,第2段階で,ショット内及びその前後のショットとの間に表れる特徴的な画像・音声の性質に基づいて異同判定を行う.スピーチショット抽出実験の結果,提案手法の有効性を確認した., We propose a method to extract speech shots from news videos using detecting the inconsisteny between a subject and the speaker focusing on features within and between shots. Speech shots in news videos contain a wealth of multimedia information, and are valuable as archived material. To extract speech shots, we have previously proposed a method to detect the inconsistency between a subject and the speaker based on the co-occurrence between a subject's lip motion and the speaker's voice. This previous method could detect the inconsistency in a shot with little audio noises. However, it is difficult to detect the inconsistency in a shot with significant amount of audio noises. In order to deal with this problem, the proposed method detects the inconsisteny between a subject and the speaker in two steps. The first step detects the inconsistency by our previous method, and the second step detects the inconsistency based on the intra- and inter- shot features. Experimental results showed the effectiveness of the proposed method., IEICE Technical Report;IE2011-147,IEICE Technical Report;MVE2011-109
- Published
- 2012