Search

This paper proposes an AI‐based multimodal architecture for anomaly detection in a video footage of a smart carrier which is used for monitoring alignment status, grinding of rollers and vibration of a carrier for manufacturing line of OLED (Organic Light Emitting Diodes) panels. The proposed multimodal architecture has a two‐stream structure. The first stream extracts a candidate anomaly region in image frames sampled from the video by DL‐based segmentation. Thereafter, the candidate region is determined whether it is a true failure or not by comparing the region with a reference image which was previously obtained in a normal setup status. The second stream calculates a dense optical flow with sparse areas using consecutive video frames. By observing change between (n)th, reference, (n‐3)th and (n‐7)th images, it is determined with sound analysis whether the smart carrier experiences vibration during movement. The overall detection rate of three anomalies is 97%, and processing takes only 40% of total video time.