Video2Entities: A computer vision-based entity extraction framework for updating the architecture, engineering and construction industry knowledge graphs.

Due to the decentralisation and complexity of knowledge in the architecture, engineering and construction (AEC) industry, the research on knowledge graphs (KGs) is still insufficient, and most of the research focuses on text-based KG structuring or updating. Entity extraction, a sub-task of knowledge extraction, is critical in general KG update approaches. While the mainstream approach for this task generally uses textual data, visual data is more readily available, more accurate and has a shorter update cycle than textual data. Therefore, this paper integrates zero-shot learning techniques with general KGs to present a novel framework called "video2entities" that can extract entities from videos to update the AEC KG. The framework combines the perceptual capabilities of computer vision with the cognitive capabilities of KG to improve the accuracy and timeliness of KG updates. Experimental results demonstrate that the framework can extract "new entities" from architectural decoration videos for AEC KG updates. • A novel video-based framework is proposed for AEC Knowledge Graph (KG) updating. • Two iterative processes are developed for the incremental extraction of entities. • Prior information is strengthened by relationship graphs fusing KG and word embedding. • General KG is updated to AEC domain KG by the application of our framework. [ABSTRACT FROM AUTHOR]