Search

Human-machine interactive systems show increasing demand for analysing fast moving objects in high-frame-rate videos. Robust foreground detection, which is able to reduce large amount of redundant background data from high-frame-rate video, becomes the essence to achieve ultra-high-speed human-machine interactions. This paper proposes a local spatial propagation based background model generation, a local linear illumination correction based background model update, and a regional central coordinates and edge keypoints constrained foreground region reselection. The three proposals make up a robust and hardware-friendly foreground detection method. Experimental results prove that the proposed hardware-friendly algorithm achieves high accuracy and robustness on various kinds of challenging cases. Meanwhile, the hardware implementation utilizes little hardware resources and achieves realtime processing of high-frame-rate (784 frame/second) video with the delay less than 1 ms/frame in image processing core. In addition, a practical system is implemented by combing a PC, a high-speed camera and a field programmable gate array (FPGA) for realworld applications. This work will significatively promote the development and application of high-speed human machine interaction. A demo of the proposed vision system working at 784 FPS is available at https://wcms.waseda.jp/em/5f84f75136a6. Note to Practitioners—This paper was motivated by the problem of high-frame-rate video contains large amount of redundant background pixels which makes ultra-high-speed human-machine interactions inaccessible. Existing approaches are mainly focused on designing complex background models, but processing speed, which is the most important issue for ultra-high-speed human-machine interactions, has received relatively little attention. This paper suggests a robust and hardware-friendly foreground detection algorithm which has been implemented as a hardware system by using an FPGA, a high-frame-rate camera, and a PC. We show that the hardware implementation utilizes less hardware resources and achieves real-time processing speed of 784 FPS with the delay less than 1 ms/frame in the image processing core. This work is a pioneering attempt of ultra-high-speed foreground detection, which will significatively speed up the wide applications of ultra-high-speed human machine interactions. [ABSTRACT FROM AUTHOR]

3D human pose estimation has many important applications in human-computer interaction and human action recognition. Simultaneously achieving real-time speed, varying human number, and high accuracy from a single RGB image is a challenging problem. To this end, this paper proposes a multi-task and multi-level neural network structure with physical constraint. The unique network structure estimates 3D human poses from single RGB image in an end-to-end way and achieves both high accuracy and high speed. Experimental results shows that the proposed system achieves 21 fps on RTX 2080 GPU with only 33 mm accuracy loss compared with conventional works. The mechanism of the network is also analyzed through network visualization. This work shows the possibility of estimating 3D human pose from a single RGB monocular camera with real-time speed. [ABSTRACT FROM AUTHOR]