Design of an action recognition system based on SoC FPGA and CNN model

Action recognition is an important area in computer vision. An action recognition system based on SoC FPGA and CNN model is presented. A pipelined LK optical flow module and a pedestrian detection module based on HOG and SVM are realized. The results of detection of dynamic pedestrian and static pedestrian are merged to recognize the area where the person is. The image and optical flow in the area are both input into the CNN model, to recognize what action it is. An ISA-NPU is proposed to realize the computation of the CNN model. The system is realized in Altera DE10-Nano Kit. With the co-operation of hardware and software, the system can recognize actions such as stand, walk, wave and squat. The system is accurate in recognition and flexible for expansion and modification of the CNN model.