Search

This paper presents a block-based embedded convolutional neural network (CNN) for gesture classification on field-programmable gate array (FPGA) in real time. Gesture recognition is an important tool to spontaneous interact with human machine interface. Many CNN architectures using RGB images have been proposed for gesture classification. RGB based gesture classification may cause incorrect results under insufficient light or similar gestures. In addition, most of the CNN architectures cannot run in real time on edge devices due to their large number of parameters and DRAM data access. In this paper, a block-based CNN using RGB-D data is proposed for gesture classification. Adding depth images to RGB images boots the classification accuracy. A CNN architecture with block-based feature maps is built for embedded FPGA implementations. The total number of parameters of the proposed RGB-D embedded CNN (eCNN) model is only 0.17M and it achieves 99.96% and 99.88% accuracy with 32-bit floating point and 8-bit fixed point implementation for America Sign Language (ASL) data set. The RTL simulation of the proposed eCNN model has the average inference speed of 0.171 milliseconds at frequency of 250MHz for a single pair RGB-D image. Implemented on a FPGA integrated with Microsoft Kinect v2 achieve an inference time in 19.42 ms which achieves high accuracy and real-time performance.

Object detection technology has received increasing research attention with recent developments in automation technology. Most studies in this field, however, use RGB images as input to deep-learning classifiers, and they rarely use depth information.So, in this paper, we use images with both RGB and depth information as input to an object detection network. We base our network on the Faster R-CNN proposed by Shih et al., and we develop a fast and accurate object detection architecture. In addition to adding depth as input, we also adjust the type of anchor boxes to improve performance on some objects. We also discuss the impact of pooling training data with multiple region proposal networks (RPN) and regions of interest (ROI).Adding depth information improved the mAP by 8.15%, from 36.86% to 45.01%, when using the SUN RGB-D dataset with 10 classes. Optimizing the anchor boxes improved the mAP from 45.01% to 45.88%. After testing various architectures with different reduced RPNs, we find that the model of 1RRPN-2ROIP performs best. The running time is 0.123 s, which is 1.8 times faster than the 3D-SSD model.

According to MD simulation results, pressing depth between two bonding materials will affect bonding strength. Alloy material (Al0.9Cu0.1) had void defect phenomenon in low bonding speed condition because the increasing chance of atom migration which will result in low bonding strength. High tensile speed causes material fracture phenomena happen earlier than low speed. Material stress in low speed is smaller than in high speed. Fracture morphology of material is different in different tensile speed. In low speed condition, material can be stretched thinner than in high speed condition. Material in high temperature has greater kinetic energy than low temperature; therefore, material in high temperature has better formability and behaves larger tensile strain than low temperature. For pure aluminum, when temperature raises to 900K which is close to melting point (933K), its crystal structure is no longer belongs to F.C.C. structure, so bonding strength is weaker than low temperature. Large size material has larger contact area than small size material; therefore, the tensile force and tensile strength of the former are larger than the latter. The order of bonding strength for these three materials is: binary alloy > pure copper > pure aluminum.