Search

Deep learning methods, especially CNNs, have achieved many promising results in a wide range of computer vision applications. However, few studies focused on designing suitable deep learning methods for parking space status inference. As we have known, it is challenging to detect parking spaces in an outdoor environment due to dynamic lighting variations, weather changes, and perspective distortion. By off-the-shelf CNNs, lighting variations might be handled well. However, to realize a practical and robust inference system, we also need to address troublesome problems, such as parking displacements, non-unified car sizes, inter-object occlusion, and perspective distortion. These problems may become even challenging if also considering the difference of space sizes. To overcome the problems, we proposed a custom–tailored deep convolutional and contrastive network with three contributions. First, we introduced a Siamese architecture to learn the contrastive and robust feature descriptor. This helps to reduce the effects owing to the variety of inter-object occlusion. Second, we integrated a convolutional Spatial Transformer Network (STN) to adaptively transform a 3-space input patch according to vehicle sizes and parking displacement. STN also helps to overcome the perspective distortion problem. Third, a multi-task loss function was designed to train the network by simultaneously considering the accuracy of inferring the status of the target space and the semantic smoothness of high-level features. Thereby, the errors caused by inter-object occlusion could be alleviated. To verify the proposed network, we visualized the learned features and analyzed their functionality. Experiments and evaluations have shown the robustness of our system in parking status inference. The real-time system currently running in public parking lots also demonstrates the effectiveness of the proposed deep network. [ABSTRACT FROM AUTHOR]

Emotion recognition is challenging due to the emotional gap between emotions and audio–visual features. Motivated by the powerful feature learning ability of deep neural networks, this paper proposes to bridge the emotional gap by using a hybrid deep model, which first produces audio–visual segment features with Convolutional Neural Networks (CNNs) and 3D-CNN, then fuses audio–visual segment features in a Deep Belief Networks (DBNs). The proposed method is trained in two stages. First, CNN and 3D-CNN models pre-trained on corresponding large-scale image and video classification tasks are fine-tuned on emotion recognition tasks to learn audio and visual segment features, respectively. Second, the outputs of CNN and 3D-CNN models are combined into a fusion network built with a DBN model. The fusion network is trained to jointly learn a discriminative audio–visual segment feature representation. After average-pooling segment features learned by DBN to form a fixed-length global video feature, a linear Support Vector Machine is used for video emotion classification. Experimental results on three public audio–visual emotional databases, including the acted RML database, the acted eNTERFACE05 database, and the spontaneous BAUM-1s database, demonstrate the promising performance of the proposed method. To the best of our knowledge, this is an early work fusing audio and visual cues with CNN, 3D-CNN, and DBN for audio–visual emotion recognition. [ABSTRACT FROM AUTHOR]