Search

Accurate objects detection in remote sensing images is very important, because security, transportation, and rescue applications in military and civilian fields require fully analyzing and using these images. To address the problem that many small-sized objects in remote sensing images are difficult to detect, this paper proposes an improved S 2 ANET-SR model based on S 2 A-NET network. In this paper, the original and reduced image are fed to the detection network at the same time, and then a super-resolution enhancement module for the reduced image is designed to enhance the feature extraction of small objects, after that, the perceptual loss and texture matching loss is proposed as supervision. Extensional experiments are conducted to evaluate the performance on the general remote sensing dataset DOTA, and the results show that our proposed method can achieve 74.47% mAP, which is 0.79% better than the accuracy of S 2 A-NET.

Missing data imputation plays an important role in the data cleansing process. Clustering algorithms have been widely used for missing data imputation, yet, there is little research done on the use of clustering ensemble for missing data imputation, which aggregates multiple clustering results. This paper proposes a novel collaborative clustering-based imputation method, called COLI, which uses the imputation quality as a key criterion for the exchange of information between different clustering results. To the best of our knowledge, this is the first study on the impact of collaborative clustering on imputation performance. The main contributions of this paper are three-fold. A novel missing value imputation based on collaborative clustering is proposed, three amputation strategies are used to induce missingness on various complete and publicly available datasets with different mechanisms, distributions, and ratios, which allows evaluating the imputation quality of the proposed method in estimating missing values of various numerical datasets with different missingness mechanisms, distributions, and ratios. The proposed method is compared to several state-of-the-art imputation methods and attained results demonstrate that the proposed method is an effective method for handling missing data.

Automatic fingerprint identification is a key technology of biological attribute measurement. Due to its sensitivity to fake fingerprint attack, its security has been widely concerned. The fingermark is an important evidence for forensic documents identification. The existence of fake fingermark seriously threatens the fairness and legitimacy in the process of forensic identification. In this paper, in view of the security risks existing in the fingermark identification of forensic documents, a database named JLW-FM-DB is introduced for detecting genuine and fake fingermarks, which consists of two sub databases, signed and unsigned, each of which covers common fake fingermark materials. Based on the database, this paper proposes a method of fingermark anti-counterfeiting based on convolution neural network(CNN). A Patch-Label training strategy is proposed, which uses the unified label as the supervision signal for the class heatmap output by the last convolution layer. This strategy realizes stronger local supervision ability to input fingermark image and enhances local coding ability of CNN feature extraction. The experiments show that our methods are suitable for the detection of genuine and fake fingermarks, achieving 97.999% average accuracy in the signed fingermark database. The effectiveness of Patch-Label training strategy in fingermark anti-counterfeiting is also proved. Moreover, a model fusion of different models can further improve the detection ability of genuine and fake fingermarks, reaching to 98.496% average accuracy.

Recently, deep learning models have achieved great success in computer vision applications, relying on large-scale class-balanced datasets. However, imbalanced class distributions still limit the wide applicability of these models due to degradation in performance. To solve this problem, in this paper, we concentrate on the study of cross entropy which mostly ignores output scores on incorrect classes. This work discovers that neutralizing predicted probabilities on incorrect classes improves the prediction accuracy for imbalanced image classification. This paper proposes a simple but effective loss named complement cross entropy based on this finding. The proposed loss makes the ground truth class overwhelm the other classes in terms of softmax probability, by neutralizing probabilities of incorrect classes, without additional training procedures. Along with it, this loss facilitates the models to learn key information especially from samples on minority classes. It ensures more accurate and robust classification results on imbalanced distributions. Extensive experiments on imbalanced datasets demonstrate the effectiveness of the proposed method., 8 pages, Accepted to Pattern Recognition Letters (PRL), August 2021

This paper presents a new model for the task of scene text visual question answering, in which questions about a given image can only be answered by reading and understanding scene text that is present in it. The proposed model is based on an attention mechanism that attends to multi-modal features conditioned to the question, allowing it to reason jointly about the textual and visual modalities in the scene. The output weights of this attention module over the grid of multi-modal spatial features are interpreted as the probability that a certain spatial location of the image contains the answer text the to the given question. Our experiments demonstrate competitive performance in two standard datasets. Furthermore, this paper provides a novel analysis of the ST-VQA dataset based on a human performance study., Comment: This paper is under consideration at Pattern Recognition Letters

In this paper, we tackle the task of improving biometric verification in the context of Human-Robot Interaction (HRI). A robot that wants to identify a specific person to provide a service can do so by either image verification or, if light conditions are not favourable, through voice verification. In our approach, we will take advantage of the possibility a robot has of recovering further data until it is sure of the identity of the person. The key contribution is that we select from both image and audio signals the parts that are of higher confidence. For images we use a system that looks at the face of each person and selects frames in which the confidence is high while keeping those frames separate in time to avoid using very similar facial appearance. For audio our approach tries to find the parts of the signal that contain a person talking, avoiding those in which noise is present by segmenting the signal. Once the parts of interest are found, each input is described with an independent deep learning architecture that obtains a descriptor for each kind of input (face/voice). We also present in this paper fusion methods that improve performance by combining the features from both face and voice, results to validate this are shown for each independent input and for the fusion methods.

Images with visual and scene text content are ubiquitous in everyday life. However, current image interpretation systems are mostly limited to using only the visual features, neglecting to leverage the scene text content. In this paper, we propose to jointly use scene text and visual channels for robust semantic interpretation of images. We do not only extract and encode visual and scene text cues, but also model their interplay to generate a contextual joint embedding with richer semantics. The contextual embedding thus generated is applied to retrieval and classification tasks on multimedia images, with scene text content, to demonstrate its effectiveness. In the retrieval framework, we augment our learned text-visual semantic representation with scene text cues, to mitigate vocabulary misses that may have occurred during the semantic embedding. To deal with irrelevant or erroneous recognition of scene text, we also apply query-based attention to our text channel. We show how the multi-channel approach, involving visual semantics and scene text, improves upon state of the art., Comment: The paper is under consideration at Pattern Recognition Letters

Facial emotion recognition (FER) methods have been proposed mainly using 2D images. These methods suffer from many problems caused by the difficult conditions of unconstrained environments such as light conditions and view variations. In this paper, we aim to recognize the emotional facial expressions independently of their identity using the 3D data and 2D depth images. Since the 3D FER is a very fine-grained recognition task, mapping the 3D images into 2D depth images may lack some geometric characteristics of the expressive face and decay the FER performance. Convolutional Neural Networks (CNN), however, have been successfully applied to the 2D depth images and improved handcrafted-based methods in computer vision and pattern recognition applications. For this reason, we combine in this paper two types of features; handcrafted and deep learning features and prove their complementarity for 3D FER. Favorably, covariance descriptors have proven a very good ability to combine features from different types into a compact representation. Therefore, we propose to use the covariance matrices of features (handcrafted and deep ones), instead of the features independently. Since covariance matrices belong to one of the manifold space types, formed by SPD (Symmetric Positive Definite) matrices, we mainly focus on the generalization of the RBF kernel to the manifold space for 3D FER using a supervised SVM classification. The achieved performance of the proposed method on the Bosphorus and BU-3DFE datasets outperforms similar state-of-the-arts.

The existence of cracks will affect the stability of the building. It is very important to identify and deal with the cracks in time to ensure the safety and stability of the building. Based on the above background, the purpose of this paper is to study the method of building crack recognition and total quality management based on deep learning. This paper focuses on the computer vision technology in artificial intelligence, studies the image classification algorithm and semantic segmentation algorithm based on the deep learning method, and applies it to the field of building crack image analysis. In this paper, we use the deep convolution neural network to design the building image crack classification model and segmentation model, realize the identification and analysis of building cracks, and build a building crack analysis system, which can significantly improve the efficiency of building crack detection. Then, based on the image processing technology, the quantitative analysis of the fracture segmentation results is carried out. Through the basic morphological methods such as corrosion, expansion, opening and closing operations, the segmentation mark map, skeleton map and geometric parameter information of the fracture are obtained, which further provides the maintenance and judgment basis for professional engineers. The experimental results show that compared with FCN, the accuracy of rfcn-a is improved by 5.98%, the precision is improved by 6.07%, and the real and f'score are improved by 3.11% and 6.01%, respectively.

With the continuous development of artificial intelligence, machine learning, the necessary way to achieve artificial intelligence, is also constantly improving, of which deep learning is one of the contents. The purpose of this paper is to evaluate and warn the security risk of large-scale group activities based on the random forest algorithm. This paper uses the methods of calculating the importance of the random forest algorithm to variables and the calculation formula of the weight of the security risk index, and combining the model parameters of the random forest algorithm The optimization experiment and the random forest model training experiment are used for risk analysis, and the classification accuracy rate reaches a maximum of 0.86, which leads to the conclusion that the random forest algorithm has good predictive ability in the risk assessment of large-scale group activities. This article takes a certain international youth environmental protection festival as an example for analysis, and better verifies the feasibility and effectiveness of this article.

With the emergence of the concept of “safe city”, security construction has gradually been valued by various cities, and video surveillance technology has also been continuously developed and applied. However, as the functional requirements of actual applications become more and more diverse, video surveillance systems also need to be more intelligent. The purpose of this article is to study methods of brute force detection and face recognition based on deep learning. Aiming at the problem of abnormal behavior detection, especially the low efficiency and low accuracy of brute force detection, a brute force detection method based on the combination of convolutional neural network and trajectory is proposed. This method uses artificial features and depth features to extract the spatiotemporal features of the video through a convolutional neural network and combines them with the trajectory features. In view of the problem that face images in surveillance video cannot be accurately recognized due to low resolution, two models are proposed: the multi-foot input CNN model and the SPP-based CNN model. By testing the performance of the brute force detection method proposed in this paper, the accuracy of the method on the Crow and Hockey datasets is as high as 92% and 97.6%, respectively. Experimental results show that the violence detection method proposed in this paper improves the accuracy of violence detection in video.

In recent years, high spatial resolution remote sensing technology has made significant progress. High-resolution remote sensing satellites provide great convenience for high-quality image acquisition. In order to adapt to changes in the appearance of the target, mainstream tracking algorithms often use pattern recognition methods to build a target appearance model with learning capabilities, and use the image frames acquired during the tracking process to update the appearance model. This paper mainly studies the object-oriented remote sensing image information extraction method based on multi-classifier combination and deep learning algorithm. In this paper, we use the splitting mechanism of the tree structure to retain the appearance model with diversity, and through the integrated learning integration strategy, the target position is collaboratively predicted. Through the comparative analysis on the OTB and VOT platforms, the algorithm works well when the requirements of the tracking standards are low (the accuracy threshold is greater than 20 pixels and the success threshold is less than 0.4 pixels). The experimental results in this paper show that compared with other advanced classification methods, the proposed method shows better generalization performance in accuracy, recall, f-measure, g-mean and AUC.

Image classification is a hot research topic in today's society and an important direction in the field of image processing research. SVM is a very powerful classification model in machine learning. CNN is a type of feedforward neural network that includes convolution calculation and has a deep structure. It is one of the representative algorithms of deep learning. Taking SVM and CNN as examples, this paper compares and analyzes the traditional machine learning and deep learning image classification algorithms. This study found that when using a large sample mnist dataset, the accuracy of SVM is 0.88 and the accuracy of CNN is 0.98; when using a small sample COREL1000 dataset, the accuracy of SVM is 0.86 and the accuracy of CNN is 0.83. The experimental results in this paper show that traditional machine learning has a better solution effect on small sample data sets, and deep learning framework has higher recognition accuracy on large sample data sets.

In recent years, computer vision, especially deep learning, has been widely used in various fields. Through the deep learning aerial image detection gradient clustering algorithm automatic recognition, it can solve the limitations of manual shooting by humans, can shoot from a high altitude to a panoramic view of a specific area, and provide a more comprehensive solution. The traditional forest resource management and management work is mainly carried out by forestry personnel to carry out a large number of investigations and investigations on the forest. This method not only consumes a lot of manpower and material resources, but also does not have real-time nature. It is difficult to deal with all kinds of forest management. Problems, causing unnecessary losses. In this regard, this paper proposes an aerial image change detection algorithm based on H-KFCM, and designs related experiments to verify and demonstrate the performance of the algorithm. In this paper, we conduct a parallel study based on deep learning on the gradient clustering algorithm of deep learning in aerial image processing. By using CUDA (Compute Unified Device Architecture) to perform large-scale parallel processing of aerial data. Can greatly shorten the time to obtain results, improve the efficiency of relevant personnel. Experiment analysis. It can be seen from the results that the deep learning parallelization program implemented in this paper has a faster calculation speed and uses less time in high-resolution images, and has a good acceleration ratio compared to the CPU.

In this paper, we investigate 3D hand pose estimation using single depth images. On the one hand, accurate hand localization is a crucial factor for pose estimation. On the other hand, multi-task learning methods have achieved great success in visual recognition tasks. Therefore, in this paper we proposed to simultaneously detect the hand location and estimate its 3D pose in a multi-task learning framework. We used 3D region proposal for 3D pose estimation, which searches possible hand locations in the 3D space. In the experimental part, the proposed method is evaluated on several benchmark datasets and shown it is comparable to most existing 3D hand pose estimation methods.

For the problems of inconsistent sample duration and unbalance of sample categories in the speech emotion corpus, this paper proposes a speech emotion recognition model based on Bi-GRU (Bidirection Gated Recurrent Unit) and Focal Loss. The model has been improved on the basis of learning CRNN (Convolutional Recurrent Neural Network) deeply. In CRNN, Bi-GRU is used to effectively lengthen the samples of the speech with short duration, and Focal Loss function is used to deal with the difficulties in classification caused by the imbalance of emotional categories of the samples. Through different methods for experimental comparison, weighted average recall (WAR), unweighted average recall (UAR) and confusion matrix (CM) are used as evaluation index of the algorithm. The experimental results show that the speech emotion recognition model proposed in this paper improves the recognition accuracy and the imbalance of IEMOCAP database samples, and can effectively prove that the improvement of speech emotion recognition performance is not due to the adjustment of model parameters or the change of the model topology.

This paper deals with the problem of computing the shortest reducts of a decision system. The shortest reducts are useful for attribute reduction in classification problems and data size reduction. Unfortunately, finding all the shortest reducts is an NP-hard problem. There are some algorithms reported in the literature to overcome the complexity of computing the shortest reducts. However, most of these algorithms relay on costly operations for candidate evaluation. In this paper, we propose a new algorithm for computing all the shortest reducts; based on binary cumulative operations over a pair-wise comparison matrix, and a fast candidate evaluation process. Binary cumulative operations save computation time by avoiding repetitive calculations. Furthermore, unlike other algorithms reported in the literature, our candidate evaluation process relays on low-cost operations which reduce the runtime in most cases. Our experiments over synthetic and real-world decision systems show that our proposal is faster than state of the art algorithms in most decision systems.

In orthodontics, a patient is collected a lot of 3D oral cavity data, including oral cavity gypsum and scan data sets. To accurately measure the patient's tooth movement, this paper proposes a robust and precise isotropic scaling registration algorithm using bi-directional distance and correntropy. Firstly, because the oral cavity gypsum data sets have a lot of gypsum tumors and bubbles, which can cause the accuracy of registration results to decrease. Then, we introduce the correntropy into the traditional scaling registration model. Secondly, since unconstrained scaling registration is an ill-posed problem, bi-directional distance is used to enhance the robustness. In this way, a registration model using bi-directional distance and correntropy is established. In order to solve this problem, this paper proposes a new registration algorithm with iterative closest point. Moreover, the convergence of the algorithm is proved theoretically. Finally, the proposed algorithm is tested on the orthodontic database, and our experimental results demonstrate that our algorithm performs robust and high accuracy.

The paper presents Imbalance-XGBoost, a Python package that combines the powerful XGBoost software with weighted and focal losses to tackle binary label-imbalanced classification tasks. Though a small-scale program in terms of size, the package is, to the best of the authors' knowledge, the first of its kind which provides an integrated implementation for the two losses on XGBoost and brings a general-purpose extension on XGBoost for label-imbalanced scenarios. In this paper, the design and usage of the package are described with exemplar code listings, and its convenience to be integrated into Python-driven Machine Learning projects is illustrated. Furthermore, as the first- and second-order derivatives of the loss functions are essential for the implementations, the algebraic derivation is discussed and it can be deemed as a separate algorithmic contribution. The performances of the algorithms implemented in the package are empirically evaluated on Parkinson's disease classification data set, and multiple state-of-the-art performances have been observed. Given the scalable nature of XGBoost, the package has great potentials to be applied to real-life binary classification tasks, which are usually of large-scale and label-imbalanced., Updated version, published in Pattern Recognition Letters 136(2020):190-197

Gliomas are characterized by high morbidity and high mortality in primary tumors. The identification of glioma type is helpful for radiologists to facilitate correct medical judgments and better prognosis for patients. In order to avoid harm to patients caused by a biopsy, radiologists attempt to classify Magnetic Resonance Images(MRI) using deep learning methods. In the present paper, we propose a deep learning convolutional neural network ResNet based on the pyramid dilated convolution for Gliomas classification. The pyramid dilated convolution is integrated into the bottom of Resnet to increase the receptive field of the original network and improve the classification accuracy. After adding the pyramid dilated convolution model, the receptive field of the original network underlying convolution was improved. A clinical dataset is used to test the pyramid dilated convolution ResNet neural network model proposed in this paper. The experimental results demonstrate that the proposed method can effectively improve glioma classification performance.

In recent days, the incidence of Diabetic Retinopathy (DR)has become high, affecting the eyes because of drastic increase in the glucose level in blood. Globally, almost half of the people under the age of 70 gets severely affected by diabetes. In the absence of earlier recognition and proper medication, the DR patients tend to lose their vision. When the warning signs are tracked down, the severity level of the disease has to be validated so to take decisions regarding appropriate treatment further. The current research paper focuses on the concept of classification of DR fundus images on the basis of severity level using a deep learning model. This paper proposes a deep learning-based automated detection and classification model for fundus DR images. The proposed method involves various processes namely preprocessing, segmentation and classification. The methods begins with preprocessing stage in which unnecessary noise that exists in the edges is removed. Next, histogram-based segmentation takes place to extract the useful regions from the image. Then, Synergic Deep Learning (SDL) model was applied to classify the DR fundus images to various severity levels. The justification for the presented SDL model was carried out on Messidor DR dataset. The experimentation results indicated that the presented SDL model offers better classification over the existing models.

Multiple Instance Learning is a weakly supervised learning technique which is particularly well suited for medical data analysis as the class labels are often not available at desired granularity. Multiple Instance Learning through Deep Neural Networks is relatively a new paradigm in machine learning. The most important part of Multiple Instance Learning through Deep Neural Networks is designing a trainable pooling function which determines the instance-to bag relationship. In this paper, we propose a Multiple Instance pooling technique based on Genetic Algorithm called Genetic Pooling. In this technique, instance labels inside a bag are optimized by minimizing bag-level losses. The main contribution of the paper is that the bag level pooling layer for generating attention weights for bag instances are replaced by random initialization of attention weights and finding the optimized attention weights through Genetic Algorithm.

The traditional K-means clustering algorithm is easily affected by the noise, outliers and falling into local optimal solution. This paper proposes a K-means clustering algorithm based on self-paced learning. Firstly, a best training subset is selected to construct the initial cluster model base on self-paced learning theory, and then enhances the generalization ability of the initial clustering model by adding sub-good subsets of samples one by one until the model performance is optimal or all training samples are used up. By analyzing the experimental results, the clustering algorithm proposed in this paper achieves better performance than the compare algorithms on the five real data sets.

The objective of this paper is to design an algorithm to maximize the learning ability and knowledge about the target class while minimizing the number of training samples for support vector data description (SVDD). With this motivation, a novel training sample reduction algorithm is proposed in this paper that selects the most promising boundary data points as training set. The proposed approach uses the local geometry of the distribution to estimate the farthest boundary points (also known as extreme points). The legitimacy of the proposed algorithm is verified via experiments performed on MNIST, Iris, UCI default credit card, svmguide and Indian Pines datasets.

In this paper, the effect of prosody-modification-based data augmentation is explored in the context of automatic speech recognition (ASR). The primary motive is to develop ASR systems that are less affected by speaker-dependent acoustic variations. Two factors contributing towards inter-speaker variability that are focused on in this paper are pitch and speaking-rate variations. In order to simulate such an ASR task, we have trained an ASR system on adults’ speech and tested it using speech data from adult as well as child speakers. Compared to adults’ speech test case, the recognition rates are noted to be extremely degraded when the test speech is from child speakers. The observed degradation is basically due to large differences in pitch and speaking-rate between adults’ and children’s speech. To overcome this problem, pitch and speaking-rate of the training speech are modified to create new versions of the data. The original and the modified versions are then pooled together in order to capture greater acoustic variability. The ASR system trained on augmented data is noted to be more robust towards speaker-dependent variations. Relative improvements of 11.5% and 27.0% over the baseline are obtained on decoding adults’ and children’s speech test sets, respectively.

This paper considers the problem of multi-label thorax disease classification on chest X-ray images. Identifying one or more pathologies from a chest X-ray image is often hindered by the pathologies unrelated to the targets. In this paper, we address the above problem by proposing a category-wise residual attention learning (CRAL) framework. CRAL predicts the presence of multiple pathologies in a class-specific attentive view. It aims to suppress the obstacles of irrelevant classes by endowing small weights to the corresponding feature representation. Meanwhile, the relevant features would be strengthened by assigning larger weights. Specifically, the proposed framework consists of two modules: feature embedding module and attention learning module. The feature embedding module learns high-level features with a convolutional neural network (CNN) while the attention learning module focuses on exploring the assignment scheme of different categories. The attention module can be flexibly integrated into any feature embedding networks with end-to-end training. The comprehensive experiments are conducted on the Chest X-ray14 dataset. CRAL yields the average AUC score of 0.816 which is a new state of the art.

While cross-media data, like text, image, audio, video and 3D model, has been the main form of big data, there is a current dearth of research on cross-media retrieval. In this paper, we focus on how to learn the common representation of heterogeneous data which is a key challenge for cross-media retrieval. Most existing approaches linearly project original low-level feature into a joint feature space for isomorphic data representation. However, linear projection cannot capture most complex cross-modal correlation with high nonlinearity. In this paper, we propose a novel feature learning algorithm, which is semi-supervised cross-modal vector-valued manifold regularization (SCVM), to explore common representation of heterogeneous data. SCVM jointly explores low-level feature correlation and semantic information in a unified framework. Based on manifold regularization, we learn cross-media features from vector-valued reproducing kernel Hilbert spaces (RKHS) by kernel transformation on both labeled and unlabeled samples; moreover, we impose smoothness constraints of possible solutions to improve retrieval accuracy. Comparing with the current state-of-the-art approaches on two public datasets, comprehensive experimental results show superior performance of our SCVM. The method is more robust and stable when extended from two media types to five media types, which is very attractive in practical application.

Most images have lots of linear information, which plays an important role in the tasks of image processing and pattern recognition. However, due to the interference of complex background in the images, the multiple directional characteristics of the linear information, and the problems of directional limitations in traditional methods, all these lead to the incomplete and discontinuities existed in the extracted linear information. To solve these problems, this paper puts forward a general model for improving the performance of the linear information extraction methods (LIEM) by utilizing the shear transformation. In this model, the shear transformation can transform an object (a filter or an image) in multiple directions, which directly or indirectly increases the directional characteristics of the traditional LIEM, and improves their performances to extract directional linear information and weak linear information. This paper elaborates on the basic principles of the model and its two implementations for the specific tasks. Furthermore, a variety of experiments are made to verify the versatility and effectiveness of the proposed model.

Component tree is a full image representation which encodes all connected components from upper (resp. lower) level sets of a given image through the inclusion relation. Information from this representation can be used in many image processing and computational vision applications, e.g. connected filtering, image segmentation, feature extraction, among others. In general, each node of a component tree represents a connected component of a level set and stores attributes which describes features of this connected component. This paper presents a review of a previously published method to compute attributes such as area, perimeter, and number of Euler by incrementally counting patterns while traversing nodes of a component tree. This method foundation is further detailed in this paper by presenting a novel theoretical background and algorithm correctness intuition. We also present a novel approach for this algorithm showing improvements for run-time execution and precision analysis.

We address the problem of spatio-temporal action localization in videos in this paper. Current state-of-the-art methods for this challenging task rely on an object detector to localize actors at frame-level firstly, and then link or track the detections across time. Most of these methods commonly pay more attention to leveraging the temporal context of videos for action detection while ignoring the importance of the object detector itself. In this paper, we prove the importance of the object detector in the pipeline of action localization, and propose a strong object detector for better action localization in videos, which is based on the single shot multibox detector (SSD) framework. Different from SSD, we introduce an anchor refine branch at the end of the backbone network to refine the input anchors, and add a batch normalization layer before concatenating the intermediate feature maps at frame-level and after stacking feature maps at clip-level. The proposed strong detector have two contributions: (1) reducing the phenomenon of missing target objects at frame-level; (2) generating deformable anchor cuboids for modeling temporal dynamic actions. Extensive experiments on UCF-Sports, J-HMDB and UCF-101 validate our claims, and we outperform the previous state-of-the-art methods by a large margin in terms of frame-mAP and video-mAP, especially at a higher overlap threshold.

Real-world data such as digital images, MRI scans and electroencephalography signals are naturally represented as matrices with structural information. Most existing classifiers aim to capture these structures by regularizing the regression matrix to be low-rank or sparse. Some other methodologies introduce factorization technique to explore nonlinear relationships of matrix data in kernel space. In this paper, we propose a multi-distance support matrix machine (MDSMM), which provides a principled way of solving matrix classification problems. The multi-distance is introduced to capture the correlation within matrix data, by means of intrinsic information in rows and columns of input data. A complex hyperplane is established upon these values to separate distinct classes. We further study the generalization bounds for i.i.d. processes and non i.i.d. process based on both SVM and SMM classifiers. For typical hypothesis classes where matrix norms are constrained, MDSMM achieves a faster learning rate than traditional classifiers. We also provide a more general approach for samples without prior knowledge. We demonstrate the merits of the proposed method by conducting exhaustive experiments on both simulation study and a number of real-word datasets., The paper is under consideration at Pattern Recognition Letters

Current 3D reconstruction systems can easily fall into the problems of insufficient memory and low efficiency when reconstructing complex large scenes, and the reconstructed model drifts a lot at the same time. To solve the above problems, a 3D reconstruction system for collaborative scanning based on multiple RGB-D Cameras (Xtion sensors) is presented. In the case of no pose estimation, this paper can obtain the camera pose initial values of the sequence images through the image acquisition platform that has been calibrated in advance, which can well cope with the problems existing in the current 3D reconstruction system. The main innovations of this paper are as follows: (1) In view of the large amount of human-computer interaction required for the current 3D reconstruction system and the high requirements for data acquisition, a collaborative scanning 3D reconstruction system based on multiple Xiton sensors (RGB-D camera) is developed. The proposed system does not require human-computer interaction and can capture high quality image sequences fully automatically; (2) In view of the problems of memory shortage and low efficiency in current 3D reconstruction systems applied to complex large-scale scenes, this paper proposes a system which can obtain camera poses in advance calibration without the need for camera pose estimation; (3) In terms of camera pose optimization, a segmented bundle adjustment method has been presented to obtain the high-precision camera poses. A large number of experiments demonstrate that the proposed system can effectively solve the existing problems. Meanwhile, the proposed reconstruction system can obtain high-precision 3D models suitable for various complex large scenes, which can be widely used in human-computer interaction, virtual reality and other fields.

Localization and reconstruction are two highly related research areas. Both of them have developed rapidly in recent years. Apparently, with the help of ground and aerial meta-data integration, the performance of both localization and reconstruction can go a step further. For localization, aerial meta-data provides a global reference, by which the ground query can achieve a cumulative error free absolute localization. As for reconstruction, a complete and detailed model can be reconstructed by integrating ground and aerial meta-data. Though with many advantages, the integration itself is non-trivial. It is difficult to obtain ground-to-aerial correspondences neither in 2D manner nor in 3D manner. That is because: (1) The differences between the ground and aerial images in viewpoint, scale, illumination, etc. are notable; (2) The discrepancies between the ground and aerial point clouds in terms of point density, accuracy, noise level, etc. are very large. To deal with these problems, lots of methods have been proposed recently. In this paper, the methods of integrating ground and aerial meta-data for localization and reconstruction are reviewed respectively. Though many intermediate results with high quality have been achieved, we hope that inspired by the reviewed methods in this paper, more thorough methods and impressive results would emerge.

There is a widely held belief that computer vision, in general, and face authentication, in particular, are to a large extent solved problems. This paper challenges this belief regarding face authentication using examples from modern art that significantly confound face detection. The challenges are made concrete using a new MAFD-150 dataset (Modern Art Face Detection) composed mostly of modern art examples that cover much diversity in style and artists. MAFD-150 challenges the belief that singleton and crowd face detection is an almost solved problem, and provides baselines and preliminary results that highlight the inadequacy of current expertise and methods to address face detection. In particular, we show that well-known face detection algorithms are only able to achieve an F1 score of less than 35% overall across the new dataset. Additionally, we discuss the performance of the selected face detectors on varying art categories (such as Impressionism, Pop Art, et al.) to show how style and face representation may impact these algorithms. The paper concludes with suggestions on how to advance face processing by leveraging the complementarity between Show-and-Tell-like methods and a context and cooperative driven visual question answering framework using relevance-based triage. The very challenges detailed throughout are then shown to be helpful with developing novel, robust, and secure access protocols that combine text and modern art images using the visual question answering framework.

The paper presents experiments with the home-made, low-cost prototype of a sensor-equipped pen for handwriting-based biometric authentication. The pen allows to capture the dynamics of user writing on normal paper, while producing a kind of password (passphrase) chosen in advance. The use of a word of any length instead of the user's signature makes the approach more robust to spoofing, since there is no repetitive pattern to steal. Moreover, if the template gets violated, this is much less harmful than signature catch. The entailed sensors are a pair of accelerometer and gyroscope and a pressure sensor. The aim is a natural yet precise interaction, that allows recognizing the user by the signals recorded while producing a specific word chosen during enrollment and possibly changed later. The pen can be exploited in a number of applications requiring user recognition, yet relieving from the need to learn complex procedures, and to undergo critical capture operations. The approach fuses the use of a kind of password, though not necessarily complex as those requested by traditional approaches, and biometric recognition. The novelty with respect to most proposals in literature is the combination of three elements at once: the matching of any handwritten text instead of user signature, the on-line capture of seven sensor signals to recognize handwriting dynamics (three from accelerometer, three from gyroscope and one from pressure sensor), and the use of normal paper instead of a digitizing tablet. Presented experiments test two different recognition techniques, implemented by two modules that can be alternatively plugged into the system. An SVM-based verification module entails to extract the most relevant features from writing dynamics, and to acquire a sufficient amount of enrolling data (30 samples per user) to train an SVM for each user. A pure Dynamic Time Warping (DTW) verification module does not require such training, and is tested using either a gallery with the same number of templates per user as those used for SVM training, or with a gallery containing a much lower number of templates per user (namely 5). Obtained results encourage further investigation of lightweight strategies for written password dynamics recognition.

An authentic Personal identification infrastructure helps to control the access in order to secure data and information. Biometric technology is mainly based on physiological or behavioural characteristics of human body. This paper elucidates Finger Knuckle Print (FKP) biometric system based on feature extraction methodology using the short and long Gabor feature extraction. This FKP authentication system involves all basic processes like pre-processing, feature extraction and classification. This feature extraction is done by Gabor filter for extracting the important features form the FKP dataset. The query FKP Gabor features are matched and compared with the enrolled template using Hamming distance [HD]. Finally this paper proposes the FKP recognition based on Support Vector Machines in accordance with score level fusion to improve the recognition performance of FKP by integrating the Gabor features. The main aim of this paper is to utilize the ability of Support Vector Machines (SVM) in pattern recognition and classifying with Hamming distance which helps to improve the False Acceptance Rate (FAR) and Genuine Acceptance Rate (GAR). This new combination (double instance) of FKP shows better results as 96.01% for MAX Rule and 92.33% for Min Rule than single instance performance as 89.11% .This idea shows good results in Finger Knuckle Print recognition of a person.

Despite of having no explicit shape model, multi-atlas approaches to image segmentation have proved to be a top-performer for several diverse datasets and imaging modalities. In this paper, we show how one can directly incorporate shape regularization into the multi-atlas framework. Unlike traditional multi-atlas methods, our proposed approach does not rely on label fusion on the voxel level. Instead, each registered atlas is viewed as an estimate of the position of a shape model. We evaluate and compare our method on two public benchmarks: (i) the VISCERAL Grand Challenge on multi-organ segmentation of whole-body CT images and (ii) the Hammers brain atlas of MR images for segmenting the hippocampus and the amygdala. For this wide spectrum of both easy and hard segmentation tasks, our experimental quantitative results are on par or better than state-of-the-art. More importantly, we obtain qualitatively better segmentation boundaries, for instance, preserving topology and fine structures.Despite of having no explicit shape model, multi-atlas approaches to image segmentation have proved to be a top-performer for several diverse datasets and imaging modalities. In this paper, we show how one can directly incorporate shape regularization into the multi-atlas framework. Unlike traditional multi-atlas methods, our proposed approach does not rely on label fusion on the voxel level. Instead, each registered atlas is viewed as an estimate of the position of a shape model. We evaluate and compare our method on two public benchmarks: (i) the VISCERAL Grand Challenge on multi-organ segmentation of whole-body CT images and (ii) the Hammers brain atlas of MR images for segmenting the hippocampus and the amygdala. For this wide spectrum of both easy and hard segmentation tasks, our experimental quantitative results are on par or better than state-of-the-art. More importantly, we obtain qualitatively better segmentation boundaries, for instance, preserving topology and fine structures.

Tensor train is a hierarchical tensor network structure that helps alleviate the curse of dimensionality by parameterizing large-scale multidimensional data via a set of network of low-rank tensors. Associated with such a construction is a notion of Tensor Train subspace and in this paper we propose a TT-PCA algorithm for estimating this structured subspace from the given data. By maintaining low rank tensor structure, TT-PCA is empirically more robust to noise as compared to PCA or Tucker-PCA. This is borne out numerically by testing the proposed approach on the Extended YaleFace Dataset B, MINIST Dataset, CIFAR-10 dataset. This paper shows that the TT-PCA methods achieve less storage requirements, and have computationally faster online implementation with improved classification performance.

This paper presents an improvement of handwriting binarization techniques on colored historical documents. We introduce a novel preprocessing step into the usual document image analysis (DIA) workflow. Before binarization, we propose a grayification step to enhance the input image with the help of a new grayscale conversion algorithm, namely the grayification algorithm. This new algorithm uses luminance and color information to improve the contrast between the foreground and the background. Especially on documents with non-black ink and moreover with diverse colors, e.g., illuminations in historical manuscripts, we expect an increased performance. The binarization give then better results on this enhanced grayscale image, and in particular color text is binarized as well as black text. In fact, by adding a preprocessing step to enhance the input grayscale image, the results on all the following tasks of the analysis chain should be improved. This modification of the usual workflow of historical document analysis eases the binarization task as well as other following tasks like layout analysis, line segmentation, OCR, etc. We demonstrate the effects of our novel preprocessing technique on a set of challenging historical documents, which we make publicly available for research purpose, and two publicly available datasets. This improvement is illustrated in this paper on the binarization task, where the results of four different binarization methods are successfully improved.

This paper presents a new flexible approach to predict the gender of the writers from their handwriting samples. Handwriting features like slant, curvature, line separation, chain code, character shapes, and more, can be extracted from different methods. Therefore, the multi-feature sets are irrelevant and redundant. The conflict of the features exists in the sets, which affects the accuracy of classification and the computing cost. This paper proposes an approach, named kernel mutual information (KMI), that focuses on feature selection. The KMI approach can decrease redundancies and conflicts. In addition, it extracts an optimal subset of features from the writing samples produced by male and female writers. To ensure that KMI can apply the various features, this paper describes the handwriting segmentation and handwritten text recognition technology used. The classification is carried out using a Support Vector Machine (SVM) on two databases. The first database comes from the ICDAR 2013 competition on gender prediction, which provides the samples in both Arabic and English. The other database contains the Registration-Document-Form (RDF) database in Chinese. The proposed and compared methods were evaluated on both databases. Results from the methods highlight the importance of feature selection for gender prediction from handwriting.

Video action recognition is widely applied in video indexing, intelligent surveillance, multimedia understanding, and other fields. Recently, it was greatly improved by incorporating the learning of deep information using Convolutional Neural Network (CNN). This motivated us to review the notable CNN-based action recognition works. Because CNN is primarily designed to extract 2D spatial features from still image and videos are naturally viewed as 3D spatiotemporal signals, the core issue of extending the CNN from image to video is temporal information exploitation. We divide the solutions for exploiting temporal information exploration into three strategies: 1) 3D CNN; 2) taking the motion-related information as the CNN input; and 3) fusion. In this paper, we present a comprehensive review of the CNN-based action recognition methods according to these strategies. We also discuss the action recognition performance on recent large-scale benchmarks and the limitations and future research directions of CNN-based action recognition. This paper offers an objective and clear review of CNN-based action recognition and provides a guide for future research.

This paper proposes a method for image segmentation using a deformable triangle mesh in the image domain. We define a piecewise constant function by labeling the mesh triangles with different phases, each representing a segment of an image. Our method finds the optimal mesh configuration and triangle labeling that minimize the piecewise constant Mumford-Shah functional. Contributions of this paper include a force model that moves mesh vertices towards the solution, and an adaptivity model that further adapts the mesh by introducing or removing vertices. The results demonstrate the advantages of our method over traditional methods like snakes and level set. Our approach supports multi-phase segmentation incurring no particular overhead. Furthermore, the use of an adaptive mesh facilitates accurate segmentation with a very compact representation. The biggest challenge of deformable meshes, changes to the topology of the segments, is handled by employing Deformable Simplical Complex (DSC), a method for explicit interface tracking.