Search

Deep similarity trackers are able to track above real-time speed. However, their accuracy is considerably lower than deep classification based trackers since they avoid valuable online cues. To feed the target-specific information for real-time object tracking, we propose a novel Siamese attention network. Different types of attention mechanisms are used to capture different contexts of target information and then learned knowledge is used to feed target cues at different representation levels of similarity tracking. In addition, an online learning mechanism is employed to utilise the available target-specific data. The proposed tracker reduces the impact of noise in the target template and improves the accuracy of similarity tracking by feeding target cues into the similarity search. Extensive evaluation performed on OTB-2013/50/100 and VOT2018 benchmark datasets demonstrate the proposed tracker outperforms state-of-the-art approaches while maintaining real-time tracking speed.

Convolutional Neural Networks (CNNs) became the de-facto standard for medical image analysis. In CNN, pooling layers are used for downsampling feature maps by aggregating features from local regions, and hence help to learn invariant features and reduce computational complexity. Various pooling techniques have been proposed, among which average and max pooling are the widely used ones because of their simplicity. Since average pooling aggregates all the information of each region in each feature map, background features may dominate in the pooled representation. On the other hand, max pooling can capture noisy features as it focuses on the most activated features. To overcome this, researchers have proposed soft pooling techniques as intermediate to max and average pooling. In addition, various other pooling techniques also have been introduced for different purposes, which include techniques to reduce overfitting, to capture higher-order information such as the correlation between features, to capture spatial/structural information, etc. In this work, we investigate different pooling mechanisms with ResNet, a widely used CNN architecture, for the classification of HEp-2 cell images. We found that average pooling generally performs better than other pooling techniques such as max pooling, soft pooling, and bilinear pooling. By just changing the local pooling operation of the ResNet architecture from max to average we show over 2% of improvement in mean class accuracy (MCA). Overall, our approach gives an MCA of 88% for the classification of HEp-2 cell images.

Diabetic Retinopathy (DR) is a retinal disease due to diabetes which damages the retina. DR became one of the important cause of blindness and vision impairment worldwide. Usually, DR is diagnosed by examining the eyes through a fundus camera. Small lesions such as microaneurysms, hemorrhages, hard exudates, etc., appear in the DR affected retina. DR grading is an important task to determine the type of treatment. Usually, lesions in the DR images are hard to detect and classify as they are smaller in size and have similar visual appearance. Most of the automated approaches for DR grading make use of the information from both eyes to determine the DR grade of a particular eye. These information fusion approaches vary from simple concatenation of the features of both retinal images (left and right eyes) to complex higher-order pooling mechanisms such as Bilinear pooling. As they are evaluated on different datasets it is hard to determine which approach is better for this purpose. In this work, we investigate different deep learning-based information fusion strategies to integrate the features from both eyes of a patient to find the DR grading of a particular eye. We found that a simple approach which concatenates the average-pooled features from both eyes to capture the overall information about the patient, with the features extracted from the eye under consideration performs equally well compared to bilinear pooling based approach, but with low memory and computational requirements. In addition, this approach gives considerable improvements in accuracy and Kappa scores compared to the widely used approach which just concatenates the features from both eyes to determine the DR grading of a particular eye. Our approach gives a Kappa score of 0.849 with ResNet-18 as the backbone architecture on the test set of Kaggle-EyePACS dataset, which is competitive to the state-of-the-art.

Quadratic Weighted Kappa (QWK) is a statistic to measure the agreement between two annotators. QWK has been widely used as the evaluation measure for various medical imaging problems, where, the class labels have a natural ordering, e.g., no Diabetic Retinopathy (DR), mild DR, and severe DR. The easiest way to treat the classification problem with these ordinal labels is to consider the problem as a multiclass classification problem and apply the Cross Entropy (CE) loss. However, when applying CE loss the order of the classes becomes meaningless, i.e., the loss will be same if a healthy image is classified into mild DR or severe DR. At the same time, the QWK score will be severely affected if a healthy image is classified into severe DR than mild DR. The most appropriate way to get a better classification score is to optimize the evaluation measure itself. i.e., directly optimize the QWK statistics. However, this optimization may hinder the learning, and may lead to sub-optimal solutions, and therefore, may give lower performance than expected. On the other hand, Ordinal Regression (OR) based approaches also can be used for such problems. The main focus of this work is to investigate which loss function (CE loss, QWK loss or OR loss) is the most appropriate one to the Convolutional Neural Network-based ordinal classification problems, where, QWK is used as the evaluation measure. Experiments on two public datasets, Diabetic Retinopathy and Prostate Cancer, with two different network architectures suggest that directly optimizing QWK is the better choice when small networks are used. On the other hand, we found that for large networks OR based loss function gives better performance.

This paper proposes a vehicle logo recognition (VLR) system centered on front-view cars, which has been largely neglected by vision community in comparison to other object recognition tasks. The study focuses on local features that describe structural characteristics by locating the logo of a car using a coarse-to-fine (CTF) strategy that first detects the bounding box of a car then the grille and at last, the logo. The detected logo is then used to recognize the make of a car in a reduced time. Our system starts to progress in detecting the bounding box of a car by means of a vocabulary voting and scale-adaptive mean-shift searching strategy. The system continues to process in locating the bounding box of an air-intake grille using a scale-adaptive sliding window searching technique. In the next level, the bounding box of a logo is located by means of cascaded classifiers and circular region detection techniques. The classification of vehicle logos is carried out on the patch-level as occurrences of similar visual words from a visual vocabulary, instead of representing the patchbased descriptors as bag-of-features and classifying them using a standard classifier. The proposed system was tested on 25 distinctive elliptical shapes of vehicle logos with 10 images per class. The system offers the advantage of accurate logo recognition of 86.3% in the presence of significant background clutter. The proposed scheme could be independently used for part recognition of grille detection and logo detection.

Bag-of-features (BoF) representation is one of the most popular image representations that is used in visual object classification, owing to its simplicity and good performance. However, the BoF representation always faces the difficulty of curse of dimensionality that leads to huge computational cost and increased storage requirement. To create a discriminative and compact BoF representation, it is desired to eliminate ambiguous features before the construction of visual codebook and to select the informative codewords from the constructed codebook. In this paper, we propose a two-staged approach to create a discriminative and compact BoF representation for object recognition. In the first step, we eliminate ambiguous patch-based descriptors using an entropy-based filtering approach to retain high-quality descriptors. In the subsequent step, we select the informative codewords based on statistical measures. We have tested the proposed technique on Xerox7, UIUC texture, PASCAL VOC 2007 and Caltech101 benchmark datasets. Testing results show that more training features and/or a high-dimensional codebook do not contribute significantly to increase the performance of classification but it increases the overall model complexity and computational cost. The proposed preprocessing step of descriptor selection increases the discriminative power of a codebook, whereas the post-processing step of codeword selection maintains the codebook to be more compact. The proposed framework would help to optimise BoF representation to be efficient with steady performance.

In a patch-based object classification system, one of the most popular image representation approach is the bag-of-features (BoF) representation. However, the number of patch-based features extracted from images to create the BoF vector is usually very large, which causes two problems: (i) Increased amount of computational needs during the vector quantisation step, and (ii) Ambiguous descriptors from training images increase false positive rate in the classification. To overcome these issues we introduce a one-pass feature selection approach followed by an entropy-based filtering technique to eliminate the ambiguous features from initial large feature set. In this work, a discriminative BoF representation for object recognition is constructed using patch-based descriptors that are informative in distinguishing object categories. Following the construction of a codebook a subset of codewords which is not activated enough in images is eliminated based on statistical measures and visual-bit representation of codewords. The proposed technique is evaluated on (i) Xerox7, (ii) UIUCTex, (iii) PASCAL VOC 2007, and (iv) Caltech101 image datasets. The proposed feature selection step increases the discriminant power of a codebook, while the codeword selection method maintains the codebook to be more compact. The proposed framework would help to optimise the BoF representation to be effective with stable performance.

This paper proposes a system that augments different techniques in identifying distinct features of Sri Lankan currency notes enabling blind people to feel safety and confidence in financial dealing. The proposed system is developed with the release of new banknote series in Sri Lanka as a working example. The recognition process of the proposed system includes skew correction, feature extraction of colour and texture, and counting the number of heavily printed dots on the currency notes. The experimental results carried out in about 270 notes demonstrate an overall recognition rate of 96%.

Visual tracking frameworks employing Convolutional Neural Networks (CNNs) have shown state-of-the-art performance due to their hierarchical feature representation. While classification and update based deep neural net tracking have shown good performance in terms of accuracy, they have poor tracking speed. On the other hand, recent matching based techniques using CNNs show higher than real-time speed in tracking but this speed is achieved at a considerably lower accuracy. To successfully manage the trade-off between accuracy and speed, we propose a novel CNN architecture for visual tracking. We achieve this trade-off balance by using an approach in which consecutive similar frames are processed with a similarity matching technique, and dissimilar frames are processed with a classification approach within the CNN architecture. The tracking speed is improved by avoiding unnecessary model updates through the measurement of similarity between adjacent frames, while the accuracy is maintained by adopting a classification approach when needed, with deeper level features. Extensive evaluation performed on a publicly available benchmark dataset demonstrates our proposed tracker shows competitive performance while maintaining near real-time speed.

Emotion recognition continues to yield more as the sophisticated track of applications in the field of human computer interaction. The main consequence of classifying the emotional states is selecting best out of a large scale of features and implementing a pattern for the classification of emotions. In this paper we propose a hierarchical classification strategy using Mel — frequency cepstral coefficients (MFCC). Statistical measurements of MFCC were explored individually in classification and the optimal model for each decision node of a binary tree is constructed using a one-versus-all based SVM classifier a the subset of statistical measures that yields the highest recognition. The overall results of experiments disclose that the basic emotions such as anger, sad, happy, and neutral can be classified in a hierarchical structure. The proposed framework has been evaluated on two benchmark speech sets: Berlin and Danish, and provides better performance in emotion classification.

Convolutional neural networks (CNNs) have been employed in visual tracking due to their rich levels of feature representation. While the learning capability of a CNN increases with its depth, unfortunately spatial information is diluted in deeper layers which hinders its important ability to localise targets. To successfully manage this trade-off, we propose a novel residual network based gating CNN architecture for object tracking. Our deep model connects the front and bottom convolutional features with a gate layer. This new network learns discriminative features while reducing the spatial information lost. This architecture is pre-trained to learn generic tracking characteristics. In online tracking, an efficient domain adaptation mechanism is used to accurately learn the target appearance with limited samples. Extensive evaluation performed on a publicly available benchmark dataset demonstrates our proposed tracker outperforms state-of-the-art approaches.

An Optical character recognition (OCR) consists of the phases: preprocessing and segmentation, feature extraction, classification and post-processing. This paper focuses on preprocessing and segmentation tasks which plays a major role in the subsequent processes of an OCR. The objective of preprocessing and segmentation is to improve the quality of the input image. In addition this phase removes unnecessary portions of the input image that would otherwise complicate the subsequent steps of OCR and reduce the overall recognition rate. Preprocessing and segmentation step consists many sub processes namely, image binarisation, noise removal, skew detection and correction, page segmentation, text or non-text classification, line segmentation, word segmentation and character segmentation. This paper proposes a new method to calculate the skew angle for skew correction. In addition this paper proposes a more accurate method to segment the input image as blocks and classify the blocks as text or non-text. The skew angle is calculated on the scanned document using Wiener filter, smearing technique and Radon transform. Document image is segmented into blocks using run length smearing algorithm and connected component analysis. Features such as basic, density and HOG are extracted from each block for text and non-text classification. The proposed methods are tested on 54 documents. The testing results show a recognition rate of 96.30% for skew detection and correction whereas the recognition rate is 99.18% for text or non-text classification with binary SVMs using RBF kernel.

Automatically detecting and tracking multiple persons in videos is one of the main research interest in computer vision based applications. This paper presents a tracking-by-detection approach for tracking people in dynamic backgrounds with frequent occlusions by combining pre-trained generic person detector, online trained person-specific detector and a motion tracker. The popular aggregate channel features (ACF) are used to train the detectors and target specific particle filter is used as motion tracker. In order to learn right appearance of a target person, person-specific detector learns positive samples from prior frames which are detected by both generic person detector and person-specific detector. Data associations among the coincident detections of the detectors and tracker are used to update the person-specific detector and motion tracker. The person-specific detector searches the target person in a reduced region, which is defined by the associate motion tracker. A careful combination of detections of both detectors and tracker are used to locate the correct target person in the video sequence. Experiments have been carried out on Caltech pedestrian benchmark dataset. The proposed method shows better performance against state-of-the-art tracker while maintaining the tracking speed in real-time.

In a patch-based object recognition system the key role of a visual vocabulary is to provide a way to map the low-level features into a fixed-length vector in histogram space to which standard classifiers can be directly applied. The discriminative power of such a visual vocabulary determines the quality of the vocabulary model, whereas the size of the vocabulary controls the complexity of the model. A compact visual vocabulary provides a lower-dimensional representation whereas a large-sized vocabulary may overfit to the distribution of visual words in an image and lead to heavy computational load. The generic framework of a bag-of-features approach follows a standard routine extracting local image descriptors and clustering with a user- designated number of clusters. The problem with this routine lies in that constructing a vocabulary for each single dataset is not efficient. Usually the construction of a vocabulary is achieved by cluster analysis using K-means algorithm. However, one of its drawbacks is the choice of a suitable value for K which determines the size of a visual vocabulary. The choice of the size of a vocabulary should be balanced between the recognition rate and computational needs. In this paper we propose a two-staged approach to map an initial high- dimensional vocabulary into a compact vocabulary while maintaining its discriminative power. Using an initial larger vocabulary we first represent the training images using a coding scheme that maps the importance of each visual word within an image as visual bits. These set of visual bits of images then form a sparse representation of every visual word with respect to the set of category- specific training images that is used for the compression. We have tested our vocabulary compression technique on four computer vision tasks: (i) Xerox7 (ii) PASCAL VOC Challenge 2007 (iii) UIUC texture and (iv) MPEG7 CE Shape-1 Part B Silhouette image datasets. Testing results show that the proposed method slightly outperforms vocabularies learnt by K-means by achieving just half the size of initial vocabulary. Our compression technique could help to optimize larger vocabularies to fewer visual words with stable performance.

Optical character recognition (OCR) is one of the important research areas in image processing and pattern recognition. OCR for printed Tamil text is considered as a challenging problem due to the large number of (i.e., 247) characters with complicated structures and, similarity between characters as well as different font styles. This paper proposes a novel approach for multiclass classification to recognise Tamil characters using binary support vector machines (SVMs) organised in a hybrid decision tree. The proposed decision tree is a binary rooted directed acyclic graph (DAG) which is succeeded by unbalanced decision trees (UDT). DAG implements OVO-based SVMs whereas UDT implements OVA-based SVMs. Each node of the hybrid decision tree exploits optimal feature subset in classifying the Tamil characters. The features used by the decision tree are basic, density, histogram of oriented gradients (HOG) and transition. Experiments have been carried out with a dataset of 12400 samples and the recognition rate observed is 98.80%with the hybrid approach of DAG and UDT SVMs using RBF kernel.

Vehicle detection has received much attention over the recent years. In this paper we mainly address on the detection of cars in front-view static images and focus on local features that describe structural characteristics in particular. Our method is based on constructing visual vocabularies for the car and non-car objects (i.e., background). Extended speeded up robust features are used in constructing the vocabularies using K-means algorithm. For a test car image, e-SURF keypoints are extracted and possible background keypoints are rejected in order to retain keypoints that of car by using a vocabulary voting strategy. The retained keypoints are then scanned using mean-shift algorithm to find a candidate bounding box of the car. The extraction of keypoints with vocabulary voting and the mean-shift searching technique are repeated to every scale of the down sampled test image. The set of bounding boxes found at every scale are then mapped in to the original test image and a non-maximum suppression technique is used to predict the final bounding box of the car. The system is evaluated on 25 distinctive vehicle classes with 20 images per class. Testing results show that the method has a high detection rate of 96.8% on front-view cars. The method is flexible and can be easily extended to other views such as side or rear view by modifying the visual vocabularies.

Images of outdoor scenes are degraded by absorption and scattering by the suspended particles and water droplets in the atmosphere. The light coming from a scene towards the camera is attenuated by fog and is blended with the airlight which adds more whiteness into the scene. Fog removal is highly desired in computer vision applications. Removing fog from images can significantly increase the visibility of the scene and is more visually pleasing. In this paper, we propose a method that can handle both homogeneous and heterogeneous fog which has been tested on several types of synthetic and real images. We formulate the restoration problem based on fusion strategy that combines two derived images from a single foggy image. One of the images is derived using contrast based method while the other is derived using statistical based approach. These derived images are then weighted by a specific weight map to restore the image. We have performed a qualitative and quantitative evaluation on 60 images. We use the mean square error and peak signal-to-noise ratio as the performance metrics to compare our technique with the state-of-the-art algorithms. The proposed technique is simple and shows comparable or even slightly better results with the state-of-the-art algorithms used for defogging a single image.

Optical Character Recognition (OCR) deals with automated recognition of characters that are in the format of digital image. OCR refers to the process by which scanned images are electronically processed and converted to an editable document. Handwritten and printed texts are the primary research areas of an OCR. Many OCR systems are commercially available for English and Arabic characters but there is still no recognition system available which yields higher recognition rate even though the scanned images are of high quality. The general framework of a Tamil OCR in the literature involves: preprocessing, line segmentation, word segmentation, character segmentation, feature extraction and recognition of characters. OCR for printed Tamil documents poses challenge owing to: one line may have different font styles, presence of pictures, multi columns, touching of adjacent characters, presence of broken characters, low print quality and complex layout. Furthermore, when comparing 26 alphabets in English, Tamil language has 247 alphabets which makes the recognition more difficult. There are few OCRs for Tamil language that are freely available with a moderate recognition rate as the performance comparisons of such OCRs are not available on a benchmark dataset. In this paper we compare OCRs for printed Tamil texts on four different types of documents: books, magazines, newspapers and pamphlets. Furthermore we propose a post-processing error correction technique to the tested OCRs which reduces the overall mean error rate by nearly 10% on those four categories.

Logo identification and classification have received considerable attention from both the machine learning and computer vision communities. Vehicle logo recognition (VLR) is used to recognise accurately the manufacturer of a vehicle by using its iconic logo. A VLR system in addition to license plate recognition aims to increase the confidence of vehicle monitoring systems in private environments such as car parks in companies, shopping malls, and institutions. VLR is a challenging process due to the presence of extensive background, clutter, different degree of illumination, varying sizes of vehicles in motion, and change of weather conditions such as fog, sunny and rainy which are present in the two-dimensional images. On the other hand, the bag-of-features (BoF) approach initiated to be used as a black box providing reliable and repeatable measurements from images for a wide range of applications such as visual object recognition. The advantages of such a BoF approach are its simplicity and state-of-the-art performance in recent VLR tasks. Most of the VLR in the literature has focused on less than twenty distinctive iconic logos of the vehicles. In this paper, we propose a novel VLR system as an alternative method to the BoF approach in which the extracted patch-based descriptors of a test image is voted against a locally merged codebooks of logos to predict the class label without the need for mapping the extracted descriptors into a fixed-length feature vector and then feeding it to a standard classifier. The proposed VLR system using SIFT descriptors is compared with the traditional BoF approach that employs a standard classifier: nearest neighbour and support vector machines. The system is evaluated on 25 distinctive vehicle classes with 20 images per class. Testing results show that our method promotes codebooks that deliver a robust classification performance of 98.6% which drastically reduces the time needed by the traditional BoF approach.

A superpixel is an image patch which is better aligned with intensity edges than a rectangular patch. Superpixels are perceptually consistent units which carry more information than pixels and adhere well to image boundaries. Nowadays superpixels are widely used for segmentation in computer vision and biomedicai applications. There are many approaches to generate superpixels such as SLIC, QuickShift, Turbopixels and Normalized cuts algorithms, each with its own advantages and drawbacks that may be better suited to a particular application. In this paper we propose a one-pass clustering (OPC) technique to efficiently generate superpixels in the combined five-dimensional feature space of CIELAB colour and XY image plane. The Berkeley segmentation dataset (BSDS500) is used to quantitatively compare the performance of OPC with SLIC superpixels method as measured by boundary recall and under-segmentation error. OPC superpixels achieve comparable performance to previously reported results which allow a trade-off between a less regular space superpixels but accurate boundaries or better efficiency.

Car detection is a challenging research area where the appearance has various changes due to different models, poses, scales, lighting conditions, background, occlusion and various image sizes. Amidst these external challenging factors, car side-views have obvious and consistent characteristics in their structure such as wheels, oblique windows and bumpers, which provide crucial cues for detection. In this paper, we describe a novel side-view car detection technique which constructs an initial contour for the Snake using the detection of wheels. Our approach is subdivided in to three stages: wheel detection, bounding box detection, and segmenting the car from its background. In the first step, we use Hough transformation to detect circles in an image and validate the wheels with a learnt visual vocabulary. In the second step, we construct an initial contour with the aid of the detected wheels and the width-to-height statistics of different types of cars. This initial contour forms the bounding box of the car. In the third step, the Snake algorithm proceeds with the initial contour to fit the boundary closer to the car. Hence, the foreground, car, is separated from the background. We empirically evaluated our method on 100 side-view cars and the testing results shows that this approach has good performance.

Support vector machine is a state-of-the-art learning machine that is used in areas, such as pattern recognition, computer vision, data mining and bioinformatics. SVMs were originally developed for solving binary classification problems, but binary SVMs have also been extended to solve the problem of multi-class pattern classification. There are different techniques employed by SVMs to tackle multi-class problems, namely oneversus-one (OVO), one-versus-all (OVA), and directed acyclic graph (DAG). When dealing with multi-class classification, one needs an appropriate technique to effectively extend these binary classification methods for multi-class classification. We address this issue by extending a novel architecture that we refer to as unbalanced decision tree (UDT). UDT is a binary decision tree arranged in a top-down manner, using the optimal margin classifier at each split to relieve the excessive time in classifying the test data when compared with the DAG-SVMs. The initial version of the UDT required a longer training time in finding the optimal model for each decision node of the tree. In this work, we have drastically reduced the excessive training time by finding the order of classifiers based on their performances during the selection of the root node and fix this order to form the hierarchy of the decision tree. UDT involves fewer classifiers than OVO, OVA and DAG -SVMs, while maintaining accuracy comparable to those standard techniques.

The codebook model-based approach, while ignoring any structural aspect in vision, nonetheless provides state-of-the-art performances on current datasets. The key role of a visual codebook is to provide a way to map the low-level features into a fixed-length vector in histogram space to which standard classifiers can be directly applied. The discriminative power of such a visual codebook determines the quality of the codebook model, whereas the size of the codebook controls the complexity of the model. Thus, the construction of a codebook is an important step which is usually done by cluster analysis. However, clustering is a process that retains regions of high density in a distribution and it follows that the resulting codebook need not have discriminant properties. This is also recognised as a computational bottleneck of such systems. In our recent work, we proposed a resource-allocating codebook, to constructing a discriminant codebook in a one-pass design procedure that slightly outperforms more traditional approaches at drastically reduced computing times. In this review we survey several approaches that have been proposed over the last decade with their use of feature detectors, descriptors, codebook construction schemes, choice of classifiers in recognising objects, and datasets that were used in evaluating the proposed methods.

The bag-of-keypoints representation started to be used as a black box providing reliable and repeatable measurements from images for a wide range of applications such as visual object recognition and texture classification. This order less bag-of-keypoints approach has the advantage of simplicity, lack of global geometry, and state-of-the-art performance in recent texture classification tasks. In such a model, the construction of a visual vocabulary plays a crucial role that not only affects the classification performance but also the construction process is very time consuming which makes it hard to apply on large datasets. This paper presents a fast approach for texture classification that integrates existing ideas to relieve the excessive time involved both in constructing a visual vocabulary and classifying unknown images using a support vector machine based decision tree. We conduct a comparative evaluation on three benchmark texture datasets: UIUCTex, Brodatz, and CUReT. Our approach achieves comparable performance to previously reported results in multi-class classification at a drastically reduced time.

Frequencies of occurrence of low-level image features is the representation of choice in the design of state-of-the-art visual object recognition systems. A crucial step in this process is the construction of a codebook of visual features, which is usually done by cluster analysis of a large number of low-level image features detected as interest points. However, clustering is a process that retains regions of high density in a distribution and it follows that the resulting codebook need not have discriminant properties. Here we extend our recent work on constructing a one-pass discriminant codebook design procedure inspired by the resource allocating network model from the artificial neural networks literature. Unlike clustering, this approach retains data spread out more widely in the input space, thereby including rare low-level features in the codebook. It simultaneously achieves increased discrimination and a drastic reduction in the computational needs. We illustrate some properties of our method and compare it to a closely related approach.