Search

Melanoma is considered the most serious type of skin cancer. All over the world, the mortality rate is much high for melanoma in contrast with other cancer. There are various computer-aided solutions proposed to correctly identify melanoma cancer. However, the difficult visual appearance of the nevus makes it very difficult to design a reliable Computer-Aided Diagnosis (CAD) system for accurate melanoma detection. Existing systems either uses traditional machine learning models and focus on handpicked suitable features or uses deep learning-based methods that use complete images for feature learning. The automatic and most discriminative feature extraction for skin cancer remains an important research problem that can further be used to better deep learning training. Furthermore, the availability of the limited available images also creates a problem for deep learning models. From this line of research, we propose an intelligent Region of Interest (ROI) based system to identify and discriminate melanoma with nevus cancer by using the transfer learning approach. An improved k-mean algorithm is used to extract ROIs from the images. These ROI based approach helps to identify discriminative features as the images containing only melanoma cells are used to train system. We further use a Convolutional Neural Network (CNN) based transfer learning model with data augmentation for ROI images of DermIS and DermQuest datasets. The proposed system gives 97.9% and 97.4% accuracy for DermIS and DermQuest respectively. The proposed ROI based transfer learning approach outperforms existing methods that use complete images for classification.

The classification of whole slide images (WSIs) provides physicians with an accurate analysis of diseases and also helps them to treat patients effectively. The classification can be linked to further detailed analysis and diagnosis. Deep learning (DL) has made significant advances in the medical industry, including the use of magnetic resonance imaging (MRI) scans, computerized tomography (CT) scans, and electrocardiograms (ECGs) to detect life-threatening diseases, including heart disease, cancer, and brain tumors. However, more advancement in the field of pathology is needed, but the main hurdle causing the slow progress is the shortage of large-labeled datasets of histopathology images to train the models. The Kimia Path24 dataset was particularly created for the classification and retrieval of histopathology images. It contains 23,916 histopathology patches with 24 tissue texture classes. A transfer learning-based framework is proposed and evaluated on two famous DL models, Inception-V3 and VGG-16. To improve the productivity of Inception-V3 and VGG-16, we used their pre-trained weights and concatenated these with an image vector, which is used as input for the training of the same architecture. Experiments show that the proposed innovation improves the accuracy of both famous models. The patch-to-scan accuracy of VGG-16 is improved from 0.65 to 0.77, and for the Inception-V3, it is improved from 0.74 to 0.79.

Scalability and ease of implementation make Peer-to-Peer (P2P) infrastructure an attractive option for live video streaming. Peer end-users or peers in these networks have extremely complex features and exhibit unpredictable behavior, i.e. any peer may join or exit the network without prior notice. Peers' dynamics is considered one of the key problems impacting the Quality of Service (QoS) of the P2P based IPTV services. Since, peer dynamics results in video disruption to consumer peers, for smooth video distribution, stable peer identification and selection is essential. Many research works have been conducted on stable peer identification using classical statistical methods. In this paper, a model based on machine learning is proposed in order to predict the length of a user session on entering the network. This prediction can be utilized in topology management such as offloading the departing peer before its exit. Consequently, this will help peers to select stable provider peers, which are the ones with longer session duration. Furthermore, it will also enable service providers to identify stable peers in a live video streaming network. Results indicate that the SVR based model performance is superior to an existing Bayesian network model.

Deep Convolutional Neural Network (DCNN) is widely used as state-of-the art models for image classification. There is a variety of applications related to image classification such as object classification, facial expression classification, and scene classification. In this paper, different activation functions are evaluated for facial expression recognition (FER). The activation functions used for evaluation are Rectified Linear Unit (ReLU), Leaky Rectified Linear Unit (Leaky ReLU), Hyperbolic Tangent Tanh, and Sigmoid. Experiments are conducted on a benchmark dataset known as Fer2013 which is publicly available on Kaggle. Our experiments show that Tanh achieved better performance compared to other activation functions.

Students' feedback is crucial for academic institutions in order to evaluate faculty performance. Handling the qualitative opinions of students efficiently while automatic report generation is a challenging task. Indeed, most organizations deal with quantitative feedback effectively, whereas qualitative feedback is either processed manually or ignored altogether. This study proposes a supervised aspect based opinion mining system based on two-layered LSTM model. The first layer predicts the aspects described within the feedback and later specifies the orientation (positive, negative, and neutral) of those predicted aspects. The model was tested on a manually tagged data set constructed from the last five years students' comments from Sukkur IBA University as well as on a standard SemEval-2014 data set. Unlike many other LSTM models proposed for other domains, the proposed model is quite simple in terms of architecture which results in less complexity. The system attains a good accuracy using the domain embedding layer in both tasks: aspect extraction (91%) and sentiment polarity detection (93%). To the best of our knowledge, this study is a first attempt that uses deep learning approach for performing aspect based sentiment analysis on students' feedback for evaluating faculty teaching performance.

In the recent few years, a lot of advancement has been made in Urdu linguistics. There are many portals and news websites that are generating a huge amount of data every day. However, there is still no publicly available dataset nor any framework available for automatic Urdu extractive summary generation. In an automatic extractive summary generation, the sentences with the highest weights are given importance to be included in the summary. The sentence weight is computed by the sum of the weights of the words in the sentence. There are two famous approaches to compute the weight of the words in the English language: local weights (LW) approach and global weights (GW) approach. The sensitivity of the weights depends on the contents of the text, the one word may have different weights in a different article, known as LW based approach. Whereas, in the case of GW, the weights of the words are computed from the independent dataset, which implies the weights of all words remain the same in different articles. In the proposed framework, LW and GW based approaches are modeled for the Urdu language. The sentence weight method and the weighted term-frequency method are LW based approaches that compute the weights of the sentences by the sum of important words and the sum of frequencies of the important words, respectively. Whereas, vector space model (VSM) is GW based approach, that computes the weight of the words from the independent dataset, and then remain the same for all types of the text; GW is widely used in the English language for various applications such as information retrieval and text classification. The extractive summaries are generated by LW and GW based approaches and evaluated with ground-truth summaries that are obtained by the experts. The VSM is used as a baseline framework for sentence weighting. Experiments show that LW based approaches are better for extractive summary generation. The F-score of the sentence weight method and the weighted term-frequency method are 80% and 76%, respectively. The VSM achieved only 62% accuracy on the same dataset. Both, the datasets with ground-truth, and the code are made publicly available for the researchers.

In this paper, the performance of Facial Expression Recognition (FER) using Deep Convolutional Neural Network (DCNN) model is evaluated. The expressions include Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral. In addition to performance evaluation, the analysis on Interclass false positives are also discussed which helps to analyze the underlying challenges to improve the model. All classifiers give low performance on Fer2013 datasets. DCNN gives 54.46% accuracy on test and 89.52% on training set, whereas, in case of different kernels of Support Vector Machines (SVM), the highest accuracy is 45% using cubic kernel on training set. Experiments show that certain facial expressions have more false positives and few of them are very dominant. In case of Disgust expression, it has Angry as dominant false positive. Based on false positives analysis, binary classifiers can be trained to improve the accuracy of the expressions with dominant false positives. Experiments on Fer2013 dataset confirms that the accuracy of expression Disgust is improved by piping the binary classifiers, Disgust vs. Angry, to existing DCNN of multi-class.

The usefulness and importance of sentiment analysis task is a widely discussed and effective technique in e-commerce. E-commerce is a very convenient way to buy things online. It saves a lot of time that is usually spent traveling and buying by visiting the shops. E-commerce provides an efficient and effective way to shop sitting right in front of one’s computer/mobile at home. For a given product, sentiment analysis captures the users views; their feelings and opinion related to that product. The reviews are categorized into three basic classes i.e. negative, positive, and neutral. This paper focuses on Urdu Roman reviews that are obtained by one of the most famous and accessed e-commerce website of Pakistan–Daraz.pk. There are total 20.286 K reviews which are annotated into three classes by three different experts. Vector space model, a.k.a bag of word model is applied for feature extraction which are later passed to Support Vector Machines (SVM) for sentiment classification. Experiments are conducted on MATLAB Linux server. The dataset is kept public for future use and experiments.

Keypoint based descriptors are widely used for various computer vision applications. During this process, key-points are initially detected from the given images which are later represented by some robust and distinctive descriptors like scale-invariant feature transform (SIFT). Keypoint based image-to-image matching has gained significant accuracy for image retrieval type of applications like image copy detection, similar image retrieval and near duplicate detection. Local keypoint descriptors are quantized into visual words to reduce the feature space which makes image-to-image matching possible for large scale applications. Bag of visual word quantization makes it efficient at the cost of accuracy. In this paper, the bag of visual word model is extended to detect frequent pair of visual words which is known as frequent item-set in text processing, also called visual phrases. Visual phrases increase the accuracy of image retrieval without increasing the vocabulary size. Experiments are carried out on benchmark datasets that depict the effectiveness of proposed scheme.

Support Vector Machines (SVMs) have shown bet-ter generalization and classification capabilities in different appli-cations of computer vision; SVM classifies underlying data by a hyperplane that can separate the two classes by maintaining the maximum margin between the support vectors of the respective classes. An empirical analysis of SVMs on the facial expression recognition task is reported with high intra and low inter class variations by conducting an extensive set of experiments on a large-scale Fer 2013 dataset. Three different kernel functions of SVM are used; linear kernel, quadratic kernel and cubic kernel, whereas, Histogram of Oriented Gradient (HoG) is used as a feature descriptor. Cubic Kernel achieves highest accuracy on Fer 2013 dataset using HoG.

Video segmentation into shots is the first step for video indexing and searching. Videos shots are mostly very small in duration and do not give meaningful insight of the visual contents. However, grouping of shots based on similar visual contents gives a better understanding of the video scene; grouping of similar shots is known as scene boundary detection or video segmentation into scenes. In this paper, we propose a model for video segmentation into visual scenes using bag of visual word (BoVW) model. Initially, the video is divided into the shots which are later represented by a set of key frames. Key frames are further represented by BoVW feature vectors which are quite short and compact compared to classical BoVW model implementations. Two variations of BoVW model are used: (1) classical BoVW model and (2) Vector of Linearly Aggregated Descriptors (VLAD) which is an extension of classical BoVW model. The similarity of the shots is computed by the distances between their key frames feature vectors within the sliding window of length L, rather comparing each shot with very long lists of shots which has been previously practiced, and the value of L is 4. Experiments on cinematic and drama videos show the effectiveness of our proposed framework. The BoVW is 25000-dimensional vector and VLAD is only 2048-dimensional vector in the proposed model. The BoVW achieves 0.90 segmentation accuracy, whereas VLAD achieves 0.83.

Image segmentation helps computer to understand visual information. A number of techniques are proposed for image segmentation that are either interactive or automatic. Interactive techniques provide satisfactory result but user interaction is the biggest limitation when large number of images are considered. On the other hand automatic image segmentation is difficult to acquire satisfactory results. We propose an unsupervised approach for automatic image segmentation that employ image color information to segment an image into foreground and background. The results confirm the effectiveness of the approach.

Image representation by set of local features are common and also state-of-the art for many applications such as image retrieval and image classification. A single image contains on average 2.5 k–3.0 k features. Searching the images based on local features are discriminative compared to global features at the cost of heavy computational overhead. Bag-of-Features (BoF), also known as bag-of-visual words, are used for feature quantization which makes searching local features feasible in very large databases at the cost of distinctiveness. Mostly, the vocabulary size in those applications is kept up-to 1 million. In this research study, we investigated the performance of Vector of Locally Aggregated Descriptors (VLAD) which is recently proposed as an alternative to BoF for different families of descriptor. The VLAD achieves similar or sometimes better performance when compared to BoF despite of limited vocabulary size. The performance of VLAD is mostly compared with BoF on gradient based descriptors in literature. In our experiments, we take gradient based descriptor, intensity based descriptor, and binary descriptor. Scale Invariant Feature Transform (SIFT), Local Intensity Order Pattern (LIOP) and BInarization of Gradient Orientation Histograms (BIGOH) are used to validate the performance of VLAD in parallel to BoF on famous benchmark dataset. VLAD outperforms BoF in gradient based family and intensity based family but non of these are feasible for binary descriptors.

Convolutional Neural Network (NN) has gained a lot of attention of the researchers due to its high accuracy in classification and feature learning. In this paper, we evaluated the performance of CNN used as feature for image retrieval with the gold standard feature, aka SIFT. Experiments are conducted on famous Oxford 5k data-set. The mAP of SIFT and CNN is 0.6279 and 0.5284, respectively. The performance of CNN is also compared with bag of visual word (BoVW) model. CNN achieves better accuracy than BoVW.

Musical sequences with actors dancing and lip-synching to songs sung by playback singers are integral parts, particularly of South Asian movies. Fans seek out movies for their songs and they often seek songs of a particular genre. In fact, song and dance sequence of South Asian movies are an industry of their own. Given the huge numbers of movies produced in South Asia over the past decades, most of which are in digital archives, it is an important problem to automatically extract and categorise their musical sequences. This paper proposes a system for musical sequences extraction from movies. Our method invokes an SVM-based classifier and makes as well a novel application of probabilistic timed automaton to distinguish musical sequences from non-musical. Our system analyses both audio and video signals to give a classifier that not only extracts musical sequences from movies but identifies their genre. We achieved a recall of 93.24% with precision of 87.34% in song extraction when applied on 10 popular Bollywood movies. An accuracy of 89.5% has been achieved on Bollywood song genre identification.

Images have become main sources for the informa-tion, learning, and entertainment, but due to the advancement and progress in multimedia technologies, millions of images are shared on Internet daily which can be easily duplicated and redistributed. Distribution of these duplicated and transformed images cause a lot of problems and challenges such as piracy, redundancy, and content-based image indexing and retrieval. To address these problems, copy detection system based on local features are widely used. Initially, keypoints are detected and represented by some robust descriptors. The descriptors are computed over the affine patches around the keypoints, these patches should be repeatable under photometric and geometric transformations. However, there exist two main challenges with patch based descriptors, (1) the affine patch over the keypoint can produce similar descriptors under entirely different scene or the context which causes “ambiguity”, and (2) the descriptors are not enough “distinctive” under image noise. Due to these limitations, the copy detection systems suffer in performance. We present a framework that makes descriptor more distinguishable and robust by influencing them with the texture and gradients in vicinity. The experimental evaluation on keypoints matching and image copy detection under severe transformations shows the effectiveness of the proposed framework.

Image segmentation is one of the most significant tasks in computer vision. Since automatic techniques are hard for this purpose, a number of interactive techniques are used for image segmentation. The result of these techniques largely depends on user feedback. It is difficult to get good interactions for large databases. On the other hand, automatic image segmentation is becoming a significant objective in computer vision and image analysis. We propose an automatic framework to detect foreground. We are applying Maximal Similarity Based Region Merging (MSRM) technique for region merging and using image boundary to identify foreground regions. The results confirm the effectiveness of the proposed framework. The proposed framework reveals its effectiveness especially to extract multiple objects from background.

Images have become one of the main sources for the information, learning and entertainment; but due to the advancement and progress in multimedia technologies, millions of images are shared daily on Internet which can be easily duplicated and redistributed. Distribution of these duplicated and transformed images causes a lot of problems and challenges such as piracy, redundancy, and content-based image indexing and retrieval. To address these problems, copy detection systems based on local features are widely used. Initially, keypoints are detected and represented by some robust descriptors. The descriptors are computed over the affine patches around the keypoints, these patches should be repeatable under photometric and geometric transformations. However, there exists two main challenges with patch based descriptors, (1) the affine patch over the keypoint can produce similar descriptors under entirely different scene or context which causes ``ambiguity'' (in-distinctiveness), and (2) the descriptors are not enough ``robust'' under image noise. In this paper, we present a framework that makes descriptor more distinguishable and robust by influencing them with the texture or gradients in vicinity by computing them on different and multiple scales. To evaluate the robustness of descriptors, an experiment on keypoint matching under severe transformations is conducted. On average the robustness of SIFT descriptor is increased up-to 12.5\%, and robustness of CSLBP descriptor is increased up-to 31\%. The distinctiveness is evaluated on image copy retrieval experiment where copies of images are retrieved under challenging transformations. On average, the performance of SIFT to retrieve all copies is increased up-to 27.27\%, and the performance of CSLBP to retrieve all copies is increased up-to 27.02\%.

With the advancement in multimedia technologies, the video databases are exponentially increasing in size which are creating many challenges for efficient indexing and retrieval of the videos. Interactive and efficient search engines allow users to query some part of the videos or search some particular scenes in the video databases. Segmentation and retrieval of videos scenes are getting popular which make video indexing more flexible and efficient. In this paper, we automatically segment the videos into scenes (visually related video shots). We first segment the videos into shots and then merge the shots which are visually similar. We represent shots by bag of visual words (BoVW) model and compute the similarity of shots with each other within the sliding window of length L, sliding window makes similarity computation efficient as the similarity of each shot is computed with its L neighbors only, instead of whole pool of shots. Experiments on cinematic videos and dramas show the effectiveness of proposed technique.

Image copy detection is widely used for many applications such as content based image retrieval, image piracy detection, object retrieval, and near duplicate detection. Local features are mostly used to represent the images in database corpus. Initially, keypoints are detected and represented by some distinctive and robust descriptors. The descriptors are computed from the affine local patches around the keypoints. These patches play vital roles for descriptors performance. The main limitation of state-of-the art descriptors include lack of robustness and distinctiveness under severe image distortions, and local patches around keypoints cannot capture distinctive spatial information and structure context. We propose an effective technique for descriptors computation for image copy detection by adding more spatial information from the vicinity of the keypoints. We show experimentally that the performance for retrieving image copies is improved under severe image distortions and attacks. On average, the robustness of SIFT is increased upto 22% and distinctiveness upto 30% for image copy detection task.

Center Symmetric-Local Binary Pattern (CSLBP) is textured based operator which is mostly used as keypoint descriptor, it is 256-length descriptor to represent single keypoint or affine patch. This operator is an extension of Local Binary Pattern (LBP) operator. The CSLBP descriptor is computationally simple, effective, and robust for various image transformations such as illumination change and image blurring. However, the space and time utilization of CSLBP can be improved by simple compression which can make CSLBP a smart selection for large databases and smart phones. In this paper, we propose simple compression of CSLBP without loss of its discriminative power. We reduce the descriptor length (dimensions) upto 50% without applying any dimensionality reduction techniques such as PCA or LDA. We evaluate our framework on state-of-the-art matching protocols and compare the effectiveness of proposed compressed descriptor (Q-CSLBP) with CSLBP, SIFT and PCA-SIFT.

In this paper we present a framework for video segmentation into scenes. Segmenting videos into scenes is the basic step for video analysis, efficient video indexing and content-based video retrieval. In our framework we used frame entropy and SURF descriptor to find shot boundaries from the videos. We extracted key frames from each shot and segmented the video into semantic scenes by key frame matching. The proposed algorithm gave promising results when applied to different genres of videos and dramas.

Video shot segmentation is an important step in key frame selection, video copy detection, video summarization, and video indexing for retrieval. Although some types of video data, e.g., live sports coverage, have abrupt shot boundaries that are easy to identify using simple heuristics, it is much more difficult to identify shot boundaries in other types such as cinematic movies. We propose an algorithm for shot boundary detection able to accurately identify not only abrupt shot boundaries, but also the fade-in and fade-out boundaries typical of cinematic movies. The algorithm is based on analysis of changes in the entropy of the gray scale intensity over consecutive frames and analysis of correspondences between SURF features over consecutive frames. In an experimental evaluation on the TRECVID-2007 shot boundary test set, the algorithm achieves substantial improvements over state of the art methods, with a precision of 97.8% and a recall of 99.3%.

Rapid increase in video databases has forced the industry to have efficient and effective frameworks for video retrieval and indexing. Video segmentation into scenes is widely used for video summarization, partitioning, indexing and retrieval. In this paper, we propose a framework for scene detection mainly based on entropy and Speeded Up Robust Features (SURF) features. First, we detect the fade and abrupt boundaries based on frame entropy analysis and SURF features matching. Fade boundaries are smart indication of scenes beginning or ending in many videos and dramas, and are detected by frame entropy analysis. Before abrupt boundary detection, unnecessary frames which are obviously not abrupt boundaries, such as blank screens, high intensity influenced images, sliding credits, are removed. Candidate boundaries are detected to make SURF features efficient for abrupt boundary detection, and SURF features between candidate boundaries and their adjacent frames are used to detect the abrupt boundaries. Second, key frames are extracted from abrupt shots. We evaluate our key frame extraction with other famous algorithms and show the effectiveness of the key frames. Finally, scene boundaries are detected using sliding window of size K over the key frames in temporal order. In experimental evaluation on the TRECVID-2007 shot boundary test set, the algorithm for shot boundary achieves substantial improvements over state-of-the-art methods with the precision of 99% and the recall of 97.8%. Experimental results for video segmentation into scenes are also promising, compared to famous state-of-the-art techniques.