Search

Large increase in audio, video and digital data in the internet signifies the importance of video annotation techniques. This paper mainly deals with the development of a hybrid algorithm for automatic Video Annotation (VA). Another aim in developing the algorithm is to improve the performance and precision as well as to reduce the amount of time required to obtain the annotations. The overall process leads to the development of efficient techniques for shot detection followed by two level key frame extractions and saliency based residual approach for feature extraction. For all the stages in VA like shot detection, keyframe extraction and feature extraction, factors relating to improve the performance are addressed here. The combination of color histogram difference (CBD) and Edge change ratio (ECR) is used here; as these two are the most promising techniques in shot detection. The new idea is proposed to fine tune the keyframe extraction, which extracts keyframe in two levels. At first level, the first frame in the shot is considered as a keyframe. But to remove redundancy, it enters into second level and finds the optimal set of keyframes by using fuzzy c-means clustering technique. Colour and texture features are used for feature extraction. Here the Video annotation process is divided into two sections, training and testing. The weight vector is found in training stage. Based on this feature vector, the similarity array is calculated in testing phase which further finds corrected annotations. The proposed method is compared with OMG-SSL and MMT-MGO and results are found better on Trechvid dataset. The significance of using weight vector is also experimentally shown here. [ABSTRACT FROM AUTHOR]

Image annotation aims at predicting labels that can accurately describe the semantic information of images. In the past few years, many methods have been proposed to solve the image annotation problem. However, the predicted labels of the images by these methods are usually incomplete, insufficient and noisy, which is unsatisfactory. In this paper, we propose a new method denoted as 2PKNN-GSR (Group Sparse Reconstruction) for image annotation and label refinement. First, we get the predicted labels of the testing images using the traditional method, i.e., a two-step variant of the classical K-nearest neighbor algorithm, called 2PKNN. Then, according to the obtained labels, we divide the K nearest neighbors of an image in the training images into several groups. Finally, we utilize the group sparse reconstruction algorithm to refine the annotated label results which are obtained in the first step. Experimental results on three standard datasets, i.e., Corel 5K, IAPR TC12 and ESP Game, show the superior performance of the proposed method compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Computer vision and natural language processing have been some of the long-standing challenges in artificial intelligence. In this paper, we explore a generative automatic image annotation model, which utilizes recent advances on both fronts. Our approach makes use of a deep-convolutional neural network to detect image regions, which later will be fed to recurrent neural network that is trained to maximize the likely-hood of the target sentence description of the given image. During our experimentation we found that better accuracy and training was achieved when the image representation from our detection model is coupled with the input word embedding, we also found out most of the information from the last layer of detection model vanishes when it is fed as thought vector for our LSTM decoder. This is mainly because the information within the last fully connected layer of the YOLO model represents the class probabilities for the detected objects and their bounding box and this information is not rich enough. We trained our model on coco benchmark for 60 h on 64,000 training and 12,800-validation dataset achieving 23% accuracy. We also realized a significant training speed drop when we changed the number of hidden units in the LSTM layer from 1470 to 4096. [ABSTRACT FROM AUTHOR]

Purpose: To describe and evaluate a segmentation method using joint adversarial and segmentation convolutional neural network to achieve accurate segmentation using unannotated MR image datasets. Theory and Methods: A segmentation pipeline was built using joint adversarial and segmentation network. A convolutional neural network technique called cycle‐consistent generative adversarial network (CycleGAN) was applied as the core of the method to perform unpaired image‐to‐image translation between different MR image datasets. A joint segmentation network was incorporated into the adversarial network to obtain additional functionality for semantic segmentation. The fully automated segmentation method termed as SUSAN was tested for segmenting bone and cartilage on 2 clinical knee MR image datasets using images and annotated segmentation masks from an online publicly available knee MR image dataset. The segmentation results were compared using quantitative segmentation metrics with the results from a supervised U‐Net segmentation method and 2 registration methods. The Wilcoxon signed‐rank test was used to evaluate the value difference of quantitative metrics between different methods. Results: The proposed method SUSAN provided high segmentation accuracy with results comparable to the supervised U‐Net segmentation method (most quantitative metrics having P > 0.05) and significantly better than a multiatlas registration method (all quantitative metrics having P < 0.001) and a direct registration method (all quantitative metrics having P< 0.0001) for the clinical knee image datasets. SUSAN also demonstrated the applicability for segmenting knee MR images with different tissue contrasts. Conclusion: SUSAN performed rapid and accurate tissue segmentation for multiple MR image datasets without the need for sequence specific segmentation annotation. The joint adversarial and segmentation network and training strategy have promising potential applications in medical image segmentation. [ABSTRACT FROM AUTHOR]

Automatic image caption generation aims to produce an accurate description of an image in natural language automatically. How- ever, Bangla, the fifth most widely spoken language in the world, is lagging considerably in the research and development of such domain. Besides, while there are many established data sets related to image annotation in English, no such resource exists for Bangla yet. Hence, this paper outlines the development of "Chittron", an automatic image captioning system in Bangla. To address the data set availability issue, a collection of 16, 000 Bangladeshi contextual images has been accumulated and manually annotated in Bangla. This data set is then used to train a model that integrates a pre-trained VGG16 image embedding model with stacked LSTM layers. The model is trained to predict the caption when the input is an image, one word at a time. The results show that the model has successfully been able to learn a working language model and to generate captions of images quite accurately in many cases. The results are evaluated mainly qualitatively. However, BLEU scores are also reported. It is expected that a better result can be obtained with a bigger and more varied data set. [ABSTRACT FROM AUTHOR]

Image annotation aims to annotate a given image with a variable number of class labels corresponding to diverse visual concepts. In this paper, we address two main issues in large-scale image annotation: 1) how to learn a rich feature representation suitable for predicting a diverse set of visual concepts ranging from object, scene to abstract concept and 2) how to annotate an image with the optimal number of class labels. To address the first issue, we propose a novel multi-scale deep model for extracting rich and discriminative features capable of representing a wide range of visual concepts. Specifically, a novel two-branch deep neural network architecture is proposed, which comprises a very deep main network branch and a companion feature fusion network branch designed for fusing the multi-scale features computed from the main branch. The deep model is also made multi-modal by taking noisy user-provided tags as model input to complement the image input. For tackling the second issue, we introduce a label quantity prediction auxiliary task to the main label prediction task to explicitly estimate the optimal label number for a given image. Extensive experiments are carried out on two large-scale image annotation benchmark datasets, and the results show that our method significantly outperforms the state of the art. [ABSTRACT FROM AUTHOR]

Highlights • Deep map network is proposed to efficiently approximate deep kernel network. • Unsupervised fine-tuning algorithm is proposed for deep map network. • The results on the three datasets show their effectiveness and efficiency. Abstract Deep kernel learning aims at designing nonlinear combinations of multiple standard elementary kernels by training deep networks. This scheme has proven to be effective, but intractable when handling large-scale datasets especially when the depth of the trained networks increases; indeed, the complexity of evaluating these networks scales quadratically w.r.t. the size of training data and linearly w.r.t. the depth of the trained networks. In this paper, we address the issue of efficient computation in Deep Kernel Networks (DKNs) by designing effective maps in the underlying Reproducing Kernel Hilbert Spaces (RKHS). Given a pretrained DKN, our method builds its associated Deep Map Network (DMN) whose inner product approximates the original network while being far more efficient. The design principle of our method is greedy and achieved layer-wise, by finding maps that approximate DKNs at different (input, intermediate and output) layers. This design also considers an extra fine-tuning step based on unsupervised learning, that further enhances the generalization ability of the trained DMNs. When plugged into SVMs, these DMNs turn out to be as accurate as the underlying DKNs while being at least an order of magnitude faster on large-scale datasets, as shown through extensive experiments on the challenging ImageCLEF, COREL5k benchmarks and the Banana dataset. [ABSTRACT FROM AUTHOR]

• This work proposes a hybrid deep learning (DL) based optimized image annotation model for AIA. • The feature dimensionality is reduced by selecting features based on the red colobuses monkey bionic model and levy and mutation operators. • The adaptive image annotation is obtained using the deep optimized convolution residual image annotation model. The advancements in Internet and Web 2.0 technologies resulted in the easier generation and distribution of digital images. Many images are stored without any form of significant labelling. This makes searching for desired images tedious and time-consuming, resulting in an increased demand for an effective image retrieval technique. Image annotation provides one or more labels to an image that describes its content. These annotations greatly help to retrieve images from large databases using different related queries. Automatic Image Annotation (AIA) applications in computer vision, data mining and web search are increasing progressively. This work proposes a hybrid Deep Learning (DL) based optimized image annotation model for AIA. Initially, the pre-processing of an input image is carried out to remove the image noise using the triple Gaussian guided filter. In the second stage, the image's features like shape, colour and texture are extracted using Slantlet transform, YCrCb Colour space and Local Binary Pattern (LBP). The feature dimensionality is reduced by selecting features based on the adaptive red colobuses monkey bionic model. Finally, the adaptive image annotation is obtained using the deep-optimized convolutional residual image annotation model. The loss function reduction and updating of weight parameters are facilitated using the hunger game bionic model. The experiments are performed in Python using Corel-5 K, ESP-Game and MIR FLICKR datasets. The proposed model attains an overall precision of 56, 99.2 and 90%, respectively. Thus, the experimental evaluation results demonstrate the excellent performance of a proposed model over existing approaches. [ABSTRACT FROM AUTHOR]

Over the last several years, in parallel with the general global advancement in mobile technology and a rise in social media network content consumption, multimedia content production and reproduction has increased exponentially. Therefore, enabled by the rapid recent advancements in deep learning technology, research on scene graph generation is being actively conducted to more efficiently search for and classify images desired by users within a large amount of content. This approach lets users accurately find images they are searching for by expressing meaningful information on image content as nodes and edges of a graph. In this study, we propose a scene graph generation method based on using the Resource Description Framework (RDF) model to clarify semantic relations. Furthermore, we also use convolutional neural network (CNN) and recurrent neural network (RNN) deep learning models to generate a scene graph expressed in a controlled vocabulary of the RDF model to understand the relations between image object tags. Finally, we experimentally demonstrate through testing that our proposed technique can express semantic content more effectively than existing approaches.

Since most computer vision approaches are now driven by machine learning, the current bottleneck is the annotation of images. This time-consuming task is usually performed manually after the acquisition of images. In this article, we assess the value of various egocentric vision approaches in regard to performing joint acquisition and automatic image annotation rather than the conventional two-step process of acquisition followed by manual annotation. This approach is illustrated with apple detection in challenging field conditions. We demonstrate the possibility of high performance in automatic apple segmentation (Dice 0.85), apple counting (88 percent of probability of good detection, and 0.09 true-negative rate), and apple localization (a shift error of fewer than 3 pixels) with eye-tracking systems. This is obtained by simply applying the areas of interest captured by the egocentric devices to standard, non-supervised image segmentation. We especially stress the importance in terms of time of using such eye-tracking devices on head-mounted systems to jointly perform image acquisition and automatic annotation. A gain of time of over 10-fold by comparison with classical image acquisition followed by manual image annotation is demonstrated.