Search

Content Based Image Retrieval is ever growing technology for many applications such as medical, remote sensing, social media search engines and surveillance monitoring, etc,. Representing image content with appropriate features is tedious task using traditional low level feature extraction methods. Deep learning models achieved high precision in classification and object detection algorithms by extracting automated high level feature extraction process. This paper proposed a hybrid feature extraction technique by combining the high level features and low level features to improve the robustness of the feature vector. The proposed model used pre-trained Googlenet model as feature extractor and combined with Gabor multiscale texture features. The final feature vector will be used for retrieving the relevant image data from the large scale image dataset. It has achieved the precision of 91 percent which shows better than state of art methods.

Question answering (QA)-based re-ranking methods for cross-modal retrieval have been recently proposed to further narrow down similar candidate images. The conventional QA-based re-ranking methods provide questions to users by analyzing candidate images, and the initial retrieval results are re-ranked based on the user's feedback. Contrary to these developments, only focusing on performance improvement makes it difficult to efficiently elicit the user's retrieval intention. To realize more useful QA-based re-ranking, considering the user interaction for eliciting the user's retrieval intention is required. In this paper, we propose a QA-based re-ranking method with considering two important factors for eliciting the user's retrieval intention: query-image relevance and recallability. Considering the query-image relevance enables to only focus on the candidate images related to the provided query text, while, focusing on the recallability enables users to easily answer the provided question. With these procedures, our method can efficiently and effectively elicit the user's retrieval intention. Experimental results using Microsoft Common Objects in Context and computationally constructed dataset including similar candidate images show that our method can improve the performance of the cross-modal retrieval methods and the QA-based re-ranking methods.

In this paper, we consider the problem of descriptors construction for the task of content-based image retrieval using deep neural networks. The idea of neural codes, based on fully connected layers activations, is extended by incorporating the information contained in convolutional layers. It is known that the total number of neurons in the convolutional part of the network is large and the majority of them have little influence on the final classification decision. Therefore, in the paper we propose a novel algorithm that allows us to extract the most significant neuron activations and utilize this information to construct effective descriptors. The descriptors consisting of values taken from both the fully connected and convolutional layers perfectly represent the whole image content. The images retrieved using these descriptors match semantically very well to the query image, and also they are similar in other secondary image characteristics, like background, textures or color distribution. These features of the proposed descriptors are verified experimentally based on the IMAGENET1M dataset using the VGG16 neural network., Comment: 8

Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) aims at searching corresponding natural images with the given free-hand sketches, under the more realistic and challenging scenario of Zero-Shot Learning (ZSL). Prior works concentrate much on aligning the sketch and image feature representations while ignoring the explicit learning of heterogeneous feature extractors to make themselves capable of aligning multi-modal features, with the expense of deteriorating the transferability from seen categories to unseen ones. To address this issue, we propose a novel Transferable Coupled Network (TCN) to effectively improve network transferability, with the constraint of soft weight-sharing among heterogeneous convolutional layers to capture similar geometric patterns, e.g., contours of sketches and images. Based on this, we further introduce and validate a general criterion to deal with multi-modal zero-shot learning, i.e., utilizing coupled modules for mining modality-common knowledge while independent modules for learning modality-specific information. Moreover, we elaborate a simple but effective semantic metric to integrate local metric learning and global semantic constraint into a unified formula to significantly boost the performance. Extensive experiments on three popular large-scale datasets show that our proposed approach outperforms state-of-the-art methods to a remarkable extent: by more than 12% on Sketchy, 2% on TU-Berlin and 6% on QuickDraw datasets in terms of retrieval accuracy. The project page is available at: https://haowang1992.github.io/publication/TCN.

To ensure the security of images outsourced to the malicious cloud without affecting searchability on such outsourced (typically encrypted) images, one could use privacy-preserving Content-Based Image Retrieval (CBIR) primitive. However, conventional privacy-preserving CBIR schemes based on Searchable Symmetric Encryption (SSE) are not capable of supporting efficient fine-grained access control and result verification simultaneously. Therefore, in this paper, we propose a Verifiable Fine-grained encrypted Image Retrieval scheme in the Multi-owner multi-user settings (VFIRM). VFIRM first utilizes a novel polynomial-based access strategy to provide efficient fine-grained access control. Then, it employs the dual secure k-nearest neighbor technique to distribute distinct keys to different data owners and data users, and finally implements an adapted homomorphic MAC technique to check the correctness of search results. Our formal security analysis shows that VFIRM is non-adaptive semantic secure if the clients search key is generated randomly and keeps in secret. Our empirical experiments using two real-world datasets (i.e., Caltech101 and Corel5k) demonstrate the practicality of VFIRM.

Cross-modal retrieval has recently attracted growing attention, which aims to match instances captured from different modalities. The performance of cross-modal retrieval methods heavily relies on the capability of metric learning to mine and weight the informative pairs. While various metric learning methods have been developed for unimodal retrieval tasks, the cross-modal retrieval tasks, however, have not been explored to its fullest extent. In this paper, we develop a universal weighting metric learning framework for cross-modal retrieval, which can effectively sample informative pairs and assign proper weight values to them based on their similarity scores so that different pairs favor different penalty strength. Based on this framework, we introduce two types of polynomial loss for cross-modal retrieval, self-similarity polynomial loss and relative-similarity polynomial loss. The former provides a polynomial function to associate the weight values with self-similarity scores, and the latter defines a polynomial function to associate the weight values with relative-similarity scores. Both self and relative-similarity polynomial loss can be freely applied to off-the-shelf methods and further improve their retrieval performance. Extensive experiments on two image-text retrieval datasets, three video-text retrieval datasets and one fine-grained image retrieval dataset demonstrate that our proposed method can achieve a noticeable boost in retrieval performance.

With the advent of cloud computing, explosively increasing images are gradually outsourced to the cloud server for costs saving and feasibility. For security and privacy concerns, images (e.g., medical diagnosis, personal photos) should be encrypted before being outsourced. However, traditional encrypted image retrieval techniques still suffer from costly access control and low search accuracy. To solve these challenging issues, in this paper, we first propose a Controllable encrypted cloud image Search scheme in Multi-user settings (namely CSM) by using the polynomial-based access strategy and proxy re-encryption technique. CSM achieves efficient access control and avoids heavy communication overhead caused by key transmission. Then, we improve the basic CSM to achieve malicious search user Tracing (namely TCSM) by utilizing the watermark technique, which can further prevent search users from illegally redistributing retrieved images to unauthorized search users. Our formal security analysis proves that our CSM (or TCSM) can guarantee the privacy of images, indexes, and search queries. Our empirical experiments using real-world datasets demonstrate the efficiency and high accuracy of our CSM (or TCSM) in practice.

Content-based image retrieval (CBIR) uses the content features for retrieving and searching the images in a given large database. Earlier, different hand feature descriptor designs are researched based on cues that are visual such as shape, colour, and texture used to represent these images. Although, deep learning technologies have widely been applied as an alternative to designing engineering that is dominant for over a decade. The features are automatically learnt through the data. This research work proposes integrated dual deep convolutional neural network (IDD-CNN), IDD-CNN comprises two distinctive CNN, first CNN exploits the features and further custom CNN is designed for exploiting the custom features. Moreover, a novel directed graph is designed that comprises the two blocks i.e. learning block and memory block which helps in finding the similarity among images; since this research considers the large dataset, an optimal strategy is introduced for compact features. Moreover, IDD-CNN is evaluated considering the two distinctive benchmark datasets the oxford dataset considering mean average precision (mAP) metrics and comparative analysis shows IDD-CNN outperforms the other existing model.

Deep hashing methods have shown their superiority to traditional ones. However, they usually require a large amount of labeled training data for achieving high retrieval accuracies. We propose a novel transductive semisupervised deep hashing (TSSDH) method which is effective to train deep convolutional neural network (DCNN) models with both labeled and unlabeled training samples. TSSDH method consists of the following four main ingredients. First, we extend the traditional transductive learning (TL) principle to make it applicable to DCNN-based deep hashing. Second, we introduce confidence levels for unlabeled samples to reduce adverse effects from uncertain samples. Third, we employ a Gaussian likelihood loss for hash code learning to sufficiently penalize large Hamming distances for similar sample pairs. Fourth, we design the large-margin feature (LMF) regularization to make the learned features satisfy that the distances of similar sample pairs are minimized and the distances of dissimilar sample pairs are larger than a predefined margin. Comprehensive experiments show that the TSSDH method can produce superior image retrieval accuracies compared to the representative semisupervised deep hashing methods under the same number of labeled training samples.

This article describes the capability of online data storage which has been enhanced by the emergence of cloud datacenter development. Distributed Hash Table (DHT) based image retrieval system using locality sensitive hash (LSH) has provided an efficient way to set up distributed Content Based Image Retrieval (CBIR) frameworks. However, with the fixed LSH function adopted, LSH and other codebook-based distributed retrieval systems are facing the problem of flexibility, and also are difficult to satisfy the user's demand. In this article, LRFMIR is proposed to introduce semantic search into DHT based CBIR system. LRFMIR is established on a DHT based network, where a flexible result truncating strategy is employed to fuse provided results by using multiple features measurements. Experiments show that LRFMIR provides a higher accuracy and recall rate than single feature employed retrieval systems, and possesses good load balancing and query efficiency performance.

In this article, we propose a novel feature selection approach via explicitly addressing the long-standing subspace sparsity issue. Leveraging l2,1-norm regularization for feature selection is the major strategy in existing methods, which, however, confronts sparsity limitation and parameter-tuning trouble. To circumvent this problem, employing the l2,0-norm constraint to improve the sparsity of the model has gained more attention recently whereas, optimizing the subspace sparsity constraint is still an unsolved problem, which only can acquire an approximate solution and without convergence proof. To address the above challenges, we innovatively propose a novel subspace sparsity discriminative feature selection (S²DFS) method which leverages a subspace sparsity constraint to avoid tuning parameters. In addition, the trace ratio formulated objective function extremely ensures the discriminability of selected features. Most important, an efficient iterative optimization algorithm is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To the best of our knowledge, such an optimization algorithm of solving the subspace sparsity issue is first proposed in this article, and a general formulation of the optimization algorithm is provided for improving the extensibility and portability of our method. Extensive experiments conducted on several high-dimensional text and image datasets demonstrate that the proposed method outperforms related state-of-the-art methods in pattern classification and image retrieval tasks.

Shenzhen Science and Technology Projects under Grant JCYJ20200109143035495; Natural Science Foundation of China under Grant 51975394.

With the emergence of intelligent terminals, the Content-Based Image Retrieval (CBIR) technique has attracted much attention from many areas (i.e., cloud computing, social networking services, etc.). Although existing privacy-preserving CBIR schemes can guarantee image privacy while supporting image retrieval, these schemes still have inherent defects (i.e., low search accuracy, low search efficiency, key leakage, etc.). To address these challenging issues, in this paper we provide a similarity Search for Encrypted Images in secure cloud computing (called SEI). First, the feature descriptors extracted by the Convolutional Neural Network (CNN) model are used to improve search accuracy. Next, an encrypted hierarchical index tree by using K-means clustering based on Affinity Propagation (AP) clustering is devised, which can improve search efficiency. Then, a limited key-leakage k-Nearest Neighbor (kNN) algorithm is proposed to protect key from being completely leaked to untrusted image users. Finally, SEI is extended to further prevent image users' search information from being exposed to the cloud server. Our formal security analysis proves that SEI can protect image privacy as well as key privacy. Our empirical experiments using a real-world dataset illustrate the higher search accuracy and efficiency of SEI.

Most of the deep quantization methods adopt unsupervised approaches, and the quantization process usually occurs in the Euclidean space on top of the deep feature and its approximate value. When this approach is applied to the retrieval tasks, since the internal product space of the retrieval process is different from the Euclidean space of quantization, minimizing the quantization error (QE) does not necessarily lead to a good performance on the maximum inner product search (MIPS). To solve these problems, we treat Softmax classification as vector quantization (VQ) with angular decision boundaries and propose angular deep supervised VQ (ADSVQ) for image retrieval. Our approach can simultaneously learn the discriminative feature representation and the updatable codebook, both lying on a hypersphere. To reduce the QE between centroids and deep features, two regularization terms are proposed as supervision signals to encourage the intra-class compactness and inter-class balance, respectively. ADSVQ explicitly reformulates the asymmetric distance computation in MIPS to transform the image retrieval process into a two-stage classification process. Moreover, we discuss the extension of multiple-label cases from the perspective of quantization with binary classification. Extensive experiments demonstrate that the proposed ADSVQ has excellent performance on four well-known image data sets when compared with the state-of-the-art hashing methods.

In this paper, we proposed a more robust supervised hashing framework based on the Cauchy loss function and Supervised Discrete Hashing (SDH) called Robust Supervised Discrete Hashing (RSDH), which can learn a robust subspace consisted of binary codes. The Cauchy loss is used to measure the error between the label matrix and the product of the decomposed matrices. RSDH can not only reduce the outliers and noise of the hashing codes, but also achieve the more satisfactory retrieval effect. Image retrieval experiments demonstrate that RSDH performs better than the other hashing methods.

In computer-based search systems, similarity plays a key role in replicating the human search process. Indeed, the human search process underlies many natural abilities such as image recovery, language comprehension, decision making, or pattern recognition. The search for images consists of establishing a correspondence between the available image and that sought by the user, by measuring the similarity between the images. Image search by content is generaly based on the similarity of the visual characteristics of the images. The distance function used to evaluate the similarity between images depends notonly on the criteria of the search but also on the representation of the characteristics of the image. This is the main idea of a content-based image retrieval (CBIR) system. In this article, first, we constructed type-2 beta fuzzy membership of descriptor vectors to help manage inaccuracy and uncertainty of characteristics extracted the feature of images. Subsequently, the retrieved images are ranked according to the novel similarity measure, noted type-2 fuzzy nearness measure (IT2FNM). By analogy to Type-2 Fuzzy Logic and motivated by near sets theory, we advanced a new fuzzy similarity measure (FSM) noted interval type-2 fuzzy nearness measure (IT-2 FNM). Then, we proposed three new IT-2 FSMs and we have provided mathematical justification to demonstrate that the proposed FSMs satisfy proximity properties (i.e. reflexivity, transitivity, symmetry, and overlapping). Experimental results generated using three image databases showing consistent and significant results.

Visual localization is a crucial component in the application of mobile robot and autonomous driving. Image retrieval is an efficient and effective technique in image-based localization methods. Due to the drastic variability of environmental conditions, e.g. illumination, seasonal and weather changes, retrieval-based visual localization is severely affected and becomes a challenging problem. In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. We also propose a new adaptive triplet loss to boost the contrastive learning of the embedding in a self-supervised manner. The final coarse-to-fine image retrieval pipeline is implemented as the sequential combination of models without and with Grad-SAM loss. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. The strong generalization ability of our approach is verified on RobotCar dataset using models pre-trained on urban part of CMU-Seasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision, especially under the challenging environments with illumination variance, vegetation and night-time images. The code and pretrained models are available on https://github.com/HanjiangHu/DISAM., Comment: Published in IEEE/CAA Journal of Automatica Sinica

Content-Based Image Retrieval (CBIR) systems have recently emerged as one of the most promising and best image retrieval paradigms. To pacify the semantic gap associated with CBIR systems, the Bag of Visual Words (BoVW) techniques are now increasingly used. However, existing BoVW techniques fail to capture the location information of visual words effectively. This paper proposes an unsupervised Content-Based Medical Image Retrieval (CBMIR) framework based on the spatial matching of the visual words. The proposed method efficiently computes the spatial similarity of visual words using a novel similarity measure called the Skip Similarity Index. Experiments on three large medical datasets reveal promising results. The location-based correlation of visual words assists in more accurate and efficient retrieval of anatomically diverse and multimodal medical images than the state-of-the-art CBMIR systems.

Retrieving out the most comparable images from huge databases is the challenging task for image retrieval systems. So, there is a great need of constructing a capable and rigorous image retrieval system. In this implementation, an exclusive and competent Content based image retrieval (CBIR) system is schemed by the integration of Color moment (CM) and Local binary pattern (LBP). A hybrid feature vector is created by the combination of these two techniques through the process of normalization. This hybrid feature vector is given as the input to the intelligent classifiers i.e. Support vector machine (SVM) and Cascade forward back propagation neural network (CFBPNN). After that, Relevance feedback (RF) technique is applied so as to get the high level information in order to reduce the semantic gap. So, here two Artificial Intelligent CBIR models are proposed, first one is (Hybrid+SVM+RF) and second is (Hybrid+CFBPNN+RF) and their performance parameters are compared. The implementations are performed on two benchmark dataset Corel-1K and Oxford flower dataset which contains 1000 and 1360 images respectively. Different parameters are figured such as accuracy, precision, average retrieval time, recall etc. The average precision obtained for the first model is 93% with Corel 1K database and 91% with Oxford flower database. And similarly for the second model, it is 97% and 94% respectively which is higher than the first model. This implemented technique is validated on both the datasets and the attained results outperforms with other related s approaches.

Due to its powerful capability of representation learning and high-efficiency computation, deep hashing has made significant progress in large-scale image retrieval. It has been recognized that deep neural networks are vulnerable to adversarial examples, which is a practical secure problem but seldom studied in deep hashing-based retrieval field. In this paper, we propose a novel prototype-supervised adversarial network (ProS-GAN), which formulates a flexible generative architecture for efficient and effective targeted hashing attack. To the best of our knowledge, this is one of the first generation-based methods to attack deep hashing networks. Generally, our proposed framework consists of three parts, i.e., a PrototypeNet, a Generator and a Discriminator. Specifically, the designed PrototypeNet embeds the target label into the semantic representation and learns the prototype code as the category-level representative of the target label. Moreover, the semantic representation and the original image are jointly fed into the generator for flexible targeted attack. Particularly, the prototype code is adopted to supervise the generator to construct the targeted adversarial example by minimizing the Hamming distance between the hash code of the adversarial example and the prototype code. Furthermore, the generator fools the discriminator to simultaneously encourage the adversarial examples visually realistic and the semantic representation informative. Extensive experiments demonstrate that the proposed framework can efficiently produce adversarial examples with better targeted attack performance and transferability over state-of-the-art targeted attack methods of deep hashing.

Similarity learning which is useful for the purpose of comparing various characteristics of images in the computer vision field has been often applied for deep metric learning (DML). Also, a lot of combinations of pairwise similarity metrics such as Euclidean distance and cosine similarity have been studied actively. However, such a local similarity-based approach can be rather a bottleneck for a retrieval task in which global characteristics of images must be considered important. Therefore, this paper proposes a new similarity metric structure that considers the local similarity as well as the global characteristic on the representation space, i.e., class variability. Also, based on an insight that better class variability analysis can be accomplished on the Stiefel (or Riemannian) manifold, manifold geometry is employed to generate class variability information. Finally, we show that the proposed method designed through in-depth analysis of generalization bound of DML outperforms conventional DML methods theoretically and experimentally.

Remote sensing image retrieval (RSIR) is the process of searching and acquiring similar images to a query image in a large-scale remote sensing image database. Several recent studies encourage the discriminative feature space by developing loss functions under the deep metric learning paradigm. However, there are still some problems that have not been well resolved. The first is that for each category in the RSIR data set, the similarity among the samples within the category is different, so the samples and the number of samples that need to be learned should also be different. We propose intraclass space sample mining, which selects ``misplaced'' samples based on the distribution of samples, instead of artificially setting the boundary threshold of sample mining as usual. The second problem is the inconsistency between the loss reduction direction and the optimization direction. We propose the cost-sensitive loss (CSL) by penalizing the error samples in the candidate list. As a plug-in, CSL combined with other local losses can effectively improve retrieval accuracy. Finally, in view of the problem that the current loss is optimized based on local space and cannot distinguish different categories with high similarity, we propose a global-aware ranking loss (GRL) model. This is a global optimization model based on the feature space and retrieval candidate list. We have conducted comprehensive experiments on two public remote sensing data sets. Compared with the baseline, our proposed algorithm can achieve state-of-the-art performance.

Recently, hashing has been successfully applied for large-scale remote sensing image retrieval (LSRSIR) due to its advantage in terms of computation and storage. In LSRSIR, existing hashing methods mainly focus on single-source remotely sensed data. They cannot effectively fuse multisource remotely sensed data, which has a large potential for LSRSIR. To fulfill this gap, this letter proposes a novel deep hashing method, dubbed Multiview Graph Convolutional Hashing (MGCH) that can successfully fuse multisource remote sensing image. Since graph convolutional network (GCN) has been applied as an effective means that expresses and integrates relationships into features, MGCH applies a GCN to explore inherent structural similarity among multiview data, which will help to generate discriminative hash codes. An asymmetric scheme is developed that optimizes the proposed deep model in an end-to-end manner to improve training efficiency. We evaluate the proposed method by fusing two different kinds of RS images, i.e., multispectral (MUL) image and panchromatic (PAN) image. The experimental results on the dual-source RS image data set (DSRSID) show that the proposed MGCH outperforms state-of-the-art multiview hashing methods.

Remote sensing (RS) crossmodal text-image retrieval has become a research hotspot in recent years for its application in semantic localization. However, since multiple inferences on slices are demanded in semantic localization, designing a crossmodal retrieval model with less computation but well performance becomes an emergent and challenging task. In this paper, considering the characteristics of multi-scale and target redundancy in RS, a concise but effective crossmodal retrieval model (LW-MCR) is designed. The proposed model incorporates multi-scale information and dynamically filters out redundant features when encoding RS image while text features are obtained via lightweight group convolution. To improve the retrieval performance of LW-MCR, we come up with a novel hidden supervised optimization method based on knowledge distillation. This method enables the proposed model to acquire dark knowledge of the multi-level layers and representation layers in the teacher network, which significantly improves the accuracy of our lightweight model. Finally, on the basis of contrast learning, we present a method employing unlabeled data to boost the performance of RS retrieval model further. The experiment results on four RS image-text datasets demonstrate the efficiency of LW-MCR in RS crossmodal retrieval tasks.

Due to high storage and calculation efficiency, hash-based methods have been widely used in image retrieval systems. Unsupervised deep hashing methods can learn the binary representations of images effectively without any annotations. The strategy of constraining hash code in the previous unsupervised methods may not fully utilize the structural information in semantic similarity. To address this problem, we propose a new strategy based on contrastive learning to capture high-level semantic similarity among features and preserve it in generated hash codes. In addition, we employ a novel framework to handle hash codes with different lengths simultaneously which is more time-saving in generating hash codes than existing methods. Extensive experiments on MIRFlickr, NUS-WIDE, and COCO benchmark datasets show that our method makes great improvement on the performance of unsupervised image retrieval.

Benefiting from low storage costs and high retrieval efficiency, hash learning has been a widely adopted technology for approximating nearest neighbor in large-scale data retrieval. Deep learning to hash greatly improves image retrieval performance by integrating feature learning and hash coding into an end-to-end framework. However, subject to application scope, most existing deep hashing methods only apply to retrieval of complete data and have undesirable results when retrieving incomplete but valuable data. In this paper we propose IDHashGAN, a novel deep hashing model with generative adversarial networks to retrieve incomplete data, in which feature restoration, feature learning and hash coding are integrated into an unified end-to-end framework. The proposed model consists of four key components: (1) reconstructive and generative loss are used to generate continuous feature of incomplete data in generative network; (2) supervised manifold similarity is proposed to improve retrieval accuracy and obtain good user acceptance; (3) adversarial and classified loss are designed to distinguish authenticity and similarity in discriminative network; and (4) encoding and quantization loss are adopted to preserve similarity and control hash quality. Extensive experiments on benchmark datasets show that IDHashGAN is competitive on complete dataset and yields substantial boosts of 70% on incomplete datasets compared to state-of-the-art hashing methods.

Content-based Image Retrieval (CBIR) techniques have been extensively studied with the rapid growth of digital images. Generally, CBIR service is quite expensive in computational and storage resources. Thus, it is a good choice to outsource CBIR service to the cloud server that is equipped with enormous recourses. However, the privacy protection becomes a big problem, as the cloud server cannot be fully trusted. In this paper, we propose an outsourced CBIR scheme based on a novel bag-of-encrypted-words (BOEW) model. The image is encrypted by color value substitution, block permutation, and intra-block pixel permutation. Then, the local histograms are calculated from the encrypted image blocks by the cloud server. All the local histograms are clustered together, and the cluster centers are used as the encrypted visual words. In this way, the bag-of-encrypted-words (BOEW) model is built to represent each image by a feature vector, i.e., a normalized histogram of the encrypted visual words. The similarity between images can be directly measured by the Manhattan distance between feature vectors on the cloud server side. Experimental results and security analysis on the proposed scheme demonstrate its search accuracy and security.

For deep networks, accurate image similarities cannot be well characterized with limited iterations, so the latent relationships between images can be embedded to enhance image retrieval performance. In this article, we propose a method named DFLLR to learn deep image features and accurate image similarities for remote sensing image retrieval (RSIR) simultaneously. First, the AlexNet is employed to extract high-level semantic features. Second, to obtain accurate image similarities, latent relationships between images are constructed with manifold learning and embedded in the AlexNet model with fully connected layers; in this way, the latent relationships and image features can be jointly learned. Third, to boost the RSIR performance further, the constraints of central and margin for jointly learning latent relationships and image features are integrated into our DFLLR. The central constraint is used to reduce the discrepancy of the latent relationships at the intraclass level and enhance the accuracies of image features. Moreover, the margin constraint is designed to enhance the accuracies of the latent relationships by maximizing the manifold margin between the latent relationships at the intraclass and interclass levels. To validate our method, we perform comprehensive experiments on three publicly available remote sensing image datasets, and the results demonstrate that it significantly outperforms other state-of-the-art methods.

The emergence of cloud computing provides new dimension for the user to perform computations and store huge amount of data say images,video,audio etc,. However,the benefits of outsourcing the tasks bring privacy issues for the data that are outsourced. Consequently,to ensure privacy,multimedia data is encrypted and offloaded to the cloud database. Though images are encrypted,during retrieval,cloud server performs similarity computation on plaintext features. Fully holomorphic shows great results in computation over encrypted data yet due to its computation burden it is not applicable for practical usage. Thus,to guarantee the secrecy of outsourced image features,the proposed paper introduced an efficient Secure Similar Image matching (SESIM) protocol under encrypted domain. The computation overhead of the proposed SESIM protocol is compared with the existing secure distance metrics. The experiments and performance analysis show the effectiveness and security of the proposed scheme under encrypted cloud database.

The methods in remote sensing image retrieval (RSIR) usually search the whole retrieval data set in the retrieval process, which takes much time and is unnecessary. To reduce the overall search time, this letter proposes a new retrieval scheme based on fuzzy rules. The proposed method calculates the fuzzy class membership of images using two ways. The first way predicts the fuzzy class membership by convolutional neural network (CNN). The other uses the image-to-class distance that is a distance between an image and each class on the training data set. The two fuzzy class memberships are used to measure the classification confidence, and a query image is classified into three fuzzy sets, namely, ``low classification confidence,'' ``medium classification confidence,'' and ``high classification confidence,'' based on the classification confidence. The fuzzy rules are built according to fuzzy classification to choose the search space for each fuzzy set. The final search space is determined by the two search spaces obtained by fuzzy rules. Moreover, the fuzzy distance between a query image and a retrieved image is used to improve the retrieval performance, which is calculated according to their fuzzy class memberships and the Euclidean distance between the two images. The experimental results on University of California, Merced data set (UCMD) and PatternNet databases show that our proposed method can not only enhance the retrieval performance but also reduce the search time in comparison to other state-of-the-art techniques.

Remote sensing image retrieval (RSIR) has broad application prospects, but related challenges still exist. One of the most important challenges is how to obtain discriminative features. In recent years, although the powerful feature learning ability of convolutional neural networks (CNNs) has significantly improved RSIR, their performance can be restricted by the complexity of remote sensing (RS) images, such as small objects, varying scales, and wide scope. To address these problems, we propose a novel wide-context attention network (W-CAN). It leverages two attention modules to adaptively learn local features correlated in the spatial and channel dimensions, respectively, which can obtain discriminative features with extensive context information. During training, a hybrid loss is introduced to enhance the intraclass compactness and interclass separability of the features. Moreover, we add a branch to learn binary descriptors and realize the end-to-end descriptor aggregation. Experiments on four RS benchmark data sets demonstrate that the proposed method can outperform some state-of-the-art RSIR methods.

Registering the 2D images (2D space) with the 3D model of the environment (3D space) provides a promising solution to outdoor Augmented Reality (AR) virtual-real registration. In this work, we use the position and orientation of the ground camera image to synthesize a corresponding rendered image from the outdoor large-scale 3D image-based point cloud. To achieve the virtual-real registration, we indirectly establish the spatial relationship between 2D and 3D space by matching the above two kinds (2D/3D space) of cross-domain images. However, matching cross-domain images goes beyond the capability of handcrafted descriptors and existing deep neural networks . To address this issue, we propose an end-to-end network, Y-Net, to learn Domain Robust Feature Representations (DRFRs) for the cross-domain images. Besides, we introduce a cross-domain-constrained loss function that balances the loss in image content and cross-domain consistency of the feature representations. Experimental results show that the DRFRs simultaneously preserve the representation of image content and suppress the influence of independent domains. Furthermore, Y-Net outperforms the existing algorithms on extracting feature representations and achieves state-of-the-art performance in cross-domain image retrieval . Finally, we validate the Y-Net-based registration approach on campus to demonstrate its possible applicability.

Hashing techniques are in great demand for a wide range of real-world applications such as image retrieval and network compression. Nevertheless, existing approaches could hardly guarantee a satisfactory performance with the extremely low-bit (e.g., 4-bit) hash codes due to the severe information loss and the shrink of the discrete solution space. In this article, we propose a novel Collaborative Learning strategy that is tailored for generating high-quality low-bit hash codes. The core idea is to jointly distill bit-specific and informative representations for a group of pre-defined code lengths. The learning of short hash codes among the group can benefit from the manifold shared with other long codes, where multiple views from different hash codes provide the supplementary guidance and regularization, making the convergence faster and more stable. To achieve that, an asymmetric hashing framework with two variants of multi-head embedding structures is derived, termed as Multi-head Asymmetric Hashing (MAH), leading to great efficiency of training and querying. Extensive experiments on three benchmark datasets have been conducted to verify the superiority of the proposed MAH, and have shown that the 8-bit hash codes generated by MAH achieve 94.3 percent of the MAP 1 1. Mean Average Precision (MAP) score on the CIFAR-10 dataset, which significantly surpasses the performance of the 48-bit codes by the state-of-the-arts in image retrieval tasks.

Fine-texture Image Retrieval, a special case in Content Based Image Retrieval, has high potential application value in many fields. However, there are very few researches focus on this special case. Due to the difference between fine-texture images and common images, general retrieval methods for common images are difficult to apply to fine-texture image retrieval. It is also a challenging issue with several obstacles: variety and complexity of appearance, as well as high requirements for retrieval accuracy. To address this issue, this study proposes a novel approach, called deep multi-view hashing (DMVH), to learn enhanced hash codes for efficient fine-texture image retrieval. We propose to use the first few layers of a deep convolutional neural network for fine-texture image presentation. To avoid overfitting, we employ L0Linear layer instead of the commonly used Linear layer in all the fully connected layers. Then we introduce a pairwise quantified similarity computed on the semantic labels. And the learning process of the model is guided by multiple labels of images. This study takes fabric as an example, and builds a dataset called MFT-fabric-v1, to validate the effectiveness of the DMVH method. On this data set, the proposed model achieved a performance with a mAP value of o.8735. The area under the pr curve is larger than other comparison methods. The experimental results demonstrate that the proposed method outperforms the state-of-the-art on the MFT-fabric-v1 dataset.

With the increasing adoption of human-computer interaction, there is a growing trend of extracting the image through hand-drawn sketches by humans to find out correlated objects from the storage unit. A review of the existing system shows the dominant use of sophisticated and complex mechanisms where the focus is more on accuracy and less on system efficiency. Hence, this proposed system introduces a simplified extraction of the related image using an attribution clustering process and a cost-effective training scheme. The proposed method uses K-means clustering and bag-of-attributes to extract essential information from the sketch. The proposed system also introduces a unique indexing scheme that makes the retrieval process faster and results in retrieving the highest-ranked images. Implemented in MATLAB, the study outcome shows the proposed system offers better accuracy and processing time than the existing feature extraction technique.

Face recognition is a geometric space recording activity that allows it to be used to distinguish the features of a face. Therefore, facial recognition can be used to identify ID cards, ATM card PINs, search for the perpetrators of theft whose faces were caught by Close-Circuit Television (CCTV), terrorists, and other criminals.  Making an application or tool is very urgent and important to assist law enforcement, such as determining a photo of the face of a suspected criminal. With a tool that can automatically provide or display a suspected face following the desired query. To overcome this, usually, law enforcers will ask for help from people who are good at drawing, especially face painters to sketch the faces of criminals based on several eyewitnesses who have seen the perpetrators of crimes even though only at a glance. Besides that, with sketches like this, it is hoped that it can help artists and witnesses. eyes to improve the sketch made based on the image of the called face (face retrieved). However, there is still a problem in how to sketch a real face image or image of someone. In this study, a digital image search technique (content-based image retrieval) was used to identify criminals, including corruptors. Often the perpetrators of crime take place very quickly and are not caught by CCTV cameras, so the only way to look at them is based on witnesses who saw the incident who had seen the criminal act. One of the problems in face recognition is that we don't have a face image of the criminals, so a face sketch is needed which is then generated it based on the sketch an actual face image is generated. The research will be carried out using no less than 1,000 facial images, both normal and abnormal faces (with obstacles).

Content-Based Image Retrieval (CBIR) is a well-known research topic from the computer vision domain which helps retrieve similar images from a dataset as per the specified query image. The retrieval performance is inadequate for benchmark datasets viz., Corel-1k, Corel-5k, Corel-10k, and Ghim-10k. In this paper, we have encountered the problem of the low retrieval rates of the CBIR system and the high dimensionality of the feature vectors. We have proposed the hybrid framework consisting of three different feature descriptors for robust retrieval performance. We have propounded the use of modified Multi-Scale Local Binary Pattern (MS-LBP), Color Coherence Vector (CCV), and Global Color Histogram (GCH) for image retrieval. We have exerted the modified MS-LBP because of its ability to capture more texture detail than Local Binary Pattern (LBP) at multiple scales. This larger filter size of MS-LBP makes it less vulnerable to noise and illumination than the conventional LBP descriptor. The CCV captures color with location information well enough, but it’s vulnerable to the brightened images whereas, the GCH operator covers brightness (less sensitive to brightness than CCV), rotation, scale, translation, camera viewpoint invariant features, but lacks spatial information. The proposed framework improves the feature selection process by blending the strength of each of these descriptors. This paper also targets the high dimensionality of the feature vector of the MS-LBP and GCH descriptors by exerting Principal Component Analysis (PCA). Moreover, Linear Discriminant Analysis (LDA) selects robust and optimal features for retrieval. The proposed method is compared with state-of-the-art literature on four benchmark datasets in terms of Average Retrieval Precision (ARP), Average Retrieval Rate (ARR), and Retrieval Time (RT). Experimental results show that the proposed method excels the examined research practices.

In lung cancer computer-aided diagnosis (CAD) systems, having an accurate ground truth is critical and time consuming. Due to lack of ground truth and semantic information, lung CAD systems are not progressing in the manner these are supposed to. In this study, we have explored Lung Image Database Consortium (LIDC) database containing annotated pulmonary computed tomography (CT) scans, and we have used semantic and content-based image retrieval (CBIR) approach to exploit the limited amount of diagnostically labeled data in order to annotate unlabeled images with diagnoses. We evaluated the method by various combinations of lung nodule sets as queries and retrieves similar nodules from the diagnostically labeled dataset. In calculating the precision of this system Diagnosed dataset and computer-predicted malignancy data are used as ground truth for the undiagnosed query nodules. Our results indicate that CBIR expansion is an effective method for labeling undiagnosed images in order to improve the performance of CAD systems while tested on PGIMER data. Also a little knowledge of biopsy confirmed cases can also assist the physician’s as second opinion to mark the undiagnosed cases and avoid unnecessary biopsies

The strong feature representation ability of deep learning enables content-based image retrieval (CBIR) to achieve higher retrieval accuracy, while there are still some challenges for CBIR such as high requirements of training labels and retrieve efficiency. In this paper, we propose an unsupervised adversarial image retrieval (UAIR) framework by breaking the limitation of training labels. The framework is composed of two opposite parts and is linked by an adversarial loss function. For each input image, a generative model is used to select “well-matched” images from the database; a discriminative model is used to distinguish whether the selected images are similar enough to the input image. During training, the generative model tries to convince the discriminative model that the selected images are similar and the discriminative model always challenges the results of the generative model. The performances of the UAIR have been compared with other state-of-the-art image retrieval methods, including recently reported GAN-based methods. Extensive experiments show that the UAIR achieves significant improvement in CBIR with unsupervised adversarial training.

Image processing is one of the vast growing scenarios. From various resources this method is used to improve raw images received from satellites, space probes, aircrafts, military reconnaissance flights or pictures taken in normal day-to-day life from normal cameras. It is a kind of signal processing. This paper is an overview of image processing methods.

A considerable volume of high-resolution remote sensing (HRRS) data is generated with the intense space explorations happening globally. Remote sensing image retrieval (RSIR) is a fundamental task ...

The retrieval of images captured by different sensors or under different conditions is a desirable work. For this task, local image descriptor is an important tool. Nevertheless, most of local descriptors are proposed for similar images, which are usually captured by the same sensor in similar condition so they are same in radiation, or there are only linear radiation changes. In this paper, we propose a novel local feature descriptor, Histogram of Orientated Edge and Texture (HOET), for the retrieval of images with radiometric changes, especially for nonlinear radiation changes. In order to increase the similarity of the images captured from the same scene with nonlinear radiation changes, we combine edge and texture features to describe structural information of the images. First, we use Sobel operators in different directions to construct edge response maps and extract edge description in spatial domain. Second, we use log-Gabor filters to extract texture feature, then construct maximum index map (MIM) and extract texture description in frequency domain. Finally, HOET is developed by combining edge and texture description. We evaluate our descriptor in two heterogeneous image datasets and experiments show that HOET outperforms the state-of-the-art methods, including hand-designed and CNN-based descriptors.

Day by day, all the research communities have been focusing on digital image retrieval due to more internet and social media uses. In this paper, a U-Net-based neural network is proposed for the segmentation process and Haar DWT and lifting wavelet schemes are used for feature extraction in content-based image retrieval (CBIR). Haar wavelet is preferred as it is easy to understand, very simple to compute, and the fastest. The U-Net-based neural network (CNN) gives more accurate results than the existing methodology because deep learning techniques extract low-level and high-level features from the input image. For the evaluation process, two benchmark datasets are used, and the accuracy of the proposed method is 93.01% and 88.39% on Corel 1K and Corel 5K. U-Net is used for the segmentation purpose, and it reduces the dimension of the feature vector and feature extraction time by 5 seconds compared to the existing methods. According to the performance analysis, the proposed work has proven that U-Net improves image retrieval performance in terms of accuracy, precision, and recall on both the benchmark datasets.

The purpose of this work is to quickly find useful information from the massive image database in view of the images, videos, and other multimedia data generated on Internet platforms, such as Wechat, Sina Weibo and Twitter. The hash algorithm is utilized to map image high-dimensional features into binary code strings to deal with problems of traditional image retrieval algorithms, such as high feature dimension, large storage space, and low retrieval efficiency. Besides, an end-to-end deep hash learning network is designed based on semantic preservation, and the deep Convolutional Neural Network (CNN) is adopted for image feature extraction and hash function learning. Moreover, a binary-constrained regularization term is added to the loss function, and a semantic reservation layer is supplemented to optimize the generation of hash code. Furthermore, a semi-supervised hash learning network is proposed based on Generative Adversarial Network (GAN), which takes the hash network as a discriminator, and a discriminating node is introduced into the output layer to discriminate the true and false samples. For each innovation of the algorithm, retrieval experiments are carried out on several public datasets to further verify the performance of the proposed algorithm to prove the effectiveness. The results reveal that: the loss function of adaptive weight based on hash network reduces the influence of imbalances positive and negative sample on retrieval performance, the constrained regular terms reduce the error caused by quantization, avoids information loss caused by "Relaxation" strategy in traditional methods, and optimizes hash structure. In this way, the obtained hash code can retain semantic similarity and improve the retrieval accuracy. Hash network based on GAN can improve the overall performance of the model by about 3%. Compared with the existing image retrieval algorithm, the deep hash learning algorithm has better effect in image retrieval than the current hash method. The study effectively solves the problems in the image and text information retrieval, and can provide some ideas and methods for the research of the text retrieval.

Pixel value access control (PVAC) was introduced to deliver a secure and simple graphical password method where it requires users to load their image as their password. PVAC extracts the image to obtain a three-octet 8-bits Red-Green-Blue (RGB) value as its password to authenticate a user. The pixel value must be matched with the record stored in the database or otherwise, the user is failed to authenticate. However, users which prefer to store images on cloud storage would unintentionally alter and as well as the pixel value due to media compression and caused faulty pixels. Thus, the K-Means clustering algorithm is adapted to fix the issue where the faulty pixel value would be recognized as having the same pixel value cluster as the original. However, most of K-Means algorithm works were mainly developed for content-based image retrieval (CBIR) which having opposite characteristics from PVAC. Thus, this study was aimed to investigate the crucial criteria of PVAC and its compatibility with the K-Means algorithm for the problem. The theoretical analysis is used for this study where the suitable characteristics of K-Means are analyze based on PVAC requirements. The compliance analysis might become a referencing work for digital image clustering techniques adaptation on security system such as image filtering, image recognition, and object detection since most of image clustering works was focused on less sensitive image retrieval.