Search

Liquid leakage can cause industrial accidents. Current liquid leakage detection methods judge the leakage state by analyzing signals from special intrusion sensors, which is not ideal for early leakage due to sensor limited sensitivity. Visual information from surveillance systems deployed in industrial environments can reflect early leakage that cannot be monitored by such pressure sensors. In this paper, we propose a visual early leakage detection system based on a visual background extractor (Vibe) and EfficientNetB0. First, we extract the translucent and small potential leakage candidates based on Vibe, which include leakage targets and environmental interference. Then, to further recognize leakage targets in potential leakage candidates, we explore CNN models and a few recently proposed methods, and compare them with two different evaluation criteria. Our model based on EfficientNetB0 performs best and achieves 99.526% accuracy. Additionally, our CNN model for leakage recognition with a smaller size is feasible for industrial application. Experiments are conducted on the leakage dataset from surveillance video from the Tianjin Binhai Heating Station, and the detection results are consistent with real leakage situations. Our leakage detection system has high sensitivity and accuracy, which meets the requirements of early leakage detection.

Recently, the visual quality evaluation of screen content images (SCIs) has become an important and timely emerging research theme. This article presents an effective and novel blind quality evaluation metric for SCIs by using stacked autoencoders (SAE) based on pictorial and textual regions. Since the SCI consists of not only the pictorial area but also the textual area, the human visual system (HVS) is not equally sensitive to their different distortion types. First, the textual and pictorial regions can be obtained by dividing an input SCI via an SCI segmentation metric. Next, we extract quality-aware features from the textual region and pictorial region, respectively. Then, two different SAEs are trained via an unsupervised approach for quality-aware features that are extracted from these two regions. After the training procedure of the SAEs, the quality-aware features can evolve into more discriminative and meaningful features. Subsequently, the evolved features and their corresponding subjective scores are input into two regressors for training. Each regressor can obtain one output predictive score. Finally, the final perceptual quality score of a test SCI is computed by these two predicted scores via a weighted model. Experimental results on two public SCI-oriented databases have revealed that the proposed scheme can compare favorably with the existing blind image quality assessment metrics.

Human beings always evaluate the perceptual quality of an image coupled with identifying the semantic content of images. This paper addresses the correlation issue between stereoscopic image quality assessment (SIQA) and semantic recognition. In contrast to the previous SIQA methods that relied on binocular quality-aware features of a stereoscopic image, our approach also extracts binocular semantic features using a pre-trained deep convolutional neural network (DCNN) on a large dataset like ImageNet dataset, as well as the manually designed binocular quality-aware features. It can solve the problem of limited SIQA dataset size and facilitate better prediction on the quality. Experimental results demonstrate that the binocular semantic features are a good predictor for the stereoscopic image quality. The proposed method outperforms the state-of-the-art SIQA methods on four benchmark SIQA datasets. Significantly, all Spearman rank-order correlation coefficients (SROCCs) between the predicted scores and the subjective scores on the four datasets exceed 0.95. The MATLAB source code of the proposed method is available at https://github.com/kyohoonsim.

Image aesthetics assessment (IAA) is a subjective and complex task. The aesthetics of different themes vary greatly in content and aesthetic results, whether they are in the same aesthetic community or not. In aesthetic evaluation tasks, the pretrained network with direct fine-tune may not be able to quickly adapt to tasks on various themes. This article introduces a metalearning-based multipatch (MetaMP) IAA method to adapt to various thematic tasks quickly. The network is trained based on metalearning to obtain content-oriented aesthetic expression. In addition, we design a complete-information patch selection scheme and a multipatch (MP) network to make the fine details fit the overall impression. Experimental results demonstrate the superiority of the proposed method in comparison with the state-of-the-art models based on aesthetic visual analysis (AVA) benchmark datasets. In addition, the evaluation of the dataset shows the effectiveness of our metalearning training model, which not only improves MetaMP assessment accuracy but also provides valuable guidance for network initialization of IAA.

Benefiting from the development of multimedia communication terminals, the visual content presented to people on mobile devices is no longer a single form, but contains text, natural images, and other computer-generated graphics, which is called screen content images (SCIs). Inspired by the different visual stimuli that text and images bring to human eyes and the concept of global-guidance and local-adjustment, we design a novel full-reference image quality assessment (IQA) model using the structural features of the text, the perceptual features of pictures, and a score integration model (SPSIM) to evaluate SCIs quality. Firstly, we split the SCIs into textual and pictorial regions through a fully convolutional network (FCN) to conduct separate analyses. For textual regions, we take advantage of narrow edge extensions and high edge steps as structural features to compute the textual score. For pictorial regions, we extract the just noticeable difference (JND) features, which measure the human eye’s ability to detect distortion as perceptual features to calculate the pictorial score. Finally, an innovative score integration method based on the global-guidance and local-adjustment is designed to better analyze the relationship between the above regional scores and the whole global SCIs score. Abundant experiments in SCIs databases have shown that the SPSIM model can achieve better consistency with the human eyes system (HVS) in predicting the visual quality of SCIs.

Recently, research on path planning for the autonomous underwater vehicles (AUVs) has developed rapidly. Heuristic algorithms have been widely used to plan a path for AUV, but most traditional heuristic algorithms are facing two problems, one is slow convergence speed, the other is premature convergence. To solve the above problems, this paper proposes a new heuristic algorithms fusion, which improves the genetic algorithm with the ant colony optimization algorithm and the simulated annealing algorithm. In addition, to accelerate convergence and expand the search space of the algorithm, some algorithms like trying to cross, path self-smoothing and probability of genetic operation adjust adaptively are proposed. The advantages of the proposed algorithm are reflected through simulated comparative experiments. Besides, this paper proposes an ocean current model and a kinematics model to solve the problem of AUV path planning under the influence of ocean currents.

As the growth of vehicles and the acceleration of urbanization, the urban traffic congestion problem becomes a burning issue in our society. Constructing a software defined Internet of things(SD-IoT) with a proper traffic control scheme is a promising solution for this issue. However, existing traffic control schemes do not make the best of the advances of the multi-agent deep reinforcement learning area. Furthermore, existing traffic congestion solutions based on deep reinforcement learning(DRL) only focus on controlling the signal of traffic lights, while ignore controlling vehicles to cooperate traffic lights. So the effect of urban traffic control is not comprehensive enough. In this article, we propose Modified Proximal Policy Optimization (Modified PPO) algorithm. This algorithm is ideally suited as the traffic control scheme of SD-IoT. We adaptively adjust the clip hyperparameter to limit the bound of the distance between the next policy and the current policy. What’s more, based on the collected data of SD-IoT, the proposed algorithm controls traffic lights and vehicles in a global view to advance the performance of urban traffic control. Experimental results under different vehicle numbers show that the proposed method is more competitive and stable than the original algorithm. Our proposed method improves the performance of SD-IoT to relieve traffic congestion.

In diverse applications of the industrial Internet of Things (IIoT), the six degrees of freedom (6-DoF) information is essential, which determines the attitude and position of a 3-D object. Nevertheless, due to the complexity and variability of the scenarios, higher requirements are imposed on the 6-DoF estimation. Among them, occlusion is one of the knottiest problems, which causes significant performance degradation and needs to be solved urgently. Therefore, in this article, we propose a completely new and universal multibranch network (MBN) for industrial applications. Our method is based on monocular vision system and convolutional neural network (CNN) framework. First and foremost, it reduces occlusion interference by focusing on the physical area characteristics of the image. Compared with the traditional CNN-based method, it owns higher accuracy and lower estimation error under occlusion. Second, we propose five algorithms to process the predictions of the independent branches, further effectively improving performance. Third, we optimize the marker to solve the inequality problem in attitude angle estimation. Furthermore, we design and conduct a series of experiments, and the experimental results sufficiently prove the superiority of MBN

Visual perception refers to the process of organizing, identifying, and interpreting visual information in environmental awareness and understanding. With the rapid progress of multimedia acquisition technology, research on visual perception has been a hot topic in the academical field and industrial applications. Especially after the introduction of artificial intelligence theory, intelligent visual perception has been widely used to promote the development of industrial production towards intelligence. In this article, we review the previous research and application of visual perception in different industrial fields such as product surface defect detection, intelligent agricultural production, intelligent driving, image synthesis, and event reconstruction. The applications basically cover most of the intelligent visual perception processing technologies. Through this survey, it will provide a comprehensive reference for research on this direction. Finally, this article also summarizes the current challenges of visual perception and predicts its future development trends.

In 3-D attitude angle estimation, monocular vision-based methods are often utilized for the advantages of short-time and high efficiency. However, the limitations of these methods lie in the complexity of the algorithm and the specificity of the scene, which needs to match the characteristics of the cooperation object and the scene. In this article, we propose a fully connected attitude detection network (FADN), which combines neural network and traditional algorithms for 3-D attitude angle estimation. FADN provides a whole process from the input of a single frame image in the industrial video stream to the output of the corresponding 3-D attitude angle estimation. Benefiting from the end-to-end estimation framework, FADN avoids tedious matching algorithms and thus has certain portability. A series of comparative experiments based on the rendering software 3-D Studio Max (3d Max) have been carried out to evaluate the performance of FADN. The experimental results show that FADN has high estimation accuracy and fast running speed. At the same time, the simulation results reliably prove the feasibility of FADN, and also promote the research in real scenarios.

Accurately detecting man-made intrusion from different events is of great significance for distributed optical fiber sensing perimeter security system. Most traditional algorithms lack the ability to reject various events of unknown class which are mainly from natural disturbance, and greatly decline the accuracy of intrusion recognition in field application. In order to solve this problem, we proposed a novel robust intrusion event recognition scheme based on convolutional prototype network (CPL), which realized end-to-end feature extraction and recognition based on the similarity of intrusion signals by integrating relevant variables of prototype learning into the training process of multiscale convolutional neural network (MSCNN) as trainable parameters, and had the ability to recognize and reject the unknown disturbance events. In field experiments, the average recognition accuracy of intrusion events as known class can reach 84.67%, with the rejection rate of disturbance events as unknown class is about 83.75%, which ensure the accuracy of intrusion events monitoring in complex field environments. And the recognition response time is about 17 ms, which also meets the need of real-time monitoring.

The small volume of the existing screen content images (SCIs) database with human ratings restricts the training processes of no-reference (NR) image quality assessment models based on traditional machine learning and deep learning. In this letter, we propose an NR model called the multi-task distortion-learning network to jointly analyse the distortion types and distortion degree of SCIs to be the prior knowledge for predicting the SCIs quality. Specifically, we first generate sufficient distorted SCIs labelled with the distortion type and degree, which does not need much effort to conduct subjective scoring experiments. Then, relying on these data, we pre-train a multi-task learning network to obtain strong prior knowledge about assessing the image quality. Finally, we further jointly train a quality assessment network with an attention module that simulates the mechanism of processing visual signals in the human eyes. The experimental results on the public SCIs databases show that the proposed model is competitive against other state-of-art approaches and achieves better consistency with the human vision system.

The continuous development of industry big data technology requires better computing methods to discover the data value. Information forecast, as an important part of data mining technology, has achieved excellent applications in some industries. However, the existing deviation and redundancy in the data collected by the sensors make it difficult for some methods to accurately predict future information. This article proposes a semisupervised prediction model, which exploits the improved unsupervised clustering algorithm to establish the fuzzy partition function, and then utilize the neural network model to build the information prediction function. The main purpose of this article is to effectively solve the time analysis of massive industry data. In the experimental part, we built a data platform on Spark, and used some marine environmental factor datasets and UCI public datasets as analysis objects. Meanwhile, we analyzed the results of the proposed method compared with other traditional methods, and the running performance on the Spark platform. The results show that the proposed method achieved satisfactory prediction effect.

When human visual system (HVS) looks at a scene, it extracts various features from the image about the scene to understand it. The extracted features are compared with the stored memory on the analogous scene to judge their similarity [1] . By analyzing to the similarity, HVS understands the scene presented on eyes. Based on the neurobiological basis, we propose a 2D full reference (FR) image quality assessment (IQA) method, named mean and deviation of deep and local similarity (MaD-DLS) that compares similarity between many original and distorted deep feature maps from convolutional neural networks (CNNs). MaD-DLS uses a deep learning algorithm, but since it uses the convolutional layers of a pre-trained model, it is free from training. For pooling of local quality scores within a deep similarity map, we employ two important descriptive statistics, (weighted) mean and standard deviation and name it mean and deviation (MaD) pooling. The two statistics each have the physical meaning: the weighted mean reflects effect of visual saliency on quality, whereas the standard deviation reflects effect of distortion distribution within the image on it. Experimental results show that MaD-DLS is superior or competitive to the existing methods and the MaD pooling is effective. The MATLAB source code of MaD-DLS will be available online soon.

Panoramic video and stereoscopic panoramic video are essential carriers of virtual reality content, so it is very crucial to establish their quality assessment models for the standardization of virtual reality industry. However, it is very challenging to evaluate the quality of the panoramic video at present. One reason is that the spatial information of the panoramic video is warped due to the projection process, and the conventional video quality assessment (VQA) method is difficult to deal with this problem. Another reason is that the traditional VQA method is problematic to capture the complex global time information in the panoramic video. In response to the above questions, this paper presents an end-to-end neural network model to evaluate the quality of panoramic video and stereoscopic panoramic video. Compared to other panoramic video quality assessment methods, our proposed method combines spherical convolutional neural networks (CNN) and non-local neural networks, which can effectively extract complex spatiotemporal information of the panoramic video. We evaluate the method in two databases, VRQ-TJU and VR-VQA48. Experiments show the effectiveness of different modules in our method, and our method outperforms state-of-the-art other related methods.

Real-time object detection and ranging of multiple objects on the road are the essential tasks in the field of autonomous driving. In this paper, we introduce a system for simultaneous detection and ranging of vehicles, people, non-motor vehicles and lanes based on RGB-D images. Among them, the detection of vehicles, people and non-motor vehicles belongs to general detection task and the lane detection belongs to segmentation task. In order to improve the accuracy and speed, we use two networks to complete these two tasks. We propose a real-time synchronization method with multi-GPU, which achieves separate training and simultaneous detection of lane detectior and vehicle, people and non-motor vehicle detector. We also propose a center-selective ranging module based on binocular ranging technology to distance the detected object. The system reaches nearly 15 FPS with four 1080Ti GPUs. We construct datasets about these problems including daytime and night in which the system achieves high accuracy. A real-time test of the system on the streets of Tianjin, China has been conducted by us, it has proved that the system can be applied to actual driving.

Due to the wide application of stereoscopic display technology, stereoscopic video quality assessment (SVQA) is facing great challenges, but worthwhile. Stereoscopic videos contain a great deal of information, which involves not only the spatial domain but also the spatio-temporal domain. Motion in stereoscopic video plays a critical role in quality perception, while the existing SVQA methods rarely refer to motion factors, and the performance of these methods is restrained. In this article, a novel SVQA based on motion perception is introduced and its performance is superior to that of existing excellent methods. Particularly, to appropriately reduce the amount of data processing, we extract the key-frame sequences according to the influence of movement intensity on binocular visual quality perception. The binocular summation and difference operations are implemented on extracted sequences, and then spatial texture and spatio-temporal texture statistic measurement are extracted simultaneously with local binary patterns from three orthogonal planes (LBP-TOP). Experiments are implemented on two publicly available databases and the results demonstrate the effectiveness and robustness of our algorithm for various categories of distortion stereoscopic video pairs.

With the wide application of stereoscopic video technology, the quality of stereoscopic video has attracted people’s attention. Objective stereoscopic video quality assessment (SVQA) is highly challenging, but essential, particularly the no-reference (NR) SVQA method, where reference information is not needed and a large number of samples are required for training and testing sets. However, as far as we know, there are only a few samples in the established stereo video database, which is unsuitable for NR quality assessment and seriously hampers the development of NR-SVQA method. For these difficulties that we encountered, we carry out a comprehensive subjective evaluation of stereoscopic video quality in our newly established TJU-SVQA databases that contain various contents, mixed resolution coding and symmetrically/asymmetrically distorted stereoscopic videos. Furthermore, we propose a new inter-frame cross map to predict the objective quality scores. We compare and analyze the performance of several state-of-the-art 2D and 3D quality evaluation methods on our new databases. The experimental results on our established databases and a public database demonstrate that the proposed method can robustly predict the quality of stereoscopic videos.

With the development of information technology, big data has had a positive impact on the development of various industries. However, the continued growth of data has led to difficulties in data sharing. Blockchain, with its information sharing characteristics and decentralized management principles, is expected to become the new engine for building a future data sharing platform. Blockchain-based data sharing is useful for recording user transactions and sharing of multi-platform data. However, this technology not only has challenges in information storage, but also risks in third-party transactions. In this article, we propose a big data sharing and transaction framework based on blockchain. This framework uses the blockchain decentralization and openness idea to establish a big data transaction platform for multiple users. For the problem of distributed secure storage of big data in the blockchain, we propose a data-tamperproof mechanism that introduces a cryptographic algorithm to prevent transaction data from being tampered with during user storage. The purpose of this article is to ensure transaction security and data reliability when trading in the blockchain.

In this paper, we find factors affecting visual comfort along the visual pathways. The statistics data used for the 3D visual discomfort predictor (3D-VDP) model extracted from the traditional optical statistical elements derived, retinal disparity and neurophysiological elements obtained from V1 and MT areas. These physiological statistics are used to create a new feature map for 3D-VDP models, known as MT importance map. The thirteen typical MT neurons are divided into four kinds of cells: tuned-excitatory cells, tuned-inhibitory cells, far cells and near cells based on their characteristics. Disparity map is also transformed by changing the viewing distance in the formula to calculate the horizontal angular disparity factor. We propose that variation of viewing distance will make the MT cells more intense reaction. As a result, three types of angular disparity maps are obtained: tuned-near, tuned-far and original disparity maps. Preliminary results on two public available databases exhibit the performance of the proposed approach outperforms previous S3D VDP models.

Accurate measurement of position and attitude information is particularly important. Traditional measurement methods generally require high-precision measurement equipment for analysis, leading to high costs and limited applicability. Vision-based measurement schemes need to solve complex visual relationships. With the extensive development of neural networks in related fields, it has become possible to apply them to the object position and attitude. In this paper, we propose an object pose measurement scheme based on convolutional neural network and we have successfully implemented end-to-end position and attitude detection. Furthermore, to effectively expand the measurement range and reduce the number of training samples, we demonstrated the independence of objects in each dimension and proposed subadded training programs. At the same time, we generated generating image encoder to guarantee the detection performance of the training model in practical applications.

In the era of industrial 4.0, industrial Internet of Things (IIoT) has brought essential changes to human society. For IIoT, communication in network can be defined as the basic condition for further development and integrated information exchange. In this way, cached-enabled heterogeneous industrial network is necessary to be optimized. In this paper, we consider the optimal geographical placement of contents in cache-enabled heterogeneous networks to minimize the total missing probability. And the probability represents that typical user cannot find requested file in the nearby base stations (BSs). In contract to existing works which only concern content placement, we jointly optimize content placement at BSs and activation densities of BSs of different tiers subject to the cache size limits and the constraint on the BSs energy consumption cost. In addition, the user distribution in this work is modeled by a homogeneous Poisson Point Process. We prove that the original optimization problem can be transformed to a convex problem. The convexity of the optimization problem allows us to apply the KKT conditions to derive useful analytical results of the optimal solution. Based on this, we propose a low-complexity near-optimal algorithm to find the approximated content placement probabilities. We further extend the optimization to heterogeneous networks with the user distribution modeled by the modified Cluster Process. Extensive simulation results show the superior performance of joint optimization of content placement and BSs activation densities compared to only optimizing content placement.

The deepening of ocean measurement work requires higher transmission bandwidth and information calculation efficiency, which provides an opportunity for fog computing. Compared with cloud computing, fog computing shows distribution because it concentrates data processing and application on devices at the edge of the network. In this article, the Ocean of Things (OoT) framework is designed for marine environment monitoring based on the Internet of Things technology. The OoT is divided into three layers: 1) data acquisition layer; 2) fog layer; and 3) cloud layer. In the fog layer, in order to complete the quality control of the sensor measurement data, we use the numerical gradient-based method to process the original acquisition data. An improved D-S algorithm is designed for multisensor information fusion, reducing the data capacity and improving data quality. In the cloud layer, we build ocean information change models based on the fog layer data to predict the dynamic ocean environment. The designed fog layer is evaluated based on marine multisensor information. The results have shown that fog-based multisensor data processing shows low time consumption and high reliability. Moreover, this article uses real temperature data sets to evaluate the prediction accuracy of the cloud model. Finally, we tested the performance of the designed OoT framework with multiple data sets. The simulation results show that the framework can improve the efficiency of data utilization at sea and improve the efficiency of information utilization.

Human kind is exposed to large amounts of data. According to statistics, more than 80% of information received by humans comes from the visual system. Therefore, image information processing is not only an important research topic but also a challenging task. The unique information processing mechanism of the human visual system provides it with fast, accurate, and efficient image processing capabilities. At present, many advanced image analysis and processing techniques have been widely used in image communication, geographic information systems, medical image analysis, and virtual reality. However, there is still a large gap between these technologies and the human visual system. Therefore, building an image system research mechanism based on the biological vision system is an attractive but difficult target. Although it is a challenge, it can also be considered as an opportunity which utilizes biologically inspired ideas. Meanwhile, through the integration of neural biology, biological perception mechanisms, and computer science and mathematical science, related research can bridge biological vision and computer vision. Finally, the biologically inspired image analysis and processing system is expected to be built on the basis of further consideration of the learning mechanism of the human brain.

For cyber-physical industrial networks, more and more multimedia data is faced in high-speed information transmission. The explosive data brings more challenges to the architecture of modern industrial networks. In this way, cache resource allocation technology is necessary for practical applications. In order to design reasonable caching framework, how to predict the multimedia data request is an important issue. In order to keep efficient and reliable data transmission in wireless industrial networks, security design is also critical for existing cache resource allocation. Based on previous works, some promising technologies have been applied, such as heterogeneous ultradense networks, wireless edge caching, and web content popularity prediction. In this paper, we summarize these promising technologies and provide a useful guidance for security design in cyber-physical cache resource allocation system. Specially, we can divide the proposed method into three main steps. First of all, a spatio-temporal multimedia content prediction based on long short-term memory is proposed for accurate prediction on multimedia data request. After that, we make use of Zipf fitting for caching model. At last, the caching optimization considering security is put forward in this paper. Experimental results show the satisfied performance of our proposed algorithm and it has obvious potential application value in cyber-physical industrial networks with cache resource allocation technology.

Most previous 2-D and 3-D image quality evaluators were based on shallow architectures. Their shallow architectures cannot model phenomenon occurring in human visual systems sufficiently. Disparities between left and right views have been importantly used for 3-D image quality assessment (IQA), but single disparity, depth, or cyclopean maps made from the disparities cannot thoroughly reflect the depth sense. In this paper, we propose a blind stereoscopic image quality evaluator using stacked auto-encoders (SAE). The proposed method is based on two theories on initial stages of stereopsis. One is cyclopean channel theory and the other is binocular summation/difference channels theory. Especially, a cyclopean image that models the former theory is computed to consider binocular suppression, whereas summation and difference images that model the latter one are utilized to treat the depth sense. We train three SAEs for quality-aware features from the three images in an unsupervised manner. Through the SAEs, the features are transformed into more meaningful features, and they are used to train two regressors. The regressors are used to obtain a final predicted score. Experimental results conducted on popular 3-D IQA databases prove that the proposed algorithm outperforms state-of-the-art 3-D IQA methods.

Blind or visually impaired people face special difficulties in daily life. With the advances in vision sensors and computer vision, the design of wearable vision assistance system is promising. In order to improve the life quality of the visually impaired group, a wearable system is proposed in this paper. Typically the performance of visual sensors is affected by a variety of complex factors in practice, resulting in a large number of noise and distortion. In this paper, we will creatively leverage image quality evaluation to select the captured images through vision sensors, which can ensure the input quality of scenes for the final identification system. First, we use binocular vision sensors to capture images in a fixed frequency and choose the informative ones based on stereo image quality assessment. Then the captured images will be sent to cloud for further computing. Specially, the detection and automatic result will be done for all the received images. Convolutional neural network based on big data will be used in this step. According to image analysis, the cloud computing can return the requested information for users, which can help them make a more reasonable decision in further action. Simulations and experiments show that the proposed method can solve the problem effectively. In addition, statistical results also demonstrate that wearable vision system can make visually impaired group more satisfied in visual needed situations.

With the development of the Internet-of-Things (IoT) industry, more and more fields are involved such as multimedia data. Currently, users rely on videos and images with high data volume, so it has brought more challenges for wireless communication and transmission. For multimedia data, it is obviously different from traditional communication data. So new method is required to solve the problem of high data volume in communication. The proactive content caching and the unmanned aerial vehicle (UAV) relaying techniques are deployed over IoT network, enabling the maximum throughput for the served IoT devices. Even though these two existing technologies are important to solve the problem of throughput, there are still other challenges for efficiently improving the system throughput. We mainly study the cache-enabled UAV to maximize throughput among IoT devices in the IoT with the placement of content caching and UAV location. Especially, we divide the joint optimization problem into two parts. First, the UAV deployment problem is decomposed into vertical and horizontal dimensions to ensure the optimal deployment height and 2-D position. The enumeration search method is employed to obtain the 2-D position. Then, we also formulate a concave problem for probabilistic caching placement. Experimental results have indicated that the cache-enabled UAV scheme can obtain a better throughput, which can bring new approach for multimedia data throughput maximization in IoT system.

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA.

This letter proposes a cache-enabled cooperation scheme to improve the performance of cognitive radio networks. We jointly optimize the content placement and the bandwidth allocation to maximize the secondary users’ average successful transmission probability under the quality of service constraint of the primary user. Although the original problem is not convex, an efficient two-level bisection search algorithm is proposed to find the optimal cache and spectrum allocation solution to the approximated problem. Simulation results verify that the cache-enabled cooperation outperforms existing schemes significantly.

With the popularity of video technology, stereoscopic video quality assessment (SVQA) has become increasingly important. Existing SVQA methods cannot achieve good performance because the videos’ information is not fully utilized. In this paper, we consider various information in the videos together, construct a simple model to combine and analyze the diverse features, which is based on saliency and sparsity. First, we utilize the 3-D saliency map of sum map, which remains the basic information of stereoscopic video, as a valid tool to evaluate the videos’ quality. Second, we use the sparse representation to decompose the sum map of 3-D saliency into coefficients, then calculate the features based on sparse coefficients to obtain the effective expression of videos’ message. Next, in order to reduce the relevance between the features, we put them into stacked auto-encoder, mapping vectors to higher dimensional space, and adding the sparse restraint, then input them into support vector machine subsequently, and finally, get the quality assessment scores. Within that process, we take the advantage of saliency and sparsity to extract and simplify features. Through the later experiment, we can see the proposed method is fitting well with the subjective scores.

Hand-gesture recognition is a key aspect to make virtual-reality (VR) interaction more convenient. Making a user's idea understood by computers (i.e., character input) plays an important role in interaction. Current methods of hand gesture and character input are too limited to make full use of the powerful capacity possessed by current computers.

With continuous progress of Internet of Things, multimedia analysis in it has attracted more and more attention. Specially, stereoscopic display technology plays an important role in the multimedia analysis processing. In the Internet of Things system, the quality of stereoscopic image will be reduced in the transmission process. In this mode, it will have a great impact on multimedia analysis to judge whether the quality of stereoscopic image meets the requirements. In this paper, a new no-reference stereoscopic image quality assessment model for multimedia analysis towards Internet of Things is built, which is based on a deep learning model to learn from the class labels and image representations. In our framework, images are represented by natural scene statistics features that are extracted from discrete cosine transform domain, and a regression model is employed to shine upon the quality from the feature vector. The training process of the proposed model contains an unsupervised pretraining phase and a supervised fine-tuning phase, enabling it to generalize over the whole distortion types and severity. The proposed model greatly shows the correlation with subjective assessment as demonstrated by experiments on the LIVE 3-D Image Quality Database and IVC 3-D Image Quality Database.

Virtual reality (VR), a new type of simulation and interaction technology, has aroused widespread attention and research interest. It is necessary to evaluate the VR quality and provide a standard for the rapidly developing technology. To the best of our knowledge, a few researchers have built benchmark databases and designed related algorithms, which has hindered the further development of the VR technology. In this paper, a free available data set (VRQ-TJU) for VR quality assessment is proposed with subjective scores for each sample data. The validity for the designed database has been proved based on the traditional multimedia quality assessment metrics. In addition, an end-to-end 3-D convolutional neural network is introduced to predict the VR video quality without a referenced VR video. This method can extract spatiotemporal features and does not require using hand-crafted features. At the same time, a new score fusion strategy is designed based on the characteristics of the VR video projection process. Taking the pre-processed VR video patches as input, the network captures local spatiotemporal features and gets the score of every patch. Then, the new quality score fusion strategy is applied to get the final score. Such approach shows advanced performance on this database.

Existing static grid resource scheduling algorithms, which are limited to minimizing the makespan, cannot meet the needs of resource scheduling required by cloud computing. Current cloud infrastructure solutions provide operational support at the level of resource infrastructure only. When hardware resources form the virtual resource pool, virtual machines are deployed for use transparently. Considering the competing characteristics of multi-tenant environments in cloud computing, this paper proposes a cloud resource allocation model based on an imperfect information Stackelberg game (CSAM-IISG) using a hidden Markov model (HMM) in a cloud computing environment. CSAM-IISG was shown to increase the profit of both the resource supplier and the applicant. Firstly, we used the HMM to predict the service provider's current bid using the historical resources based on demand. Through predicting the bid dynamically, an imperfect information Stackelberg game (IISG) was established. The IISG motivates service providers to choose the optimal bidding strategy according to the overall utility, achieving maximum profits. Based on the unit prices of different types of resources, a resource allocation model is proposed to guarantee optimal gains for the infrastructure supplier. The proposed resource allocation model can support synchronous allocation for both multi-service providers and various resources. The simulation results demonstrated that the predicted price was close to the actual transaction price, which was lower than the actual value in the game model. The proposed model was shown to increase the profits of service providers and infrastructure suppliers simultaneously.

In this paper, a novel image quality assessment (IQA) model using statistic features based on image information theory is proposed. Firstly, the image is decomposed into non-overlapped patches and transformed into the saliency information, specific information and entanglement information. Then considering the perception characteristics of human visual system, e.g. the center-surrounded of the receptive field and the contrast-gain masking process, have an important influence on image quality evaluation, the statistic features of the image information were employed to describe the image local contrast, structure, multi-scale and multi-direction properties, include the mean subtracted and contrast normalized (MSCN) features, the gradient magnitude (GM) features and the Laplacian of Gaussian (LOG) features. Finally, the mapping between the statistic features and the human subjective perception is established and used to measure the image quality. Experimental results on benchmark databases (LIVE, TID2013, CSIQ) indicate the rationality and validity of the proposed method.

In the field of big data applications, image information is widely used. The value density of information utilization in big data is very low, and how to extract useful information quickly is very important. So we should transform the unstructured image data source into a form that can be analyzed. In this paper, we proposed a fast image retrieval method which designed for big data. First of all, the feature extraction method is necessary and the feature vectors can be obtained for every image. Then, it is the most important step for us to encode the image feature vectors and make them into database, which can optimize the feature structure. Finally, the corresponding similarity matching is used to determined the retrieval results. There are three main contributions for image retrieval in this paper. New feature extraction method, reasonable elements ranking, and appropriate distance metric can improve the algorithm performance. Experiments show that our method has a great improvement in the effective performance of feature extraction and can also get better search matching results.