Search

Affective computing is an important research area for developing an emotion recognition system. In such systems, stimuli for emotion elicitation are standard dynamic range (SDR) multimedia content. This study uses high dynamic range (HDR) multimedia content that effectively presents the scene's inherent colors and high dynamic range of luminance as stimuli for emotion elicitation, and its impact on human emotional experience with different personality traits is explored. For this purpose, four low and high-valence multimedia clips in SDR and HDR versions were shown to sixty subjects. Their emotional experience was recorded on a valence-arousal plane, which is then statistically analyzed in terms of the type of content and human personality traits. The results of the t-test with a 95% confidence interval show that HDR multimedia content elicits better emotions than SDR multimedia content and can be effectively utilized in emotion recognition systems for better emotion elicitation. [ABSTRACT FROM AUTHOR]

No-referenceimage quality assessment (NR-IQA) techniques try to assess the quality of images without anyinformation regarding the pristine version of the image. NR-IQA becomes more challenging for images affected by multiple distortions and images taken in real-world scenarios. Most convolutional neural network (CNN) based NR-IQA techniques do not use visual saliency (VS) and the ones that do require a reference image for computation of VS map. This paper proposes a completely blind end-to-end NR-IQA methodology based on VS and convolutional neural network (CNN). It utilizes VS models to identify the significant region of the image that can be used for quality assessment. The VS methodology used in the proposed approach does not require a reference image for the extraction of VS map. The regions proposed by the VS model are then utilized by a CNN to predict the quality score. The proposed methodology is tested on three publicly available multi-distorted image quality assessment databases i.e., live multiply distorted image quality (MDIQ) database, multiply distorted image database (MDID), and the live in the wild challenge (CLIVE) database. The proposed methodology is also tested over three VS models extract VS maps without the reference image. The experimental results show that using VS models improve the performance of CNN for predicting the image quality score. [ABSTRACT FROM AUTHOR]

Considerable progress has been made toward developing standard dynamic range (SDR) blind video quality assessment (BVQA) models that do not require any baseline reference for quality prediction. However, there is no such method for the high dynamic range (HDR) content. Unlike SDR video, HDR video represents a high-fidelity representation of the real-world scene by preserving the wide luminance range and color gamut. Therefore, SDR BVQA models are not suitable for HDR BVQA. Towards ameliorating this, a first-of-its-kind BVQA model for HDR content is presented in this work. The proposed HDR blind video quality model (HDR-BVQM) is inspired by the spatio-temporal natural scene statistics model, previously employed in SDR blind quality assessment metrics. To build our proposed model, we first develop a comprehensive subjective HDR video quality dataset, including 228 distorted videos generated through three different (H.264, HEVC, packet drop) distortion processes from 19 pristine HDR videos. The developed dataset is then used to extract HDR relevant features, which vary in different distortion types, to train and test the proposed HDR-BVQM. The features are based on the pointwise, pairwise log-derivative, and motion coherence based statistics. Finally, detailed validation and performance comparison is performed with full-reference HDR and no-reference SDR quality assessment methods. The results reveal that the quality prediction by HDR-BVQM correlates with the human judgment of quality. [ABSTRACT FROM AUTHOR]

Considerable progress has been made toward developing standard dynamic range (SDR) blind video quality assessment (BVQA) models that do not require any baseline reference for quality prediction. However, there is no such method for the high dynamic range (HDR) content. Unlike SDR video, HDR video represents a high-fidelity representation of the real-world scene by preserving the wide luminance range and color gamut. Therefore, SDR BVQA models are not suitable for HDR BVQA. Towards ameliorating this, a first-of-its-kind BVQA model for HDR content is presented in this work. The proposed HDR blind video quality model (HDR-BVQM) is inspired by the spatio-temporal natural scene statistics model, previously employed in SDR blind quality assessment metrics. To build our proposed model, we first develop a comprehensive subjective HDR video quality dataset, including 228 distorted videos generated through three different (H.264, HEVC, packet drop) distortion processes from 19 pristine HDR videos. The developed dataset is then used to extract HDR relevant features, which vary in different distortion types, to train and test the proposed HDR-BVQM. The features are based on the pointwise, pairwise log-derivative, and motion coherence based statistics. Finally, detailed validation and performance comparison is performed with full-reference HDR and no-reference SDR quality assessment methods. The results reveal that the quality prediction by HDR-BVQM correlates with the human judgment of quality. [ABSTRACT FROM AUTHOR]

No reference image quality assessment (NR-IQA) has received considerable importance in the last decade due to a rise in the use of multimedia content in our daily lives. Due to limitations in technology, multiple distortions may be introduced in the images that need to be assessed. Recently feature selection has shown promising results for single distorted NR-IQA and their effectiveness on multiple distorted images still need to be addressed. In this paper, impact of feature level fusion and feature selection on multiple distorted image quality assessment is presented. To this end features are extracted from multiple distorted images using six NR-IQA techniques (BLIINDS-II, BRISQUE, CurveletQA, DIIVINE, GM-LOG, SSEQ) that extract features in different (discrete cosine transform, spatial, curvelet transform, wavelet transform, spatial and gradient, spatial and spectral) domains. The extracted features from different domains are fused to generate a single feature vector. All combinations of feature-level fusion from six different techniques have been evaluated. Three different feature selection algorithms (genetic search, linear forward search, particle swarm optimization) are then applied to select optimum features for NR-IQA. The selected features are then used by the support vector regression model to predict the quality score. The performance of the proposed methodology is evaluated for two multiple distorted IQA databases (LIVE multiple distorted image dataset (LIVEMD), multiply distorted image database (MDID2017)), two singly synthetically distorted IQA databases (Tampere image database (TID2013), Computational and subjective image quality database (CSIQ)), and one screen content IQA database (Screen content image quality database (SIQAD)). Experimental results show that the fusion of features from different domains gives better performance in comparison to existing multiple-distorted NR-IQA techniques with SROCC scores of 0.9555, 0.9587, 0.6892, 0.9452, and 0.7682 on the LIVEMD, MDID, TID2013, CSIQ, and SIQAD databases respectively. Moreover, the performance is further improved when the genetic search feature selection algorithm is applied to fused features to remove the redundant and irrelevant features. The SROCC scores are improved to 0.9691, 0.9723, and 0.6897 for LIVEMD, MDID, and TID2013 databases respectively. [ABSTRACT FROM AUTHOR]

Most of no-reference image quality assessment (NR-IQA) techniques reported in literature have utilized transform coefficients, which are modeled using curve fitting to extract features based on natural scene statistics (NSS). The performance of NR-IQA techniques that utilize curve-fitting suffers from degradation in performance because the distribution of curve fitted NSS features deviate from the statistical distribution of a distorted image. Although deep convolutional neural networks (DCNNs) have been used for NR-IQA that are NSS model-independent but their performance is dependent upon the size of training data. The available datasets for NR-IQA are small, therefore data augmentation is used that affects the performance of DCNN based NR-IQA techniques and is also computationally expensive. This work proposes a new patch-based NR-IQA technique, which utilizes features extracted from discrete cosine transform coefficients. The proposed technique is curve fitting independent and helps in avoiding errors in the statistical distribution of NSS features. It relies on global statistics to estimate image quality based on local patches, which allow us to decompose the statistics of images. The proposed technique divides the image into patches and extracts nine handcrafted features i.e., entropy, mean, variance, skewness, kurtosis, mobility, band power, energy, complexity, and peak to peak value. The extracted features are used with a support vector regression model to predict the image quality score. The experimental results have shown that the proposed technique is database and image content-independent. It shows better performance over a majority of distortion types and on images taken in real-time. [ABSTRACT FROM AUTHOR]

The aim of no-reference image quality assessment (NR-IQA) is to assess the quality of an image, which is consistent with the mean opinion score, without any prior knowledge about the reference image. This work proposes a new NR-IQA technique based on natural scene statistics properties of the bag-of-features representation and feature selection algorithms. The proposed bag-of-features technique utilizes Harris affine detector and scale invariant feature transform to compute points, which are clustered using the k-means clustering algorithm to extract features for IQA. The extracted features are utilized with a support vector regression model to assess the quality of the image. The proposed technique outperforms state-of-the-art NR-IQA techniques, when tested on three commonly used subjective image quality assessment databases. The experimental results have shown that the features extracted using the proposed technique are database independent and shows high correlation with the mean opinion score. [ABSTRACT FROM AUTHOR]

No-reference image quality assessment (NR-IQA) aims to evaluate the perceived quality of distorted images without prior knowledge of pristine version of the images. The quality score is predicted based on the features extracted from the distorted image, which needs to correlate with the mean opinion score. The prediction of an image quality score becomes a trivial task, if the noise affecting the quality of an image can be modeled. In this paper, gradient magnitude and Wiener filtered discrete wavelet coefficients are utilized for image quality assessment. In order to reconstruct an estimated noise image, Wiener filter is applied to discrete wavelet coefficients. The estimated noise image and the gradient magnitude are modeled as conditional Gaussian random variables. Joint adaptive normalization is applied to the conditional random distribution of the estimated noise image and the gradient magnitude to form a feature vector. The feature vector is used as an input to a pre-trained support vector regression model to predict the image quality score. The proposed NR-IQA is tested on five commonly used image quality assessment databases and shows better performance as compared to the existing NR-IQA techniques. The experimental results show that the proposed technique is robust and has good generalization ability. Moreover, it also shows good performance when training is performed on images from one database and testing is performed on images from another database. [ABSTRACT FROM AUTHOR]

The goal of this study is to enhance the emotional experience of a viewer by using enriched multimedia content, which entices tactile sensation in addition to vision and auditory senses. A user-independent method of emotion recognition using electroencephalography (EEG) in response to tactile enhanced multimedia (TEM) is presented with an aim of enriching the human experience of viewing digital content. The selected traditional multimedia clips are converted into TEM clips by synchronizing them with an electric fan and a heater to add cold and hot air effect. This would give realistic feel to a viewer by engaging three human senses including vision, auditory, and tactile. The EEG data is recorded from 21 participants in response to traditional multimedia clips and their TEM versions. Self assessment manikin (SAM) scale is used to collect valence and arousal score in response to each clip to validate the evoked emotions. A t-test is applied on the valence and arousal values to measure any significant difference between multimedia and TEM clips. The resulting p-values show that traditional multimedia and TEM content are significantly different in terms of valence and arousal scores, which shows TEM clips have enhanced evoked emotions. For emotion recognition, twelve time domain features are extracted from the preprocessed EEG signal and a support vector machine is applied to classify four human emotions i.e., happy, angry, sad, and relaxed. An accuracy of 43.90% and 63.41% against traditional multimedia and TEM clips is achieved respectively, which shows that EEG based emotion recognition performs better by engaging tactile sense. [ABSTRACT FROM AUTHOR]

Video delivery over error prone channels suffer from packet loss, which significantly deteriorates the reconstructed video quality. Multiple description coding (MDC) is an effective error resilient coding scheme for transmission of video over unreliable networks, where video is encoded into multiple descriptions having redundancy between them. The amount of redundancy between these descriptions has an important role in MDC, because it provides error resilience. In this paper, a new redundancy allocation scheme using visual saliency model is presented with an aim of achieving high decoding quality, when one or more than one descriptions are received at the decoder. The proposed MDC method splits the input video sequence into even and odd subsequences, which are independently encoded by using high efficiency video coding (HEVC) encoder. The missing frames in each description are predicted by using pixel interpolation and motion interpolation. The residual information is generated by using the interpolated frame and its adjacent frame from the other description, which represents the redundancy in each description. Residual information is encoded based on a visual saliency mask, which is generated using a global contrast based visual saliency model. In this work, two different modes are proposed for visual saliency based redundancy allocation to provide better perceptual quality, while decoding the descriptions. The proposed scheme is implemented on HEVC reference software HM 16.2 and compared with two state-of-the-art temporal subsampling based multiple description video coding methods. Experiments show that the proposed method performs better than the reference methods, both in lossless and packet erasure channel conditions. [ABSTRACT FROM AUTHOR]

Rapid increase in digital images demands effective and efficient image retrieval systems. In text based image retrieval, images are annotated with keywords based on human perception. A user query is composed of keywords according to his/her requirements. Query keywords are matched with the keywords associated with images, for retrieval. This process has been extended with ontology to resolve semantic heterogeneities. However, crisp annotation and retrieval processes could not produce the desired results because both processes involve human perception. To overcome this problem, we have proposed a retrieval system that makes use of fuzzy ontology for improving retrieval performance. For modeling the semantic description of an image, it is divided into regions in our dataset and then regions are classified into concepts. The concepts are combined into categories. The concepts, categories and images are linked among themselves with fuzzy values in ontology. The retrieved results are ranked based on the relevancy between the keywords of a query and images. For evaluating the performance of the proposed methodology, we have used both the objective and subjective measures. Experimental results show that the proposed system performs better than the existing systems in terms of retrieval performance. [ABSTRACT FROM AUTHOR]

The online version of the original article can be found at https://doi.org/10.1007/s11042-022-12060-6 B Correction to: Multimedia Tools and Applications b https://doi.org/10.1007/s11042-022-12060-6 The affiliations in the original publication were incorrectly assigned to all authors. Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. [Extracted from the article]