Search

Paper folding-and-punching tests are used to measure spatial abilities in humans. This paper presents a qualitative model for paper folding (QPF) and a computer game (Paper Folding Game) developed to apply and show the reasoning capabilities of the QPF model. This interactive game presents paper-folding activities intended to help users train and understand how to fold a paper to get a specific shape. Then, it presents paper-folding-and-punching tests to the players. The Paper Folding Game can automatically generate paper-folding-and-punching questions with varying degrees of difficulty depending on the number of folds and holes made, thus producing additional levels for training. The reasoning mechanisms in the QPF model are used by the Paper Folding Game to infer the right answer to each paper-folding-and-punching question. This reasoning capability allows the game to provide feedback to the players when they are wrong and also to create other plausible answers automatically, so that random question-answers are shown to the players in the master-mode. The Paper Folding Game has been implemented using Unity engine and it is available to download from GooglePlay and AppleStore for everyone to train their spatial reasoning skills.

Headline generation is a challenging subtask of abstractive text summarization, which its output should be a summary, shorter than one sentence. It would be precious to develop a dataset for the evaluation of abstractive summarization methods on this task in the Persian language. There are several datasets for headline generation in Persian, most of which are not large enough to be used by more sophisticated methods of text summarization, such as deep learning models. Moreover, all of these datasets are focused on daily news and there is no dataset for summarizing scientific Persian papers. In this article, we present “ElmNet,” a headline generation dataset of about 400,000 abstract/headline pairs of scientific papers, gathered from six major publishers for scientific articles in Persian. We, moreover, evaluate the performance of the most important deep learning-based headline generation methods, on the proposed dataset. The results prove the comparability of the performance of the state-of-the-art methods on this task, to their results on the existing English datasets.

In this paper, we aim to implement augmented reality (AR) on distant text documents or books. For this purpose, we propose a new paper-based AR system that can detect text documents in real scenes, markerize and identify them, estimate their relative 3D poses to the camera, and augment them with virtual contents. Unlike the previous paper-based AR systems (applicable to only close documents), the proposed system not only requires no detection of words or characters, but allows partial occlusions like the previous systems. In our experiments, the proposed system worked at 24 fps and could consistently achieve high identification rates for both occluded and unoccluded pages.

Physical paper, in its various forms (e.g. books, leaflets, catalogues), is extensively used in everyday activities, despite any advancements in digital technology and ICT. Inspired by the popularity of this medium, several research efforts have envisioned and pursued a new era of interactive paper, however several challenges still remain to be addressed. On the other hand, recent advancements towards Ambient Intelligence (AmI) and Smart environments, bear the promise of seamless integration of the physical and digital worlds in an intuitive and user-friendly manner. This paper presents InPrinted, a systematic and generic framework supporting physical paper augmentation and user interaction in Ambient intelligence environments. InPrinted enables natural multimodal user interaction with any kind of printed matter in smart environments, providing context aware and anticipation mechanisms, as well as tools and interaction techniques that support the development of applications incorporating printed matter augmentation. InPrinted has been put to practice in the development of various systems, including the Interactive Maps system, facilitating interaction with printed maps as well as their augmentation with digital information. The results of an in-situ observation experiment of the Interactive Maps system are reported, highlighting that interacting with augmented paper is quite easy and natural, while the overall User Experience is positive.

Paper checks may have complex background features including fine lines and patterns, which make forgery more difficult. Also, halftoning techniques are used to produce continuous tones and to prevent copies with void pantograph features. When these kinds of checks are scanned, Moire patterns may occur. These patterns make it difficult for customers to examine the scanned check images on ATM (Automated Teller Machine) displays. They also can decrease the classification accuracy of check recognition systems. In this paper, we propose an algorithm to enhance the perceptual quality of scanned check images by reducing the Moire patterns. The proposed algorithm consists of foreground extraction, Moire detection and Moire removal. Subjective image quality assessment was performed to evaluate the degree of improvement. Experimental results show that the proposed algorithm improves perceptual quality while maintaining check recognition accuracy.

The distortion introduced in the cover image is one of the most important factors influencing image steganography. The lower the distortion is, the higher the quality of the stego image, and thus the higher the resistance to steganalysis would be. This paper proposes an image steganographic method which employs the Hamiltonian paths in order to decrease the distortion produced by the LSB substitution scheme. The proposed method divides the cover image into non-overlapping blocks. For each block, the arrangement of the pixels of the block is first changed according to a Hamiltonian path. Next, the pixels whose LSB is not equal to the secret data are determined. The path that results in minimum distortion is chosen, and its binary code as well as the mismatched pixels are recorded. Repeating this process for all blocks, the pixels that should be modified during data embedding and a binary key that includes the codes of the best Hamiltonian path in all blocks are determined. Then, the new values of the second LSB of the changeable pixels are computed using writing on wet paper approach with regard to the compressed key. Finally, +1 or ?1 is added to each changeable pixel according to the value computed for its second LSB. By this strategy, the distortion of each pixel of the stego image is at most 1. Experimental results evaluated on 2000 natural images reveal that the proposed method can significantly decrease the image distortion and thus enhance security.

Multimedia field has dramatically grown in the last 20 years. We still remember the first ACM Multimedia Conference in Anaheim, CA in 1993, at whichmost participants even did not know the meaning of the word “multimedia”. In its early age, research in multimedia was focused on video and image compression and developing relevant standards, multimedia communication and synchronization, storage and retrieval, multimedia modeling, and creating first multimedia applications. Today, multimedia is interdisciplinary field, which is in tremendous explosion finding applications in practically all parts of our lives, from art, education, advertisement, and entertainment to engineering, medicine, business, scientific research, and many others. We created two special issues: Survey Papers in Multimedia by World Experts and Hot Research Topics in Multimedia We invited experts in the field both from academia and industry to contribute with their articles, and the response was overwhelming. This special issue “Survey Papers” contains 13 articles presenting the state of the art in the various areas of multimedia, from context modeling, semantic retrieval, personalized multimedia, to multimedia data mining, augmented reality, and others. The main objective of this special issue is to present the current status of multimedia field from leading researchers and practitioners worldwide. We thank all authors for their contributions in creating these two special issues. Guest Editors Ramesh Jain, Albert Del Bimbo, Tat-Seng Chua, and Borko Furht Multimed Tools Appl (2011) 51:1–4 DOI 10.1007/s11042-010-0681-1

Content-Based Image Retrieval (CBIR) is a well-known research topic from the computer vision domain which helps retrieve similar images from a dataset as per the specified query image. The retrieval performance is inadequate for benchmark datasets viz., Corel-1k, Corel-5k, Corel-10k, and Ghim-10k. In this paper, we have encountered the problem of the low retrieval rates of the CBIR system and the high dimensionality of the feature vectors. We have proposed the hybrid framework consisting of three different feature descriptors for robust retrieval performance. We have propounded the use of modified Multi-Scale Local Binary Pattern (MS-LBP), Color Coherence Vector (CCV), and Global Color Histogram (GCH) for image retrieval. We have exerted the modified MS-LBP because of its ability to capture more texture detail than Local Binary Pattern (LBP) at multiple scales. This larger filter size of MS-LBP makes it less vulnerable to noise and illumination than the conventional LBP descriptor. The CCV captures color with location information well enough, but it’s vulnerable to the brightened images whereas, the GCH operator covers brightness (less sensitive to brightness than CCV), rotation, scale, translation, camera viewpoint invariant features, but lacks spatial information. The proposed framework improves the feature selection process by blending the strength of each of these descriptors. This paper also targets the high dimensionality of the feature vector of the MS-LBP and GCH descriptors by exerting Principal Component Analysis (PCA). Moreover, Linear Discriminant Analysis (LDA) selects robust and optimal features for retrieval. The proposed method is compared with state-of-the-art literature on four benchmark datasets in terms of Average Retrieval Precision (ARP), Average Retrieval Rate (ARR), and Retrieval Time (RT). Experimental results show that the proposed method excels the examined research practices.

This paper mainly studies the Neural Style Transfer (NST) problem based on convolutional neural networks (CNN). Existing deep style migration algorithms do not mimic the styles to a reasonable position. To solve the problem, this paper proposes a multi-scale style transfer algorithm based on deep semantic matching. For purpose of guiding the correct migration of the style, we use the priori spatial segmentation and illumination information of the input image to integrate the deep semantic information. First, we find that spatial division and illumination analysis are two important visual understanding approaches for artists to make each painting decision. In order to simulate these two visual understanding approaches, this paper defines the DSS (deep semantic space), which contains spatial segmentation and contextual illumination information. The semantic exists in the form of CNN (convolution neural network) characteristic graph. Second, we propose a deep semantic loss function based on DSS matching and nearest neighbor search to optimize the effect of deep style migration. Third, we propose a multi-scale optimization strategy for improving the speed of our method. The experiments show that our method can reasonably synthesize images in spatial structures. The placement of each style is more reasonable and has a good visual aesthetic.

Hyperspectral imaging has been extensively investigated as an emerging, promising technique for measuring the quality and protection of horticultural and agricultural products over the past 20 years. This technology evolved from remote sensing and joins the machine vision and point spectroscopy realms to provide superior image segmentation for defect and contamination detection. In this paper, we have incorporated spatial and spectral information into hyperspectral imaging techniques. It can efficiently and non-destructively provide helpful information on both external physical and internal chemical characteristics of agricultural and food products. This paper has reviewed the sum of crucial aspect applications of hyperspectral imaging in precision agriculture. This paper reviews hyperspectral imaging’s current and past development in the agriculture industry, such as classification, chromatic, climatic, convergence, etc. This analysis aims to refer to potential work on key issues and promote deployed end-user solutions to fulfill the existing global sustainability objectives.

An important goal of color image gray-scale is to keep the edge details of the original color image as much as possible. In many cases, the degree of feature discrimination is maintained, but in some cases, edge details are still lost or blurred. Therefore, this paper first uses an improved non-linear global mapping grayscale method to grayscale the color image, and then proposes a grayscale image detail enhancement algorithm based on rolling guided filtering. The method in this paper is to enhance the edge details of the grayscale image by rolling guided filter processing on the basis of the grayscale image. In addition, the rolling-guided filter is a local linear model with better edge retention characteristics, which can overcome the defect that other filters are prone to gradient flips on the edges where the gray level of the image changes sharply, causing the image to appear “false edges”. The experimental results show that when the traditional method loses or blurs the detailed features, the method in this paper can maintain better detailed features.

In the recent digitization era, image hashing is a key technology, including image recognition, authentication and manipulation detection, among many multimedia security applications. The primary challenge in hashing schemes is to extract its robust feature. For a better understanding and design of a robust image hashing algorithm, it is very crucial to look into few important parameters like discrimination, robustness, reliability, etc. This paper reflects a detailed study of the existing literature on hashing-based image authentication techniques. This work provides a systematic overview and highlights the merits and demerits associated with various state-of-the-art techniques. In particular, the basic features and categories of image authentication techniques based on hashing are explored along with their properties. Moreover, different performance measures such as output metrices, receiver operating characteristics (ROC) parameters, execution time, etc., have been discussed in this work. The paper also compares the performances of various existing algorithms related to different content preserving operations on diverse data sets. This paper summarizes all the techniques and provides the most optimum solutions in regard to image hashing techniques based on different parameters.

Optic disc (OD) analysis is an important stage in detecting retinal diseases and existing approaches are not suitable for analyzing multiple examinations in a single process. This paper proposes a new unified method to detect (i) OD center, (ii) OD boundary, and (iii) enhancement of vasculature within the OD region in a single algorithm. This paper presents a unique strategy, which involves L2 norm of contourlet subbands of retinal images to locate the OD center. A novel OD boundary tracing approach that integrates morphological operations, and segmentation region growing techniques, with the aid of the OD center as a seed point, is presented. Among the two novel OD vasculature enhancement techniques, one of which involves image sharpening to improve the local contrast followed by histogram equalization to increase the overall contrast of the fundus image. Edge superimposing with the flood fill technique is another approach used for vasculature enhancement. The algorithm is tested on DRIVE (40), STARE (81), MESSIDOR (1200), E-ophtha (87), Diaretdb1 (89) images, out of which 40, 77, 1182, 87, and 87 images are detected correctly. The Figure of Merit (FOM) is used to confirm the correctness of the boundary tracking performance (97.15%). EME, SSIM, and other parameters are used to assess the efficiency of the vasculature enhancement technique.

Hyperspectral face recognition plays an important role in remote sensing. However, it faces many challenges such as difficulty in data acquisition, low signal to noise ratio (SNR), and high dimensionality. In this paper, we develop a novel method for hyperspectral face recognition by extracting histogram of oriented features (HOG) and using collaborate representation-based classifier (CRC) to classify unknown face data cubes. To improve overall classification rates, we also implement noise reduction in hyperspectral face data cubes. We also crop the face images by a bounding box and use this bounding box image to classify the testing faces. Experiments show that our new method outperforms several existing methods for both the PolyU-HSFD dataset and the CMU-HSFD dataset for hyperspectral face recognition. The contribution of this paper is the following: We introduce the MNF-based denoising method in this paper, which is new to the best of our knowledge. We also combine it with HOG features and CRC classifier so that better recognition rate can be achieved.

Fog computing is considered a formidable next-generation complement to cloud computing. Nowadays, in light of the dramatic rise in the number of IoT devices, several problems have been raised in cloud architectures. By introducing fog computing as a mediate layer between the user devices and the cloud, one can extend cloud computing's processing and storage capability. Offloading can be utilized as a mechanism that transfers computations, data, and energy consumption from the resource-limited user devices to resource-rich fog/cloud layers to achieve an optimal experience in the quality of applications and improve the system performance. This paper provides a systematic and comprehensive study to evaluate fog offloading mechanisms' current and recent works. Each selected paper's pros and cons are explored and analyzed to state and address the present potentialities and issues of offloading mechanisms in a fog environment efficiently. We classify offloading mechanisms in a fog system into four groups, including computation-based, energy-based, storage-based, and hybrid approaches. Furthermore, this paper explores offloading metrics, applied algorithms, and evaluation methods related to the chosen offloading mechanisms in fog systems. Additionally, the open challenges and future trends derived from the reviewed studies are discussed.

There have been considerable advancements in multimedia technologies over the past 5 years. It has been observed that state-of-the-art multimedia systems face three broad categories of challenges: (1) Dependency on continuous network connections, (2) Data-sharing applications & collaboration, and (3) Security issues. Among these, security vulnerability poses a major threat to modern multimedia systems. Therefore, it is imperative to carefully investigate the security issues that can endanger wireless and mobile communications. At present, multimedia security research mainly focuses on wireless traffic monitoring, wireless system attacks, and wireless and mobile security. In this paper, we have used the network attack-type, “Reconnaissance”, which contains two types of malicious activities: (1) OS scanning, and (2) Fuzzing. The goal of this paper is to quantify multimedia security risks due to Fuzzing by using various types of machine learning models. The highest accuracy i.e., 96.8%, is obtained using the XGBoost classifier, which is good compared to the existing models present in the literature. This is the first paper, to the best of our knowledge, that uses machine learning methods to differentiate between benign and malignant Fuzzing attacks.

This paper mainly considers the humanoid robot vision system and the type of its joint structure, and designs a set of effective planning strategies for the robots grabbing the chess pieces from the chess board and places them into the designated positions and ensures their success in the chess game. The core content of this paper is the design of the humanoid robot performing the operation of chess. The main procedure of the humanoid robot performing the operation of the next chess game is as follows: firstly, the image is acquired by the vision system mounted on the camera of the humanoid robot; secondly, the pixel position of the checkerboard corner point is obtained through the image processing; furthermore, a target position is set for the humanoid robot and then the improvement is performed to let it become more efficiencies. The monocular distance ranging algorithm is used to obtain the actual position of the corner point of the checkerboard. Finally, the robot kinematics accurately controls the humanoid robot to perform an operation of grabbing the chess to place the chess to the specified position successfully. The superiority of the proposed work is that it can be for various complex conditions. In the three-dimensional environment, the visual system of the NAO robot was firstly used to perceive its surrounding environment, and then the image processing technology was used to identify the chess position. The first general remark is that this work can be conceived as a good technological/engineering achievement.

Human Action Recognition is a research hotspot in the field of computer vision. However, due to the complexity of the environment and the diversity of actions, Human Action Recognition still faces many challenges. At the same time, traditional CNN has problems such as single feature scale, decreased accuracy of deep network, and excessive network parameters. Aiming at the above research problems, this paper proposes a novel residual network model based on Multi-scale Feature Fusion and Global Average Pooling. The model uses a Multi-scale Feature Fusion module to extract feature information of different scales, enriches spatial-time information. At the end of the network, Global Average Pooling is used to instead of a Fully Connected layer. Compared with a Fully Connected layer, Global Average Pooling will dilute the combination of the relative positions of different features. Therefore, the features trained by convolution are more effective. In addition, Global Average Pooling can realize direct mapping between output channels and feature categories to reduce excessive model parameters. The model in this paper is verified on the UT-interaction dataset, UCF11 (YouTube Action dataset), UCF101 dataset and CAVIAR dataset. The results show that compared with the state-of-the-art approaches, this approach has high recognition accuracy and excellent robustness, and has excellent performance on datasets with complex backgrounds and diverse action categories.

This paper presents a novel encryption method based on individual block ciphering using Catalan random walks. This paper aims to offer some new possibilities of multimedia data protection to realize the rights of participants in the multimedia distribution chain (image, text, video, sound). Also, the emphasis is on advanced analysis of Catalan numbers and their combinatorial representations in multimedia security. The proposed method consists of five phases: conversion, division, selection, encryption, and generation. We presented the application of our method in ensuring the security of multimedia content. The proposed method was implemented in Java. In the experimental testing, we provide the time and space complexity of Catalan keys generation and Maurer’s universal statistical test for the proposed method. Also, we state security analysis using machine learning methods and comparative analysis with existing methods of encrypting data into a blockchain.

Securing the network is a top priority for all organizations. Many high-profile breaches in the field of cybersecurity claim to threaten the global economy. In addition to that, they impose a threat for system infiltration and data theft, clearly warning organizations and companies to strengthen the existing cybersecurity frameworks and policies. In this paper, a real-time blockchain based integrated solution is proposed that fortifies the security against cyber-attacks. The scheme introduces the concept of Cyber-Soldiers that work together to update the Artificial Neural Network (ANN) based framework, which exists on the cloud for incentives. The enterprises can subscribe to this scheme monthly. This research work introduces its cryptocurrency CyberCent for the network. Also, smart contracts are deployed on the conditionally public blockchain to carry out monetary transactions. The experimental analysis with state-of-art compression of the framework is also demonstrated in the paper.

In smart grid infrastructure, multimedia communication plays an important role in various applications, for instance, load monitoring, automatic smart meter reading, and energy management. Energy management has gained widespread popularity with the increasing energy demand. The goal of this paper is to explore multiagent-based Reinforcement Learning (RL) for multicarrier (i.e., electricity and gas) Residential Energy Management (REM) systems with data security. It facilitates the possibility for a separate Demand Response Program (DRP) for every energy component accelerates with computational aspects of RL. This paper proposes Q-MSEM, a Q-learning-based Multiagent and Secure Energy Management scheme for the optimal solution of REM problems using RL and Ethereum Blockchain (EB) to reduce energy load and decrease energy costs. Then, Q-MSEM uses Ethereum Smart Contract (ESC) to covenants data security issues using off-chain storage, i.e., InterPlanetary File System (IPFS) to handle data storage costs issues. Experimental results disclose the effectiveness of the proposed Q-MSEM scheme in terms of reduction in load, energy cost (15.82%), and data storage cost.

Contrast enhancement is an important pre-processing task in any Image Analysis (IA) system. In this paper, we formulate the image contrast enhancement problem as an optimization problem where the goal is to optimize the pixel intensity values of an input image to obtain a contrast enhanced version of the same. This optimization task is executed by suitably customizing a nature-inspired optimization algorithm called Selfish Herd Optimizer (SHO). The optimization problem is solved using two different solution representations: pixel wise optimization (SHO(direct)) and transformation function based optimization (SHO(transformation)). Moreover, an ablation study is performed to select the most appropriate parameters which can be used in fitness measure for this optimization problem. On experimenting over the popular Kodak image dataset, it has been observed that the proposed methods outperform many existing methods published recently. Further comparisons indicate that the direct approach performs better than its transformation counterpart. This paper further investigates the robustness of SHO(direct) approach by applying it to enhance the degraded document images of H-DIBCO 2018.

A novel color image encryption algorithm based on the fractional order laser chaotic system and DNA mutation principle is proposed in this paper. Using phase diagram, Lyapunov exponential spectrum, bifurcation diagram and C0 complexity, the dynamic characteristics of the fractional-order laser chaotic system are analyzed, and the hardware circuit simulation of the chaotic system is realized on the DSP platform. Based on the comprehensive analysis of the system, we have designed an image encryption algorithm. Firstly, the chaotic sequence and Arnold matrix are used to scramble the value of the image. Then, in the diffusion link, DNA diffusion algorithm and DNA mutation theory were used for the first time, which increased the randomness of the matrix. The performances of the designed encryption scheme are analyzed by key space, correlation coefficients, information entropy, histogram, differential attacks and robustness analysis., the experimental results prove that the algorithm has strong encryption capabilities and can withstand multiple decryption methods. The new color image encryption scheme proposed in this paper can realize the secure communication of digital images.

Smart city systems do not only handle information but also handle many critical aspects of citizens’ lives, including automobiles, healthcare, banking, etc. Smart city systems need to be both safe and secure systems. Therefore, this paper uses formal methods to develop safe and secure systes for smart city. However, because the creation of formal models is considered a difficult task that requires experience in modeling and a strong mathematical background, many techniques and patterns should be introduced to simplify the construction of formal models to make them more accessible for the development of systems, especially for systems pertaining to smart city services. In this paper, we propose an approach that uses Event-B theories to support data refinement and reduce the burden of proof for constructing Event-B formal models based on reused modeling components. In particular, we develop Event-B theories for common data structures based on pointers that could be used as patterns to carry out the refinement of abstract data structures, such as sets, lists and sequences. These data structures can offer solutions for the management of smart city services. We applied the proposed approach to construct Event-B models for task lists in a real-time operating system (FreeRTOS) to evaluate our approach. We successfully were able to model five task lists in FreeRTOS and developing data refinement levels without the need of spending long time in modeling or re-carrying any proofs.

The prediction of abnormality in the heart functionality at an early stage increases the chances of saving the life of people. Thus, this paper proposes a technique which predicts the abnormality in the functionality of the heart using heart rate in the form of beats per minute using wearable devices and deep learning model. The devices used are wrist strap and devices that can be fixed near the chest of the person or back of the person where heart beat can be detected. The proposed system is divided into 3 modules: (1) data collection and processing module, (2) prediction module and (3) communication module. First module is used to collect data and process, while prediction module predicts the abnormal functionality of the heart using deep learning model. One of the advantage of the proposed work in this paper is communication module as the communication is given to the doctor who can perform analysis before the patient reaches the hospital. A message is sent to the ambulance so that it reaches the destination on time. The message related to first aid is sent to two dear ones and the patient such that appropriate measures can be taken. The proposed technique is evaluated in terms of sensitivity, specificity, F1–Score, time and ROC curve metrics. It is also compared with the Two Stage Neural Network and TSNN and proved to be performing better.

With the development of science and technology and the progress of human beings, intelligence is gradually integrated into human daily life. The smart city uses innovative technology to manage and operate cities intelligently. Through the research of facial expression recognition technology, this paper explores the application of facial expression recognition in smart city construction. In this paper, a hybrid neural network structure is proposed, which includes Sparse Autoencoder and Convolutional Neural Network (SCNN). The network reconstructs the input data by Sparse Autoencoder, so as to learn the approximate value between the original data and the reconstructed data, and obtain more high-dimensional abstract features. Then, combined with the Convolutional Neural Network, the features are further extracted and dimensionally reduced. The model can effectively solve the problem that the shallow network structure can not fully extract image features and train the model with a small number of samples. In this paper, CK+, FER2013 and Oulu-CASIA databases are used for Cross-Validation of the model. The experimental results show that the model has achieved good results in both databases. Compared with other methods, the accuracy of this model has been greatly improved.

Generation of a clear, correct, concise, complete, and coherent linguistic description of the visual patterns in a medical image is a challenging task. Unfortunately, many radiologists fail to satisfactorily perform this task due to various reasons such as workload, scant time, and fatigue. Although AI-based computer-aided detection (CADe) and computer-aided diagnosis (CADx) systems have been developed for observing and interpreting patterns in medical images, they do not generate radiological reports. In recent years, a lot of research has been done to develop automated report generation methods. This paper presents a comprehensive survey of all such methods specifically developed for chest radiographs. It consolidates information about standard chest X-ray datasets, state-of-the-art report generation methods, evaluation metrics, and their results. Deep learning-based techniques for automatically generating chest radiographic reports have been classified and discussed in detail. The encoder-decoder-based techniques have been meticulously categorized for a better understanding of the developments in this area. This paper is also beneficial for the researchers interested in developing automatic report generation systems for imaging modalities other than chest radiographs.

The classification of birdsong has very important signification to monitor the bird population in the habitats. Aiming at the birdsong dataset with complex and diverse audio background, this paper attempts to introduce an acoustic feature for voice and music analysis: Chroma. It is spliced and fused with the commonly used birdsong features, Log-Mel Spectrogram (LM) and Mel Frequency Cepstrum Coefficient (MFCC), to enrich the representational capacity of single feature; At the same time, in view of the characteristic that birdsong has continuous and dynamic changes in time, a 3DCNN-LSTM combined model is proposed as a classifier to make the network more sensitive to the birdsong information that changes with time. In this paper, we selected four bird audio data from the Xeno-Canto website to evaluate how LM, MFCC and Chroma were fused to maximize the birdsong audio information. The experimental results show that the LM-MFCC-C feature combination achieves the best result of 97.9% mean average precision (mAP) in the experiment.

A convenient, accurate and well-known way toward any supervised task is using Nearest Neighbor approach or its variants. However, there has been little attempt toward improving interpretability by human and providing a classical optimization of Synthetic Reduced Nearest Neighbor. To tackle these issues, this paper provides a novel optimization of Synthetic Reduced Nearest Neighbor based on Expectation Maximization (EM-SRNN). Reduced Nearest Neighbor model consists of a subset of the samples from the trainset that has similar accuracy to Nearest Neighbor. Synthetic Reduced Nearest Neighbor relaxes the model to learn K synthetic samples (or prototypes/centroids) in the space of dataset. Therefore, inspired by EM algorithm for K-means, we propose a novel optimization based on EM algorithm to learn EM-SRNN by iterating over the centroids of the model and assignment of training samples to the centroids. The first step consists of optimizing the position of each centroid based on the assignment of the samples to the centroid and the second step consists finding optimal assignments and labels of the centroids. The EM-SRNN is interpretable since the centroids exist inside the space of training samples. Additionally, the centroids represent the multiple modalities (or sub-clusters) of the classes that are interpretable by human. These properties make this type of interpretability unique, hence, making this model suitable for various studies that are related to interpretability by human, such as image processing and epidemiological studies. In this paper, analytical aspects of problem are explored and it is shown that computational complexity of proposed optimization is linear over size of the trainset. Finally, EM-SRNN shows superior or similar performance when compared with several other interpretable and similar state-of-the-art models, such as trees and kernel SVMs.

The video file is a collection of image sequential; this image sequence holds both spatial and temporal information. Optical flow and motion history images are two well-known methods for the identification of human activities. Optical flow describes the speed of every individual pixel point in the picture. Still, this information about the motion cannot represent the complete action and different movement speeds. The durations of Local body parts show almost similar intensity in the Motion history image. Therefore, similar actions are not identifying with good precision. In this paper, a deep convolutional neural model for human activities recognition video has been proposed in which multiple CNN streams are combined. The model combines spatial and temporal information. Two fusion schemes, i.e. Average fusion and convolution fusion of spatial and temporal stream, are discussed in this paper. The proposed method performs better than other approaches based on human activity recognition methods on a benchmark dataset, namely UCF101 and HMDB51.Average fusion score 95.4% test accuracy and convolution fusion score 97.2% test accuracy on UCF101 and for HMDB51, average fusion score 84.3% and convolution fusion score 85.1% respectively.

Video standards are crucial for exchanging video content, enabling a myriad of services and supporting a wide variety of devices ranging from personal devices to clouds and IoT. One of the core requirements in video standards is the rate control that regulates the bit allocation and picture quality. This paper presents an overview of rate control techniques in the HEVC video coding standard. While providing an insight into the rate control mechanism specific to HEVC, it describes the basic operating principle of rate control algorithms, including their essential parameter, outputs, and performance measures. We review rate control in past coding standards and bring out the basic features of HEVC that drive the need for new rate control algorithms. Alongside, we delineate the Rate-Distortion model-based taxonomy of various algorithms, including their classification criteria. The paper gives out another classification of the rate control algorithms based on their basic principle and mechanisms. The article also explains the scalable extension of HEVC, namely SHVC, while highlighting some of the possible SHVC rate control design challenges. Finally, we present some of the unresolved research issues in HEVC rate control and outline possible future research directions.

In recent years, there have been frequent jellyfish outbreaks in many offshore areas worldwide, which have severely affected marine fishery production, coastal tourism, coastal industrial cooling water systems, and marine ecology. Achieving the monitoring of jellyfish plays a vital role in solving the problems mentioned above. However, the research on jellyfish is still in the primary stage. Jellyfish detection technology based on deep learning is gradually being applied to jellyfish detection due to its high efficiency and accuracy, but it is not systematic enough and can identify few jellyfish species. So this paper studies a jellyfish detection algorithm based on deep learning. Based on convolution neural network theory and digital image processing technology, 10 species of jellyfish and fish are detected. Because the quality of underwater images affects the detection accuracy, to further improve the accuracy of the detection algorithm, this paper studies the underwater image processing algorithm. Experimental results show that the image quality is better after applying the three algorithms of dark channel prior algorithm, quadratic combining gray world and perfect reflection algorithm, and contrast-limited adaptive histogram equalization algorithm, which is more conducive to detection. Then, deep learning theory is applied to classify jellyfish. By comparing the AlexNet and GoogLeNet backbone networks’ classification results, the accuracy of the jellyfish classification task based on the GoogLeNet backbone network is 96.21%, which is better than AlexNet. Finally, the Faster R-CNN algorithm is used to detect jellyfish, and its detection performance is analyzed based on the two backbone networks mentioned above. The results show that the Faster R-CNN algorithm based on GoogLeNet has a higher detection accuracy in the jellyfish detection task, with an average detection accuracy of 74.96%. In addition, we set up a new jellyfish data set, which includes 25,344 images. The images were divided into 11 species, including 10 species of jellyfish and one fish species. The paper’s research lays a theoretical and technical foundation for the subsequent construction of a real-time monitoring system for underwater jellyfish optical imaging, plays an important role in the development of jellyfish monitoring technology, and provides valuable information for marine biologists.

The paper aims to show how blockchain, particularly Ethereum private blockchain, performs as a record keeper for Malaysian Muslim marriage records as a case study. The study also aims to demonstrate how blockchain technology conserves marriage data and improves data sharing efficiency involves accessing data from the blockchain and multiple users through proof of concept (POC). This POC also intended to seek evidence of immutability, transparency, and authenticity claims by the technology to ensure the record's security is a high priority. The study implements a smart contract on the Ethereum platform in the Government of Malaysia network environment and, the results provide significant evidence of private Ethereum blockchain principles of high availability, efficiency, immutability, transparency, and truthfulness particularly in record keeping. Future research can investigate other blockchain technology for other blockchain initiatives, such as blockchain disruptors. This paper is useful for management researchers and guides management practice for decision-making when determining whether the technology can use as a record-keeper. This paper proposes POC to design and develop record-keeping of Malaysian Muslim marriage records using Ethereum blockchain technology.

International audience; This paper presents a novel method for image manipulation iden- tification of natural images in JPEG format. The image forgery detection technique is based on a signal-dependent noise model that is relevant to de- scribe a natural image acquired by a digital camera. This parametric model is characterized by two fingerprints which are used to falsification identification. The problem is cast in the framework of the hypothesis testing theory. For practical use, the Generalized Likelihood Ratio Tests (GLRT) are presented and their performance is theoretically established. There are different types of image forgery which have been considered in this paper for example re- sampling, Gaussian filtering and median filtering. Experiments with real and simulated images highlight the relevance of the proposed approach.

The emergence of biometric technology provides enhanced security compared to the traditional identification and authentication techniques that were less efficient and secure. Despite the advantages brought by biometric technology, the existing biometric systems such as Automatic Speaker Verification (ASV) systems are weak against presentation attacks. A presentation attack is a spoofing attack launched to subvert an ASV system to gain access to the system. Though numerous Presentation Attack Detection (PAD) systems were reported in the literature, a systematic survey that describes the current state of research and application is unavailable. This paper presents a systematic analysis of the state-of-the-art voice PAD systems to promote further advancement in this area. The objectives of this paper are two folds: (i) to understand the nature of recent work on PAD systems, and (ii) to identify areas that require additional research. From the survey, a taxonomy of voice PAD and the trend analysis of recent work on PAD systems were built and presented, whereby the recent and relevant articles including articles from Interspeech and ICASSP Conferences, mostly indexed by Scopus, published between 2015 and 2021 were considered. A total of 172 articles were surveyed in this work. The findings of this survey present the limitation of recent works, which include spoof-type dependent PAD. Consequently, the future direction of work on voice PAD for interested researchers is established. The findings of this survey present the limitation of recent works, which include spoof-type dependent PAD. Consequently, the future direction of work on voice PAD for interested researchers is established.

Capturing of infrared images is an easy task but perceptual visualization is difficult due to environmental conditions such as light rain, partly cloudy, mostly cloudy, haze, poor lightening conditions, noise generated by the sensors, geographical distance and appearances of the objects. To improve the human perception and quality of the infrared images for further processing like image analysis, image enhancement is an essential process. This paper provides a detailed review of various image enhancement techniques from contrast stretching to optimization methods used in infrared images. It also discusses the existing infrared image enhancement techniques as group such as histogram based methods, filter based methods, transform domain based methods, morphological based methods, saliency extraction methods, fuzzy based methods, learning methods, optimization methods and its popular algorithms also address the countless issues. Some of the existing image enhancement methods (Histogram Equlization, Max-median filter, Top-Hat transform) and infrared image enhancement methods (multi-scale top-hat transform, adaptive infrared image enhancement) are implemented along with the adaptive fuzzy based infrared image enhancement method and its obtained results evaluation is done on subjective and objective ways. From the results observed that the fuzzy based method works well for both subjective and objective evaluation. The paper aims to provide a complete study on image enhancement techniques and how they specially utilized while dealing with infrared images. In addition, the paper helps the researchers to select the suitable infrared image enhancement techniques for their infrared image application needs.

The transfer of IPv6 packets over the low-power personal area network (6LoWPAN) using IPv6 routing protocol enables small devices with limited processing power to transmit wirelessly. Privacy enabled disjoint and dynamic address auto-configuration (PeDAAC) is one of the 6LoWPAN protocols designed to ensure the privacy of the nodes by changing IPv6 and MAC addresses. One of the main purposes of this protocol is to keep the identity of the sender and receiver secret, so it can have different applications, such as hiding the identity of patients in the hospital or military applications to send data anonymously. PeDAAC is a dynamic protocol whose purpose is to create a conflict-free, auto-configuring IPv6 addressing scheme, eliminating the need for duplicate address detection. It results in lower latency and optimal communication costs, and the IPV6 addresses of each node will be unique. Two successful attacks on the PeDAAC protocol are discussed in this paper. An adversary can attack the availability of the nodes and can cause a denial of service attack. Also, by detecting the node’s identity, changes over time can gather more messages, which increases the likelihood of successful eavesdropping. The greatest common divisor-based lightweight improvement (GCDLi) is a proposed lightweight extension of this paper to PeDAAC protocol to protect against these attacks. Beyond that, it counteracts two types of security threats: eavesdropping and denial of service attack. The proposed extension of the protocol improves the privacy and anonymity of the nodes. The time complexity of the proposed method is $$ \theta \left(\log n\right) $$ , and due to the limited range of input values (maximum 16 bit), it is considered a lightweight improvement.

The availability of videos has grown rapidly in recent years. Finding and browsing relevant information to be automatically extracted from videos is not an easy task, but today it is an indispensable feature due to the immense number of digital products available. In this paper, we present a system which provides a process to automatically extract information from videos. We describe a system solution that uses a re-trained OpenNLP model to locate all the places and famous people included in a specific video. The system obtains information from the Google Knowledge Graph related to relevant named entities such as places or famous people. In this paper we will also present the Automatic Georeferencing Video (AGV) system developed by RAI (Radiotelevisione italiana, which is the national public broadcasting company of Italy, owned by the Ministry of Economy and Finance) Teche for the European Project “La Citta Educante” (The Educating City: teaching and learning processes in cross-media ecosystem) Our system contributes to The Educating City project by providing the technological environment to create statistical models for automatic named entity recognition (NER), and has been implemented in the field of education, in Italian initially. The system has been applied to the learning challenges facing the world of educational media and has demonstrated how beneficial combining topical news content with scientific content can be in education.

Semantic similarity assessment between concepts is an important task in many language related applications. In the past, many approaches to assess similarity of concepts have been proposed by using one knowledge source. In this paper, some limitations of the existing similarity measures are identified. To tackle these problems, we propose an extensive study for semantic similarity of concepts from which a unified framework for semantic similarity computation is presented. Based on our framework, we give some generic and flexible approaches to semantic similarity measures resulting from instantiations of the framework. In particular, we obtain some new approaches to similarity measures that existing methods cannot deal with by introducing multiple knowledge sources. The evaluation based on eight benchmarks, three widely used benchmarks (i.e., M&C, R&G, and WordSim-353 benchmarks) and five benchmarks developed in ourselves (i.e, Jiang-1, Jiang-2, Jiang-3, Jiang-4, and Jiang-5 benchmarks), sustains the intuitions with respect to human judgements. Overall, some methods proposed in this paper have a good human correlation (Pearson correlation with human judgments and Spearman correlation with human judgments) and constitute some effective ways of determining semantic similarity between concepts.

Color music has attracted great interest in real applications. However, the mismatching problem between music and color has not been resolved. This paper proposes a novel mechanism to map the corresponding relation between music and color, which is embedded into a device with a micro-processor to play music color flashing. The proposed mechanism deduces perfect fifth relation among the wavelengths of lights and determines 12 colors corresponding to musical notes of Twelve-tone equal temperament is determined. Specifically, when a piece of music is playing, the audio signal is sampled and transformed by Fast Fourier Transform (FFT). The method can judge color corresponding to a note and research the mixed light effect of RGB LED driven by PWM outputs. Extended experiments show that the effect of music playing with matching colors flashing in real time is reached, and the color of the mixed lights can automatically match with arbitrary music being played. The paper can reveal relationships between music and color from the perspective of frequency spectrum, and promote the development of the color music, which has broad applications.

The usage of productive baseband receivers are portrayed by reasonable computational load is an urgent point in the advancement of transmission frameworks misusing assorted qualities in various spaces.Several research works have been published in the literature ofspace-time block codes multi-carrier code-division multiple-access (STBC MC-CDMA)Systems based on minimum conditional bit error rate Criterion. As like, in our previous paper, an efficient STBC communication system was performed using oppositional krill herd algorithm (OKHA). To overcome the drawbacks in the previous work, in this work we proposed a hybrid Particle swarm optimization (PSO) and group search optimization (GSO), called PSGSO based Multi-User Detection in STBC MC-CDMA Systems. In this paper, we use the Alamouti’s STBC encoder for transmitting sequence and the channel estimation matrix is optimally evaluated using PSGSO algorithm. The simulation results show that the proposed algorithm is properly reducing the bit error rate (BER), symbol error rate (SER) and properly increase the spectral efficiency value compare to other techniques.

Visual cryptography is an encryption technique that decomposes secret images into multiple shares. These shares are digitally or physically overlapped to recover the original image, negating the need for complex mathematical operations or additional hardware. There have been many variations of visual cryptography proposed over the years, each addressing different problems or to fulfill different security requirements. Existing review papers on the area only cover certain types of visual cryptography or lack comparisons between the various schemes. To address this gap, this paper provides broad overview of the area to aid new researchers in identifying research problems or to select suitable visual cryptography methods for their desired applications. For more veteran researchers in the area, our paper provides the most up-to-date coverage of the state-of-the-art. We first provide an introduction to the various categories of visual cryptography techniques, including a discussion on recently proposed schemes. These schemes are then compared in terms of their features, performance metrics, advantages and disadvantages. Compared to prior work, we extend the number of comparison metrics to include signal-to-noise ratio and the type of shares. Over 40 visual cryptography schemes that have been proposed in the past two decades were analyzed and compared. Our findings indicate that existing problems such as pixel expansion, poor quality of recovered image quality, computational and memory complexities still exist, and a optimizing the trade-off between these requirements still requires further investigation. We conclude the paper with a discussion of these open problems and future research directions.

As a research hotspot in the field of robotics, Simultaneous localization and mapping (SLAM) has made great progress in recent years, but few SLAM algorithms take dynamic or movable targets in the scene into account. In this paper, a robust new RGB-D SLAM method with dynamic area detection towards dynamic environments named GMSK-SLAM is proposed. Most of the existing related papers use the method of directly eliminating the whole dynamic targets. Although rejecting dynamic objects can increase the accuracy of robot positioning to a certain extent, this type of algorithm will result in the reduction of the number of available feature points in the image. The lack of sufficient feature points will seriously affect the subsequent precision of positioning and mapping for feature-based SLAM. The proposed GMSK-SLAM method innovatively combines Grid-based Motion Statistics (GMS) feature points matching method with K-means cluster algorithm to distinguish dynamic areas from the images and retain static information from dynamic environments, which can effectively increase the number of reliable feature points and keep more environment features. This method can achieve a highly improvements on localization accuracy in dynamic environments. Finally, sufficient experiments were conducted on the public TUM RGB-D dataset. Compared with ORB-SLAM2 and the RGB-D SLAM, our system, respectively, got 97.3% and 90.2% improvements in dynamic environments localization evaluated by root-mean-square error. The empirical results show that the proposed algorithm can eliminate the influence of the dynamic objects effectively and achieve a comparable or better performance than state-of-the-art methods.

The integration of the Internet of Things in medical systems referred to as the Internet of Medical Things (IoMT), which supports medical events for instance real-time diagnosis, remote monitoring of patients, real-time drug prescriptions, among others. This aids the quality of services provided by the health workers thereby improve patients’ satisfaction. However, the integrity and confidentiality of medical information on the IoMT platform remain one of the contentions that causes problems in medical services. Another serious concern with achieving protection for medical records is information confidentiality for patient’s records over the IoMT environment. Therefore, this paper proposed a Crypto-Stegno model to secure medical information on the IoMT environment. The paper validates the system on healthcare information datasets and revealed extraordinary results in respect to the quality of perceptibility, extreme opposition to data loss, extreme embedding capability and security, which made the proposed system an authentic strategy for resourceful and efficient medical information on IoTM platform.

Authenticity in multimedia information retrieval system is one such parameter which is of utmost in demand. One of such parameters is the biometric based authentication system, as seen in case of human the system proves its importance in almost each and every field. But in case of animals it becomes quite complex to recognize them using unique biometric parameter. This paper presents a modified 10 layered DenseNet deep learning framework which is implemented for the identification of Humpback whale, in an online challenge conducted by the kaggle, using biometric parameter. The proposed dense model minimizes the traditional limitation of vanishing gradient problem and outperforms as compared with other methods exists in the literature. This paper also proposed an hand free flow algorithm for ROI segmentation. In addition, the proposed methodology also uses CLAHE as a preprocessing method and focused on the feature re-usability with an optimized number of parameters used for recognition. During the experimentation, validation of the proposed model is found satisfactory with an accuracy of 93.21%. The test set results are provided by the kaggle with an AUC-ROC result over public and private scoreboard with an accuracy of 94.47% and 92.54% respectively. The comparative result with other deep methodologies suggests that the highest accuracy gained is 95.55% by Se-ResNet50 by some other learderboard and proposed model is found at second place.

This paper considers the area of digital forensics (DF). One of the problem in DF is the issue of identification of digital cameras based on images. This aspect has been attractive in recent years due to popularity of social media platforms like Facebook, Twitter etc., where lots of photographs are shared. Although many algorithms and methods for digital camera identification have been proposed, there is lack of research about their robustness. Therefore, in this paper the robustness of digital camera identification with the use of convolutional neural network is discussed. It is assumed that images may be of poor quality, for example, degraded by Poisson noise, Gaussian blur, random noise or removing pixels’ least significant bit. Experimental evaluation conducted on two large image datasets (including Dresden Image Database) confirms usefulness of proposed method, where noised images are recognized with almost the same high accuracy as normal images.

Robotics is one of the most emerging technologies today, and are used in a variety of applications, ranging from complex rocket technology to monitoring of crops in agriculture. Robots can be exceptionally useful in a smart hospital environment provided that they are equipped with improved vision capabilities for detection and avoidance of obstacles present in their path, thus allowing robots to perform their tasks without any disturbance. In the particular case of Autonomous Nursing Robots, major essential issues are effective robot path planning for the delivery of medicines to patients, measuring the patient body parameters through sensors, interacting with and informing the patient, by means of voice-based modules, about the doctors visiting schedule, his/her body parameter details, etc. This paper presents an approach of a complete Autonomous Nursing Robot which supports all the aforementioned tasks. In this paper, we present a new Autonomous Nursing Robot system capable of operating in a smart hospital environment area. The objective of the system is to identify the patient room, perform robot path planning for the delivery of medicines to a patient, and measure the patient body parameters, through a wireless BLE (Bluetooth Low Energy) beacon receiver and the BLE beacon transmitter at the respective patient rooms. Assuming that a wireless beacon is kept at the patient room, the robot follows the beacon’s signal, identifies the respective room and delivers the needed medicine to the patient. A new fuzzy controller system which consists of three ultrasonic sensors and one camera is developed to detect the optimal robot path and to avoid the robot collision with stable and moving obstacles. The fuzzy controller effectively detects obstacles in the robot’s vicinity and makes proper decisions for avoiding them. The navigation of the robot is implemented on a BLE tag module by using the AOA (Angle of Arrival) method. The robot uses sensors to measure the patient body parameters and updates these data to the hospital patient database system in a private cloud mode. It also makes uses of a Google assistant to interact with the patients. The robotic system was implemented on the Raspberry Pi using Matlab 2018b. The system performance was evaluated on a PC with an Intel Core i5 processor, while the solar power was used to power the system. Several sensors, namely HC-SR04 ultrasonic sensor, Logitech HD 720p image sensor, a temperature sensor and a heart rate sensor are used together with a camera to generate datasets for testing the proposed system. In particular, the system was tested on operations taking place in the context of a private hospital in Tirunelveli, Tamilnadu, India. A detailed comparison is performed, through some performance metrics, such as Correlation, Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE), against the related works of Deepu et al., Huh and Seo, Chinmayi et al., Alli et al., Xu, Ran et al., and Lee et al. The experimental system validation showed that the fuzzy controller achieves very high accuracy in obstacle detection and avoidance, with a very low computational time for taking directional decisions. Moreover, the experimental results demonstrated that the robotic system achieves superior accuracy in detecting/avoiding obstacles compared to other systems of similar purposes presented in the related works.