Search

Computer vision is one of the artificial intelligence’s most challenging fields, enabling computers to interpret, analyse and derive meaningful information from the visual world. There are various utilizations of computer vision algorithms, and most of them, from simpler to more complicated, have an object and shape recognition in common. Traditional pen and paper tests are designed in a pre-established format and consist of numerous basic shapes, which designate the important parts of the test itself. With that in mind, many computer vision applications regarding pen and paper tests arise as an opportunity. Massive courses and large schooling organizations mostly conduct their exams in paper format and assess them manually, which imposes a significant burden on the teaching staff. Any kind of automatization that will facilitate the grading process is highly desirable. Hence, an automated answer recognition system in assessment was developed to mitigate the problems above. The system uses images of scanned test pages obtained from the test scanning process and performs the necessary image manipulation steps to increase target recognition accuracy. Further, it manages to identify regions of interest containing multiple-choice questions and contours. Finally, the system verifies obtained results using the knowledge of the whereabouts of the test template regions of interest.

Supplementary Information The online version contains supplementary material available at https://doi.org/ 10.1007/s11042-023-14451-9, Authors, editors, and reviewers need to have a good perception regarding the quality of a manuscript in order to improve their skills, save effort, and prevent errors that can affect the submission procedure. In this paper, we compared the author’s perception of a manuscript’s quality with the manuscript’s actual impact. In addition, we analyzed the uncertainty of the author’s perception of the manuscript’s quality. From there,we defined ‘partition’ as the author’s ability to perceive the actual quality. We did this by launching a website for the use of the scientific community. This webpage provided a tool to help improve an investigator’s skill in understanding and recognizing the quality of a manuscript so as to help researchers improve and maximize their works’ potential impact. We carried out the experiment with 106 experienced users who tested our webpage. We found that the Abstract, the Title, and the Keywords were enough to perform a substantially decent evaluation of a manuscript. Most of the researchers were able to determine the quality of a paper in less than a minute from this small amount of information., Universidad de Granada/CBUA

Headline generation is a challenging subtask of abstractive text summarization, which its output should be a summary, shorter than one sentence. It would be precious to develop a dataset for the evaluation of abstractive summarization methods on this task in the Persian language. There are several datasets for headline generation in Persian, most of which are not large enough to be used by more sophisticated methods of text summarization, such as deep learning models. Moreover, all of these datasets are focused on daily news and there is no dataset for summarizing scientific Persian papers. In this article, we present “ElmNet,” a headline generation dataset of about 400,000 abstract/headline pairs of scientific papers, gathered from six major publishers for scientific articles in Persian. We, moreover, evaluate the performance of the most important deep learning-based headline generation methods, on the proposed dataset. The results prove the comparability of the performance of the state-of-the-art methods on this task, to their results on the existing English datasets.

Paper folding-and-punching tests are used to measure spatial abilities in humans. This paper presents a qualitative model for paper folding (QPF) and a computer game (Paper Folding Game) developed to apply and show the reasoning capabilities of the QPF model. This interactive game presents paper-folding activities intended to help users train and understand how to fold a paper to get a specific shape. Then, it presents paper-folding-and-punching tests to the players. The Paper Folding Game can automatically generate paper-folding-and-punching questions with varying degrees of difficulty depending on the number of folds and holes made, thus producing additional levels for training. The reasoning mechanisms in the QPF model are used by the Paper Folding Game to infer the right answer to each paper-folding-and-punching question. This reasoning capability allows the game to provide feedback to the players when they are wrong and also to create other plausible answers automatically, so that random question-answers are shown to the players in the master-mode. The Paper Folding Game has been implemented using Unity engine and it is available to download from GooglePlay and AppleStore for everyone to train their spatial reasoning skills.

Physical paper, in its various forms (e.g. books, leaflets, catalogues), is extensively used in everyday activities, despite any advancements in digital technology and ICT. Inspired by the popularity of this medium, several research efforts have envisioned and pursued a new era of interactive paper, however several challenges still remain to be addressed. On the other hand, recent advancements towards Ambient Intelligence (AmI) and Smart environments, bear the promise of seamless integration of the physical and digital worlds in an intuitive and user-friendly manner. This paper presents InPrinted, a systematic and generic framework supporting physical paper augmentation and user interaction in Ambient intelligence environments. InPrinted enables natural multimodal user interaction with any kind of printed matter in smart environments, providing context aware and anticipation mechanisms, as well as tools and interaction techniques that support the development of applications incorporating printed matter augmentation. InPrinted has been put to practice in the development of various systems, including the Interactive Maps system, facilitating interaction with printed maps as well as their augmentation with digital information. The results of an in-situ observation experiment of the Interactive Maps system are reported, highlighting that interacting with augmented paper is quite easy and natural, while the overall User Experience is positive.

In this paper, we aim to implement augmented reality (AR) on distant text documents or books. For this purpose, we propose a new paper-based AR system that can detect text documents in real scenes, markerize and identify them, estimate their relative 3D poses to the camera, and augment them with virtual contents. Unlike the previous paper-based AR systems (applicable to only close documents), the proposed system not only requires no detection of words or characters, but allows partial occlusions like the previous systems. In our experiments, the proposed system worked at 24 fps and could consistently achieve high identification rates for both occluded and unoccluded pages.

Paper checks may have complex background features including fine lines and patterns, which make forgery more difficult. Also, halftoning techniques are used to produce continuous tones and to prevent copies with void pantograph features. When these kinds of checks are scanned, Moire patterns may occur. These patterns make it difficult for customers to examine the scanned check images on ATM (Automated Teller Machine) displays. They also can decrease the classification accuracy of check recognition systems. In this paper, we propose an algorithm to enhance the perceptual quality of scanned check images by reducing the Moire patterns. The proposed algorithm consists of foreground extraction, Moire detection and Moire removal. Subjective image quality assessment was performed to evaluate the degree of improvement. Experimental results show that the proposed algorithm improves perceptual quality while maintaining check recognition accuracy.

The distortion introduced in the cover image is one of the most important factors influencing image steganography. The lower the distortion is, the higher the quality of the stego image, and thus the higher the resistance to steganalysis would be. This paper proposes an image steganographic method which employs the Hamiltonian paths in order to decrease the distortion produced by the LSB substitution scheme. The proposed method divides the cover image into non-overlapping blocks. For each block, the arrangement of the pixels of the block is first changed according to a Hamiltonian path. Next, the pixels whose LSB is not equal to the secret data are determined. The path that results in minimum distortion is chosen, and its binary code as well as the mismatched pixels are recorded. Repeating this process for all blocks, the pixels that should be modified during data embedding and a binary key that includes the codes of the best Hamiltonian path in all blocks are determined. Then, the new values of the second LSB of the changeable pixels are computed using writing on wet paper approach with regard to the compressed key. Finally, +1 or ?1 is added to each changeable pixel according to the value computed for its second LSB. By this strategy, the distortion of each pixel of the stego image is at most 1. Experimental results evaluated on 2000 natural images reveal that the proposed method can significantly decrease the image distortion and thus enhance security.

Multimedia field has dramatically grown in the last 20 years. We still remember the first ACM Multimedia Conference in Anaheim, CA in 1993, at whichmost participants even did not know the meaning of the word “multimedia”. In its early age, research in multimedia was focused on video and image compression and developing relevant standards, multimedia communication and synchronization, storage and retrieval, multimedia modeling, and creating first multimedia applications. Today, multimedia is interdisciplinary field, which is in tremendous explosion finding applications in practically all parts of our lives, from art, education, advertisement, and entertainment to engineering, medicine, business, scientific research, and many others. We created two special issues: Survey Papers in Multimedia by World Experts and Hot Research Topics in Multimedia We invited experts in the field both from academia and industry to contribute with their articles, and the response was overwhelming. This special issue “Survey Papers” contains 13 articles presenting the state of the art in the various areas of multimedia, from context modeling, semantic retrieval, personalized multimedia, to multimedia data mining, augmented reality, and others. The main objective of this special issue is to present the current status of multimedia field from leading researchers and practitioners worldwide. We thank all authors for their contributions in creating these two special issues. Guest Editors Ramesh Jain, Albert Del Bimbo, Tat-Seng Chua, and Borko Furht Multimed Tools Appl (2011) 51:1–4 DOI 10.1007/s11042-010-0681-1

Sign language recognition is one of the fundamental ways to assist deaf people to communicate with others. An accurate vision-based sign language recognition system using deep learning is a fundamental goal for many researchers. Deep convolutional neural networks have been extensively considered in the last few years, and a slew of architectures have been proposed. Recently, Vision Transformer and other Transformers have shown apparent advantages in object recognition compared to traditional computer vision models such as Faster R-CNN, YOLO, SSD, and other deep learning models. In this paper, we propose a Vision Transformer-based sign language recognition method called DETR (Detection Transformer), aiming to improve the current state-of-the-art sign language recognition accuracy. The DETR method proposed in this paper is able to recognize sign language from digital videos with a high accuracy using a new deep learning model ResNet152 + FPN (i.e., Feature Pyramid Network), which is based on Detection Transformer. Our experiments show that the method has excellent potential for improving sign language recognition accuracy. For instance, our newly proposed net ResNet152 + FPN is able to enhance the detection accuracy up to 1.70% on the test dataset of sign language compared to the standard Detection Transformer models. Besides, an overall accuracy 96.45% was attained by using the proposed method.

Deep learning (DL) is becoming a fast-growing field in the medical domain and it helps in the timely detection of any infectious disease (IDs) and is essential to the management of diseases and the prediction of future occurrences. Many scientists and scholars have implemented DL techniques for the detection and prediction of pandemics, IDs and other healthcare-related purposes, these outcomes are with various limitations and research gaps. For the purpose of achieving an accurate, efficient and less complicated DL-based system for the detection and prediction of pandemics, therefore, this study carried out a systematic literature review (SLR) on the detection and prediction of pandemics using DL techniques. The survey is anchored by four objectives and a state-of-the-art review of forty-five papers out of seven hundred and ninety papers retrieved from different scholarly databases was carried out in this study to analyze and evaluate the trend of DL techniques application areas in the detection and prediction of pandemics. This study used various tables and graphs to analyze the extracted related articles from various online scholarly repositories and the analysis showed that DL techniques have a good tool in pandemic detection and prediction. Scopus and Web of Science repositories are given attention in this current because they contain suitable scientific findings in the subject area. Finally, the state-of-the-art review presents forty-four (44) studies of various DL technique performances. The challenges identified from the literature include the low performance of the model due to computational complexities, improper labeling and the absence of a high-quality dataset among others. This survey suggests possible solutions such as the development of improved DL-based techniques or the reduction of the output layer of DL-based architecture for the detection and prediction of pandemic-prone diseases as future considerations.

In this paper, a computer-aided method is proposed for abnormality detection Wireless Capsule Endoscopy (WCE) video frames. Common abnormalities in WCE images include ulcers, bleeding, Angiodysplasia, Lymphoid Hyperplasia, and polyp. In this paper, deep features and Hand-crafted features are combined to detect these abnormalities in WCE images. There are no sufficient images to train deep structures therefore the ResNet50 pertained model is used to extract deep features. Hand-crafted features are associated with color, shape, and texture. They are extracted from the region of interest (ROI), i.e. suspicious region. The expectation Maximization (EM) algorithm is used to extract more distinct areas in the background as ROI. The expectation Maximization (EM) algorithm is configured in a way that can extract the area with a distinct texture and color as ROI. The EM algorithm is also initialized with a new fast method which leads to an increase in the accuracy of the method. We also used a novel idea to reveal unexpected color changes in the background due to existing lesions as a feature set. A large number of features are created by the method, so the minimum redundancy maximum relevance approach is used to select a subset of more effective features. These selected features are then fed to a Support Vector Machine for classification. The results show that the proposed approach can detect mentioned abnormalities in WCE frames with the accuracy of 97.82%

Motion-based technology (MBT) has been applied in the last decades with enormous success in a high number of applications. Its use continues growing and is specially interesting in the health area. Nowadays, its employment is being more and more specialised with respect to the profile of the end user (i.e., child, adolescent/teenager, adult or elderly). This paper first reviews the use of MBT centered in the end user from a global perspective. It also proposes a taxonomy that allows cataloguing the MBT employment directed to the end user. Then, from these results, the paper centers the review on the MBT application aiming to improve the health of elderly. The results highlighted in this paper can help to a better understanding of MBT, especially when it is applied thinking in elderly as the end users.

Object detection is the most important problem in computer vision tasks. After AlexNet proposed, based on Convolutional Neural Network (CNN) methods have become mainstream in the computer vision field, many researches on neural networks and different transformations of algorithm structures have appeared. In order to achieve fast and accurate detection effects, it is necessary to jump out of the existing CNN framework and has great challenges. Transformer’s relatively mature theoretical support and technological development in the field of Natural Language Processing have brought it into the researcher’s sight, and it has been proved that Transformer’s method can be used for computer vision tasks, and proved that it exceeds the existing CNN method in some tasks. In order to enable more researchers to better understand the development process of object detection methods, existing methods, different frameworks, challenging problems and development trends, paper introduced historical classic methods of object detection used CNN, discusses the highlights, advantages and disadvantages of these algorithms. By consulting a large amount of paper, the paper compared different CNN detection methods and Transformer detection methods. Vertically under fair conditions, 13 different detection methods that have a broad impact on the field and are the most mainstream and promising are selected for comparison. The comparative data gives us confidence in the development of Transformer and the convergence between different methods. It also presents the recent innovative approaches to using Transformer in computer vision tasks. In the end, the challenges, opportunities and future prospects of this field are summarized.

In the last few decades, there has been an increase in food safety and traceability issues. To prevent accidents and misconduct, it became essential to establish Food Safety Traceability System (FSTS) to trace the food from producer to consumer. The traceability systems can help track food in supply chains from farms to retail. Numerous technologies such as Radio Frequency Identification (RFID), sensor networks, and data mining have been integrated into traditional food supply chain systems to remove unsafe food products from the chain. But, these are not adequate for the current supply chain market. The emerging technology of blockchain can overcome safety and tracking issues. This can be possible with the help of blockchain features like transparent, decentralized, distributed, and immutable. Most of the previous works missed the discussion of the systematic process and technology involved in implementing the FSTS using blockchain. In this paper, we have discussed an organized state of research of the existing FSTS using blockchain. This survey paper aims to outline a detailed analysis of blockchain technology, FSTS using blockchain, consensus algorithms, security attacks, and solutions. Several survey papers and solutions based on blockchain are included in this research paper. Also, this work discusses some of the open research issues related to FSTS.

The digital image proves critical evidence in the fields like forensic investigation, criminal investigation, intelligence systems, medical imaging, insurance claims, and journalism to name a few. Images are an authentic source of information on the internet and social media. But, using easily available software or editing tools such as Photoshop, Corel Paint Shop, PhotoScape, PhotoPlus, GIMP, Pixelmator, etc. images can be altered or utilized maliciously for personal benefits. Various active, passive and other new deep learning technology like GAN approaches have made photo-realistic images difficult to distinguish from real images. Digital image tamper detection now focuses on determining the authenticity and consistency of digital photos. The major research problems use generic solutions and strategies, such as standardized data sets, benchmarks, evaluation criteria and generalized approaches.This paper overviews the evaluation of various image tamper detection methods. A brief discussion of image datasets and a comparative study of image criminological (forensic) methods are included in this paper. Furthermore, recently developed deep learning techniques along with their limitations have also been addressed. This study aims to comprehensively analyze image forgery detection methods using conventional and advanced deep learning approaches.

Effective mining of social media, which consists of a large number of users is a challenging task. Traditional approaches rely on the analysis of text data related to users to accomplish this task. However, text data lacks significant information about the social users and their associated groups. In this paper, we propose CommuNety, a deep learning system for the prediction of cohesive networks using face images from photo albums. The proposed deep learning model consists of hierarchical CNN architecture to learn descriptive features related to each cohesive network. The paper also proposes a novel Face Co-occurrence Frequency algorithm to quantify existence of people in images, and a novel photo ranking method to analyze the strength of relationship between different individuals in a predicted social network. We extensively evaluate the proposed technique on PIPA dataset and compare with state-of-the-art methods. Our experimental results demonstrate the superior performance of the proposed technique for the prediction of relationship between different individuals and the cohesiveness of communities.

Multilevel image thresholding is a well-known technique for image segmentation. Recently, various metaheuristic methods have been proposed for the determination of the thresholds for multilevel image segmentation. These methods are mainly based on metaphors and they have high complexity and their convergences are comparably slow. In this paper, a multilevel image thresholding approach is proposed that simplifies the thresholding problem by using a simple optimization technique instead of metaphor-based algorithms. More specifically, in this paper, Chaotic enhanced Rao (CER) algorithms are developed where eight chaotic maps namely Logistic, Sine, Sinusoidal, Gauss, Circle, Chebyshev, Singer, and Tent are used. Besides, in the developed CER algorithm, the number of thresholds is determined automatically, instead of manual determination. The performances of the developed CER algorithms are evaluated based on different statistical analysis metrics namely BDE, PRI, VOI, GCE, SSIM, FSIM, RMSE, PSNR, NK, AD, SC, MD, and NAE. The experimental works and the related evaluations are carried out on the BSDS300 dataset. The obtained experimental results demonstrate that the proposed CER algorithm outperforms the compared methods based on PRI, SSIM, FSIM, PSNR, RMSE, AD, and NAE metrics. In addition, the proposed method provides better convergence regarding speed and accuracy.

Left subclavian artery occlusion is frequently treated by stent placement. The most commented on In-Stent Restenosis is excessive thrombus that squeezes the stent and leads to vessel occlusion. In this paper, an automatic system is provided to segment the left subclavian arteries and left aortic arches from chest MRI (Magnetic Resonance Imaging) images, and then to identify the occlusion of left subclavian artery based on the gray-levels of the extracted left subclavian artery and left aortic arch. Experimental results show that the system obtains the accuracy rate 97.33% of detecting the occlusion of left subclavian artery. The other task in this paper is to explore the relationship between stent/vascular diameter ratio and restenosis of left subclavian artery after stenting. Besides that, the experimental results also show that Assurance is useful for reducing the restenosis of left subclavian artery with stent.

The past decade is known as the era of integrations where multiple technologies had integrated, and new research trends were seen. The security of data and information in the digital world has been a challenge to everyone; Blockchain technology has attracted many researchers in these scenarios. This paper focuses on finding the current trends in Blockchain technology to help the researchers select an area to carry future research. The data related to Blockchain Technologies have been collected from IEEE, Springer, ACM, and other digital databases. Then, the formulated corpus is used for topic modelling, and Latent Dirichlet Allocation is deployed. The outcomes of the Latent Dirichlet Allocation model are then analyzed based on various extracted key terms and key documents found for each topic. All the topic solution has been identified from the bag of words. The extracted topics are thereafter semantically mapped. Thus, based on the analysis of more than 900 papers, the most recent research trends have been discussed in this paper, ultimately focusing on the areas that need more attention from the research community. Also, the meta data analysis has been accomplished, evaluating the year wise and publication source wise research growth. More than 15 research directions are elaborated in this paper, which can direct and guide the researchers to pursuit the research in specific trends and also, find the research gaps in various technologies associated with Blockchain Technology.

Digital platforms are becoming critical infrastructures for supporting a variety of innovative services that enhance our everyday lives. These platforms need to offer not only rational services but also ludic or slow services that focus on human pleasure. One important aspect of creating innovative digital platforms is that their concrete requirements and potential opportunities are vague before they are designed. Thus, designing, prototyping and evaluating digital platforms iteratively is essential for refining or customizing them, as knowledge is gradually gained throughout these iterations. However, it is costly to develop prototype platforms and evaluate them with traditional methods. A better tool that can be used to reveal these platforms’ potential opportunities by conceiving them in a simple and rapid way is needed. In this paper, we present our journey to develop nine digital platforms that share collective human sight and hearing with the Human-Material-Pleasure (HMP) annotation method, which is a tool that we use to describe the visually structured annotations of multiple digital platforms based on the annotated portfolio method. The most significant part of the paper presents annotated portfolios based on the HMP annotation method for the nine digital platforms that we develop and shows how these annotated portfolios play an essential role in revealing and exploring the potential opportunities of our platforms during the refinement process. We also discuss how the HMP annotation method is used in the context of exploring the potential opportunities of wearable shape-changing robotic devices; these devices have significantly different characteristics from our digital platforms, which allows for showing insights more objectively by extracting diverse insights from an alternative angle.

The year 2020 and 2021 was the witness of Covid 19 and it was the leading cause of death throughout the world during this time period. It has an impact on a large geographic area, particularly in countries with a large population. Due to the fact that this novel coronavirus has been detected in all countries around the world, the World Health Organization (WHO) has declared Covid-19 to be a pandemic. This novel coronavirus spread quickly from person to person through the saliva droplets and direct or indirect contact with an infected person. The tests carried out to detect the Covid-19 are time-consuming and the primary cause of rapid growth in Covid19 cases. Early detection of Covid patient can play a significant role in controlling the Covid chain by isolation the patient and proper treatment at the right time. Recent research on Covid-19 claim that Chest CT and X-ray images can be used as the preliminary screening for Covid-19 detection. This paper suggested an Artificial Intelligence (AI) based approach for detecting Covid-19 by using X-ray and CT scan images. Due to the availability of the small Covid dataset, we are using a pre-trained model. In this paper, four pre-trained models named VGGNet-19, ResNet50, InceptionResNetV2 and MobileNet are trained to classify the X-ray images into the Covid and Normal classes. A model is tuned in such a way that a smaller percentage of Covid cases will be classified as Normal cases by employing normalization and regularization techniques. The updated binary cross entropy loss (BCEL) function imposes a large penalty for classifying any Covid class to Normal class. The experimental results reveal that the proposed InceptionResNetV2 model outperforms the other pre-trained model with training, validation and test accuracy of 99.2%, 98% and 97% respectively.

The Part-Of-Speech tagging is widely used in the natural language process. There are many statistical approaches in this area. The most popular one is Hidden Markov Model. In this paper, an alternative approach, linear-chain Conditional Random Fields, is introduced. The Conditional Random Fields is a factor graph approach that can naturally incorporate arbitrary, non-independent features of the input without conditional independence among the features or distributional assumptions of inputs. This paper applied the Conditional Random Fields for the car review word Part-Of-Speech tagging and then the feature extraction, which can be used as an input to an opinion mining system. To reduce the computational time, we also proposed applying the Limited-memory BFGS algorithm to train the Conditional Random Fields. Furthermore, this paper evaluated the Conditional Random Fields and the classical graph approach using the car review dataset to demonstrate that the Conditional Random Fields have a more robust result with a smaller training dataset.

The healthcare industry has always been an early adopter of new technology and a big benefactor of it. The use of reinforcement learning in the healthcare system has repeatedly resulted in improved outcomes.. Many challenges exist concerning the architecture of the RL method, measurement metrics, and model choice. More significantly, the validation of RL in authentic clinical settings needs further work. This paper presents a new Effective Resource Allocation Strategy (ERAS) for the Fog environment, which is suitable for Healthcare applications. ERAS tries to achieve effective resource management in the Fog environment via real-time resource allocating as well as prediction algorithms. Comparing the ERAS with the state-of-the-art algorithms, ERAS achieved the minimum Makespan as compared to previous resource allocation algorithms, while maximizing the Average Resource Utilization (ARU) and the Load Balancing Level (LBL). For each application, we further compared and contrasted the architecture of the RL models and the assessment metrics. In critical care, RL has tremendous potential to enhance decision-making. This paper presents two main contributions, (i) Optimization of the RL hyperparameters using PSO, and (ii) Using the optimized RL for the resource allocation and load balancing in the fog environment. Because of its exploitation, exploration, and capacity to get rid of local minima, the PSO has a significant significance when compared to other optimization methodologies.

Face recognition plays the significant role in many human-computer interaction decvices and applications, whose access control systems are based on the verification of face biometrical features. Though great improvement in the recognition performances have been achieved, when under some specific conditions like faces with occlusions, the performance would suffer a severe drop. Occlusion is one of the most significant reasons for the performance degrade of the existing general face recognition systems. The biggest problem in occluded face recognition (OFR) lies in the lack of the occluded face data. To mitigate this problem, this paper has proposed one new OFR network DOMG-OFR (Dynamic Occlusion Mask Generator based Occluded Face Recognition), which keeps trying to generate the most informative occluded face training samples on feature level dynamically, in this way, the recognition model would always be fed with the most valuable training samples so as to save the labor in preparing the synthetic data while simultaneously improving the training efficiency. Besides, this paper also proposes one new module called Decision Module (DM) in an attempt to combine both the merits of the two mainstream methodologies in OFR which are face image reconstruction based methodologies and the face feature filtering based methodologies. Furthermore, to enable the existing face deocclusion methods that mostly target at near frontal faces to work well on faces under large poses, one head pose aware deocclusion pipeline based on the Condition Generative Adversarial Network (CGAN) is proposed. In the experimental parts, we have also investigated the effects of the occlusions upon face recognition performance, and the validity and the efficiency of our proposed Decision based OFR pipeline has been fully proved. Through comparing both the verification and the recognition performance upon both the real occluded face datasets and the synthetic occluded face datasets with other existing works, our proposed OFR architecture has demonstrated obvious advantages over other works.

The first step to reducing the effect of viral disease is to prevent the spread which could be achieved by implementing social distancing (reducing the number of close physical interactions between peoples). Almost every viral disease whose means of communication is air, and enters through mouth or nose, definitely will affect our vocal organs which cause changes in features of our voice and could be traceable using feature analysis of voice using deep learning. The detection of an affected person using deep neural networks and tracking him would help us in the implementation of the social distancing rule in an area where it is needed. The aim of this paper is to study different solutions which help in enabling, encouraging, and even enforcing social distancing. In this paper, we implemented and analyzed scenarios on the basis of COVID-19 patient detection using cough and tracking him using smart cameras, or emerging wireless technologies with deep learning techniques for prediction and preventing the spread of disease. Thus these techniques are easy to be implemented in the initial stage of any pandemic as well and will help us in the implementation of smart social distancing (apply whenever needed).

Artificial Intelligent (AI) systems generally require training data of sufficient quantity and appropriate quality to perform efficiently. However, in many areas, such training data is simply not available or incredibly difficult to acquire. The recent developments in Virtual Reality (VR) have opened a new door for addressing this issue. This paper demonstrates the use of VR for generating training data for AI systems through a case study of human fall detection. Fall detection is a challenging problem in the public healthcare domain. Despite significant efforts devoted to introducing reliable and effective fall detection algorithms and enormous devices developed in the literature, minimal success has been achieved. The lack of recorded fall data and the data quality have been identified as major obstacles. To address this issue, this paper proposes an innovative approach to remove the afformentioned obstacle using VR technology. In this approach, a framework is, first, proposed to generate human fall data in virtual environments. The generated fall data is then tested with state-of-the-art visual-based fall detection algorithms to gauge its effectiveness. The results have indicated that the virtual human fall data generated using the proposed framework have sufficient quality to improve fall detection algorithms. Although the approach is proposed and verified in the context of human fall detection, it is applicable to other computer vision problems in different contexts, including human motion detection/recognition and self-driving vehicles.

Dataset complexity assessment aims to predict classification performance on a dataset with complexity calculation before training a classifier, which can also be used for classifier selection and dataset reduction. The training process of deep convolutional neural networks (DCNNs) is iterative and time-consuming because of hyperparameter uncertainty and the domain shift introduced by different datasets. Hence, it is meaningful to predict classification performance by assessing the complexity of datasets effectively before training DCNN models. This paper proposes a novel method called cumulative maximum scaled Area Under Laplacian Spectrum (cmsAULS), which can achieve state-of-the-art complexity assessment performance on six datasets., Published as a journal paper at Springer MTAP

Garbage management is an essential task in the everyday life of a city. In many countries, dumpsters are owned and deployed by the public administration. An updated what-and-where list is in the core of the decision making process when it comes to remove or renew them. Moreover, it may give extra information to other analytics in a smart city context. In this paper, we present a capsule network-based architecture to automate the visual classification of dumpsters. We propose different network hyperparameter settings, such as reducing convolutional kernel size and increasing convolution layers. We also try several data augmentation strategies, as crop and flip image transformations. We succeed in reducing the number of network parameters by 85% with respect to the best previous method, thus decreasing the required training time and making the whole process suitable for low cost and embedded software architectures. In addition, the paper provides an extensive experimental analysis including an ablation study that illustrates the contribution of each component in the proposed method. Our proposal is compared with the state-of-the-art method, which is based on a Google Inception V3 architecture pretrained with Imagenet. Experimental results show that our proposal achieves a 95.35% accuracy, 2.35% over the previous best method.

Cervical cell classification has important clinical significance in cervical cancer screening at early stages. However, there are fewer public cervical cancer smear cell datasets, the weights of each classes' samples are unbalanced, the image quality is uneven, and the classification research results based on CNN tend to overfit. To solve the above problems, we propose a cervical cell image generation model based on taming transformers (CCG-taming transformers) to provide high-quality cervical cancer datasets with sufficient samples and balanced weights, we improve the encoder structure by introducing SE-block and MultiRes-block to improve the ability to extract information from cervical cancer cells images; we introduce Layer Normlization to standardize the data, which is convenient for the subsequent non-linear processing of the data by the ReLU activation function in feed forward; we also introduce SMOTE-Tomek Links to balance the source data set and the number of samples and weights of the images we use Tokens-to-Token Vision Transformers (T2T-ViT) combing transfer learning to classify the cervical cancer smear cell image dataset to improve the classification performance. Classification experiments using the model proposed in this paper are performed on three public cervical cancer datasets, the classification accuracy in the liquid-based cytology Pap smear dataset (4-class), SIPAKMeD (5-class), and Herlev (7-class) are 98.79%, 99.58%, and 99.88%, respectively. The quality of the images we generated on these three data sets is very close to the source data set, the final averaged inception score (IS), Fréchet inception distance (FID), Recall and Precision are 3.75, 0.71, 0.32 and 0.65 respectively. Our method improves the accuracy of cervical cancer smear cell classification, provides more cervical cell sample images for cervical cancer-related research, and assists gynecologists to judge and diagnose different types of cervical cancer cells and analyze cervical cancer cells at different stages, which are difficult to distinguish. This paper applies the transformer to the generation and recognition of cervical cancer cell images for the first time.The online version contains supplementary material available at 10.1007/s11042-022-12670-0.

Recently, the detection and segmentation of salient objects that attract the attention of human visual in images is determined by using salient object detection (SOD) techniques. As an essential computer vision problem, SOD has increasingly attracted the researchers’ interest over the years. While a lot of SOD models and applications have been proposed, there is still a lack of deep understanding of the issues and achievements. A comprehensive study on the recent techniques of SOD is provided in this paper. Precisely, this paper presents a review of SOD techniques from various perspectives. Various image segmentation techniques are presented such as segmentation based on machine learning or deep learning, the second perspective concentrates on classifying them into supervised and unsupervised learning techniques and the last one based on manual approach, semi-automatic approach, and fully automatic approach and so on. Then, the paper presents a summarization of datasets used for SOD. Finally, analyses of SOD models and comparison results are presented.

Coronavirus Disease 2019 (COVID-19) is an evolving communicable disease caused due to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) which has led to a global pandemic since December 2019. The virus has its origin from bat and is suspected to have transmitted to humans through zoonotic links. The disease shows dynamic symptoms, nature and reaction to the human body thereby challenging the world of medicine. Moreover, it has tremendous resemblance to viral pneumonia or Community Acquired Pneumonia (CAP). Reverse Transcription Polymerase Chain Reaction (RT-PCR) is performed for detection of COVID-19. Nevertheless, RT-PCR is not completely reliable and sometimes unavailable. Therefore, scientists and researchers have suggested analysis and examination of Computing Tomography (CT) scans and Chest X-Ray (CXR) images to identify the features of COVID-19 in patients having clinical manifestation of the disease, using expert systems deploying learning algorithms such as Machine Learning (ML) and Deep Learning (DL). The paper identifies and reviews various chest image features using the aforementioned imaging modalities for reliable and faster detection of COVID-19 than laboratory processes. The paper also reviews and compares the different aspects of ML and DL using chest images, for detection of COVID-19.

Intelligent Transportation System (ITS), including unmanned vehicles, has been gradually matured despite on road. How to eliminate the interference due to various environmental factors, carry out accurate and efficient traffic sign detection and recognition, is a key technical problem. However, traditional visual object recognition mainly relies on visual feature extraction, e.g., color and edge, which has limitations. Convolutional neural network (CNN) was designed for visual object recognition based on deep learning, which has successfully overcome the shortcomings of conventional object recognition. In this paper, we implement an experiment to evaluate the performance of the latest version of YOLOv5 based on our dataset for Traffic Sign Recognition (TSR), which unfolds how the model for visual object recognition in deep learning is suitable for TSR through a comprehensive comparison with SSD (i.e., single shot multibox detector) as the objective of this paper. The experiments in this project utilize our own dataset. Pertaining to the experimental results, YOLOv5 achieves 97.70% in terms of mAP@0.5 for all classes, SSD obtains 90.14% mAP in the same term. Meanwhile, regarding recognition speed, YOLOv5 also outperforms SSD.

The coronavirus is an irresistible virus that generally influences the respiratory framework. It has an effective impact on the global economy specifically, on the financial movement of stock markets. Recently, an accurate stock market prediction has been of great interest to investors. A sudden change in the stock movement due to COVID -19 appearance causes some problems for investors. From this point, we propose an efficient system that applies sentiment analysis of COVID-19 news and articles to extract the final impact of COVID-19 on the financial stock market. In this paper, we propose a stock market prediction system that extracts the stock movement with the COVID spread. It is important to predict the effect of these diseases on the economy to be ready for any disease change and protect our economy. In this paper, we apply sentimental analysis to stock news headlines to predict the daily future trend of stock in the COVID-19 period. Also, we use machine learning classifiers to predict the final impact of COVID-19 on some stocks such as TSLA, AMZ, and GOOG stock. For improving the performance and quality of future trend predictions, feature selection and spam tweet reduction are performed on the data sets. Finally, our proposed system is a hybrid system that applies text mining on social media data mining on the historical stock dataset to improve the whole prediction performance. The proposed system predicts stock movement for TSLA, AMZ, and GOOG with average prediction accuracy of 90%, 91.6%, and 92.3% respectively.

This paper proposes a method to evaluate the effectiveness of the eye message therapy. The existing methods are via the diagnoses conducted by the medical professions based on the measurements acquired by the optical instruments. However, this approach is very expensive. To address this issue, this paper performs the classification between the periocular images taken before performing the eye massage therapy and those after performing the eye massage therapy to address the above difficulty. First, the median filtering is used to suppress the solitary point noise with preserving the edges of the image without causing the significant blurring. Then, the Canny operator is employed to accurately locate the edges. Next, the circle Hough transform (CHT) is used for performing the iris segmentation. Finally, various classifiers are used to perform the classification. The computer numerical simulation results show that our proposed method can achieve the high classification accuracies. This implies that there is a significant difference on the iris before performing the eye massage therapy and after performing the eye massage therapy. In addition, the comparisons with the state of art Daugman method have been performed. It is found that the classification performance achieved by the CHT based method is better than those achieved by the Daugman method.

Urban complex scenarios are the most challenging situations in the field of Autonomous Driving (AD). In that sense, an AD pipeline should be tested in countless environments and scenarios, escalating the cost and development time exponentially with a physical approach. In this paper we present a validation of our fully-autonomous driving architecture using the NHTSA (National Highway Traffic Safety Administration) protocol in the CARLA simulator, focusing on the analysis of our decision-making module, based on Hierarchical Interpreted Binary Petri Nets (HIBPN). First, the paper states the importance of using hyper-realistic simulators, as a preliminary help to real test, as well as an appropriate design of the traffic scenarios as the two current keys to build safe and robust AD technology. Second, our pipeline is introduced, which exploits the concepts of standard communication in robotics using the Robot Operating System (ROS) and the Docker approach to provide the system with isolation, flexibility and portability, describing the main modules and approaches to perform the navigation. Third, the CARLA simulator is described, outlining the steps carried out to merge our architecture with the simulator and the advantages to create ad-hoc driving scenarios for use cases validation instead of just modular evaluation. Finally, the architecture is validated using some challenging driving scenarios such as Pedestrian Crossing, Stop, Adaptive Cruise Control (ACC) and Unexpected Pedestrian. Some qualitative (video files: Simulation Use Cases) and quantitative (linear velocity and trajectory splitted in the corresponding HIBPN states) results are presented for each use case, as well as an analysis of the temporal graphs associated to the Vulnerable Road Users (VRU) cases, validating our architecture in simulation as a preliminary stage before implementing it in our real autonomous electric car.

Content-Based Image Retrieval (CBIR) is a well-known research topic from the computer vision domain which helps retrieve similar images from a dataset as per the specified query image. The retrieval performance is inadequate for benchmark datasets viz., Corel-1k, Corel-5k, Corel-10k, and Ghim-10k. In this paper, we have encountered the problem of the low retrieval rates of the CBIR system and the high dimensionality of the feature vectors. We have proposed the hybrid framework consisting of three different feature descriptors for robust retrieval performance. We have propounded the use of modified Multi-Scale Local Binary Pattern (MS-LBP), Color Coherence Vector (CCV), and Global Color Histogram (GCH) for image retrieval. We have exerted the modified MS-LBP because of its ability to capture more texture detail than Local Binary Pattern (LBP) at multiple scales. This larger filter size of MS-LBP makes it less vulnerable to noise and illumination than the conventional LBP descriptor. The CCV captures color with location information well enough, but it’s vulnerable to the brightened images whereas, the GCH operator covers brightness (less sensitive to brightness than CCV), rotation, scale, translation, camera viewpoint invariant features, but lacks spatial information. The proposed framework improves the feature selection process by blending the strength of each of these descriptors. This paper also targets the high dimensionality of the feature vector of the MS-LBP and GCH descriptors by exerting Principal Component Analysis (PCA). Moreover, Linear Discriminant Analysis (LDA) selects robust and optimal features for retrieval. The proposed method is compared with state-of-the-art literature on four benchmark datasets in terms of Average Retrieval Precision (ARP), Average Retrieval Rate (ARR), and Retrieval Time (RT). Experimental results show that the proposed method excels the examined research practices.

This paper mainly studies the Neural Style Transfer (NST) problem based on convolutional neural networks (CNN). Existing deep style migration algorithms do not mimic the styles to a reasonable position. To solve the problem, this paper proposes a multi-scale style transfer algorithm based on deep semantic matching. For purpose of guiding the correct migration of the style, we use the priori spatial segmentation and illumination information of the input image to integrate the deep semantic information. First, we find that spatial division and illumination analysis are two important visual understanding approaches for artists to make each painting decision. In order to simulate these two visual understanding approaches, this paper defines the DSS (deep semantic space), which contains spatial segmentation and contextual illumination information. The semantic exists in the form of CNN (convolution neural network) characteristic graph. Second, we propose a deep semantic loss function based on DSS matching and nearest neighbor search to optimize the effect of deep style migration. Third, we propose a multi-scale optimization strategy for improving the speed of our method. The experiments show that our method can reasonably synthesize images in spatial structures. The placement of each style is more reasonable and has a good visual aesthetic.

Hyperspectral imaging has been extensively investigated as an emerging, promising technique for measuring the quality and protection of horticultural and agricultural products over the past 20 years. This technology evolved from remote sensing and joins the machine vision and point spectroscopy realms to provide superior image segmentation for defect and contamination detection. In this paper, we have incorporated spatial and spectral information into hyperspectral imaging techniques. It can efficiently and non-destructively provide helpful information on both external physical and internal chemical characteristics of agricultural and food products. This paper has reviewed the sum of crucial aspect applications of hyperspectral imaging in precision agriculture. This paper reviews hyperspectral imaging’s current and past development in the agriculture industry, such as classification, chromatic, climatic, convergence, etc. This analysis aims to refer to potential work on key issues and promote deployed end-user solutions to fulfill the existing global sustainability objectives.

An important goal of color image gray-scale is to keep the edge details of the original color image as much as possible. In many cases, the degree of feature discrimination is maintained, but in some cases, edge details are still lost or blurred. Therefore, this paper first uses an improved non-linear global mapping grayscale method to grayscale the color image, and then proposes a grayscale image detail enhancement algorithm based on rolling guided filtering. The method in this paper is to enhance the edge details of the grayscale image by rolling guided filter processing on the basis of the grayscale image. In addition, the rolling-guided filter is a local linear model with better edge retention characteristics, which can overcome the defect that other filters are prone to gradient flips on the edges where the gray level of the image changes sharply, causing the image to appear “false edges”. The experimental results show that when the traditional method loses or blurs the detailed features, the method in this paper can maintain better detailed features.

In the recent digitization era, image hashing is a key technology, including image recognition, authentication and manipulation detection, among many multimedia security applications. The primary challenge in hashing schemes is to extract its robust feature. For a better understanding and design of a robust image hashing algorithm, it is very crucial to look into few important parameters like discrimination, robustness, reliability, etc. This paper reflects a detailed study of the existing literature on hashing-based image authentication techniques. This work provides a systematic overview and highlights the merits and demerits associated with various state-of-the-art techniques. In particular, the basic features and categories of image authentication techniques based on hashing are explored along with their properties. Moreover, different performance measures such as output metrices, receiver operating characteristics (ROC) parameters, execution time, etc., have been discussed in this work. The paper also compares the performances of various existing algorithms related to different content preserving operations on diverse data sets. This paper summarizes all the techniques and provides the most optimum solutions in regard to image hashing techniques based on different parameters.

Optic disc (OD) analysis is an important stage in detecting retinal diseases and existing approaches are not suitable for analyzing multiple examinations in a single process. This paper proposes a new unified method to detect (i) OD center, (ii) OD boundary, and (iii) enhancement of vasculature within the OD region in a single algorithm. This paper presents a unique strategy, which involves L2 norm of contourlet subbands of retinal images to locate the OD center. A novel OD boundary tracing approach that integrates morphological operations, and segmentation region growing techniques, with the aid of the OD center as a seed point, is presented. Among the two novel OD vasculature enhancement techniques, one of which involves image sharpening to improve the local contrast followed by histogram equalization to increase the overall contrast of the fundus image. Edge superimposing with the flood fill technique is another approach used for vasculature enhancement. The algorithm is tested on DRIVE (40), STARE (81), MESSIDOR (1200), E-ophtha (87), Diaretdb1 (89) images, out of which 40, 77, 1182, 87, and 87 images are detected correctly. The Figure of Merit (FOM) is used to confirm the correctness of the boundary tracking performance (97.15%). EME, SSIM, and other parameters are used to assess the efficiency of the vasculature enhancement technique.

Hyperspectral face recognition plays an important role in remote sensing. However, it faces many challenges such as difficulty in data acquisition, low signal to noise ratio (SNR), and high dimensionality. In this paper, we develop a novel method for hyperspectral face recognition by extracting histogram of oriented features (HOG) and using collaborate representation-based classifier (CRC) to classify unknown face data cubes. To improve overall classification rates, we also implement noise reduction in hyperspectral face data cubes. We also crop the face images by a bounding box and use this bounding box image to classify the testing faces. Experiments show that our new method outperforms several existing methods for both the PolyU-HSFD dataset and the CMU-HSFD dataset for hyperspectral face recognition. The contribution of this paper is the following: We introduce the MNF-based denoising method in this paper, which is new to the best of our knowledge. We also combine it with HOG features and CRC classifier so that better recognition rate can be achieved.

Fog computing is considered a formidable next-generation complement to cloud computing. Nowadays, in light of the dramatic rise in the number of IoT devices, several problems have been raised in cloud architectures. By introducing fog computing as a mediate layer between the user devices and the cloud, one can extend cloud computing's processing and storage capability. Offloading can be utilized as a mechanism that transfers computations, data, and energy consumption from the resource-limited user devices to resource-rich fog/cloud layers to achieve an optimal experience in the quality of applications and improve the system performance. This paper provides a systematic and comprehensive study to evaluate fog offloading mechanisms' current and recent works. Each selected paper's pros and cons are explored and analyzed to state and address the present potentialities and issues of offloading mechanisms in a fog environment efficiently. We classify offloading mechanisms in a fog system into four groups, including computation-based, energy-based, storage-based, and hybrid approaches. Furthermore, this paper explores offloading metrics, applied algorithms, and evaluation methods related to the chosen offloading mechanisms in fog systems. Additionally, the open challenges and future trends derived from the reviewed studies are discussed.

There have been considerable advancements in multimedia technologies over the past 5 years. It has been observed that state-of-the-art multimedia systems face three broad categories of challenges: (1) Dependency on continuous network connections, (2) Data-sharing applications & collaboration, and (3) Security issues. Among these, security vulnerability poses a major threat to modern multimedia systems. Therefore, it is imperative to carefully investigate the security issues that can endanger wireless and mobile communications. At present, multimedia security research mainly focuses on wireless traffic monitoring, wireless system attacks, and wireless and mobile security. In this paper, we have used the network attack-type, “Reconnaissance”, which contains two types of malicious activities: (1) OS scanning, and (2) Fuzzing. The goal of this paper is to quantify multimedia security risks due to Fuzzing by using various types of machine learning models. The highest accuracy i.e., 96.8%, is obtained using the XGBoost classifier, which is good compared to the existing models present in the literature. This is the first paper, to the best of our knowledge, that uses machine learning methods to differentiate between benign and malignant Fuzzing attacks.