Search

Continuous Sign Language Recognition (CSLR) refers to the challenging problem of recognizing sign language glosses and their temporal boundaries from weakly annotated video sequences. Previous methods focus mostly on visual feature extraction neglecting text information and failing to effectively model the intra-gloss dependencies. In this work, a cross-modal learning approach that leverages text information to improve vision-based CSLR is proposed. To this end, two powerful encoding networks are initially used to produce video and text embeddings prior to their mapping and alignment into a joint latent representation. The purpose of the proposed cross-modal alignment is the modelling of intra-gloss dependencies and the creation of more descriptive video-based latent representations for CSLR. The proposed method is trained jointly with video and text latent representations. Finally, the aligned video latent representations are classified using a jointly trained decoder. Extensive experiments on three well-known sign language recognition datasets and comparison with state-of-the-art approaches demonstrate the great potential of the proposed approach.

AI technologies can play an important role in breaking down the communication barriers of deaf or hearing-impaired people with other communities, contributing significantly to their social inclusion. Recent advances in both sensing technologies and AI algorithms have paved the way for the development of various applications aiming at fulfilling the needs of deaf and hearing-impaired communities. To this end, this survey aims to provide a comprehensive review of state-of-the-art methods in sign language capturing, recognition, translation and representation, pinpointing their advantages and limitations. In addition, the survey presents a number of applications, while it discusses the main challenges in the field of sign language technologies. Future research direction are also proposed in order to assist prospective researchers towards further advancing the field.

Background and objective Electrocardiogram is commonly used as a diagnostic tool for the monitoring of cardiac health and the detection of possible heart diseases. However, the procedure followed for the diagnosis of heart abnormalities is time consuming and prone to human errors. Thus, the development of computer-aided techniques for the automatic analysis of electrocardiogram signals is of vital importance for the diagnosis and prevention of heart diseases. The most serious outcome of coronary heart disease is the myocardial infarction, i.e., the rapid and irreversible damage of cardiac muscles, which, if not diagnosed and treated in time, continues to damage further the myocardial structure and function. In this paper we propose a novel approach for the automatic detection and localization of myocardial infarction from multi-lead electrocardiogram signals. Methods The proposed method initially reshapes the multidimensional signal into a third-order tensor structure and subsequently extracts feature representations in both Euclidean and Grassmannian space. In addition, two different methods are proposed for the mapping of the two different feature representations into a common Hilbert space before the final classification of signals. The first approach is based on the mapping of both Grassmannian and Euclidean features in a Reproducing Kernel Hilbert Space (RKHS), while the second one attempts to initially apply Vector of Locally Aggregated Descriptors (VLAD) encoding directly to Grassmann manifold and then concatenate the two VLAD representations. Results For the evaluation of the proposed method, we have conducted extensive tests using a publicly available dataset, namely PTB Diagnostic ECG database, containing 549 multi-lead ECG data recordings from 290 subjects and from different diagnostic classes. The method provides an excellent detection rate of 100%, and localization rate, i.e., 100% with the first fusion method and 99.7% with the second one. Conclusions The Experimental results presented in this paper show the superiority of the proposed methodology against a number of state-of-the-art approaches. The main advantage of the proposed approach is that it exploits better the intercorrelations between signals of different ECG leads, by extracting feature representations that lie in different geometrical spaces and contain complementary information with regard to the dynamics of signals.

In this paper, we propose a novel multi-stage deep learning methodology to accurately tackle the problem of hand pose estimation. More specifically, we initially propose a disentanglement stage to differentiate the significant pose-specific information from the irrelevant background noise and illumination variations of RGB images. Then, we introduce a variational alignment stage to accurately align the latent spaces of the pose-specific and the true hand pose information, effectively improving the discrimination ability of the proposed methodology. Finally, we propose the use of two loss terms to impose physiological and geometrical kinematic constraints to the predicted hand poses, empowering the proposed methodology to avoid non-plausible poses. During all stages, a novel injection decoder is introduced, improving significantly the decoding accuracy of the latent space. Extensive experimentation on two well-known datasets (i.e., RHD and STB) validate the ability of the proposed methodology to achieve accurate hand pose estimation results, overcoming current state-of-the-art methods.

Continuous sign language recognition is a weakly supervised task dealing with the identification of continuous sign gestures from video sequences, without any prior knowledge about the temporal boundaries between consecutive signs. Most of the existing methods focus mainly on the extraction of spatio-temporal visual features without exploiting text or contextual information to further improve the recognition accuracy. Moreover, the ability of deep generative models to effectively model data distribution has not been investigated yet in the field of sign language recognition. To this end, a novel approach for context-aware continuous sign language recognition using a generative adversarial network architecture, named as Sign Language Recognition Generative Adversarial Network (SLRGAN), is introduced. The proposed network architecture consists of a generator that recognizes sign language glosses by extracting spatial and temporal features from video sequences, as well as a discriminator that evaluates the quality of the generator’s predictions by modeling text information at the sentence and gloss levels. The paper also investigates the importance of contextual information on sign language conversations for both Deaf-to-Deaf and Deaf-to-hearing communication. Contextual information, in the form of hidden states extracted from the previous sentence, is fed into the bidirectional long short-term memory module of the generator to improve the recognition accuracy of the network. At the final stage, sign language translation is performed by a transformer network, which converts sign language glosses to natural language text. Our proposed method achieved word error rates of 23.4%, 2.1%, and 2.26% on the RWTH-Phoenix-Weather-2014 and the Chinese Sign Language (CSL) and Greek Sign Language (GSL) Signer Independent (SI) datasets, respectively.

Food analysis resides at the core of modern nutrition recommender systems, providing the foundation for a high-level understanding of users’ eating habits. This paper focuses on the sub-task of ingredient recognition from food images using a variational framework. The framework consists of two variational encoder-decoder branches, aimed at processing information from different modalities (images and text), as well as a variational mapper branch, which accomplishes the task of aligning the distributions of the individual branches. Experimental results on the Yummly-28K data-set showcase that the proposed framework performs better than similar variational frameworks, while it surpasses current state-of-the-art approaches on the large-scale Recipe1M data-set.

The environmental challenges the world faces have never been greater or more complex. Global areas that are covered by forests and urban woodlands are threatened by large-scale forest fires that have increased dramatically during the last decades in Europe and worldwide, in terms of both frequency and magnitude. To this end, rapid advances in remote sensing systems including ground-based, unmanned aerial vehicle-based and satellite-based systems have been adopted for effective forest fire surveillance. In this paper, the recently introduced 360-degree sensor cameras are proposed for early fire detection, making it possible to obtain unlimited field of view captures which reduce the number of required sensors and the computational cost and make the systems more efficient. More specifically, once optical 360-degree raw data are obtained using an RGB 360-degree camera mounted on an unmanned aerial vehicle, we convert the equirectangular projection format images to stereographic images. Then, two DeepLab V3+ networks are applied to perform flame and smoke segmentation, respectively. Subsequently, a novel post-validation adaptive method is proposed exploiting the environmental appearance of each test image and reducing the false-positive rates. For evaluating the performance of the proposed system, a dataset, namely the “Fire detection 360-degree dataset”, consisting of 150 unlimited field of view images that contain both synthetic and real fire, was created. Experimental results demonstrate the great potential of the proposed system, which has achieved an F-score fire detection rate equal to 94.6%, hence reducing the number of required sensors. This indicates that the proposed method could significantly contribute to early fire detection.

The field of 3D hand pose estimation has been gaining a lot of attention recently, due to its significance in several applications that require human-computer interaction (HCI). The utilization of technological advances, such as cost-efficient depth cameras coupled with the explosive progress of Deep Neural Networks (DNNs), has led to a significant boost in the development of robust markerless 3D hand pose estimation methods. Nonetheless, finger occlusions and rapid motions still pose significant challenges to the accuracy of such methods. In this survey, we provide a comprehensive study of the most representative deep learning-based methods in literature and propose a new taxonomy heavily based on the input data modality, being RGB, depth, or multimodal information. Finally, we demonstrate results on the most popular RGB and depth-based datasets and discuss potential research directions in this rapidly growing field.

Being the second most common neurodegenerative disease, Parkinson's disease (PD) can be symptomatically treated, although, unfortunately, it cannot be cured yet. Moreover, diagnosing and assessing PD patients is a complex process, requiring continuous monitoring. In this vein, the design, development, and validation of innovative assessment tools may be helpful in the management of patients with PD, in particular. Based on intelligent ICT interventions, the i-PROGNOSIS project intends to mitigate PD's specific symptoms, such as neurological movement disorders of gait, balance, coordination, and posture, already characterized in the early phase of the disease. From this perspective, an innovative iPrognosis motor assessment tool is presented here, taking into consideration the Unified Parkinson Disease Rating Scale (UPDRS) Part III motor skills testing items, for evaluating the motor skills status. The efficiency of the proposed Assessment Tests to reflect the motor skills status, similarly to the UPDRS Part III items, was validated via 27 participants (18 males; mean age = 62 years, SD = 10.36 years; range, 43–79 years) with early (n = 10) and moderate (n = 17) PD who performed the Assessment Tests. Features from the latter were then correlated with the corresponding clinically assessed UPDRS Part III items, and statistically significant negative correlations (range, −0.364 to −0.802) were identified between the median values of the Assessment Tests and the UPDRS Part III items. In this vein, the iPrognosis Assessment Tests were integrated within the personalized interventions of the i-PROGNOSIS project, providing alternative means of assessing their effect on the PD patient's motor skills enhancement. The promising results presented here elaborate on the concept of using ICT-based assessment means to achieve comparable outcomes with the clinical standards in motor skills assessment.

Parkinson's disease (PD) is a chronic and progressive neurodegenerative disease that affects ~7 million people worldwide, without any cure to date; however, it can be symptomatically treated. In this vein, innovative technologies can be used for the objective assessment of clinical symptoms and to provide supportive therapies at home. The present work explores the processes and the outcomes of the i-PROGNOSIS (www.i-prognosis.eu) intervention deployment in three PD clinical centres (Greece, UK, and Germany). For that purpose, 36 PD patients were recruited to voluntarily participate in the i-PROGNOSIS feasibility study, spread across the three different countries. The PD patients interacted with the i-PROGNOSIS system for up-to-three months, mainly within the clinical environment, using the provided iPrognosis Games in dedicated gaming stations that were setup in the corresponding clinical centres. Overall, the results show that the iPrognosis Games were positively evaluated by medical experts. Moreover, based on the collected feedback, the iPrognosis Games have achieved their main goals of providing an innovative, objective and usable system for the monitoring of early PD (motor and non-motor) symptomatology, by providing tools for complementing existing clinical interventions for the improvement of PD patients' quality of life.

People with various degrees of disabilities (e.g., visually or hearing impaired) often find it difficult to access mainstream products and services and thus they are excluded from enjoying audio-visual services on an equal basis as people without disabilities. These people feel marginalized in today's digital society and they are unable to reach their maximum potential personally, professionally and socially. EasyTV aims to address these issues by providing numerous services that not only ease the access of people with disabilities to multimedia services, but also enhance interaction with the media through a multi-language approach that adapts content based on user's preferences. In this work, the main EasyTV services are described in detail. Furthermore, an evaluation of the main services by end users is presented and discussed in order to highlight the importance of offering accessibility services to people with disabilities.

In this paper, we propose a novel cross-modal variational alignment method in order to process and relate information across different modalities. The proposed approach consists of two variational autoencoder (VAE) networks which generate and model the latent space of eachmodality. The first network is a multi-modal variational-autoencoder that maps directly one modality to the other, while the second one is a single-modal variational autoencoder. In order to associate the two spaces, we apply variational alignment, which acts as a translation mechanismthat projects the latent space of the first VAE onto the one ofthe single-modal VAE through an intermediate distribution.Experimental results on four well-known datasets, covering two different application domains (food image analysisand 3D hand pose estimation), show the generality of theproposed method and its superiority against a number of state-of-the-art approaches.

In this paper, the problem of pruning and compressing the weights of various layers of deep neural networks is investigated. The proposed method aims to remove redundant filters from the network to reduce computational complexity and storage requirements, while improving the performance of the original network. More specifically, a novel filter selection criterion is introduced based on the fact that filters whose weights follow a Gaussian distribution correspond to hidden units that do not capture important aspects of data. To this end, Higher Order Statistics (HOS) are used and filters with low cumulant values that do not deviate significantly from Gaussian distribution are identified and removed from the network. In addition, a novel pruning strategy is proposed aiming to decide on the pruning ratio of each layer using the Shapiro-Wilk normality test. The use of auxiliary MSE losses (intermediate and after the softmax layer) during the fine-tuning phase further improves the overall performance of the compressed network. Extensive experiments with different network architectures and comparison with state-of-the-art approaches on well-known public datasets, such as CIFAR-10, CIFAR-100 and ILSCVR-12, demonstrate the great potential of the proposed approach.

Millions of people suffering from partial or complete hearing loss use variants of sign language to communicate with each other or hearing people in their everyday life. Thus, it is imperative to develop systems to assist these people by removing the barriers that affect their social inclusion. These systems should aim towards capturing sign language in an accurate way, classifying sign language to natural words and representing sign language by having avatars or synthesized videos execute the exact same moves that convey a meaning in the sign language. This chapter reviews current state-of-the-art approaches that attempt to solve sign language recognition and representation and analyzes the challenges they face. Furthermore, this chapter presents a novel AI-based solution to the problem of robust sign language capturing and representation, as well as a solution to the unavailability of annotated sign language datasets before limitations and directions for future work are discussed.

Wood recognition is a crucial task for wood sciences and industries, since it leads to the identification of the anatomical features and physical properties of wood. Traditionally, the recognition process relies almost exclusively on human experts, who are based on various characteristics of wood, such as color, structure and texture. However, there are numerous types of wood species in the nature that are difficult to be identified even by experienced scientists. Towards this end, in this paper we propose a novel approach for automated wood species recognition through multidimensional texture analysis. By taking advantage of the fact that static wood images contain periodic spatially-evolving characteristics, we introduce a new spatial descriptor considering each wood image as a collection of multidimensional signals. More specifically, the proposed methodology enables the representation of wood images as concatenated histograms of higher order linear dynamical systems produced by vertical and horizontal image patches. The final classification of images, i.e., histogram representations, into wood species, is performed using a Support Vector Machines (SVM) classifier. For the evaluation of the proposed method, a dataset, namely “WOOD-AUTH”, consisting of more than 4200 wood images (from cross, radial and tangential sections of normal wood structure) of twelve common wood species existing in Greek territory, was created. Experimental results presented in this paper show the great potential of the proposed methodology, which, despite a small number of misclassification cases with regards to both anatomically similar and different species, outperforms a number of state of the art approaches, yielding a classification rate of 91.47% in wood cross sections.

In this paper, we consider the problem of multi-dimensional dynamic texture analysis, and we introduce a new higher order linear dynamical system (h-LDS) descriptor. The proposed h-LDS descriptor is based on the higher order decomposition of the multidimensional image data and enables the analysis of dynamic textures by using information from various image elements. In addition, we propose a methodology for its application to video-based early warning systems that focus on smoke identification. More specifically, the proposed methodology enables the representation of video subsequences as histograms of h-LDS descriptors produced by the smoke candidate image patches in each subsequence. Finally, to further improve the classification accuracy, we propose the combination of multidimensional dynamic texture analysis with the spatiotemporal modeling of smoke by using a particle swarm optimization approach. The ability of the h-LDS to analyze the dynamic texture information is evaluated through a multivariate comparison against the standard LDS descriptor. The experimental results that use two video datasets have shown the great potential of the proposed smoke detection method.

Past research has now provided compelling evidence pointing towardscorrelations among individual eating styles and the development of (un)healthyeating patterns, obesity and other medical conditions. In this setting, an automatic,non-invasive food bite detection system can be a really useful tool in the handsof nutritionists, dietary experts and medical doctors in order to explore real-lifeeating behaviors and dietary habits.Unfortunately, the automatic detection of foodbites can be challenging due to occlusions between hands and mouth, use of differentkitchen utensils and personalized eating habits. On the other hand, althoughaccurate, manual bite detection is time-consuming for the annotator, making itinfeasible for large scale experimental deployments or real-life settings. To thisregard, we propose a novel deep learning methodology that relies solely on humanbody and face motion data extracted from videos depicting people eating meals.The purpose is to develop a system that can accurately, robustly and automaticallyidentify food bite instances,with the long-term goal to complement or even replacemanual bite-annotation protocols currently in use. The experimental results on alarge dataset reveal the superb classification performance of the proposed methodologyon the task of bite detection and paves the way for additional research onautomatic bite detection systems.

Parkinson’s disease (PD) is a progressive neurological disorder and the second most common age-related neurodegenerative disease after Alzheimer's disease. The primary symptoms of the disease are associated with the loss of motor skills affecting patients’ movement and coordination and disrupting their daily life. Unfortunately, such motor symptoms cannot be fully relieved by therapeutic options. On the other hand, studies have shown that regular training and exercising can prove neuroprotective in PD patients helping them maintain independent longer. Based on recent studies stating that computer-based physical therapy games can be used as an option for facilitating PD rehabilitation exercise programs, we present the development of a body motion based videogame, using the Kinect sensor, targeted for PD patients. We tested twelve patients with advanced forms of PD motor symptoms (UPDRS motor score>20) and six initial stage PD patients (UPDRS motor score

In this paper, we propose a novel image-based fire detection approach, which combines the power of modern deep learning networks with multidimensional texture analysis based on higher-order linear dynamical systems. The candidate fire regions are identified by a Faster R-CNN network trained for the task of fire detection using a set of annotated images containing actual fire as well as selected negatives. The candidate fire regions are projected to a Grassmannian space and each image is represented as a cloud of points on the manifold. Finally, a vector representation approach is applied aiming to aggregate the Grassmannian points based on a locality criterion on the manifold. For evaluating the performance of the proposed methodology, we performed experiments with annotated images of two different databases containing fire and fire-coloured objects. Experimental results demonstrate the potential of the proposed methodology compared to other state of the art approaches.

In this paper, we propose a novel framework for the detection and classification of centroblasts (CB) in follicular lymphoma (FL) tissue samples stained with PAX5 and H&E stains and sliced at 1 $$\upmu $$ m thickness level. By employing PAX5 immunohistochemistry, we facilitate the segmentation of nuclei, while the use of H&E stain enables us to extract textural information related to histological characteristics used by pathologists in the diagnosis of FL grading. For the segmentation of nuclei in PAX5-stained images, we initially apply an energy minimization technique based on graph cuts and then we propose a novel algorithm for the separation of overlapped nuclei inspired by the clustering of large-scale visual vocabularies. The morphological characteristics of nuclei extracted from PAX5-stained images are combined with a number of textural characteristics identified in H&E images through a Bayesian network classifier, which aims to model pathologists’ knowledge used in FL grading. Experimental results have already shown the great potential of the proposed methodology providing an average F-score of $$94.56\,\%$$ .

Sign language recognition (SLR) refers to theclassification of signs with a specific meaning performed by thedeaf and/or hearing-impaired people in their everydaycommunication. In this work, we propose a deep learningbased framework, in which we examine and analyze thecontribution of video (image and optical flow) and skeletal(body, hand and face) features in the challenging task of isolated SLR, in which each signed video corresponds to asingle word. Moreover, we employ various fusion schemes inorder to identify the optimal way to combine the informationobtained from the various feature representations and proposea robust SLR methodology. Our experimentation on two signlanguage datasets and the comparison with state-of-the-artSLR methods reveals the superiority of optimally combiningskeletal and video features for SLR tasks.

The accuracy of modern depth sensors, the robustnessof skeletal data to illumination variations and the superbperformance of deep learning techniques on several classificationtasks have sparkled a renewed interest towards skeleton basedaction recognition. In this paper, we propose a four streamdeep neural network based on two types of spatialskeletal features and their corresponding temporal representationsextracted by the novel Grassmannian Pyramid Descriptor(GPD). The performance of the proposed action recognitionmethodology is further enhanced by the use of a meta-learnerthat takes advantage of the meta knowledge extracted fromthe processing of the different features. Experiments on severalwell-known action recognition datasets reveal that ourproposed methodology outperforms a number of state-of-the-artskeleton-based action recognition methods.

Sign language recognition (SLR) is a challenging, but highly important research field for several computer vision systems that attempt to facilitate the communication among the deaf and hearing impaired people. In this work, we propose an accurate and robust deep learning-based methodology for sign language recognition from video sequences. Our novel method relies on hand and body skeletal features extracted from RGB videos and, therefore, it acquires highly discriminative for gesture recognition skeletal data without the need for any additional equipment, such as data gloves, that may restrict signer’s movements. Experimentation on a large publicly available sign language dataset reveals the superiority of our methodology with respect to other state of the art approaches relying solely on RGB features.

ProsocialLearn is a digital pro-social games platform. The ProsocialLearn project has delivered a series of disruptive innovations for the production and distribution of pro-social digital games that engage children (7-10 years old). Additionally it has stimulated technology transfer from the games industry to the educational sector. ProsocialLearn fosters the creation of a new market for digital games aimed at increasing social inclusion and academic performance, as well as a distribution channel to deliver prosocial games to children and teachers in European schools. Furthermore, it provides a proven pro-social methodology to design digital games. The ProsocialLearn platform makes available a series of APIs, which game developers use to integrate many of the ProsocialLearn functions into games, i.e. emotion and engagement monitoring, in-game achievements, games adaptation based on Prosocial Learning Objectives (PLOs), and micro-transactions.

Mixed Reality as display technology, gamification as motivational element, storytelling as interaction metaphor while maintaining the feeling of Presence, are identified as key elements for defining the next generation of virtual museums.

Intangible Cultural Heritage (ICH) creations include, amongst other, music, dance, singing, theatre, human skills, and craftsmanship. These cultural expressions are usually transmitted orally and/or using gestures and are modified over a period of time, through a process of collective recreation. As the world becomes more interconnected and many different cultures come into contact, local communities run the risk of losing important elements of their ICH, while young people find it difficult to maintain the connection with the cultural heritage treasured by their elders. In this paper, we present a novel holistic approach for the safeguarding and transmission of ICH that goes beyond the mere digitization of ICH content. Based on multisensory technology for the capturing of ICH, the proposed approach enables the generation of completely novel cultural content. High-level semantics are extracted from the acquired data, enabling researchers to identify possible implicit or hidden correlations between different ICH expressions or interpretation styles and study the evolution of a specific ICH. These data, coupled with other cultural resources, are accessible through the i-Treasures Web-platform, which provides the means for supporting knowledge exchange between researchers as well as know-how transmission from ICH bearers to apprentices.

Residual networks (ResNets) have introduced a milestone for the deep learning community due to their outstanding performance in diverse applications. They enable efficient training of increasingly deep networks, reducing the training difficulty and error. The main intuition behind them is that, instead of mapping the input information, they are mapping a residual part of it. Since the original work, a lot of extensions have been proposed to improve information mapping. In this paper, a novel extension of the residual block is proposed inspired by linear dynamical systems (LDSs), called LDS-ResNet. Specifically, a new module is presented that improves mapping of residual information by transforming it in a hidden state and then mapping it back to the desired feature space using convolutional layers. The proposed module is utilized to construct multi-branch residual blocks for convolutional neural networks. An exploration of possible architectural choices is presented and evaluated. Experimental results show that LDS-ResNet outperforms the original ResNet in image classification and object detection tasks on public datasets such as CIFAR-10/100, ImageNet, VOC, and MOT2017. Moreover, its performance boost is complementary to other extensions of the original network such as pre-activation and bottleneck, as well as stochastic training and Squeeze-Excitation.

In this paper, we propose a method for the partitioning of dance sequences into multiple periods and motion patterns. The proposed method deploys features in the form of a skeletal representation of the dancer observed through time using multiple depth sensors. This representation is the fusion of skeletal features captured using multiple sensors and combined into a single, more robust, skeletal representation. Using this information, initially we partition the dance sequence into periods and subsequently into motion patterns. Partitioning into periods is based on observing the horizontal displacement of the dancer while each period is subsequently partitioned into motion patterns by using an exemplar-based Hidden Markov Model that classifies each frame into an exemplar representing a hidden state of the HMM. The proposed method was tested on dance sequences comprising multiple periods and motion patterns providing promising results.

Every year, a large number of wildfires all over the world burn forested lands, causing adverse ecological, economic, and social impacts. Beyond taking precautionary measures, early warning and immediate response are the only ways to avoid great losses. To this end, in this paper we propose a computer vision approach for fire-flame detection to be used by an early-warning fire monitoring system. Initially, candidate fire regions in a frame are defined using background subtraction and color analysis based on a nonparametric model. Subsequently, the fire behavior is modeled by employing various spatio-temporal features, such as color probability, flickering, spatial, and spatio-temporal energy, while dynamic texture analysis is applied in each candidate region using linear dynamical systems and a bag-of-systems approach. To increase the robustness of the algorithm, the spatio-temporal consistency energy of each candidate fire region is estimated by exploiting prior knowledge about the possible existence of fire in neighboring blocks from the current and previous video frames. As a final step, a two-class support vector machine classifier is used to classify the candidate regions. Experimental results have shown that the proposed method outperforms existing state-of-the-art algorithms.

As of the early 20th century, a significant body of research has been published that shows how effective game-based learning and gamification techniques can be compared to other methods. However, creating games can be time consuming and usually demands a significant effort. Therefore, this paper focuses on the design and development of a novel framework for the rapid design of body-motion-based customizable game-like applications. This framework consists of two components: i) an interface that allows the user to design the game and capture the motion data, and ii) a customizable game for learning and training using off-the-shelf motion capture sensors like the Microsoft Kinect. The game is automatically configured based on the output of the game design interface. In order to evaluate the proposed system, a pilot use case for the Latin dance Salsa has been selected. Preliminary small-scaled experiments with latin dance students have shown the great potential of the proposed application.

The use of serious games in health care interventions sector has grown rapidly in the last years, however, there is still a gap in the understanding on how these types of interventions are used for the management of the Parkinson Disease (PD), in particular. Targeting intelligent early detection and intervention in PD area, the Personalized Game Suite (PGS) design process approach is presented as part of the H2020 i-PROGNOSIS project that introduces the integration of different serious games in a unified platform (i.e., ExerGames, DietaryGames, EmoGames, and Handwriting/Voice Games). From the methodological point of view, to facilitate the visualization of 14 game-scenarios, the system interface and the PD contexts, the storyboarding technique was adopted here. Overall, the realization of the PGS sets the basis for establishing a holistic framework that could aim at improving motor and non-motor symptoms, in order to inform health care providers and policy makers for its inclusion in routine management for PD.

In this paper we address the problem of automated grading of invasive breast carcinoma through the encoding of histological images as VLAD (Vector of Locally Aggregated Descriptors) representations on the Grassmann manifold. The proposed method considers each image as a set of multidimensional spatially-evolving signals that can be efficiently modeled through a higher-order linear dynamical systems analysis. Subsequently, each H&E (Hematoxylin and Eosin) stained breast cancer histological image is represented as a cloud of points on the Grassmann manifold, while a vector representation approach is applied aiming to aggregate the Grassmannian points based on a locality criterion on the manifold. To evaluate the efficiency of the proposed methodology, two datasets with different characteristics were used. More specifically, we created a new medium-sized dataset consisting of 300 annotated images (collected from 21 patients) of grades 1, 2 and 3, while we also provide experimental results using a large dataset, namely BreaKHis, containing 7,909 breast cancer histological images, collected from 82 patients, of both benign and malignant cases. Experimental results have shown that the proposed method outperforms a number of state of the art approaches providing average classification rates of 95.8% and 91.38% with our dataset and the BreaKHis dataset, respectively.

In this paper we address the problem of recognizing student engagement in prosocial games by exploiting engagement cues from different input modalities. Since engagement is a multifaceted phenomenon with different dimensions, i.e., behavioral, cognitive and affective, we propose the modeling of student engagement using real-time data from both the students and the game. More specifically, we apply body motion and facial expression analysis to identify the affective state of students, while we extract features related to their cognitive and behavioral engagement based on the analysis of their interaction with the game. For the automatic recognition of engagement, we adopt a machine learning approach based on artificial neural networks, while for the annotation of the engagement data, we introduce a novel approach based on the use of games with different degrees of challenge in conjunction with a retrospective self-reporting method. To evaluate the proposed methodology, we conducted real-life experiments in four classes, in three primary schools, with 72 students and 144 gameplay recordings in total. Experimental results show the great potential of the proposed methodology, which improves the classification accuracy of the three distinct dimensions with a detection rate of 85%. A detailed analysis of the role of each component of the Game Engagement Questionnaire (GEQ), i.e., immersion, presence, flow and absorption, in the classification process is also presented in this paper.

In this paper, a complete methodology for automatic detection of centroblasts (CBs) in microscopic images acquired from tissue biopsies of follicular lymphoma is presented. In the proposed method, tissue sections are sliced at a low thickness level, around 1–1.5 $$\upmu $$ m, which provides a more detailed depiction of the nuclei and other textural information. Initially, images are segmented into their basic cytological components, i.e., blood cells, nuclei and extra-cellular material, and then a novel touching-cell splitting algorithm is applied using a Gaussian mixture model and expectation–maximization algorithm. Additionally, a morphological and textural analysis of CBs is applied in order to extract various features related to their nuclei, nucleoli and cytoplasm. In the final step, a novel classification scheme is proposed based on adaptive neuro-fuzzy inference systems to classify the candidate cells. The methodology yielded promising results with an average detection rate of 90.35 %.

Cataloged from PDF version of article. This is a review article describing the recent developments in Video based Fire Detection (VFD). Video surveillance cameras and computer vision methods are widely used in many security applications. It is also possible to use security cameras and special purpose infrared surveillance cameras for fire detection. This requires intelligent video processing techniques for detection and analysis of uncontrolled fire behavior. VFD may help reduce the detection time compared to the currently available sensors in both indoors and outdoors because cameras can monitor “volumes” and do not have transport delay that the traditional “point” sensors suffer from. It is possible to cover an area of 100 km2 using a single pan-tiltzoom camera placed on a hilltop for wildfire detection. Another benefit of the VFD systems is that they can provide crucial information about the size and growth of the fire, direction of smoke propagation. © 2013 Elsevier Inc. All rights reserved

A challenging research issue, which has recently attracted a lot of attention, is the incorporation of emotion recognition technology in serious games applications, in order to improve the quality of interaction and enhance the gaming experience. To this end, in this paper, we present an emotion recognition methodology that utilizes information extracted from multimodal fusion analysis to identify the affective state of players during gameplay scenarios. More specifically, two monomodal classifiers have been designed for extracting affective state information based on facial expression and body motion analysis. For the combination of different modalities a deep model is proposed that is able to make a decision about player’s affective state, while also being robust in the absence of one information cue. In order to evaluate the performance of our methodology, a bimodal database was created using Microsoft’s Kinect sensor, containing feature vectors extracted from users' facial expressions and body gestures. The proposed method achieved higher recognition rate in comparison with mono-modal, as well as early-fusion algorithms. Our methodology outperforms all other classifiers, achieving an overall recognition rate of 98.3%.

In this paper, we present an emotion recognition methodology that utilizes information extracted from body motion analysis to assess affective state during gameplay scenarios. A set of kinematic and geometrical features are extracted from joint-oriented skeleton tracking and are fed to a deep learning network classifier. In order to evaluate the performance of our methodology, we created a dataset with Microsoft Kinect recordings of body motions expressing the five basic emotions (anger, happiness, fear, sadness and surprise) which are likely to appear in a gameplay scenario. In this five emotions recognition problem, our methodology outperformed all other classifiers, achieving an overall recognition rate of 93 %. Furthermore, we conducted a second series of experiments to perform a qualitative analysis of the features and assess the descriptive power of different groups of features.

Automated centroblast (CB) detection in Follicular Lymphoma (FL) tissue samples has recently attracted significant research interest. Most of the methods described in the literature are based on the use of Hematoxilin and Eosin (H&E) stain. However, the automated detection of CBs from H&E stained images remains a challenging issue. To this end, this paper presents a novel approach which is based on the use of both PAX5 and H&E stains in tissue sections sliced at the thickness of 1μm. The goal of PAX5 is three-fold: to facilitate the segmentation of nuclei, to remove a number of follicular dendritic cells and finally to extract morphological characteristics of nuclei. Furthermore, the use of H&E stain enables us to extract textural information related to histological characteristics used by pathologists in diagnosis of FL grading. In our method we propose a novel algorithm for the separation of overlapped nuclei inspired by the clustering of large scale visual vocabularies. Finally, aiming to model pathologists’ knowledge used in FL grading, we use a Bayesian Network classifier to combine the morphological and textural characteristics. Experiments conducted on a dataset of ten pairs of PAX5 and H&E images demonstrate the potential of the proposed approach providing an average detection rate of 93.46%.

Outdoor large-scale cultural sites are mostly sensitive to environmental, natural and human made factors, implying an imminent need for a spatio-temporal assessment to identify regions of potential cultural interest (material degradation, structuring, conservation). On the other hand, in Cultural Heritage research quite different actors are involved (archaeologists, curators, conservators, simple users) each of diverse needs. All these statements advocate that a 5D modelling (3D geometry plus time plus levels of details) is ideally required for preservation and assessment of outdoor large scale cultural sites, which is currently implemented as a simple aggregation of 3D digital models at different time and levels of details. The main bottleneck of such an approach is its complexity, making 5D modelling impossible to be validated in real life conditions. In this paper, a cost effective and affordable framework for 5D modelling is proposed based on a spatial-temporal dependent aggregation of 3D digital models, by incorporating a predictive assessment procedure to indicate which regions (surfaces) of an object should be reconstructed at higher levels of details at next time instances and which at lower ones. In this way, dynamic change history maps are created, indicating spatial probabilities of regions needed further 3D modelling at forthcoming instances. Using these maps, predictive assessment can be made, that is, to localize surfaces within the objects where a high accuracy reconstruction process needs to be activated at the forthcoming time instances. The proposed 5D Digital Cultural Heritage Model (5D-DCHM) is implemented using open interoperable standards based on the CityGML framework, which also allows the description of additional semantic metadata information. Visualization aspects are also supported to allow easy manipulation, interaction and representation of the 5D-DCHM geometry and the respective semantic information. The open source 3DCityDB incorporating a PostgreSQL geo-database is used to manage and manipulate 3D data and their semantics.

Human body motion tracking is important for a large variety of applications e.g., entertainemnt, human computer interaction, creative intudstry etc. Μarker-based motion capture systems offer accurate motion tracking, however, they cannot be widely used due to their high cost and operational complexity. To this end, emerging depth sensing technologies, such as Microsoft Kinect, can be employed in many cases as an alternative low cost marker-less solution. Such sensors are mainly used to track the frontal view of the user, since it provides high tracking quality. To assess the potential tracking abilities of Kinect nonfrontal views, we compare the tracking results from four views (front, left side, right side, rear) with those from an off-the-self optical motion capture system. The experimental results show that the skeleton tracking accuracy of the rear view is close to the accuracy of the frontal view, although Kinect sensor cannot distinguish between the front and back sides of the user. In addition, the left and right side views can produce valuable tracking results for the body parts closer to each sensor.

In this paper we introduce Multi-Entity Bayesian Networks (MEBNs) as the means to combine first-order logic with probabilistic inference and facilitate the semantic analysis of Intangible Cultural Heritage (ICH) content. First, we mention the need to capture and maintain ICH manifestations for the safeguarding of cultural treasures. Second, we present the MEBN models and stress their key features that can be used as a powerful tool for the aforementioned cause. Third, we present the methodology followed to build a MEBN model for the analysis of a traditional dance. Finally, we compare the efficiency of our MEBN model with that of a simple Bayesian network and demonstrate its superiority in cases that demand for situation-specific treatment.

Game-based learning and gamification techniques are recently becoming a popular trend in the field of Technology Enhanced Learning. In this paper, we mainly focus on the use of game design elements for the transmission of Intangible Cultural Heritage (ICH) knowledge and, especially, for the learning of traditional dances. More specifically, we present a 3D game environment that employs an enjoyable natural human computer interface, which is based on the fusion of multiple depth sensors data in order to capture the body movements of the user/learner. In addition, the system automatically assesses the learner’s performance by utilizing a combination of Dynamic Time Warping (DTW) with Fuzzy Inference System (FIS) approach and provides feedback in a form of a score as well as instructions from a virtual tutor in order to promote self-learning. As a pilot use case, a Greek traditional dance, namely Tsamiko, has been selected. Preliminary small-scaled experiments with students of the Department of Physical Education and Sports Science at Aristotle University of Thessaloniki have shown the great potential of the proposed application.

An aircraft is most at risk for an accident when it's still on the ground - when taxiing before takeoff or after landing. This is because traffic throughput on the ground is limited by inadequate airport infrastructures and is often incapacitated during conditions of poor visibility. The INTERVUSE project, funded by the European Commission, aims to address these problems by developing a cost-effective artificial intelligence system based on a network of intelligent digital cameras. The system uses image-processing techniques to detect traffic and correlates and fuses data to generate a synthetic ground-situation display.

In this paper we present a novel interactive platform for the operational support and training of the forest fire volunteer groups that are accredited through the mechanisms of Civil Protection Services. The proposed system is based on mobile and web technologies, fire simulation techniques and shortest-path routing calculation. The system consists of a mobile application (client) allowing the volunteers to report fire incidents and vegetation types of an area to a Control Center (server). The Control Center is also capable of organizing Training Scenarios, based on fire simulations and detailed safe routes, which are sent to the volunteers' mobile devices. The system was tested by Greek and Bulgarian volunteer firefighters in the context of the `OUTLAND' project. The architecture and the functionalities of the proposed system are presented accompanied by figures and indicative screenshots.

In this paper, we describe a novel methodology for dance learning and evaluation using multi-sensor and 3D gaming technology. The learners are captured during dancing, while an avatar visualizes their motion using fused input from multiple sensors. Motion analysis and fuzzy-logic are employed for the evaluation of the learners’ performance against the performance of an expert. Specifically, a two level Fuzzy Inference System is proposed which uses as input low level skeletal data and high level motion recognition probabilities for the evaluation of dancer’s performance. Tests with real dancers, both learners and experts, dancing Tsamiko, a very popular traditional Greek dance, are presented showing the potential of the proposed method.