Search

This paper provides an overview of some of the most relevant deep learning approaches to pattern extraction and recognition in visual arts, particularly painting and drawing. Recent advances in deep learning and computer vision, coupled with the growing availability of large digitized visual art collections, have opened new opportunities for computer science researchers to assist the art community with automatic tools to analyse and further understand visual arts. Among other benefits, a deeper understanding of visual arts has the potential to make them more accessible to a wider population, ultimately supporting the spread of culture.

In this paper, we propose a novel method for mild cognitive impairment detection based on jointly exploiting the complex network and the neural network paradigm. In particular, the method is based on ensembling different brain structural “perspectives” with artificial neural networks. On one hand, these perspectives are obtained with complex network measures tailored to describe the altered brain connectivity. In turn, the brain reconstruction is obtained by combining diffusion-weighted imaging (DWI) data to tractography algorithms. On the other hand, artificial neural networks provide a means to learn a mapping from topological properties of the brain to the presence or absence of cognitive decline. The effectiveness of the method is studied on a well-known benchmark data set in order to evaluate if it can provide an automatic tool to support the early disease diagnosis. Also, the effects of balancing issues are investigated to further assess the reliability of the complex network approach to DWI data.

Cultural heritage, especially the fine arts, plays an invaluable role in the cultural, historical, and economic growth of our societies [...]

Unmanned aerial vehicles (UAVs) also known as drones are increasingly populating our skies. This represents a relevant issue both for the legislator and the researcher. While the regulation plans often assume precautionary approaches, stating restrictive conditions of use for the sake of public safety, the applied research is exploring novel strategies to develop autonomous vehicles endowed with trusty operating mechanisms. The challenge is to let drones overflying even populated areas while keeping a steady control of the situation on the ground, thus enabling the possibility of safe landing with no harm for people. This can be done employing on-board cameras and embedded GPUs which allow for the execution in real-time of computer vision applications. In this paper, we introduce a crowd detection method for drone safe landing. The pivotal points of our proposal are related to the computational limitations imposed by the currently available hardware resources of UAVs. In this sense, our method is based on the light-weight scheme of a fully-convolutional neural network which conjugates nimble computations and effectiveness. We propose a two-loss model where a classification task (oriented to distinguish between crowded/non-crowded scenes) is supported by a regression task (aimed at better focusing the agglomeration tendency of the persons). This latter job is realized by resorting to the construction of a spatial graph for each analysed image and to the evaluation of the corresponding clustering coefficient. As a further element, our model is endowed with the capability to produce class activation heatmaps which contribute to the semantic enrichment of the flight maps. We tested our model on a large dataset of aerial images and we observed how it compares favorably with other approaches proposed in literature.

Computer aided diagnosis systems can provide non-invasive, low-cost tools to support clinicians. These systems have the potential to assist the diagnosis and monitoring of neurodegenerative disorders, in particular Parkinson’s disease (PD). Handwriting plays a special role in the context of PD assessment. In this paper, the discriminating power of “dynamically enhanced” static images of handwriting is investigated. The enhanced images are synthetically generated by exploiting simultaneously the static and dynamic properties of handwriting. Specifically, we propose a static representation that embeds dynamic information based on: (i) drawing the points of the samples, instead of linking them, so as to retain temporal/velocity information; and (ii) adding pen-ups for the same purpose. To evaluate the effectiveness of the new handwriting representation, a fair comparison between this approach and state-of-the-art methods based on static and dynamic handwriting is conducted on the same dataset, i.e. PaHaW. The classification workflow employs transfer learning to extract meaningful features from multiple representations of the input data. An ensemble of different classifiers is used to achieve the final predictions. Dynamically enhanced static handwriting is able to outperform the results obtained by using static and dynamic handwriting separately.

Handwriting dynamics is relevant to discriminate people affected by neurodegenerative dementia from healthy subjects. This can be possible by administering simple and easy-to-perform handwriting/drawing tasks on digitizing tablets provided with electronic pens. Encouraging results have been recently obtained; however, the research community still lacks an acquisition protocol aimed at (i) collecting different traits useful for research purposes and (ii) supporting neurologists in their daily activities. This work proposes a handwriting-based protocol that integrates handwriting/drawing tasks and a digitized version of standard cognitive and functional tests already accepted, tested, and used by the neurological community. The protocol takes the form of a modular framework which facilitates the modification, deletion, and incorporation of new tasks in accordance with specific requirements. A preliminary evaluation of the protocol has been carried out to assess its usability. Successively, the protocol has been administered to more than 100 elderly MCI and match controlled subjects. The proposed protocol intends to provide a “cognitive model” for evaluating the relationship between cognitive functions and handwriting processes in healthy subjects as well as in cognitively impaired patients. The long-term goal of this research is the development of an easy-to-use and non-invasive methodology for detecting and monitoring neurodegenerative dementia during screening and follow-up.

The analysis of gene expression data is a complex task, and many tools and pipelines are available to handle big sequencing datasets for case-control (bivariate) studies. In some cases, such as pilot or exploratory studies, the researcher needs to compare more than two groups of samples consisting of a few replicates. Both standard statistical bioinformatic pipelines and innovative deep learning models are unsuitable for extracting interpretable patterns and information from such datasets. In this work, we apply a combination of fuzzy rule systems and genetic algorithms to analyze a dataset composed of 21 samples and 6 classes, useful for approaching the study of expression profiles in ovarian cancer, compared to other ovarian diseases. The proposed method is capable of performing a feature selection among genes that is guided by the genetic algorithm, and of building a set of if-then rules that explain how classes can be distinguished by observing changes in the expression of selected genes. After testing several parameters, the final model consists of 10 genes involved in the molecular pathways of cancer and 10 rules that correctly classify all samples.

Parkinson’s disease (PD) is commonly characterized by several motor symptoms, such as bradykinesia, akinesia, rigidity, and tremor. The analysis of patients’ fine motor control, particularly handwriting, is a powerful tool to support PD assessment. Over the years, various dynamic attributes of handwriting, such as pen pressure, stroke speed, in-air time, etc., which can be captured with the help of online handwriting acquisition tools, have been evaluated for the identification of PD. Motion events, and their associated spatio-temporal properties captured in online handwriting, enable effective classification of PD patients through the identification of unique sequential patterns. This paper proposes a novel classification model based on one-dimensional convolutions and Bidirectional Gated Recurrent Units (BiGRUs) to assess the potential of sequential information of handwriting in identifying Parkinsonian symptoms. One-dimensional convolutions are applied to raw sequences as well as derived features; the resulting sequences are then fed to BiGRU layers to achieve the final classification. The proposed method outperformed state-of-the-art approaches on the PaHaW dataset and achieved competitive results on the NewHandPD dataset.

Automatic age estimation from facial images is attracting increasing interest due to its many potential applications. Several deep learning-based methods have been proposed to tackle this task; however, they usually require prohibitive resources to run in real-time. In this work, we propose a fully automated system based on YOLOv5 and EfficientNet to perform face detection and subsequent age estimation in real-time. Also, to make the model more robust, EfficientNet was trained on the new MIVIA Age Dataset, released as part of a challenge. The results obtained in the contest are promising, and are strengthened by the lightness of the overall system which in fact is not only effective but also efficient.

Virtual Learning Environments (VLEs) are web platforms where educational content is delivered, along with tools to support individual study. Logs that record how students interact with the platform are collected daily, so automated methods can be used to extract useful knowledge from these data. All stakeholders involved in the learning activities of the VLEs, especially students and teachers, can benefit from the insights derived from the educational data and valuable information can be extracted using machine learning algorithms. Usually, educational data are examined as stationary data using conventional batch methods. However, these data are non-stationary by nature and could be better treated as data streams. This paper reports the results of a classification study in which Random Forests, applied in both batch and adaptive mode, are used to build a model for predicting student exam failure/success. In addition, an analysis of the most important features is performed to detect the most discriminating attributes related to the student’s result. Experiments conducted on a subset of the Open University Learning Analytics (OULAD) dataset demonstrate the reliability of the adaptive version of Random Forest in accurately classifying the evolving educational data.

This paper provides a brief overview of some of the most relevant deep learning approaches to visual art pattern extraction and recognition, particularly painting and drawing. Indeed, recent advances in deep learning and computer vision, coupled with the growing availability of large digitized visual art collections, have opened new opportunities for computer science researchers to assist the art community with automatic tools to analyze and further understand visual arts. Among other benefits, a deeper understanding of visual arts has the potential to make them more accessible to a wider population, both in terms of fruition and creation, thus supporting the spread of culture.

Although several vaccination campaigns have been launched to combat the ongoing COVID-19 pandemic, the primary treatment of suspected infected people is still symptomatic. In particular, the analysis of images derived from computed tomography (CT) appears to be useful for retrospectively analyzing the novel coronavirus and the chest injuries it causes. The growing body of literature on this topic shows the predominance of supervised learning methods that are typically adopted to automatically discriminate pathological patients from normal controls. However, very little work has been done from an unsupervised perspective. In this paper, we propose a new pipeline for automatic clustering of CT scans of COVID-19 patients based on deep learning. A pre-trained convolutional neural network is used for feature extraction; then, the extracted features are used as input to a deep embedding clustering model to perform the final clustering. The method was tested on the publicly available SARS-CoV-2 CT-Scan dataset that not only provides scans of COVID patients but also of patients with other lung conditions. The results obtained indicate that the radiological features of COVID patients largely overlap with those of other lung diseases. Unsupervised approaches to COVID analysis are promising, as they reduce the need for hard-to-collect human annotations and allow for deeper analysis not tied to a binary or multiclass classification task.

As network cyber attacks continue to evolve, traditional intrusion detection systems are no longer able to detect new attacks with unexpected patterns. Deep learning is currently addressing this problem by enabling unprecedented breakthroughs to properly detect unexpected network cyber attacks. However, the lack of decomposability of deep neural networks into intuitive and understandable components makes deep learning decisions difficult to interpret. In this paper, we propose a method for leveraging the visual explanations of deep learning-based intrusion detection models by making them more transparent and accurate. In particular, we consider a CNN trained on a 2D representation of historical network traffic data to distinguish between attack and normal flows. Then, we use the Grad-CAM method to produce coarse localization maps that highlight the most important regions of the traffic data representation to predict the cyber attack. Since decisions made on samples belonging to the same class are expected to be explained with similar localization maps, we base the final classification of a new network flow on the class of the nearest-neighbour historical localization map. Experiments with various benchmark datasets demonstrate the effectiveness of the proposed method compared to several state-of-the-art methods.

Unmanned aerial vehicles (UAVs), more commonly known as drones, are increasingly used as a technological support tool for search-and-rescue (SAR) operations and also for post-disaster area explorations. UAVs can be equipped with high-resolution cameras and embed GPUs powerful enough to provide effective and efficient aid to emergency rescue operations. Victims can be unconscious or injured, and any means of locating them as quickly as possible is critical to increasing their chances of survival. In particular, using drones that can automatically detect people in scenes can increase detection rate, while reducing rescue time. In this work, we present a new dataset specifically designed for SAR operations from drones using computer vision. Being small in size, the dataset is currently intended for testing and evaluation purposes only. We also provide baseline results on this dataset, obtained with the state-of-the-art TinyYOLOv3 object detector. The purpose of this preliminary work is to stimulate interest and encourage contributions to this research topic.

This study aims to develop a CNN-based system for dolphins’ photo-identification. Starting from 1,960 photos of dolphin fins each one labeled with the related dolphin name, we have developed a predictive model able to classify unknown fins. In particular, the system is composed by two different CNNs. The first one automatically learns how to create a mask to extract the foreground, i.e. the fin, from each original image. Then, these masks are used as starting point for the GrabCut algorithm, which creates more precise binary masks. Finally, the obtained masked images are given as input to the second CNN, which aims to recognize an unknown dolphin’s fin, providing a probability for each known individual.

Crowd analysis is receiving an increasing attention in the last years because of its social and public safety implications. One of the building blocks of crowd analysis is crowd counting and the associated crowd density estimation. Several commercially available drones are equipped with onboard cameras and embed powerful GPUs, making them an excellent platform for real-time crowd counting tools. This paper proposes a light-weight and fast fully-convolutional neural network to learn a regression model for crowd counting in images acquired from drones. A robust model is derived by training the network from scratch on a subset of the very challenging VisDrone dataset, which is characterized by a high variety of locations, environments, perspectives and lighting conditions. The derived model achieves an MAE of 8.86 and an RMSE of 15.07 on the test images, outperforming models developed by state-of-the-art light-weight architectures, that are MobileNetV2 and YOLOv3.

Virtual Learning Environments (VLEs) are Web-based platforms where educational contents, together with study support tools, are provided. Logs recording the interactions between students and VLEs are collected on a daily basis, thus automatic techniques are needed to manage and analyze such huge quantities of data. Students, teachers, managers, and in general all stakeholders involved in the VLEs’ learning activities, can take advantage of the insights coming from educational data and useful information can be extracted by using machine learning techniques. Traditionally, educational data have been studied as stationary data by using conventional machine learning methods. However, educational data are non-stationary by nature and they can be better treated as data streams. In this paper, we show the results of a classification study where the random forest algorithm, applied both in batch and adaptive mode, is used to develop a model for predicting the failure/success of students’ exams. Moreover, a feature importance analysis is carried out to detect the most discriminant attributes for the predictive task. Experiments were performed on the Open University Learning Analytics Dataset (OULAD) showing the reliability of adaptive random forest in creating accurate classification models from evolving educational data.

Multiple Sclerosis (MS) is a demyelinating autoimmune disease that usually affects young adults; however, recently some symptoms of cognitive impairment have been recognized as early signs of MS onset in pediatric patients (PedMS). The underlying relationships between these two conditions, as well as their molecular markers, have not been fully understood yet. In this work, we analyze microRNAs (miRNAs) expression profiles of PedMS patients with machine learning algorithms in order to create effective models able to detect the presence of cognitive impairment. In particular, we compare three different classification algorithms, fed with features automatically selected by a feature selection strategy. Experimental results show that linear support vector machines achieved the best performance. Moreover, we discuss the importance of ten of the most discriminant automatically selected miRNAs. A graphical analysis of these features highlights the relationships among miRNAs and the two classes the patients belongs to.

This paper proposes a novel technique for the automatic detection of dementia based on the attentional matrices test (AMT) for selective attention assessment. The original test provides three matrices, of increasing difficulty, and the test taker is asked to mark target digits assigned. In our proposal, AMT was developed on a digitizing tablet, equipped with an electronic pen. Tablet technology enables the acquisition of additional measures to those that can be obtained by observing the execution of the traditional paper-based test. These measures reflect the dynamics of the handwriting process, particularly the pauses and hesitations while the pen is not in contact with the pad surface. Handwriting measures can then serve as an input to machine learning algorithms to automatize disease detection. In contrast to the traditional approach, dynamic handwriting analysis can provide a means to better evaluate the visual search of the patient, as well as motor planning. To evaluate the effectiveness of the proposal, a classification study was carried out involving 29 healthy control subjects and 36 demented patients. We employed different machine learning algorithms and an ensemble scheme. We observed the first matrix to be the most discriminating, while the ensemble of the best classification models over the three matrices provided the best classification performance [i.e., an area under the ROC curve (AUC) of 87.30% and a sensitivity of 86.11%]. Our proposal has the potential to provide a cost-effective and easy-to-use diagnostic tool, which may also support mass screening of the population.

Clustering artworks is difficult for several reasons. On the one hand, recognizing meaningful patterns in accordance with domain knowledge and visual perception is extremely difficult. On the other hand, applying traditional clustering and feature reduction techniques to the highly dimensional pixel space can be ineffective. To address these issues, we propose to use a deep convolutional embedding model for digitized painting clustering, in which the task of mapping the raw input data to an abstract, latent space is jointly optimized with the task of finding a set of cluster centroids in this latent feature space. Quantitative and qualitative experimental results show the effectiveness of the proposed method. The model is also capable of outperforming other state-of-the-art deep clustering approaches to the same problem. The proposed method can be useful for several art-related tasks, in particular visual link retrieval and historical knowledge discovery in painting datasets., Accepted at ICPR2020. Added references. Corrected typos. Added new results and observations according to reviewers

Forensic handwriting examination is often criticized for its lack of objective standards and rigorous scientific validation. On the other hand, cutting-edge techniques for biometric handwriting and signature verification are often perceived as perfect black boxes and are not used by forensic handwriting examiners in their work environment. This paper presents an easy-to-explain yet effective framework to support semi-automatic signature verification in forensic settings. The proposed approach is based on measuring similarities between signatures by applying Dynamic Time Warping on easy-to-derive dynamic features. The goal is to provide forensic handwriting examiners with a decision support tool for making reproducible and less questionable inferences, while being both intuitive and easy to explain. The method is tested on a newly proposed dataset that also takes into account the so-called disguised signatures which are of extreme importance in this scenario.

This paper presents a preliminary easy to explain and effective framework for supporting dynamic signature analysis in forensic settings. The proposed approach is based on measuring similarities among signatures by applying Dynamic Time Warping on easy to derive dynamic measures. The long term goal of our research is to provide forensic handwriting examiners with a decision support tool to perform reproducible and less questionable inference.

Clustering artworks is a very difficult task. Recognizing meaningful patterns in accordance with domain expertise and visual perception, in fact, can be extremely hard. On the other hand, applying traditional clustering and feature reduction techniques to the highly dimensional raw pixel space can be ineffective. To overcome these problems, we propose to use a deep convolutional embedding clustering framework. The model simultaneously optimizes the task of mapping the input pixel data to a latent feature space and the task of finding cluster centroids in this latent space. A quantitative and qualitative preliminary study on a collection of artworks made by Pablo Picasso shows the effectiveness of the model. The proposed method may assist in art-related tasks, in particular visual link retrieval and historical knowledge discovery in painting datasets.

In this paper, we propose a novel crowd detection method for drone safe landing, based on an extremely light and fast fully convolutional neural network. Such a computer vision application takes advantage of the technical tools some commercial drones are equipped with. The proposed architecture is based on a two-loss model in which the main classification task, aimed at distinguishing between crowded and non-crowded scenes, is simultaneously assisted by a regression task, aimed at people counting. In addition, the proposed method provides class activation heatmaps, useful to semantically augment the flight maps. To evaluate the effectiveness of the proposed approach, we used the challenging VisDrone dataset, characterized by a very large variety of locations, environments, lighting conditions, and so on. The model developed by the proposed two-loss deep architecture achieves good values of prediction accuracy and average precision, outperforming models developed by a similar one-loss architecture and a more classic scheme based on MobileNet. Moreover, by lowering the confidence threshold, the network achieves very high recall, without sacrificing too much precision. The method also compares favorably with the state-of-the-art, providing an effective and efficient tool for several safe drone applications.

This paper presents a preliminary investigation aimed at developing a tool for visual link retrieval and knowledge discovery in painting datasets. The proposed framework is based on a deep convolutional network to perform feature extraction and on a fully-unsupervised nearest neighbor approach to retrieve visual links among digitized paintings. Moreover, the proposed method makes it possible to study influences among artists by means of graph analysis. The tool is intended to help art historians better understand visual arts.

This paper proposes a novel lightweight and fast convolutional neural network to learn a regression model for crowd counting in images captured from drones. The learning system is initially based on a multi-input model trained on two different views of the same input for the task at hand: (i) real-world images; and (ii) corresponding synthetically created “crowd heatmaps”. The synthetic input is intended to help the network focus on the most important parts of the images. The network is trained from scratch on a subset of the VisDrone dataset. During inference, the synthetic path of the network is disregarded resulting in a traditional single-view model optimized for resource-constrained devices. The derived model achieves promising results on the test images, outperforming models developed by state-of-the-art lightweight architectures that can be used for crowd counting and detection.

Diagnosing and monitoring Parkinson’s disease (PD) is a topic of current research in many fields, including AI. The innovative challenge is to develop a low-cost, non-invasive tool to support clinicians at the point of care. In particular, since handwriting difficulties in PD patients are well-known, changes in handwriting have emerged as a powerful discriminant factor for PD assessment. A crucial step in designing a decision support system based on handwriting concerns the choice of the most appropriate handwriting tasks to be administered for data acquisition. When data are collected, traditional approaches assume that different tasks, although not with the same impact, are all important for classification. However, not all tasks are likely to be useful for diagnosis, and the inclusion of these tasks may be detrimental to prediction accuracy. This work investigates the potential of an optimal subset of tasks for a more accurate PD classification. The evaluation is carried out by adopting a performance-driven multi-expert approach on different handwriting tasks performed by the same subjects. The multi-expert system is based on similar or conceptually different classifiers trained on features related to the dynamics of the handwriting process. The proposed approach improves baseline results on the PaHaW data set.

Machine learning techniques are tailored to build intelligent systems to support clinicians at the point of care. In particular, they can complement standard clinical evaluations for the assessment of early signs and manifestations of Parkinson’s disease (PD). Patients suffering from PD typically exhibit impairments of previously learned motor skills, such as handwriting. Therefore, handwriting can be considered a powerful marker to develop automatized diagnostic tools. In this paper, we investigated if and to which extent dynamic features of the handwriting process can support PD diagnosis at earlier stages. To this end, a subset of the publicly available PaHaW dataset has been used, including those patients showing only early to mild degree of disease severity. We developed a classification framework based on different classifiers and an ensemble scheme. Some encouraging results have been obtained, in particular, good specificity performances have been observed. This indicates that a handwriting-based decision support tool could be used to administer screening tests useful for ruling in disease.

The inherent features of Mobile Ad-hoc NETworks (MANETs) make them vulnerable to various kinds of security attacks. In particular, in a so-called blackhole attack, one or more malicious hosts can send fake routing information towards an initiator, compromising the reliability of the network in the whole. In this paper, we propose a refinement of the NACK-based Ad-hoc On-demand Distance Vector (N-AODV) protocol, namely Blackhole-free N-AODV (BN-AODV), as a solution to intercept (cooperative) blackhole attacks in MANETs. Thanks to a formalization through an Abstract State Machine-based model, the correctness of the proposed protocol is formally proved.

Mobile Ad-hoc NETworks (MANETs) are wireless networks designed for communications among nomadic hosts in absence of fixed infrastructure. Different reactive protocols for MANETs, in which routes are established only when needed, provide different network topology awareness (NTA) to each host, depending on their algorithmic features. NACK-based AODV (N-AODV) is a variant of the well-known Ad-hoc On-demand Distance Vector (AODV) reactive protocol for MANETs we proposed with the aim of improving NTA of the original protocol. In this paper, a performance comparison between AODV and N-AODV is conducted in order to investigate whether N-AODV effectively improves NTA with respect to AODV, and how this improvement affects both its effectiveness and efficiency. The experiment is executed within a simulation environment. The obtained results show that a MANET adopting N-AODV exploits higher NTA than a MANET adopting AODV. Moreover, the improved awareness impacts effectiveness and efficiency of routing activities because, in the long-run, it results in a lower need to activate the route discovery process for establishing new communications sessions.

The present paper proposes a variant of the Ad-hoc On-demand Distance Vector (AODV) routing protocol for Mobile Ad-hoc NETworks (MANETs) by means of an Abstract State Machine (ASM)-based model. The variant introduces a new unicast message, which makes each host aware about the network topology more quickly than in the original AODV.