Search

Artificial intelligence (AI) is predicted to have profound effects on the future of video capsule endoscopy (VCE) technology. The potential lies in improving anomaly detection while reducing manual labour. Existing work demonstrates the promising benefits of AI-based computer-assisted diagnosis systems for VCE. They also show great potential for improvements to achieve even better results. Also, medical data is often sparse and unavailable to the research community, and qualified medical personnel rarely have time for the tedious labelling work. We present Kvasir-Capsule, a large VCE dataset collected from examinations at a Norwegian Hospital. Kvasir-Capsule consists of 117 videos which can be used to extract a total of 4,741,504 image frames. We have labelled and medically verified 47,238 frames with a bounding box around findings from 14 different classes. In addition to these labelled images, there are 4,694,266 unlabelled frames included in the dataset. The Kvasir-Capsule dataset can play a valuable role in developing better algorithms in order to reach true potential of VCE technology., Measurement(s) Gastrointestinal Tract • gastrointestinal system disease Technology Type(s) Capsule Endoscope • visual assessment of in vivo video recording Sample Characteristic - Organism Homo sapiens Sample Characteristic - Environment alimentary part of gastrointestinal system Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.14178905

Nowadays, recommendation systems play an indispensable role in many fields, including e-commerce, finance, economy, and gaming. There is emerging research on publication venue recommendation systems to support researchers when submitting their scientific work. Several publishers such as IEEE, Springer, and Elsevier have implemented their submission recommendation systems only to help researchers choose appropriate conferences or journals for submission. In this work, we present a demo framework to construct an effective recommendation system for paper submission. With the input data (the title, the abstract, and the list of possible keywords) of a given manuscript, the system recommends the list of top relevant journals or conferences to authors. By using state-of-the-art techniques in natural language understanding, we combine the features extracted with other useful handcrafted features. We utilize deep learning models to build an efficient recommendation engine for the proposed system. Finally, we present the User Interface (UI) and the architecture of our paper submission recommendation system for later usage by researchers.

In this paper, we present PMData: a dataset that combines traditional lifelogging data with sports-activity data. Our dataset enables the development of novel data analysis and machine-learning applications where, for instance, additional sports data is used to predict and analyze everyday developments, like a person's weight and sleep patterns; and applications where traditional lifelog data is used in a sports context to predict athletes' performance. PMData combines input from Fitbit Versa 2 smartwatch wristbands, the PMSys sports logging smartphone application, and Google forms. Logging data has been collected from 16 persons for five months. Our initial experiments show that novel analyses are possible, but there is still room for improvement.

Mathematical morphology provides a large set of powerful non-linear image operators, widely used for feature extraction, noise removal or image enhancement. Although morphological filters might be used to remove artifacts produced by image manipulations, both on binary and gray level documents, little effort has been spent towards their forensic identification. In this paper we propose a non-trivial extension of a deterministic approach originally detecting erosion and dilation of binary images. The proposed approach operates on grayscale images and is robust to image compression and other typical attacks. When the image is attacked the method looses its deterministic nature and uses a properly trained SVM classifier, using the original detector as a feature extractor. Extensive tests demonstrate that the proposed method guarantees very high accuracy in filtering detection, providing 100% accuracy in discriminating the presence and the type of morphological filter in raw images of three different datasets. The achieved accuracy is also good after JPEG compression, equal or above 76.8% on all datasets for quality factors above 80. The proposed approach is also able to determine the adopted structuring element for moderate compression factors. Finally, it is robust against noise addition and it can distinguish morphological filter from other filters. publishedVersion

In this paper, we propose a deep learning based system for food recognition from personal life archive im- ages. The system first identifies the eating moments based on multi-modal information, then tries to focus and enhance the food images available in these moments, and finally, exploits GoogleNet as the core of the learning process to recognise the food category of the images. Preliminary results, experimenting on the food recognition module of the proposed system, show that the proposed system achieves 95.97% classification accuracy on the food images taken from the personal life archive from several lifeloggers, which potentially can be extended and applied in broader scenarios and for different types of food categories.

Modern computer graphics technologies brought realism in computer-generated characters, making them achieve truly natural appearance. Besides traditional virtual reality applications such as avatars, games, or cinema, these synthetic characters may be used to generate realistic fakes, which may lead to improper use of the technology. This fact raises the demand for advanced tools able to discriminate real and artificial human faces in digital media. In this paper, we propose a method to distinguish between computer generated and natural faces by modeling and evaluating their dynamic behavior. Because of a 3D-model-based video analysis, the proposed technique allows identifying synthetic characters by detecting their more limited variability over time. Experimental results demonstrate the effectiveness of the proposed approach also on very challenging and realistic video sequences.

Face authentication has been shown to be vulnerable against three main kinds of attacks: print, replay, and 3D mask. Among those, video replay attacks appear more challenging to be detected. There exist in the literature many countermeasures to face spoofing attacks, but a sophisticated detector is still needed to deal with particularly high-quality video based attacks. In this work, we perform analysis on the noise residual in frequency domain, and extract discriminative features by using a dynamic texture descriptor to characterize video based spoofing attacks. We propose a promising detector, which produces competitive results on the most challenging dataset of video based spoofing.

Automatic detection of diseases by use of computers is an important, but still unexplored field of research. Such innovations may improve medical practice and refine health care systems all over the world. However, datasets containing medical images are hardly available, making reproducibility and comparison of approaches almost impossible. In this paper, we present KVASIR, a dataset containing images from inside the gastrointestinal (GI) tract. The collection of images are classified into three important anatomical landmarks and three clinically significant findings. In addition, it contains two categories of images related to endoscopic polyp removal. Sorting and annotation of the dataset is performed by medical doctors (experienced endoscopists). In this respect, KVASIR is important for research on both single- and multi-disease computer aided detection. By providing it, we invite and enable multimedia researcher into the medical domain of detection and retrieval.

Analysis of medical videos for detection of abnormalities and diseases requires both high precision and recall, but also real-time processing for live feedback and scalability for massive screening of entire populations. Existing work on this field does not provide the necessary combination of retrieval accuracy and performance.; AB@In this paper, a multimedia system is presented where the aim is to tackle automatic analysis of videos from the human gastrointestinal (GI) tract. The system includes the whole pipeline from data collection, processing and analysis, to visualization. The system combines filters using machine learning, image recognition and extraction of global and local image features. Furthermore, it is built in a modular way so that it can easily be extended. At the same time, it is developed for efficient processing in order to provide real-time feedback to the doctors. Our experimental evaluation proves that our system has detection and localisation accuracy at least as good as existing systems for polyp detection, it is capable of detecting a wider range of diseases, it can analyze video in real-time, and it has a low resource consumption for scalability.

In this paper, we propose a novel approach for face spoofing detection using the high-order Local Derivative Pattern from Three Orthogonal Planes (LDP-TOP). The proposed method is not only simple to derive and implement, but also highly efficient, since it takes into account both spatial and temporal information in different directions of subtle face movements. According to experimental results, the proposed approach outperforms state-of-the-art methods on three reference datasets, namely Idiap REPLAY-ATTACK, CASIA-FASD, and MSU MFSD. Moreover, it requires only 25 video frames from each video, i.e., only one second, and thus potentially can be performed in real time even on low-cost devices.

In this paper, we present a novel method that can produce a visual description of a landmark by choosing the most diverse pictures that best describe all the details of the queried location from community-contributed datasets. The main idea of this method is to filter out non-relevant images at a first stage and then cluster the images according to textual descriptors first, and then to visual descriptors. The extraction of images from different clusters according to a measure of user's credibility, allows obtaining a reliable set of diverse and relevant images. Experimental results performed on the MediaEval 2014 “Retrieving Diverse Social Images” dataset show that the proposed approach can achieve very good performance outperforming state-of-art techniques.

Digital graphics tools are nowadays capable of rendering highly photorealistic imagery, which easily puzzle our perception of reality. This poses serious ethical and legal issues, which in turn create the need for further technologies able to ensure the trustworthiness of digital media as a true representation of reality, especially when depicting humans. In this work, we propose a novel forensic technique to tackle the problem of distinguishing computer generated (CG) from real humans in videos. It exploits the temporal information inherent of a video sequence by analyzing the spatio-temporal appearance of facial expressions in both CG and real humans. Even if rendering facial expression has reached outstanding performances, CG face appearance over time still presents some underlying mechanical properties that greatly differ from the natural muscle movements of real humans. We build an efficient classifier on a set of features describing facial dynamics and spatio-temporal changes during smiling to distinguish CG from human faces. Experimental results demonstrate the effectiveness of the proposed approach.

Given the recent development of advanced multimedia techniques able to support the creation of realistic computer generated characters, there is the parallel need of automatic tools allowing users to verify the source of the multimedia data they are observing, thus discriminating between artificial and natural information. In this paper, we focus on video representing human beings and we propose a novel method to identify computer generated characters by analysing the evolution of the face model in chronological order. Experimental results show that photorealistic facial animations, which are usually performed following fixed patterns, can be distinguished from natural ones, which follow much more complicated and various geometric distortions.

Median filtering is a well-known non linear denoising filter often used as an harmless post-processing, sometimes also employed to affect the reliability of some forensic techniques. In this work, we present a novel counter-forensic method able to conceal the characteristic traces left by median filtering. By exploiting the knowledge of features used in existing median filtering detectors, we are able to remove the characteristic footprints via suitable random pixel modification, while keeping the quality of the counter-attacked image high. Experimental results show that the proposed method is very effective, computationally efficient and competitive with other state-of-the-art techniques.

In this paper we propose a new method to infer human social interactions using typical techniques adopted in literature for visual search and information retrieval. The main piece of information we use to discriminate among different types of interactions is provided by proxemics cues acquired by a tracker, and used to distinguish between intentional and casual interactions. The proxemics information has been acquired through the analysis of two different metrics: on the one hand we observe the current distance between subjects, and on the other hand we measure the O-space synergy between subjects. The obtained values are taken at every time step over a temporal sliding window, and processed in the Discrete Fourier Transform (DFT) domain. The features are eventually merged into an unique array, and clustered using the K-means algorithm. The clusters are reorganized using a second larger temporal window into a Bag Of Words framework, so as to build the feature vector that will feed the SVM classifier.

Significant improvements have been recently achieved in both quality and realism of computer generated characters, which are nowadays often very difficult to be distinguished from real ones. However, generating highly realistic facial expressions is still a challenging issue, since synthetic expressions usually follow a repetitive pattern, while in natural faces the same expression is usually produced in similar but not equal ways. In this paper, we propose a method to distinguish between computer generated and natural faces based on facial expressions analysis. In particular, small variations of the facial shape models corresponding to the same expression are used as evidence of synthetic characters.

Nowadays, large-scale networked social media need better search technologies to achieve suitable performance. Multimodal approaches are promising technologies to improve image ranking. This is particularly true when metadata are not completely reliable, which is a rather common case as far as user annotation, time and location are concerned. In this paper, we propose to properly combine visual information with additional multi-faceted information, to define a novel multimodal similarity measure. More specifically, we combine visual features, which strongly relate to the image content, with semantic information represented by manually annotated concepts, and geo tagging, very often available in the form of object/subject location. Furthermore, we propose a supervised machine learning approach, based on Support Vector Machines (SVMs), to automatically learn optimized weights to combine the above features. The resulting models is used as a ranking function to sort the results of a multimodal query.

In this work we introduce a new application of saliency detection as a support for forensic analysis in digital images. Indeed, although there are some attempts to achieve both detection and localization of image tampering, the majority of forensics tools require hypothesis about the modified area. Since we claim that salient objects/subjects, conveying the semantic content of the image, are the regions whose integrity is more critical, we propose to combine salient object detection and forensic analysis on the corresponding output. To this aim we introduce here an improved version of a saliency map extractor based on segmentation and describe the application of a tampering detection method allowing digital composite detection. Experimental results are presented and discussed.

Quick reorganizing and draft annotating personal photo albums under event scheme is an emerging trend. In this research, a method has been developed to meet such requirements using the idea of gist and mosaic art so that viewers could understand the meaning of a whole scene without paying much attention in individual details. First, given a photo album, all chronologically ordered images are normalized to a smaller size, and then mosaicked side-by-side to create a signature image representing for that album. Next, by integrating the optimized linear programming with the color descriptor of the signature image, not only the event-type of the album but also all sub-event-types of the sub-sequence photos are decided. More than 19,000 images of five varied event-types have been used to evaluate the proposed method. Experimental results show that the proposed method could detect events towards annotation and re-organization of personal photo albums with high accuracy at a rapid speed.