Search

In this work, we propose a 3D fully convolutional architecture for video saliency prediction that employs hierarchical supervision on intermediate maps (referred to as conspicuity maps) generated using features extracted at different abstraction levels. We provide the base hierarchical learning mechanism with two techniques for domain adaptation and domain-specific learning. For the former, we encourage the model to unsupervisedly learn hierarchical general features using gradient reversal at multiple scales, to enhance generalization capabilities on datasets for which no annotations are provided during training. As for domain specialization, we employ domain-specific operations (namely, priors, smoothing and batch normalization) by specializing the learned features on individual datasets in order to maximize performance. The results of our experiments show that the proposed model yields state-of-the-art accuracy on supervised saliency prediction. When the base hierarchical model is empowered with domain-specific modules, performance improves, outperforming state-of-the-art models on three out of five metrics on the DHF1K benchmark and reaching the second-best results on the other two. When, instead, we test it in an unsupervised domain adaptation setting, by enabling hierarchical gradient reversal layers, we obtain performance comparable to supervised state-of-the-art. Source code, trained models and example outputs are publicly available at https://github.com/perceivelab/hd2s.

Human behavior understanding in videos is a complex, still unsolved problem and requires to accurately model motion at both the local (pixel-wise dense prediction) and global (aggregation of motion cues) levels. Current approaches based on supervised learning require large amounts of annotated data, whose scarce availability is one of the main limiting factors to the development of general solutions. Unsupervised learning can instead leverage the vast amount of videos available on the web and it is a promising solution for overcoming the existing limitations. In this paper, we propose an adversarial GAN-based framework that learns video representations and dynamics through a self-supervision mechanism in order to perform dense and global prediction in videos. Our approach synthesizes videos by (1) factorizing the process into the generation of static visual content and motion, (2) learning a suitable representation of a motion latent space in order to enforce spatio-temporal coherency of object trajectories, and (3) incorporating motion estimation and pixel-wise dense prediction into the training procedure. Self-supervision is enforced by using motion masks produced by the generator, as a co-product of its generation process, to supervise the discriminator network in performing dense prediction. Performance evaluation, carried out on standard benchmarks, shows that our approach is able to learn, in an unsupervised way, both local and global video dynamics. The learned representations, then, support the training of video object segmentation methods with sensibly less (about 50%) annotations, giving performance comparable to the state of the art. Furthermore, the proposed method achieves promising performance in generating realistic videos, outperforming state-of-the-art approaches especially on motion-related metrics.

Immunotherapy is one of the most interesting and promising cancer treatments. Encouraging results have confirmed the effectiveness of immunotherapy drugs for treating tumors in terms of long-term survival and a significant reduction in toxicity compared to more traditional chemotherapy approaches. However, the percentage of patients eligible for immunotherapy is rather small, and this is likely related to the limited knowledge of physiological mechanisms by which certain subjects respond to the treatment while others have no benefit. To address this issue, the authors propose an innovative approach based on the use of a non-linear cellular architecture with a deep downstream classifier for selecting and properly augmenting 2D features from chest-abdomen CT images toward improving outcome prediction. The proposed pipeline has been designed to make it usable over an innovative embedded Point of Care system. The authors report a case study of the proposed solution applied to a specific type of aggressive tumor, namely Metastatic Urothelial Carcinoma (mUC). The performance evaluation (overall accuracy close to 93%) confirms the proposed approach effectiveness.

12th International Workshop on Machine Learning in Medical Imaging, MLMI 2021, held in conjunction with 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 -- 27 September 2021 through 27 September 2021 -- -- 266089, Visual explanation methods have an important role in the prognosis of the patients where the annotated data is limited or unavailable. There have been several attempts to use gradient-based attribution methods to localize pathology from medical scans without using segmentation labels. This research direction has been impeded by the lack of robustness and reliability. These methods are highly sensitive to the network parameters. In this study, we introduce a robust visual explanation method to address this problem for medical applications. We provide an innovative visual explanation algorithm for general purpose and as an example application we demonstrate its effectiveness for quantifying lesions in the lungs caused by the Covid-19 with high accuracy and robustness without using dense segmentation labels. This approach overcomes the drawbacks of commonly used Grad-CAM and its extended versions. The premise behind our proposed strategy is that the information flow is minimized while ensuring the classifier prediction stays similar. Our findings indicate that the bottleneck condition provides a more stable severity estimation than the similar attribution methods. The source code will be publicly available upon publication. © 2021, Springer Nature Switzerland AG.

This study shows the results of an interdisciplinary research aimed at devising artificial intelligence methods to support data understanding in Cultural Heritage. The objective is to create automated methods for automatically classifying and successively retrieving photographic data leveraging the current breed of AI methods based on Deep Learning paradigm. In this work, the lack of images to be used for the training of the AI system is addressed by testing an approach based on Generative Adversarial Networks (GANs) for automatically synthesizing unrealistic photos, which, in turn, can be used for training image classification and retrieval systems. Specifically, we propose a method to drive the generation of realistic classical order images using GAN approaches anchored to semantic ontology domain representation. More specifically, the proposed AI leverages the advantages of knowledge-based and data-driven methods and foresees a level image generation: the first one that synthesizes isolated objects corresponding to each ontology concept and the second one that, instead, combines the generated objects according to the spatial information provided by the ontology in order to generate realistic scenes. The resulting images can then be employed to train automated classifiers and more reliable retrieval methods by minimizing the efforts usually required by human operators to manually annotate data. In addition, the proposed generative strategy allows for a deeper understanding of cultural heritage visual data enabling the possibility to enhance design capabilities in simulation contexts.

Integrating human-provided location priors into video object segmentation has been shown to be an effective strategy to enhance performance, but their application at large scale is unfeasible. Gamification can help reduce the annotation burden, but it still requires user involvement. We propose a video object segmentation framework that leverages the combined advantages of user feedback for segmentation and gamification strategy by simulating multiple game players through a reinforcement learning (RL) model that reproduces human ability to pinpoint moving objects and using the simulated feedback to drive the decisions of a fully convolutional deep segmentation network. Experimental results on the DAVIS-17 benchmark show that: 1) including user-provided prior, even if not precise, yields high performance; 2) our RL agent replicates satisfactorily the same variability of humans in identifying spatiotemporal salient objects; and 3) employing artificially generated priors in an unsupervised video object segmentation model reaches state-of-the-art performance.

In the last decade, deep learning models have yielded impressive performance on visual object recognition and image classification. However these methods still rely on learning visual data distributions and show difficulties in dealing with complex scenarios where visual appearance only is not enough to effectively tackle them. This is the case, for instance, of fine-grained image classification in domain-specific applications for which it is very complex to employ data-driven models because of the lack of large amounts of samples and that, instead, can be solved by resorting to specialized human knowledge. However, encoding this specialized knowledge and injecting it into deep models is not trivial. In this paper, we address this problem by: a) employing computational ontologies to model specialized knowledge in a structured representation and, b) building a hybrid visual-semantic classification framework. The classification method performs inference over a Bayesian Network graph, whose structure depends on the knowledge encoded in an ontology and evidences are built using the outputs of deep networks. We test our approach on a fine-grained classification task, employing an extremely complex dataset containing images from several fruit varieties as well as visual and semantic annotations. Since the classification is done at the variety level (e.g., discriminating between different cherry varieties), appearance changes slightly and expert domain knowledge — making using of contextual information — is required to perform classification accurately. Experimental results show that our approach significantly outperforms standard deep learning–based classification methods over the considered scenario as well as existing methods leveraging semantic information for classification. These results demonstrate, on one hand, the difficulty of purely-visual deep methods in tackling small and highly-specialized datasets and, on the other hard, the capabilities of our approach to effectively encode and use semantic knowledge for enhanced accuracy.

Video Capsule Endoscopy (VCE) is a diagnostic imaging technology, based on capsule with a built-in camera, that enables screening of the gastro-intestinal tract by reducing the invasiveness of traditional endoscopy procedures. Despite VCE has been designed mainly for investigations on small intestine, it is a powerful tool, because of its low invasiveness and usage simplicity, for supporting large scale screening. However, each VCE video is typically long about eight hours, and endoscopists usually take about two hours, using simple computing methods, for its analysis, thus limiting its application for large scale studies. In this paper, we propose a novel computational framework leveraging the recent advances in artificial intelligence based on the deep learning paradigm to support effectively the whole screening procedure from video transmission to automated lesion identification to reporting. More specifically, our approach handles multiple video uploads at the same time, processes them automatically with the objective of identifying key video frames with potential lesions (for subsequent analysis by endoscopists) and provides physicians with means to compare the findings with either previously detected lesions or with images and scientific information from relevant retrieved documents for a more accurate final diagnosis.

Label propagation by means of nearest-neighbor search and "mutual local similarity".Effectiveness and efficiency of quantized HoG for large scale image retrieval.Example application to low-quality underwater images and fish classification. The shift from model-based approaches to data-driven ones is opening new frontiers in computer vision. Several tasks which required the development of sophisticated parametric models can now be solved through simple algorithms, by offloading the complexity of the task to the amount of available data. However, in order to develop data-driven approaches, it is necessary to have large annotated datasets. Unfortunately, manual labeling of large scale datasets is a complex, error prone and tedious task, especially when dealing with noisy images or with fine-grained visual tasks.In this paper we present an automatic label propagation approach that transfers labels from a small set of manually labeled images to a large set of unlabeled items by means of nearest-neighbor search operating on HoG image descriptors. In particular, we introduce the concept of mutual local similarity between the labeled query image and its nearest neighbors as the condition to be verified for propagating labels.The performance evaluation, carried out on the COREL 5K dataset and on a dataset of 20 million underwater low-quality images, showed how big data combined to simple nonparametric approaches allows to solve effectively complex visual tasks.

Analysis of medical videos for detection of abnormalities and diseases requires both high precision and recall, but also real-time processing for live feedback and scalability for massive screening of entire populations. Existing work on this field does not provide the necessary combination of retrieval accuracy and performance.; AB@In this paper, a multimedia system is presented where the aim is to tackle automatic analysis of videos from the human gastrointestinal (GI) tract. The system includes the whole pipeline from data collection, processing and analysis, to visualization. The system combines filters using machine learning, image recognition and extraction of global and local image features. Furthermore, it is built in a modular way so that it can easily be extended. At the same time, it is developed for efficient processing in order to provide real-time feedback to the doctors. Our experimental evaluation proves that our system has detection and localisation accuracy at least as good as existing systems for polyp detection, it is capable of detecting a wider range of diseases, it can analyze video in real-time, and it has a low resource consumption for scalability.

The importance of having high quality ground truth annotations for a variety of multimedia applications is widely recognised. Indeed, one of the most time-consuming steps in methods’ development is represented by the generation of accurate truth and comparing this truth to the output of applications to provide evidence that the devised methods are performing well in the targeted domain. However, the cost of creating labeled data, which implies asking a human to examine multimedia data thoroughly and provide labels, becomes impractical as datasets to be labeled grow. This can lead to the creation of disparate datasets which are often too small for both learning and evaluating the underlining data distribution. To build up large scale datasets, recently, methods exploiting the collaborative effort of a large population of users annotators (e.g. Labelme, CalTech, Pascal VOC, Trecvid) have been devised. Nevertheless, the creation of a common and large scale ground truth data to train, test and evaluate algorithms for multimedia processing is still a major concern. In particular, the research in ground truth labelling still lacks both in developing user-oriented tools and in automatic methods for supporting annotators in accomplishing their labelling tasks. In fact, tools for ground truth annotation must be user-oriented, providing visual interfaces and methods that are able to guide and speed-up the process of ground truth creation. Under this scenario, multimedia processing methods and collaborative methods play a crucial role. Further, setting up requirements and standards for the creation of multimedia dataset allows other researchers in the field to continue efforts and to contribute to the creation and annotation of multimedia data. This allows researchers to share and extend each others’ work, which is beneficial for the research community.

Large scale labeled datasets are of key importance for the development of automatic video analysis tools as they, from one hand, allow multi-class classifiers training and, from the other hand, support the algorithms' evaluation phase. This is widely recognized by the multimedia and computer vision communities, as witnessed by the growing number of available datasets; however, the research still lacks in annotation tools able to meet user needs, since a lot of human concentration is necessary to generate high quality ground truth data. Nevertheless, it is not feasible to collect large video ground truths, covering as much scenarios and object categories as possible, by exploiting only the effort of isolated research groups. In this paper we present a collaborative web-based platform for video ground truth annotation. It features an easy and intuitive user interface that allows plain video annotation and instant sharing/integration of the generated ground truths, in order to not only alleviate a large part of the effort and time needed, but also to increase the quality of the generated annotations. The tool has been on-line in the last four months and, at the current date, we have collected about 70,000 annotations. A comparative performance evaluation has also shown that our system outperforms existing state of the art methods in terms of annotation time, annotation quality and system's usability.

Health care has a long history of adopting technology to save lives and improve the quality of living. Visual information is frequently applied for disease detection and assessment, and the established fields of computer vision and medical imaging provide essential tools. It is, however, a misconception that disease detection and assessment are provided exclusively by these fields and that they provide the solution for all challenges. Integration and analysis of data from several sources, real-time processing, and the assessment of usefulness for end-users are core competences of the multime- dia community and are required for the successful improvement of health care systems. For the benefit of society, the multimedia community should recognize the challenges of the medical world that they are uniquely qualified to address. We have conducted initial investigations into two use cases surrounding diseases of the gastrointestinal (GI) tract, where the detection of abnormali- ties provides the largest chance of successful treatment if the initial observation of disease indicators occurs before the patient notices any symptoms. Although such detection is typically provided vi- sually by applying an endoscope, we are facing a multitude of new multimedia challenges that differ between use cases. In real-time assistance for colonoscopy, we combine sensor information about camera position and direction to aid in detecting, investigate means for providing support to doctors in unobtrusive ways, and assist in reporting. In the area of large-scale capsular endoscopy, we investigate questions of scalability, performance and energy efficiency for the recording phase, and combine video summarization and retrieval questions for analysis.

International audience; Using multimedia identification tools is considered as one of the most promising solutions to help bridge the taxonomic gap and build accurate knowledge of the identity, the geographic distribution and the evolution of living species. Large and structured communities of nature observers (e.g., iSpot, Xeno-canto, Tela Botanica, etc.) as well as big monitoring equipment have actually started to produce outstanding collections of multimedia records. Unfortunately, the performance of the state-of-the-art analysis techniques on such data is still not well understood and is far from reaching real world requirements. The LifeCLEF lab proposes to evaluate these challenges around 3 tasks related to multimedia information retrieval and fine-grained classification problems in 3 domains. Each task is based on large volumes of real-world data and the measured challenges are defined in collaboration with biologists and environmental stakeholders to reflect realistic usage scenarios. For each task, we report the methodology, the data sets as well as the results and the main outcome

Human annotated data is a prerequisite for the training and evaluation of computer vision algorithms. Such data is referred to as “ground truth” data. In this chapter, we describe the strategies and systems we have devised in order to obtain the ground truth data required by each of the computer vision components within the Fish4Knowledge system, including fish detection, tracking, and recognition.

In computer vision and machine learning, the availability of annotated datasets is of crucial importance for both learning and performance evaluation. However, annotating visual datasets is a tedious and error-prone task and computer vision researchers usually dedicate a large amount of their time for collecting and generating annotations, which most of the time cannot be re-used in other scenarios. In this paper, we propose a simple, but effective, interactive video object segmentation method exploiting large noisy data gathered from crowd of users while playing a web game. Experimental results, carried out on two challenging video benchmarks, show how it is possible to generate reliable object segmentations in videos with a small human effort, achieving an accuracy comparable to the one obtained with manually-labeled annotations and also outperforming state-of-the-art video object segmentation approaches.

This paper proposes a novel method for text mining on the Grid, aimed at pointing out hidden relationships for hypothesis generation and suitable for semi-interactive querying. The method is based on unsupervised clustering and the outputs are visualized with contextual information. Grid implementation is crucial for feasibility. We demonstrate it with a mining run for discovering genes-diseases associations from bibliographic sources and annotated databases. The proposed methodology is in view of a Grid architecture specialized in bioinformatics mining tasks. Some performance considerations are provided.

Objective: To describe the techniques used for automatic landmarking of cephalograms, highlighting the strengths and weaknesses of each one and reviewing the percentage of success in locating each cephalometric point. Materials and Methods: The literature survey was performed by searching the Medline, the Institute of Electrical and Electronics Engineers, and the ISI Web of Science Citation Index databases. The survey covered the period from January 1966 to August 2006. Abstracts that appeared to fulfill the initial selection criteria were selected by consensus. The original articles were then retrieved. Their references were also hand-searched for possible missing articles. The search strategy resulted in 118 articles of which eight met the inclusion criteria. Many articles were rejected for different reasons; among these, the most frequent was that results of accuracy for automatic landmark recognition were presented as a percentage of success. Results: A marked difference in results was found between the included studies consisting of heterogeneity in the performance of techniques to detect the same landmark. All in all, hybrid approaches detected cephalometric points with a higher accuracy in contrast to the results for the same points obtained by the model-based, image filtering plus knowledge-based landmark search and “soft-computing” approaches. Conclusions: The systems described in the literature are not accurate enough to allow their use for clinical purposes. Errors in landmark detection were greater than those expected with manual tracing and, therefore, the scientific evidence supporting the use of automatic landmarking is low.

Label propagation consists in annotating an unlabeled dataset starting from a set of labeled items. However, most current methods exploit only image similarity between labeled and unlabeled images in order to find propagation candidates, which may result, especially in very large datasets, in retrieving mostly near-duplicate images. While such approaches are technically correct, as they maximize the propagation precision, the resulting annotated dataset may not be as useful, since they lack intra-class variability within the set of images sharing the same label. In this paper, we propose an approach for label propagation which favors the propagation of an object's label to a set of images representing as many different views of that object as possible, while at the same time preserving the relevance of the retrieved items to the query. Our method is based on a diversity-based clustering technique using a random forest framework and a label propagation approach which is able to effectively and efficiently propagate annotations using a similarity-based approach operating on clusters. The method was tested on a very large dataset of fish images achieving good performance in automated label propagation, ensuring diversification of the annotated items while preserving precision.

HighlightsAutomated segmentation of X-ray images for skeletal bone age assessment.Hidden Markov models for bone development modelling.A software tool for supporting clinicians in the X-Ray investigation according to the TW2 clinical method. This paper presents a tool for automatic assessment of skeletal bone age according to a modified version of the Tanner and Whitehouse (TW2) clinical method. The tool is able to provide an accurate bone age assessment in the range 0-6 years by processing epiphysial/metaphysial ROIs with image-processing techniques, and assigning TW2 stage to each ROI by means of hidden Markov models.The system was evaluated on a set of 360 X-rays (180 for males and 180 for females) achieving a high success rate in bone age evaluation (mean error rate of 0.41?0.33 years comparable to human error) as well as outperforming other effective methods. The paper also describes the graphical user interface of the tool, which is also released, thus to support and speed up clinicians' practices when dealing with bone age assessment.

In this paper we present a system for automatic generation of summaries of patients' unstructured medical reports. The system employs Natural Language Processing techniques in order to determine the most interesting points and uses the MetaMap module for recognizing the medical concepts in a medical report. Afterwards the sentences that do not contain interesting concepts are removed and a summary is generated which contains URL links to the Linked Life Data pages of the identified medical concepts, enabling both medical doctors and patients to further explore what is reported in. Such integration also allows the tool to interface with other semantic web-based applications. The performance of the tool were also evaluated, achieving remarkable results in sentence identification, polarity detection and concept recognition. Moreover, the accuracy of the generated summaries was evaluated by five medical doctors, proving that the summaries keep the same relevant information as the medical reports, despite being much more concise.

International audience; Using multimedia identification tools is considered as one of the most promising solutions to help bridging the taxonomic gap and build accurate knowledge of the identity, the geographic distribution and the evolution of living species. Large and structured communities of nature observers (e.g. eBird, Xeno-canto, Tela Botanica, etc.) as well as big monitoring equipments have actually started to produce outstanding collections of multimedia records. Unfortunately, the performance of the state-of-the-art analysis techniques on such data is still not well understood and is far from reaching the real world's requirements. The LifeCLEF lab proposes to evaluate these challenges around 3 tasks related to multimedia information retrieval and fine-grained classification problems in 3 living worlds. Each task is based on large and real-world data and the measured challenges are defined in collaboration with biologists and environmental stakeholders in order to reflect realistic usage scenarios. This paper presents more particularly the 2015 edition of Life-CLEF. For each of the three tasks, we report the methodology and the data sets as well as the raw results and the main outcomes.

Chapitre : Information Access Evaluation. Multilinguality, Multimodality, and Interaction; International audience; Using multimedia identification tools is considered as one of the most promising solutions to help bridging the taxonomic gap and build accurate knowledge of the identity, the geographic distribution and the evolution of living species. Large and structured communities of nature observers (e.g. eBird, Xeno-canto, Tela Botanica, etc.) as well as big monitoring equipments have actually started to produce outstanding collections of multimedia records. Unfortunately, the performance of the state-of-the-art analysis techniques on such data is still not well understood and is far from reaching the real world’s requirements. The LifeCLEF lab proposes to evaluate these challenges around three tasks related to multimedia information retrieval and fine-grained classification problems in three living worlds. Each task is based on large and real-world data and the measured challenges are defined in collaboration with biologists and environmental stakeholders in order to reflect realistic usage scenarios. This paper presents more particularly the 2014 edition of LifeCLEF, i.e. the pilot one. For each of the three tasks, we report the methodology and the datasets as well as the official results and the main outcomes.

Ecology is, nowadays, an interdisciplinary, collabo- rative and data-intensive science, therefore, discovering, integrat- ing and analysing daily-produced data is necessary to support researchers to investigate complex questions, ranging from single particles to animals to the biosphere [1]. As a consequence, ecology-related multimedia content has been produced massively in recent years: for example, the Xeno-canto project1 and the Pl@ntNet project2 respectively collected 140,000 audio records of 8,700 bird species and about 60,000 thousand images covering thousand of plant species, to be used by scientists or professionals. Unfortunately, a manual analysis of such amount of generated data is impossible: automatic analysis tools combined with high- performance computing (HPC) solutions are therefore heavily demanded for making sense of such big ecological data. In this paper we present a case study of large-scale video processing on HPC facilities for underwater fish monitoring in the context of the Fish4Knowledge project 3, where a system to analyse long-term underwater camera footage has been developed. The paper is meant to report on the employed hardware/software architecture, the design and deployment of the parallel job manager, and the problems encountered during the whole process, from load balancing to job submission policies to bottlenecks.

The study of fish populations in their own natural environment is a task that has usually been tackled in invasive ways which inevitably influenced the behavior of the fish under observation. Recent projects involving the installation of permanent underwater cameras (e.g. the Fish4Knowledge (F4K) project, for the observation of Taiwan’s coral reefs) allow to gather huge quantities of video data, without interfering with the observed environment, but at the same time require the development of automatic processing tools, since manual analysis would be impractical for such amounts of videos. Event detection is one of the most interesting aspects from the biologists’ point of view, since it allows the analysis of fish activity during particular events, such as typhoons. In order to achieve this goal, in this paper we present an automatic video analysis approach for fish behavior understanding during typhoon events. The first step of the proposed system, therefore, involves the detection of “typhoon” events and it is based on video texture analysis and on classification by means of Support Vector Machines (SVM). As part of our behavior understanding efforts, trajectory extraction and clustering have been performed to study the differences in behavior when disruptive events happen. The integration of event detection with fish behavior understanding surpasses the idea of simply detecting events by low-level features analysis, as it supports the full semantic comprehension of interesting events.

We present a research tool that supports marine ecologists' research by allowing analysis of long-term and continuous fish monitoring video content. The analysis can be used for instance to discover ecological phenomena such as changes in fish abundance and species composition over time and area. Two characteristics set our system apart from traditional ecological data collecting and processing methods. First, the continuous video recording results in enormous data volumes of monitoring data. Currently around a year of video recordings (containing over the 4 million fish observations) have been processed. Second, different from traditional manual recording and analysing the ecological data, the whole recording, analysing and presentation of results is automated in this system. On one hand, it saves the effort of manually examining every video, which is infeasible. On the other hand, no automatic video analysis method is perfect, so the user interface provides marine ecologists with multiple options to verify the data. Marine ecologists can examine the underlying videos, check results of automatic video analysis at different certainty levels computed by our system, and compare results generated by multiple versions of automatic video analysis software to verify the data in our system. This research tool enables marine ecologists for the first time to analyse long-term and continuous underwater video records.

Understanding and analyzing fish behaviour is a fundamental task for biologists that study marine ecosystems because the changes in animal behaviour reflect environmental conditions such as pollution and climate change. To support investigators in addressing these complex questions, underwater cameras have been recently used. They can continuously monitor marine life while having almost no influence on the environment under observation, which is not the case with observations made by divers for instance. However, the huge quantity of recorded data make the manual video analysis practically impossible. Thus machine vision approaches are needed to distill the information to be investigated. In this paper, we propose an automatic event detection system able to identify solitary and pairing behaviours of the most common fish species of the Taiwanese coral reef. More specifically, the proposed system employs robust low-level processing modules for fish detection, tracking and recognition that extract the raw data used in the event detection process. Then each fish trajectory is modeled and classified using hidden Markov models. The events of interest are detected by integrating end-user rules, specified through an ad hoc user interface, and the analysis of fish trajectories. The system was tested on 499 events of interest, divided into solitary and pairing events for each fish species. It achieved an average accuracy of 0.105, expressed in terms of normalized detection cost. The obtained results are promising, especially given the difficulties occurring in underwater environments. And moreover, it allows marine biologists to speed up the behaviour analysis process, and to reliably carry on their investigations.

The 3rd ACM International Workshop on Multimedia Anal- ysis for Ecological Data (MAED'14) is held as part of ACM Multimedia 2014.MAED'14, following the previous two workshops of the MAED series (MAED'12 and MAED'13) held as part of, respec- tively, ACM Multimedia 2012 and ACM Multimedia 2013, is concerned with the processing, interpretation, and visuali- sation of ecology-related multimedia content with the aim to support biologists in their investigations for analysing and monitoring natural environments.

Current Location Based Services (LBS) don’t take into account the status of the services neither the real time traffic conditions. Aim of the paper is to show how LBSs may evolve towards location intelligence services based on Horn clauses expressed in Prolog to find the most suitable services for the users and the relevant paths to reach them by taking into account the current traffic situation. Fuzzy logic and semantic web technologies are also used to improve the LBS applications. The former to find the services that meet the user expectations, the latter to take advantage from all the information stored on the distributed databases at urban/metropolitan scale. A case study developed using Ruby on Rails (RoR) and JQueryMobile illustrates how such a web service may work in practice. A comparison between the RoR version and another one where mobiles play a more active role is also discussed.

Many Location Based Services (LBSs) have been proposed, but they are developed mainly with a proprietary approach, i.e., they are specialized for managing specific tasks and are not able to make available the collected information to the services offered by other providers, thus limiting the potential benefits for the users. For this reason LBSs should be extended with a semantic layer able to provide the mobile users with the real time and off line information coming from the multitude of databases relevant for LBS activities, hopefully by using the same mobile device. Aim of the paper is to show how this can be accomplished in practice by integrating three methodologies in a single web architecture: a) a scenario based design methodology to implement the use stories of the mobile users, b) a semantic data management methodology to allow data integration, and c) a suitable JQuery Mobile based interface to allow the RoR web server to inform people on the available services independently on the adopted mobile. Keywords-component; Mobile Networked Applications; Intelligent Transport System; Scenario-based Design, Web

Visual tracking is a topic on which a lot of scientific work has been carried out in the last years. An important aspect of tracking algorithms is the performance evaluation, which has been carried out typically through hand-labeled ground-truth data. Since the manual generation of ground truth is a time-consuming, error-prone and tedious task, recently many researchers have focused their attention on self-evaluation techniques for performance analysis. In this paper we propose a novel tool that enables image processing researchers to test the performance of tracking algorithms without resorting to hand-labeled ground truth data. The proposed approach consists of computing a set of features describing shape, appearance and motion of the tracked objects and combining them through a naive Bayesian classifier, in order to obtain a probability score representing the overall evaluation of each tracking decision. The method was tested on three different targets (vehicles, humans and fish) with three different tracking algorithms and the results show how this approach is able to reflect the quality of the performed tracking.

The ACM International Workshop on Multimedia Analysis for Ecological Data (MAED'12) is held as part of ACM Multimedia 2012. MAED'12 is concerned with the processing, interpretation, and visualization of ecology-related multimedia content with the aim to support biologists in their investigations for analyzing and monitoring natural environments, with particular attention to living organisms and pollution effects.

In this paper, we present BioWizard, a bioinformatics knowledge discovery tool for extracting and validating implicit associations between biological entities. By mining specialized scientific literature, BioWizard not only generates biological hypotheses in the form of associations between genes, proteins and diseases, but also validates the plausibility of such associations against high-throughput biological data (microarrays) and annotated databases. The main novelties of the proposed approach are that: 1) it infers associations between biological entities by mining full text papers instead of only abstracts as usually performed by the existing tools, 2) a named entity recognition that improves the precision of the derived associations by enriching the vocabularies used in the mining loop with terms extracted directly from the text and, 3) the inferred associations are filtered according to their evidence in experimental data. We tested the precision and the recall of our system in retrieving known-associations (which did not appear in the same document) from gold standards and the results shown the ability of BioWizard in retrieving valid associations, thus providing a valuable tool for the use of biomedical researchers to speed up scientific progress.

As known, the new generation of Intelligent Transportation Systems (ITSs) is evolving towards a geo-referenced semantic system, denoted in the paper as ITS 3.0, highly permeated by the Web 3.0 technology. More explicitly, ITS 3.0 is a set of databases dealing with geospatial, administrative and business information to be interconnected by a standard data scheme to allow mobile users to access the relevant ITS information through a suitable geo-referenced semantic interface. Although many models have been proposed for semantically structuring the main location based services by means of conceptual taxonomies or entity relations, few real ITSs work in this way. Thus, aim of the paper is to discuss how effective semantic ITSs can be implemented in practice by three technologies: the first devoted to implement user centered processes (Ruby on Rails), the second aiming at integrating the disparate data sources featuring the ITS applications by a suitable data ontology (Protege), the third to provide the user processes with the needed data (Sesame). Both theoretical considerations and practical issues will be taken into account to build safe and live ITS applications that meet the user practices.

In this paper we present a domain-independent multimedia retrieval (MMR) platform. Currently, the use of MMR systems for different domains poses several limitations, mainly related to the poor flexibility and adaptability to different domains and user requirements. A semantic-based platform that uses ontologies for describing not only the application domain but also the processing workflow to be followed for the retrieval, according to user's requirements and domain characteristics is here proposed. In detail, an ontological model (domain-processing ontology) that integrates domain peculiarities and processing algorithms allows self-adaptation of the retrieval mechanism to the specified application domain. According to the instances generated for each user request, our platform generates the appropriate interface (GUI) for the specified application domain (e.g. music, sport video, medical images, etc…) by a procedure guided by the defined domain-processing ontology. A use case on content based music retrieval is here presented in order to show how the proposed platform also facilitates the process of multimedia retrieval system implementation.

The paper outlines a methodology and presents a tool to help biomedical researchers in interpreting complex experiments by automatically discovering gene networks and underlying biological processes (revealed by gene-expression patterns) that usually are extracted manually using existing tools. The proposed method, first, starts by mining specialized medical literature available on the Web to discover possible associations between genes and diseases. Discovered gene-disease associations are subsequently explored by analyzing abnormally expressed genes using microarray data analysis. Afterwards, relevant gene networks are built by clustering these genes on the basis of the similarity of their profile expressions in microarrays data. Finally, molecular, biological processes, cellular components and molecular functions, which may have a role in the disease, are pointed out by querying the Gene Ontology (GO) database. The methodology is illustrated by a case study on neuromuscular disorders.

In this paper we propose a novel telemedicine system capable of medical data acquisition, transmission, integration and analysis. In detail, the proposed system consists of: 1) a wearable acquisition system for ECG (electrocardiogram), EEG (electroencephalogram), body temperature and blood pressure, 2) a storage/analysis center where the acquired medical data are integrated into a Personal Health Record (PHR) system that also provides data analysis tools that can assist medical doctors in their diagnosis, and 3) a TV-centric set-top-box that manages both patient-medical doctor communication and interactive services, such as customized TV channels promoting individual adoption of healthy lifestyles and self-care. The proposed system uses standard semantic web technologies, such as RDF, and common ontologies, such as FOAF and CCR, for data description and integration. This makes the system a “Medicine 2.0” application, interoperable with existing medical data storage/analysis systems.

Computational grids have become an imperative rising platform for high-performance computing. However, the grid and the grid applications development are still far from being affirmed, which is mainly due to the undeveloped grid-enabled computing environments. For that reason in this paper we propose a toolbox, called GrIPLab 1.0 (Grid Image Processing Laboratory), that aims at providing high performance image-processing platform in a grid computing environment by using the GLite middleware developed in the EGEE project. GrIPLab 1.0 is a combination of vision algorithms (the most common and some novel approaches) on which complex distributed vision applications can be modeled as a simple sequence of choices in a user friendly interface. Therefore, the main advantage of the presented dynamic grid toolbox is that provides a novel and comfortable access for scientific software developers and users without prior knowledge of grid technologies or even the underlying architecture. In this paper, we discuss the infrastructure that provides flexible and useful mechanism to achieve series image processing operations and we analyze the advantages of using such a system.

This paper proposes a traffic monitoring architecture based on a high-speed communication network whose nodes are equipped with fuzzy processors and cellular neural network (CNN) embedded systems. It implements a real-time mobility information system where visual human perceptions sent by people working on the territory and video-sequences of traffic taken from webcams are jointly processed to evaluate the fundamental traffic parameters for every street of a metropolitan area. This paper presents the whole methodology for data collection and analysis and compares the accuracy and the processing time of the proposed soft computing techniques with other existing algorithms. Moreover, this paper discusses when and why it is recommended to fuse the visual perceptions of the traffic with the automated measurements taken from the webcams to compute the maximum traveling time that is likely needed to reach any destination in the traffic network.

The aim of this paper is to present a novel distributed genetic algorithm architecture implemented on grid computing by using the G-Lite middleware developed in the EGEE project. Genetic algorithms are known for their capability to solve a wide range of optimization problems and one of the most relevant feature of GAs is their structural parallelism, that fits well the intrinsically distributed GRID architecture. The proposed architecture is based on different specialized autonomous entities able to interact in order to carry out a global optimization task. The interaction is based on exchange of knowledge on the problem and solutions. In this way the main problem can be solved by using many cooperative small entities that can be classified into different specialized families that cover only one aspect of the global problem. The topology is based on archipelagos of islands that interact by chromosomes migrating with an user-definable strategy. Grid has been mainly used in the high performance computing area. The properties of the proposed GAs architecture and its related computing properties have great potential in solving big instances of optimization problems. Furthermore this implementation (distributed genetic algorithms with grid computing) is suitable to solve time consuming problems reducing by executing different instances on many virtual organizations (VOs) according to the Grid philosophy. The proposed parallel algorithm has been tested on denoising problems applied to image processing which are known to be time consuming. The paper reports some results about the time performance compared to traditional denoising filter algorithms.

In this paper we introduce a novel multi-view annotation tool for generating 3D ground truth data of the real location of people in the scene. The proposed tool allows the user to accurately select the ground occupancy of people by aligning an oriented rectangle on the ground plane. In addition, the height of the people can also be adjusted. In order to achieve precise ground truth data the user is aided by the video frames of multiple synchronized and calibrated cameras. Finally, the 3D annotation data can be easily converted to 2D image positions using the available calibration matrices. One key advantage of the proposed technique is that different methods can be compared against each other, whether they estimate the real world ground position of people or the 2D position on the camera images. Therefore, we defined two different error metrics, which quantitatively evaluate the estimated positions. We used the proposed tool to annotate two publicly available datasets, and evaluated the metrics on two state of the art algorithms.