Search

Using open data and digital footprint analysis to identify the most vulnerable communities in Wales

Reproducibility of AI models on biomedical data still stays as a major concern for their acceptance into the clinical practice. Initiatives for reproducibility in the development of predictive biomarkers as the MAQC Consortium already underlined the importance of appropriate Data Analysis Plans (DAPs) to control for different types of bias, including data leakage from the training to the test set. In the context of digital pathology, the leakage typically lurks in weakly designed experiments not accounting for the subjects in their data partitioning schemes. This issue is then exacerbated when fractions or subregions of slides (i.e. "tiles") are considered. Despite this aspect is largely recognized by the community, we argue that it is often overlooked. In this study, we assess the impact of data leakage on the performance of machine learning models trained and validated on multiple histology data collection. We prove that, even with a properly designed DAP (10x5 repeated cross-validation), predictive scores can be inflated up to 41% when tiles from the same subject are used both in training and validation sets by deep learning models. We replicate the experiments for $4$ classification tasks on 3 histopathological datasets, for a total of 374 subjects, 556 slides and more than 27,000 tiles. Also, we discuss the effects of data leakage on transfer learning strategies with models pre-trained on general-purpose datasets or off-task digital pathology collections. Finally, we propose a solution that automates the creation of leakage-free deep learning pipelines for digital pathology based on histolab, a novel Python package for histology data preprocessing. We validate the solution on two public datasets (TCGA and GTEx).

Nowcasting (very short-term forecasting) in meteorology is a very important topic for agriculture, human safety, and renewable energy production. Recently, a precipitation nowcasting method has been proposed that achieves state of the art results using deep learning techniques. In this work, we present the application of the method on a novel dataset acquired from echo patterns of weather radar at the regional scale in Trentino-Suedtirol, in the Italian Alps. The model can forecast rainfall up to 75’ ahead at a spatial resolution of 1.65km based on 5 frames of recent (25’) radar data. Further, we show that the model can manage blocking effects due to orography. We also introduce an evolution of the method that is applied to lighting prediction of the same area using a dataset of 95.7K lightning strikes gathered from a collaborative lightning location network. This evolved model can forecast lightning strikes up to 30’ ahead at a spatial resolution of 6.6Km. The performances of the models are assessed by CSI (Critical success index), FAR (False alarm rate), POD (Probability of detection) and Correlation measures commonly used in meteorology, achieving state of the art results. We describe a deep learning framework for precipitation and lightning nowcasting, applied to weather echo radar and lightning data at a regional scale in Trentino-Suedtirol, in the Italian Alps. Nowcasting, i.e. forecasts obtained by extrapolation for a period of 0 to 6 hours ahead (World Meteorological Organization) relies here on a Conv-LSTM model [1] and it is embedded in an operational context. The model is able to forecast reflectivity up to 75’ ahead at a spatial resolution of 1.65km based on 5 frames of recent (25’) radar data. Further, the model can manage blocking effects due to orography. The framework is then applied to 95.7K events from a collaborative lightning location network in the same region.&nbsp

Artificial Intelligence is exponentially increasing its impact on healthcare. As deep learning is mastering computer vision tasks, its application to digital pathology is natural, with the promise of aiding in routine reporting and standardizing results across trials. Deep learning features inferred from digital pathology scans can improve validity and robustness of current clinico-pathological features, up to identifying novel histological patterns, e.g. from tumor infiltrating lymphocytes. In this study, we examine the issue of evaluating accuracy of predictive models from deep learning features in digital pathology, as an hallmark of reproducibility. We introduce the DAPPER framework for validation based on a rigorous Data Analysis Plan derived from the FDA’s MAQC project, designed to analyse causes of variability in predictive biomarkers. We apply the framework on models that identify tissue of origin on 787 Whole Slide Images from the Genotype-Tissue Expression (GTEx) project. We test 3 different deep learning architectures (VGG, ResNet, Inception) as feature extractors and three classifiers (a fully connected multilayer, Support Vector Machine and Random Forests) and work with 4 datasets (5, 10, 20 or 30 classes), for a total 53000 tiles at 512 × 512 resolution. We analyze accuracy and feature stability of the machine learning classifiers, also demonstrating the need for random features and random labels diagnostic tests to identify selection bias and risks for reproducibility. Further, we use the deep features from the VGG model from GTEx on the KIMIA24 dataset for identification of slide of origin (24 classes) to train a classifier on 1060 annotated tiles and validated on 265 unseen ones. The DAPPER software, including its deep learning backbone pipeline and the HINT (Histological Imaging - Newsy Tiles) benchmark dataset derived from GTEx, is released as a basis for standardization and validation initiatives in AI for Digital Pathology.Author summaryIn this study, we examine the issue of evaluating accuracy of predictive models from deep learning features in digital pathology, as an hallmark of reproducibility. It is indeed a top priority that reproducibility-by-design gets adopted as standard practice in building and validating AI methods in the healthcare domain. Here we introduce DAPPER, a first framework to evaluate deep features and classifiers in digital pathology, based on a rigorous data analysis plan originally developed in the FDA’s MAQC initiative for predictive biomarkers from massive omics data. We apply DAPPER on models trained to identify tissue of origin from the HINT benchmark dataset of 53000 tiles from 787 Whole Slide Images in the Genotype-Tissue Expression (GTEx) project. We analyze accuracy and feature stability of different deep learning architectures (VGG, ResNet and Inception) as feature extractors and classifiers (a fully connected multilayer, SVMs and Random Forests) on up to 20 classes. Further, we use the deep features from the VGG model (trained on HINT) on the 1300 annotated tiles of the KIMIA24 dataset for identification of slide of origin (24 classes). The DAPPER software is available together with the HINT benchmark dataset.

The annual conference CLIC–it (''Italian Conference on Computational Linguistics'') is an initiative of the ''Italian Association of Computational Linguistics'' (AILC – www.ai-lc.it) which is intended to meet the need for a national and international forum for the promotion and dissemination of high-level original research in the field of Computational Linguistics (CL), with particular emphasis on Italian. The volume gathers the Proceedings of the ''Third Italian Conference on Computational Linguistics'' (CLiC–it 2016), held in Naples on 5-6 December 2016. The CLiC–it 2016 papers cover a wide range of topics in the area of computational linguistics and natural language (both written and spoken) processing, by targeting state–of–art theoretical results, experimental methodologies, technologies and application perspectives, and by addressing challenges, open issues and new perspectives related to current and novel trends of the discipline.

Software maintenance is an essential step in the evolution of software systems and represents one of the most expensive, time consuming, and challenging phases of the whole development process. In particular, the cost and the effort necessary for both the maintenance and the evolution operations (e.g., corrective, adaptive, etc.) are mainly related to the effort necessary to comprehend the system and its source code. As a consequence many "reverse engineering" tools and solutions have been proposed to support the maintainers in their activities. An important resource for maintainers is represented by the architectural information of the system. However such information is usually not documented, or the documentation is outdated. Therefore, the existing code remains the most updated source of information to exploit in order to automatically retrieve and reconstruct the architecture of a system. Many research efforts are being devoted to support this task, in order to define solutions that are able to "re-modularise" a given software application. The main purpose of re-modularisation techniques is to automatically partition the system into meaningful subsystems, in order to locate and group together software components that are in some way related, e.g., they implement the same functionalities. A number of these approaches generally attempt to discover these groups (or clusters) by exploiting the lexical information provided in the source code, such as terms in comments, as well as names of identifiers (e.g., variable, methods and classes). Nevertheless, the source code lexicon has some specific peculiarities that make it conceptually different from a typical textual resource: identifiers are often created by concatenating multiple words (e.g. getAttribute, MINHEIGHT), which may be additionally shortened (e.g., getAttr, MINHGT) to avoid long names. As a consequence, tools and techniques that analyse the source code lexicon must integrate algorithms to "normalise" its vocabulary. Another well known and largely investigated issue in software maintenance is "clone detection": it is focused on the identification of source code duplications. Software clones might affect the reliability and the maintainability of large software systems. For example, errors affecting a fragment of code must be fixed in everyone of its possible duplications. Clones are usually not documented, and their identification is usually complicated since programmers adapt software copies by applying multiple modifications (e.g., adding new statements and renaming variables). Therefore, automatic and reliable approaches are required in order to tackle this problem. In this thesis we proposed new Machine Learning (ML) based approaches that mine the relevant information directly from the source code to cope with the three introduced issues, namely the software re-modularisation, the source code vocabulary normalisation, and the clone detection. In particular, proposed contributions leverages the benefits of ML algorithms, which have been properly tailored and customised in order to make them suitable for the considered domain. All the presented approaches have been extensively assessed with empirical evaluations conducted on large software systems, and results have been compared with other related techniques, whenever possible. Achieved results outperform the state-of-the-art solutions for all the three considered problems, thus confirming the benefits derived from the definition and the application of ML algorithms to maintenance tasks.

In this paper, we present the results of a manual assessment on the coherence between the comments and the implementation of 3636 methods in three open source soft- ware applications (for one of these applications, we considered two different subsequent versions) implemented in Java. The results of this assessment have been collected in a dataset we made publicly available on the Web. The creation of this dataset is based on a protocol that is detailed in this paper. We present that protocol to let researchers evaluate the goodness of our dataset and to ease its future possible extensions. Another contribution of this paper consists in preliminarily investigating on the effectiveness of adopting a Vec- tor Space Model (VSM) with the tf-idf schema to discriminate coherent and non-coherent methods. We observed that the lexical similarity alone is not sufficient for this distinc- tion, while encouraging results have been obtained by applying an Support Vector Machine (SVM) classifier on the whole vector space.

In literature some approaches have been proposed to partition software systems into meaningful subsystems exploiting the lexical information provided by programmers into the source code. However these techniques usually do not consider the programming language constructs in which the lexicon appears (e.g.: comments, class names, method names) even if it is a common experience that programmers place di erent care in choosing terms for different constructs. In this paper we present a novel lexical-based software clustering technique which exploits the contribution of terms placed in six different parts of the source code (i.e. zones), namely Class Names, Attribute Names, Method Names, Parameter Names, Comments and Source Code Statements. These zones convey information with different levels of relevance, and so their contribution should be di erently weighted according to the specificities of the analyzed software system. To this aim we dene a probabilistic model of the data whose parameters are estimated automatically by the Expectation-Maximization algorithm. These weights are then exploited to generate the software partitions with two distinct clustering algorithms properly customized to make them more suitable for the specic domain. The overall technique has been assessed in a case study conducted on a dataset of 16 open source software systems whose results are presented in the paper. In particular, we experimentally observed that the use of both the dened zones and the Expectation-Maximization algorithm improves the overall quality of results.

Source code comments provide useful information on the implementation of a software and on the intent behind design decisions and goals. Writing informative and useful comments is far from being a trivial task. Moreover, source code comments tend to remain mostly unchanged during maintenance activities. As a consequence, the information provided in the comment of a method and in its corresponding implementation may be not coherent with each other (i.e., The comment does not properly describe the implementation). In this paper, we present the results of a manual assessment on the coherence between comments and implementations of 3636 methods, gathered from 3 Java open source software systems (for one of these systems, we considered 2 different subsequent versions). Resulting evaluations have been collected in a dataset, we made publicly available on the web. The defined protocol used for the creation of this dataset is also described. This lets researchers evaluate the goodness of our dataset and eases its future possible extensions. Another contribution of our paper consists in investigating on a possible link between coherence and lexical similarity between source code and comments. Our preliminary outcomes suggest that this similarity is higher in case the comment of methods and their implementations are coherent. However, the obtained similarity values are generally low and are not much higher than those for non-coherent method implementations and comments.

In this paper, we investigate some ideas based on Machine Learning, Natural Language Processing, and Information Retrieval to outline possible research directions in the field of software architecture recovery and clone detection. In particular, after presenting an extensive related work, we illustrate two proposals for addressing these two issues, that represent hot topics in the field of Software Maintenance. Both proposals use Kernel Methods for exploiting structural representation of source code and to automate the detection of clones and the recovery of the actually implemented architecture in a subject software system.

Search-Based Software Testing is a well-established research area, whose goal is to apply meta-heuristic approaches, like Genetic Algorithms, to address optimization problems in the testing domain. Even if many interesting results have been achieved in this field, the heavy computational resources required by these approaches are limiting their practical application in the industrial domain. In this chapter, the authors propose the migration of Search-Based Software Testing techniques to the Cloud aiming to improve their performance and scalability. Moreover, they show how the use of the MapReduce paradigm can support the parallelization of Genetic Algorithms for test data generation and their migration in the Cloud, thus relieving software company from the management and maintenance of the overall IT infrastructure and developers from handling the communication and synchronization of parallel tasks. Some preliminary results are reported, gathered by a proof-of-concept developed on the Google’s Cloud Infrastructure.

Information Retrieval (IR) techniques are being exploited by an increasing number of tools suited to support Software Maintenance activities. This is because the lexical information embedded in the source code by programmers can be valuable for tasks such as concept location, clustering or recovery of traceability links. However, the application of such IR-based techniques relies on the consistency of lexicon available in the different artifacts, and their effectiveness can worsen if programmers introduce abbreviations (e.g: rect) and/or did not strictly follow naming conventions such as Camel Case (e.g: UTFtoASCII). In this paper we propose an approach useful for all of these IR-based tools, suited to automatically split identifiers in their composing words, and expand abbreviations. The solution is able to perform in linear time, taking advantage of an approximate pattern matching technique applied in a graph-based model. Linear complexity allows exploiting a number of different dictionaries, referring to increasingly broader contexts, in order to achieve a disambiguation strategy based on the knowledge gathered from the most appropriate domain. The proposed approach has been compared to other splitting and expansion techniques, using freely available oracles for the identifiers extracted from a number of C/C++ and Java open source systems. Results show an improvement in both splitting and expanding performance, in addition to a strong enhancement in the computational efficiency.

In the field of Software Maintenance the definition of effective approaches to partition a software system into meaningful subsystems is a longstanding and relevant research topic. These techniques are very important as they can significantly support a Maintainer in his/her tasks by grouping related entities of a large system into smaller and easier to comprehend subsystems. In this paper we investigate the effectiveness of combining information retrieval and machine learning techniques in order to exploit the lexical information provided by programmers for software clustering. In particular, differently from any related work, we employ indexing techniques to explore the contribution of the combined use of six different dictionaries, corresponding to the six parts of the source code where programmers introduce lexical information, namely: class, attribute, method and parameter names, comments, and source code statements. Moreover their relevance is estimated on the basis of the project characteristics, by applying a machine learning approach based on a probabilistic model and on the Expectation-Maximization algorithm. To group source files accordingly, two clustering algorithms have been compared, i.e. the K-Medoids and the Group Average Agglomerative Clustering, and the investigation has been conducted on a dataset of 9 open source Java software systems.