Search

The reuse and recycling of critical raw materials is limited, as waste electrical and electronic recycling focuses on base and precious metals, and device component reuse is in its infancy. To help to address this issue this paper provides the conceptual design of a Digital Product Passport based circular supply management system. To enable the recovery of critical raw materials at component and material levels for reuse and recycling. The works include an assessment of existing critical raw materials information management and an information needs identification survey, with 10 manufacturers, producer responsibility organisations, collectors and recyclers. The needs were used to generate 14 key product information management processes and exchanges that when implemented form a Digital Product Passport based circular supply management system. Information managed via a physical-digital linkage through individual product tags includes product registrations, materials declarations, life cycle status updates, the sorting of products at collection points based on critical raw material contents, and flagging of products for critical raw materials component extraction. A dataspace-based IT systems architecture is proposed for the implementation of the supply management system taking into account global and European information standards. Finally, key challenges to implement such an IT architecture are discussed.

The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. The experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.

While in-situ workflow formulations have addressed some of the data-related challenges associated with extreme-scale scientific workflows, these workflows involve complex interactions and different modes of data exchange. In the context of increasing system complexity, such workflows present significant resource management challenges, requiring complex cost-performance tradeoffs. This paper presents RISE, an intelligent staging-based data management middleware, which builds on the DataSpaces framework and performs intelligent scheduling of data management operations to reduce I/O contention. In RISE, data are always written immediately to local buffers to reduce the effect of the transfer impact upon application performance. RISE identifies applications’ data access patterns and moves data towards data consumers only when the network is expected to be idle, reducing the impact of asynchronous background data movement upon critical data read/write requests. We experimentally demonstrate that RISE can take advantage of staging nodes to offload data during writes without degrading application data movement performance.

Οι τεχνολογικές εξελίξεις κατά τη διάρκεια των τελευταίων χρόνων έχουν οδηγήσει στην αλλαγή στον τρόπο με τον οποίο οι άνθρωποι χρησιμοποιούν και να αλληλεπιδρούν με το διαδίκτυο. Οι νέες τεχνολογίες ιστού καθώς και οι εφαρμογές που έχουν προκύψει, οδηγούν σε ένα οικοσύστημα το οποίο όταν χρησιμοποιηθεί σωστά, θα οδηγεί σε χρήσιμες στον άνθρωπο υπηρεσίες. Η χρήση των κινητών τηλεφώνων για την κοινωνική αλληλεπίδραση, οδηγεί στην παραγωγή τεράστιων ποσοτήτων δεδομένων. Αυτό, σε συνδυασμό με την ικανότητα των κινητών τηλεφώνων να ανιχνεύουν τη γεωγραφική τους θέση, οδηγεί στην αναζήτηση πιο προηγμένων υπηρεσιών που λαμβάνουν υπόψη όχι μόνο τη γεωγραφική θέση αλλά και το σύνολο των δεδομένων που δημιουργούνται από τους χρήστες. Σε αυτή την διδακτορική διατριβή, μελετούμε την περίπτωση αυτού του είδους των Υπηρεσιών Θέσης και συγκεκριμένα το πώς μπορούν να βελτιωθούν τα τυπικά ερωτήματα υπηρεσιών θέσης με περιεχόμενο που προέρχεται από πηγές στο διαδίκτυο μέσω των χρηστών, με τη μορφή των κειμένων. Προς την κατεύθυνση αυτή έχουμε εστιάσει την έρευνά μας σε τρεις κύριες κατευθύνσεις: (α) Πρόταση μεθόδων ευρετηρίασης που επωφελούνται από τις πρόσθετες πληροφορίες μέσω του περιεχομένου, (β) πρόταση νέων ερωτημάτων και αλγορίθμων για βελτιωμένες υπηρεσίες θέσης, που εκμεταλλεύονται τις νέες μεθόδους ευρετηρίασης, και (γ) την αποτελεσματικότητα της εκτέλεσης των ερωτημάτων αξιοποιώντας την πιο πρόσφατη έρευνα σε αλγόριθμους εγγύτητας, αλγόριθμους για την εύρεση κοντινότερων μονοπατιών σε γράφους και την έρευνα σχετικά με ομοιότητα συνόλων. Μια εκτενής ανασκόπηση της βιβλιογραφίας έχει πραγματοποιηθεί για μια σε βάθος έρευνα σχετικά με τις περιοχές ενδιαφέροντος που αφορούν αυτή τη διδακτορική διατριβή. Νέες μέθοδοι πρόσβασης και αλγόριθμοι επεξεργασίας επερωτήσεων έχουν προταθεί για να υποστηρίξουν υπηρεσίες θέσης που είναι εμπλουτισμένες με περιεχόμενο που παράγεται από χρήστες και εξάγεται από διαδικτυακές πλατφόρμες. Η πειραματική αξιολόγηση των προτεινόμενων αλγορίθμων και τεχνικών αποδεικνύουν την αποτελεσματικότητά τους. Προτείνεται επίσης ένα σύστημα που λειτουργεί εξ' ολοκλήρου εντός της βάσης δεδομένων, προκειμένου να στηρίξει ερωτήματα απόστασης σε γράφους μεγάλης κλίμακας. Τέλος, παρουσιάζουμε ένα σύστημα για τη συλλογή, ανάλυση, υποβολή επερωτήσεων και την οπτικοποίηση των δεδομένων των κοινωνικών δικτύων, που στοχεύει στην υποστήριξη των προτεινόμενων από αυτή τη διατριβή υπηρεσιών θέσης.

This paper presents the Trusted Integrated Knowledge Dataspace (TIKD), a new dataspace, based on linked data technologies and trusted data sharing, that supports integrated knowledge graphs for sensitive application environments such as healthcare. State-of-the-art shared dataspaces do not consider sensitive data and privacy-aware log records as part of their solutions, defining only how to access data. TIKD complements dataspace security approaches through trusted data sharing that considers personal data handling, data privileges, pseudonymization of user activity logging, and privacy-aware data interlinking services. TIKD was implemented on the Access Risk Knowledge(ARK) Platform, a socio-technical risk governance system, and deployed as part of the ARK-Virus Project which aims to govern the risk management of Personal Protection Equipment (PPE)across a group of collaborating healthcare institutions. The ARK Platform was evaluated, both before and after implementing the TIKD, using the ISO 27001 Gap Analysis Tool (GAT) which determines compliance with the information security standard.The results of the evaluation indicated that compliance with ISO 27001 increased from 50% to 85%. The evaluation also provided a set of recommended actions to meet the remaining requirements of the ISO 27001 standard. TIKD provides a collaborative environment, based on knowledge graph integration and GDPR-compliant personal data handling, as part of the data security infrastructure. As a result of this work, a new trusted data security methodology, based on personal data handling,data privileges, access control context specification, and privacy-aware data interlinking, was developed using a knowledge graph approach

Summary Most of the knowledge in materials science literature is in the form of unstructured data such as text and images. Here, we present a framework employing natural language processing, which automates text and image comprehension and precision knowledge extraction from inorganic glasses’ literature. The abstracts are automatically categorized using latent Dirichlet allocation (LDA) to classify and search semantically linked publications. Similarly, a comprehensive summary of images and plots is presented using the caption cluster plot (CCP), providing direct access to images buried in the papers. Finally, we combine the LDA and CCP with chemical elements to present an elemental map, a topical and image-wise distribution of elements occurring in the literature. Overall, the framework presented here can be a generic and powerful tool to extract and disseminate material-specific information on composition–structure–processing–property dataspaces, allowing insights into fundamental problems relevant to the materials science community and accelerated materials discovery., Highlights • Natural language processing is used for information extraction from research papers • Caption cluster plots are used for exploring figure captions across the entire corpus • Elemental maps are used to identify the chemical elements reported in a study • A framework to extract domain-specific queries from the literature, The bigger picture Most knowledge generated through scientific enquiry in materials domain is presented in the form of unstructured data. Among the available sources such as online websites, digital data, and publications, peer-reviewed journals serve as the undisputed source of reliable information regarding materials synthesis, characterization, and properties. Despite the availability of large data, only a limited fraction is compiled in the form of machine-readable databases, most of which are manually curated. Here, applying natural language processing on a large corpus of journal publications on inorganic glasses, we present a framework of information extraction from text and images, which answers queries related to synthesis and characterization techniques, and even chemical elements used. The scalable approach presented here can be applied to other domains for efficient information retrieval from scientific literature., This work demonstrates the use of natural language processing tools to extract information from a large corpus of scientific literature in the field of inorganic glasses. The scalable approach presented here combines information from the text and images to answer domain-specific queries, thereby enabling accelerated materials discovery.

Fair evaluation of text clustering methods needs to clarify the relations between 1)pre-processing, resulting in raw term occurrence vectors, 2)data transformation, and 3)method in the strict sense. We have tried to empirically compare a dozen well-known methods and variants in a protocol crossing three contrasted open-access corpora in a few tens transformed dataspaces. We compared the resulting clusterings to their supposed "ground-truth" classes by means of four usual indices. The results show both a confirmation of well-established implicit combinations, and good performances of unexpected combinations, mostly in spectral or kernel dataspaces. The rich material resulting from these some 450 runs includes a wealth of intriguing facts, which needs further research on the specificities of text corpora in relation to methods and dataspaces.

With the increasing fidelity and resolution enabled by high-performance computing systems, simulation-based scientific discovery is able to model and understand microscopic physical phenomena at a level that was not possible in the past. A grand challenge that the HPC community is faced with is how to handle the large amounts of analysis data generated from simulations. In-memory computing, among others, is recognized to be a viable path forward and has experienced tremendous success in the past decade. Nevertheless, there has been a lack of a complete study and understanding of in-memory computing as a whole on HPC systems. This paper presents a comprehensive study, which goes well beyond the typical performance metrics. In particular, we assess the in-memory computing with regard to its usability, portability, robustness and internal design trade-offs, which are the key factors that of interest to domain scientists. We use two realistic scientific workflows, LAMMPS and Laplace, to conduct comprehensive studies on state-of-the-art in-memory computing libraries, including DataSpaces, DIMES, Flexpath and Decaf. We conduct cross-platform experiments at scale on two leading supercomputers, Titan at ORNL and Cori at NERSC, and summarize our key findings in this critical area.

As applications move towards extreme scales, data-related challenges are becoming significant concerns for scientific workflows, and in-situ/in-transit data processing have been proposed to address these challenges. However, increasing scales are expected to result in an increase in the rate of failures and the cost of resilience. Even worse, since coupled applications in workflows frequently interact and exchange a large amount of data, simply applying state of the art fault tolerance techniques to individual application components can not guarantee data consistency in workflows after failure recovery. Furthermore, naive use of fault tolerance techniques, such as checkpoint/restart, to the entire workflows prohibits the diversity of resilience of application components in workflows, and finally incurs a significant latency, storage overheads, and performance degradation. This paper addressed fault tolerance challenge for extreme scale in-situ scientific workflows. We present a loose coupled checkpoint/restart framework for in-situ workflows. This proposed approach provides a scalable and flexible fault tolerance scheme for in-situ workflows while still maintaining the data consistency and low resiliency cost. Specifically, we introduce a data logging mechanism in data staging which is composed by the queue based algorithm and user interface to keep data/events consistent during failure recovery. We have implemented our approach within the DataSpaces, an open-source data staging middleware, and evaluated it using synthetic workflows on a Cray XC40 system (Cori) at different scales. We demonstrated that, in the presence of failures, uncoordinated checkpoint and hybrid checkpoint with data logging scheme improved the workflow execution time by up to 13.48% in comparison with global coordinated checkpoint/restart approach.

Within dataspaces, data sources are not necessarily fully integrated or homogeneous in their schematics and semantics. For dataspaces to support a real-time response to situations of interest when a set of events take place, for example from sensor readings, there is a need for a principled approach to tackling data heterogeneity within real-time data processing. In this chapter, we detail techniques for developing dataspace support services for dealing with the semantic heterogeneity of real-time data.

The proliferation of sensor devices and services along with the advances in event processing brings many new opportunities as well as challenges for intelligent systems. It is now possible to provide, analyse, and react upon real-time, complex events in smart environments. When existing event services do not provide such complex events directly, an event service composition may be required. However, it is difficult to determine which event service candidates (or service compositions) best suit users’ and applications’ quality-of-service requirements. A sub-optimal service composition may lead to inaccurate event detection and lack of system robustness. In this chapter, we address these issues by first providing a Quality-of-Service (QoS) aggregation schema for complex event service compositions, and then developing a genetic algorithm to create near-optimal event service compositions efficiently. The approach is evaluated with both real sensor data collected via Internet of Things services and synthesised datasets.

Real-time predictive data analytics is a very important tool for effective decision support within intelligent systems. When making decisions using data, it is critical to use the most appropriate data. When creating predictive analytics, the selection of data sources is important as the quality of the sources influences the accuracy of the predictive model. Within a smart environment, a dataspace is valuable for data scientists as it provides a one-stop shop of all the data required for creating their analytical models: enterprise data, Internet of Things (IoT), sensor data, and open data. However, the increase in the number of data sources presents a challenge in selecting the most appropriate data source to use. The co-existence approach of dataspaces results in them containing much more data sources than within traditional data management approaches. This means that the need to perform source selection is an ongoing activity; as the dataspace is incrementally improved, sources will need to be re-examined to determine their suitability for tasks. We propose an autonomic source selection service for predictive analytics for intelligent systems within a smart environment. This service has been evaluated in real-world settings using a Real-time Linked Dataspace for energy predictions using IoT sensor data and open weather data.

The Internet of Things (IoT) envisions smart objects and intelligent systems collecting and sharing data on a global scale to enable smart environments. One challenging data management issue is how to disseminate data to relevant consumers efficiently. This chapter leverages semantic technologies, such as linked data, which can facilitate machine-to-machine communications to build an efficient stream dissemination system for Semantic IoT. The system integrates linked data streams generated from various collectors and disseminates matched data to relevant consumers based on user queries registered in the system. We design two new data structures to suit the needs of high-performance linked data stream dissemination in the following two scenarios: (1) stream dissemination in point-to-point systems; and (2) stream dissemination in wireless broadcast systems. The evaluation of the approaches using real-world datasets shows that they can disseminate linked data streams more efficiently than existing techniques.

Dataspaces can provide an approach to enable data management in smart environments that can help to overcome technical, conceptual, and social/organisational barriers to information sharing. However, there has been limited work on the use of dataspaces within smart environments and the necessary support services for real-time events and data streams. This chapter introduces the Real-time Linked Dataspace (RLD) as a data platform for intelligent systems within smart environments. The RLD combines the pay-as-you-go paradigm of dataspaces with linked data, knowledge graphs, and (near) real-time processing capabilities. The RLD has been specifically designed to support the sharing and processing of data between intelligent systems within smart environments. We propose a set of specialised dataspace support services to enable the requirements of loose administrative proximity and semantic integration for event and stream systems. These requirements form the foundation of the techniques and models used to process events and streams within the RLD.