Search

As a GPU has become an essential component in high performance computing, it has been attempted by many works to leverage GPU computing in Hadoop. However, few works considered to fully utilize the GPU in Hadoop and only a few works studied utilizing both CPU and GPU at the same time. In this paper, we propose a CPU-GPU hybrid scheduling in Hadoop, where both CPUs and GPUs in a node are exploited as much as possible in an adaptive manner. The technical barrier stands in that the optimal number of GPU tasks is not known in advance, and the total number of Containers in a node cannot be changed once a Hadoop job starts. In the proposed approach, we first determine the initial number of Containers as well as the hybrid execution mode, then the proposed dynamic scheduler adjusts the number of Containers for a GPU and a CPU with the help of a GPU monitor during the job execution. It also employs a load-balancing algorithm for the tail. The experiments with various benchmarks show that the proposed CPU-GPU hybrid scheduling achieves 3.87 × of speedup on average against the 12-core CPU-only Hadoop. [ABSTRACT FROM AUTHOR]

In the present paper, we propose a new approach on 'distributed systems': the processes are represented through total orders and the communications are characterized by means of biorders. The resulting distributed systems capture situations met in various fields (such as computer science, economics and decision theory). We investigate questions associated to the numerical representability of order structures, relating concepts of economics and computing to each other. The concept of 'quasi-finite partial orders' is introduced as a finite family of chains with a communication between them. The representability of this kind of structure is studied, achieving a construction method for a finite (continuous) Richter–Peleg multi-utility representation. [ABSTRACT FROM AUTHOR]

Summary With the rapid development of the global renewable energy source field, the importance of dynamic index processing technology in distributed energy systems has become more and more obvious. To better improve the real‐time dynamic interaction means of microgrids in the energy Internet and optimize the relevant methods for microgrid energy consumption detection, this article proposes to introduce the distributed Hadoop platform into the electrical thermal coupling multivariate data in the form of cluster configuration, and then use the Spark framework to detect and capture real‐time data, to complete the tracking and analysis of energy consumption data. At the same time, the Internet of Things and the cloud intelligent monitoring system are combined to further clean and explore the data, to achieve the in‐depth detection of the energy consumption problem of the microgrid under the premise of reducing the initial investment, and achieve the purpose of reducing the operating cost. In this case, the outliers are detected according to the photovoltaic indicators of photovoltaic power stations, the filtration and purification functions of photovoltaic indicators are used by the nuclear density curve, and the sustainable solar energy is optimized by combining multiple indicators such as wind direction and temperature. Based on reducing energy consumption, the overfitting phenomenon of the controller is controlled, and an optimized controller‐led cloud platform is established. By establishing the objective function model, the robustness of the controller is guaranteed and the detection expectation is satisfied by the experiment of energy consumption data. In addition, when the cloud platform is created, this study uses a genetic algorithm to optimize the controller index and then builds a cloud console detection mechanism that collaborates with the Internet. Through the research, it is found that outliers may lead to the redundancy of energy consumption indicators in the non‐processing state. This study adopts the optimization of energy consumption parameters and the help of a distributed data framework to deal with and effectively solve this problem. In terms of interpolation simulation verification combined with experimental data, this paper proposes to use the Internet of Things, wearable devices, sensors, and other means to monitor the cost of energy consumption, to realize the distributed dynamic storage of massive real‐time data in the process of parallel processing, as well as the evaluation and detection of real‐time data replacement. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) consists of millions of devices deployed over hundreds of thousands of different networks, providing an ever-expanding resource to improve our understanding of and interactions with the physical world. Global service discovery is key to realizing the opportunities of the IoT, spanning disparate networks and technologies to enable the sharing, discovery, and utilisation of services and data outside of the context in which they are deployed. In this paper, we present Decentralised Service Registries (DSRs), a novel trustworthy decentralised approach to global IoT service discovery and interaction, building on DSF-IoT to allow users to simply create and share public and private service registries, to register and query for relevant services, and to access both current and historical data published by the services they discover. In DSR, services are registered and discovered using signed objects that are cryptographically associated with the registry service, linked into a signature chain, and stored and queried for using a novel verifiable DHT overlay. In contrast to existing centralised and decentralised approaches, DSRs decouple registries from supporting infrastructure, provide privacy and multi-tenancy, and support the verification of registry entries and history, service information, and published data to mitigate risks of service impersonation or the alteration of data. This decentralised approach is demonstrated through the creation and use of a DSR to register and search for real-world IoT devices and their data as well as qualified using a scalable cluster-based testbench for the high-fidelity emulation of peer-to-peer applications. DSRs are evaluated against existing approaches, demonstrating the novelty and utility of DSR to address key IoT challenges and enable the sharing, discovery, and use of IoT services. [ABSTRACT FROM AUTHOR]

Mutation testing is a valuable technique for measuring the quality of test suites in terms of detecting faults. However, one of its main drawbacks is its high computational cost. For this purpose, several approaches have been recently proposed to speed-up the mutation testing process by exploiting computational resources in distributed systems. However, bottlenecks have been detected when those techniques are applied in large-scale systems. This work improves the performance of mutation testing using large-scale systems by proposing a new load distribution algorithm, and parallelising different steps of the process. To demonstrate the benefits of our approach, we report on a thorough empirical evaluation, which analyses and compares our proposal with existing solutions executed in large-scale systems. The results show that our proposal outperforms the state-of-the-art distribution algorithms up to 35% in three different scenarios, reaching a reduction of the execution time of—at best—up to 99.66%. [ABSTRACT FROM AUTHOR]

One of the primary challenges in distributed systems, such as cloud computing, lies in ensuring that data objects are accessible within a reasonable timeframe. To address this challenge, the data objects are replicated across multiple servers. Estimating the minimum quantity of data replicas and their optimal placement is considered an NP-complete optimization problem. The primary objectives of the current research include minimizing data processing costs, reducing the quantity of replicas, and maximizing the applied algorithms’ reliability in replica placement. This paper introduces a hybrid chaos-based swarm approach using the modified shuffle-frog leaping algorithm with a new local search strategy for replicating data in distributed systems. Taking into account the algorithm’s performance in static settings, the introduced method reduces the expenses associated with replica placement. The results of the experiment conducted on a standard data set indicate that the proposed approach can decrease data access time by about 33% when using approximately seven replicas. When executed several times, the suggested method yields a standard deviation of approximately 0.012 for the results, which is lower than the result existing algorithms produce. Additionally, the new approach’s success rate is higher in comparison with existing algorithms used in addressing the problem of replica placement. [ABSTRACT FROM AUTHOR]

State Machine Replication (SMR) solutions often divide time into rounds, with a designated leader driving decisions in each round. Progress is guaranteed once all correct processes synchronize to the same round, and the leader of that round is correct. Recently suggested Byzantine SMR solutions such as HotStuff, and LibraBFT achieve progress with a linear message complexity and a constant time complexity once such round synchronization occurs. But round synchronization itself incurs an additional cost. By Dolev and Reischuk's lower bound, any deterministic solution must have Ω (n 2) communication complexity. Yet the question of randomized round synchronization with an expected linear message complexity remained open. We present an algorithm that, for the first time, achieves round synchronization with expected linear message complexity and expected constant latency. Existing protocols can use our round synchronization algorithm to solve Byzantine SMR with the same asymptotic performance. [ABSTRACT FROM AUTHOR]

Distributed systems have been widely used in many safety‐critical areas. Any abnormalities (e.g., service interruption or service quality degradation) could lead to application crashes or decrease user satisfaction. These things may cause serious economic losses. Among the various quality assurance approaches for distributed systems, log‐based anomaly detection (LAD) has become a popular research topic. Its popularity relates to system logs being able to record and reveal important run‐time information. This paper presents a general LAD framework for distributed systems. Log grouping and feature‐pattern mining are two crucial LAD components that impact on the anomaly‐detection effectiveness. We also present a systematic survey of techniques in these two directions; propose classification frameworks for log grouping and feature patterns; and summarize four log‐grouping techniques and five feature patterns (which refer to invariant relationships among logs that can be used for anomaly detection). To evaluate their applicability, we report on the findings when applying existing techniques to Ray, a popular industrial distributed system. Based on these findings, several open issues are identified, which provide potential guidance for future research and development. [ABSTRACT FROM AUTHOR]

The continued growth of the cyber-physical system (CPS) and Internet of Things technologies raises device security and monitoring concerns. For device identification, authentication, conditioning, and security, device fingerprint/fingerprinting (DFP) is increasingly used. However, finding the correct DFP features and sources to establish a unique and stable fingerprint is challenging. We present a state-of-the-art survey of DFP techniques for CPS device applications. We investigate the numerous DFP features, their origins, characteristics, and applications. Additionally, we discuss the DFP characteristics and their sources in detail, taking into account the physical contexts of various entities (i.e., machines, sensors, networks, and computational devices), as well as their software and applications for the CPS. We believe that this article will provide researchers and developers with insights into the DFP and its applications, sources, aggregation methods, and factors affecting its use in CPS domains. [ABSTRACT FROM AUTHOR]

Reliable systems require effective monitoring techniques for fault identification. System-level diagnosis was originally proposed in the 1960s as a test-based approach to monitor and identify faulty components of a general system. Over the last decades, several diagnosis models and strategies have been proposed, based on different fault models, and applied to the most diverse types of computer systems. In the 1990s, unreliable failure detectors emerged as an abstraction to enable consensus in asynchronous systems subject to crash faults. Since then, failure detectors have become the de facto standard for monitoring distributed systems. The purpose of the present work is to fill a conceptual gap by presenting a distributed diagnosis model that is consistent with unreliable failure detectors. Properties are proven for the number of tests/monitoring messages required, latency for event detection, as well as completeness and accuracy. Three different failure detectors compliant with the proposed model are presented, including vRing and vCube, which provide scalable alternatives to the traditional all-monitor-all strategy adopted by most existing failure detectors. [ABSTRACT FROM AUTHOR]

Load imbalance severely affects cluster performance, and the polarization of resources due to load skewing leads to further worsening of system throughput and latency problems. The proliferation of tasks to be processed in the big data era leads to more severe load skewing. How to cope with the surge of skewed data stream in the context of big data is a new challenge now. In this paper, we propose a coordinated load balancing strategy on skewed data streams (referred to as St-Stream), which is a two-tier hierarchical system for handling data streams. The proposed strategy is characterized by performing a migration pairing strategy for resources at the task allocation stage by cutting and moving out the tasks of high-load nodes in a hierarchical manner, and the moved-out operators are placed in the routing table, and the routing table operators are moved out to these nodes sequentially according to the tasks required by low-load nodes. We further design a two-tier coordination scheme for the resource allocation problem, which can adjust the skewed load from within the nodes and then dynamically restore the balance between the nodes. We implemented St-Stream on Apache Storm, which achieves a 21% coordination in processing CPU utilization, a 17.6% reduction in latency, and a 0.3 improvement in load balance recovery compared to the baseline design. Our experimental results demonstrate that the proposed load balancing strategy better balances the cluster load and improves the performance of the stream processing system. [ABSTRACT FROM AUTHOR]

One of the main problems of big distributed systems, like IoT, is the high access time to data objects. Replicating the data objects on various servers is a traditional strategy. Replica placement, which can be implemented statically or dynamically, is generally crucial to the effectiveness of distributed systems. Producing the minimum number of data copies and placing them on appropriate servers to minimize access time is an NP-complete optimization problem. Various heuristic techniques for efficient replica placement in distributed systems have been proposed. The main objectives of this research are to decrease the cost of data processing operations, decrease the number of copies, and improve the accessibility of the data objects. In this study, a discretized and group-based gray wolf optimization algorithm with swarm and evolutionary features was developed for the replica placement problem. The proposed algorithm includes swarm and evolutionary features and divides the wolves' population into subgroups, and each subgroup was locally searched in a different solution space. According to experiments conducted on the standard benchmark dataset, the suggested method provides about a 40% reduction in the data access time with about five replicas. Also, the reliability of the suggested method during different executions is considerably higher than the previous methods. [ABSTRACT FROM AUTHOR]

We describe implementing and testing a security monitoring system for Blockchain-based applications by performing Log Analysis through natural language processing (NLP) techniques and enabling queries via REST API. The study focuses on the Hyperledger Fabric framework, a highly reliable and scalable open-source software platform for private blockchains. The real-time monitoring system aims to detect and prevent attack scenarios on the network structure and smart contracts, such as DDOS, Sybil, and Eclipse. The vulnerability analysis is extended to inspecting Docker containers to detect network-based attacks. Although the case study focuses on Hyperledger Fabric, the techniques used are general and applicable to all Blockchain-based systems and fulfill the need to provide adequate monitoring capabilities for blockchain implementations. [ABSTRACT FROM AUTHOR]

We present an extensive study focused on partial network partitioning. Partial network partitions disrupt the communication between some but not all nodes in a cluster. First, we conduct a comprehensive study of system failures caused by this fault in 13 popular systems. Our study reveals that the studied failures are catastrophic (e.g., lead to data loss), easily manifest, and are mainly due to design flaws. Our analysis identifies vulnerabilities in core systems mechanisms including scheduling, membership management, and ZooKeeper-based configuration management. Second, we dissect the design of nine popular systems and identify four principled approaches for tolerating partial partitions. Unfortunately, our analysis shows that implemented fault tolerance techniques are inadequate for modern systems; they either patch a particular mechanism or lead to a complete cluster shutdown, even when alternative network paths exist. Finally, our findings motivate us to build Nifty, a transparent communication layer that masks partial network partitions. Nifty builds an overlay between nodes to detour packets around partial partitions. Nifty provides an approach for applications to optimize their operation during a partial partition. We demonstrate the benefit of this approach through integrating Nifty with VoltDB, HDFS, and Kafka. [ABSTRACT FROM AUTHOR]

Cross-domain applications are rapidly adopting blockchain techniques for immutability, availability, integrity, and interoperability. However, for most applications, global consensus is unnecessary and may not even provide sufficient guarantees. We propose a new distributed data structure: Attested Data Structures (ADS), which generalize not only blockchains but also many other structures used by distributed applications. As in blockchains, data in ADSs is immutable and self-authenticating. ADSs go further by supporting application-defined proofs (attestations). Attestations enable applications to plug in their own mechanisms to ensure availability and integrity. We present Charlotte, a framework for composable ADSs. Charlotte deconstructs conventional blockchains into more primitive mechanisms. Charlotte can be used to construct blockchains but does not impose the usual global-ordering overhead. Charlotte offers a flexible foundation for interacting applications that define their own policies for availability and integrity. Unlike traditional distributed systems, Charlotte supports heterogeneous trust: different observers have their own beliefs about who might fail, and how. Nevertheless, each observer has a consistent, available view of data. Charlotte's data structures are interoperable and composable: applications and data structures can operate fully independently or can share data when desired. Charlotte defines a language-independent format for data blocks and a network API for servers. To demonstrate Charlotte's flexibility, we implement several integrity mechanisms, including consensus and proof of work. We explore the power of disentangling availability and integrity mechanisms in prototype applications. The results suggest that Charlotte can be used to build flexible, fast, composable applications with strong guarantees. [ABSTRACT FROM AUTHOR]

With the proliferation of cloud native services, the need for efficient software design strategies has become of the utmost importance. The hypothesis of this article is that domain driven design approaches, when integrated into cloud native service architecture, provide a valuable methodology for building modular, scalable, and maintainable systems. The goal of the article is to analyse how these approaches can improve software design while also contributing to system availability, reliability, and resilience. The methodology employed in this study involves the analysis of domain-driven design approaches and their integration with cloud native technologies. The paper emphasizes the importance of clean domain models, well-defined bounded contexts, and the separation of concerns in enterprise-grade software. While focusing on foundational concepts, the paper suggests the potential for a future case study to illustrate the domain driven software development process in action. While the paper does not provide specific empirical results, it highlights the potential benefits of adopting domain-driven design and cloud native architectures. That is why the article examines the fundamental components of domain driven design, their integration with cloud native technologies, benefits, and challenges. In addition, the study sets the stage for further research in this area to help software architects and developers. [ABSTRACT FROM AUTHOR]

Student archives are records of students' learning activities during school and important proof of students' learning experiences, so the construction of a credible digital archives system is of great significance. Blockchain is a data structure and information storage technology that has become well-known with the development of Bitcoin and is currently widely used to build a trusted digital environment. The use of blockchain technology can solve the problems of low confidentiality and easy tampering of traditional archives management. Therefore, this paper proposes a blockchain-based student archives management scheme, designs the key data structures and algorithms of the system model, and tests the functions and performance of the system model on the ESXi platform. Experimental results show that the scheme proposed in this paper can effectively improve the confidentiality and difficulty of archive tampering, and at the same time have good resource sharing and retrieval efficiency, making archives management safer, more credible and more intelligent. [ABSTRACT FROM AUTHOR]

Availability and accessibility of data objects in a reasonable time is a main issue in distributed systems like cloud computing services. As a result, the reduction of data-related operation times in distributed systems such as data read/write has become a major challenge in the development of these systems. In this regard, replicating the data objects on different servers is one commonly used technique. In general, replica placement plays an essential role in the efficiency of distributed systems and can be implemented statically or dynamically. Estimation of the minimum number of data replicas and the optimal placement of the replicas is an NP-complete optimization problem. Hence, different heuristic algorithms have been proposed for optimal replica placement in distributed systems. Reducing data processing costs as well as the number of replicas, and increasing the reliability of the replica placement algorithms are the main goals of this research. This paper presents a discrete and swarm-evolutionary method using a combination of shuffle-frog leaping and genetic algorithms to data-replica placement problems in distributed systems. The experiments on the standard dataset show that the proposed method reduces data access time by up to 30% with about 14 replicas; whereas the generated replicas by the GA and ACO are, respectively, 24 and 30. The average reduction in data access time by GA and ACO 21% and 18% which shows less efficiency than the SFLA–GA algorithm. Regarding the results, the SFLA–GA converges on the optimal solution before the 10th iteration, which shows the higher performance of the proposed method. Furthermore, the standard deviation among the results obtained by the proposed method on several runs is about 0.029, which is lower than other algorithms. Additionally, the proposed method has a higher success rate than other algorithms in the replica placement problem. [ABSTRACT FROM AUTHOR]

Regression testing is indispensable, especially for real-time distributed systems to ensure that existing functionalities are not affected by changes. Despite recent advances, regression testing for distributed systems remains challenging and extremely costly. Existing techniques often require running a failing system several times before detecting a regression. As a result, conventional approaches that use re-execution without considering the inherent non-determinism of distributed systems, and providing no (or low) control over execution are inadequate in many ways. In this paper, we present MRegTest, a replay-based regression testing framework in the context of model-driven development to facilitate deterministic replay of traces for detecting regressions while offering sufficient control for the purpose of testing over the execution of the changed system. The experimental results show that compared to the traditional approaches that annotate traces with timestamps and variable values MRegTest detects almost all regressions while reducing the size of the trace significantly and incurring similar runtime overhead. [ABSTRACT FROM AUTHOR]

Achieving timely access to data objects is a major challenge in big distributed systems like the Internet of Things (IoT) platforms. Therefore, minimizing the data read and write operation time in distributed systems has elevated to a higher priority for system designers and mechanical engineers. Replication and the appropriate placement of the replicas on the most accessible data servers is a problem of NP-complete optimization. The key objectives of the current study are minimizing the data access time, reducing the quantity of replicas, and improving the data availability. The current paper employs the Olympiad Optimization Algorithm (OOA) as a novel population-based and discrete heuristic algorithm to solve the replica placement problem which is also applicable to other fields such as mechanical and computer engineering design problems. This discrete algorithm was inspired by the learning process of student groups who are preparing for the Olympiad exams. The proposed algorithm, which is divide-and-conquer-based with local and global search strategies, was used in solving the replica placement problem in a standard simulated distributed system. The 'European Union Database' (EUData) was employed to evaluate the proposed algorithm, which contains 28 nodes as servers and a network architecture in the format of a complete graph. It was revealed that the proposed technique reduces data access time by 39% with around six replicas, which is vastly superior to the earlier methods. Moreover, the standard deviation of the results of the algorithm's different executions is approximately 0.0062, which is lower than the other techniques' standard deviation within the same experiments. [ABSTRACT FROM AUTHOR]

An overview of PARCS (Parallel Asynchronous Recursive Control Space) technology developments is provided. The concept of the control space, i.e., a model apparatus, based on which the logical structure of the investigated problem (system) is described, and dynamic changes in it are reflected, is considered. The PARCS model is proposed whose application leads to flexible and unified adaptation to emerging programming technologies. PARCS-extension of the following programming languages is considered: PASCAL, C, FORTRAN, MODULA2, Java, CUDA, OpenCL, PYTHON,.NET, GO/PYTHON. [ABSTRACT FROM AUTHOR]

Service choreographies are a versatile approach for building service-based distributed systems. Many approaches can be found in the literature tackling different aspects of service choreographies, such as choreography realizability and conformance checking, distributed coordination, formal choreographic languages, and scalability. As of today, choreography scalability has not been specifically addressed through approaches that also consider coordination issues while still decoupling these two related aspects. Scalability is one of the most important properties to be considered when building distributed systems. It enhances the user-perceived performances and influences the overall dependability of the system. In particular, load scalability allows distributed service-oriented systems to effectively handle varying loads without suffering performance degradation. In this direction, microservice-based systems are able to scale thanks to the possibility of replicating those microservices exposed to growing loads, distributing their workload among different instances. By leveraging on our experience in coordinating service choreographies, in this paper, we propose a layered architectural style that allows to realize scalable microservice-oriented choreographies. The architecture integrates a fully-distributed coordination layer capable of ensuring the correct interactions and a load-balancing layer that allows to balance of coordinated requests. We discuss the properties of the proposed architectural style and evaluate its benefits on user-perceived performances. [ABSTRACT FROM AUTHOR]

The development of Distributed Ledger Technology (DLT) is pushing toward automating decentralized data exchange processes. One of the key components of this evolutionary step is facilitating smart contracts that, in turn, come with several additional vulnerabilities. Despite the existing tools for analyzing smart contracts, keeping these systems running and preserving performance while maintaining a decent level of security in a constantly increasing number of contracts becomes challenging. Machine Learning (ML) methods could be utilized for analyzing and detecting vulnerabilities in DLTs. This work proposes a new ML-based two-phase approach for the detection and classification of vulnerabilities in smart contracts. Firstly, the system's operation is set up to filter the valid contracts. Secondly, it focuses on detecting a vulnerability type, if any. In contrast to existing approaches in this field of research, our algorithm is more focused on vulnerable contracts, which allows to save time and computing resources in the production environment. According to the results, it is possible to detect vulnerability types with an accuracy of 0.9921, F1 score of 0.9902, precision of 0.9883, and recall of 0.9921 within reasonable execution time, which could be suitable for integrating existing DLTs. [ABSTRACT FROM AUTHOR]

Availability is crucial to the security of distributed systems, but guaranteeing availability is hard, especially when participants in the system may act maliciously. Quorum replication protocols provide both integrity and availability: data and computation is replicated at multiple independent hosts, and a quorum of these hosts must agree on the output of all operations applied to the data. Unfortunately, these protocols have high overhead and can be difficult to calibrate for a specific application's needs. Ideally, developers could use high-level abstractions for consensus and replication to write fault-tolerant code that is secure by construction. This paper presents Flow-Limited Authorization for Quorum Replication (FLAQR), a core calculus for building distributed applications with heterogeneous quorum replication protocols while enforcing end-to-end information security. Our type system ensures that well-typed FLAQR programs cannot fail (experience an unrecoverable error) in ways that violate their type-level specifications. We present noninterference theorems that characterize FLAQR 's confidentiality, integrity, and availability in the presence of consensus, replication, and failures, as well as a liveness theorem for the class of majority quorum protocols under a bounded number of faults. Additionally, we present an extension to FLAQR that supports secret sharing as a form of declassification and prove it preserves integrity and availability security properties. [ABSTRACT FROM AUTHOR]

The digitalization of production in smart grids entails challenges related to data collection, coordination, privacy protection, and anomaly detection. Machine learning techniques offer effective tools for processing Big Data, but identifying critical system states amidst vast amounts of data remains a challenge. To expedite data analysis, preprocessing through machine learning algorithms becomes essential. This paper introduces the advanced FedSVD algorithm, utilizing Singular Value Decomposition (SVD), which efficiently decomposes large datasets, establishes relationships, and identifies irrelevant data. The algorithm operates in federated machine learning systems, enabling local data processing on private devices while sharing only results with the global learning model. This approach enhances information processing confidentiality and facilitates the exchange of anomaly detection outcomes among network devices. The results of the study demonstrate that the modified FedSVD processing is 5 ms faster on average in comparison to the non-modified one. The proposed FedSVD algorithm calculates anomaly detection with higher accuracy by an average of 1–3% compared to the non-modified FedSVD and SVD ones. The advanced FedSVD algorithm proves to be a decentralized, confidential, and efficient solution for anomaly detection in smart grid systems. [ABSTRACT FROM AUTHOR]

This paper solves the problem of detecting violations of predicates over distributed continuous-time and continuous-valued signals in cyber-physical systems (CPS). CPS often operate in a safety-critical context, where their correctness is crucially important. Large CPS that consist of many autonomous and communicating components distributed across a geographical area must maintain global correctness and safety. We assume a partially synchronous setting, where a clock synchronization algorithm guarantees a bound on clock drifts among all signals. We introduce a novel retiming method that allows reasoning about the correctness of predicates among continuous-time signals that do not share a global view of time. The resulting problem is encoded as an SMT problem and we introduce techniques to solve the SMT encoding efficiently. Leveraging simple knowledge of physical dynamics allows further runtime reductions. We fully implement our approach on three distributed CPS applications: monitoring of a network of autonomous ground vehicles, a network of aerial vehicles, and a water distribution system. The results show that in some cases it is even possible to monitor a distributed CPS sufficiently fast for online deployment on fleets of autonomous vehicles. [ABSTRACT FROM AUTHOR]

Nowadays, the Artificial Intelligent (AI) techniques are applied in enterprise software to solve Big Data and Business Intelligence (BI) problems. But most AI techniques are computationally excessive, and they become unfeasible for common business use. Therefore, specific high performance computing is needed to reduce the response time and make these software applications viable on an industrial environment. The main objective of this paper is to demonstrate the improvement of an aquaculture BI tool based in AI techniques, using parallel programming. This tool, called AquiAID, was created by the research group of Economic Management for the Sustainable Development of Primary Sector of the Universidad de Cantabria. The parallelisation reduces the computation time up to 60 times, and the energy efficiency by 600 times with respect to the sequential program. With these improvements, the software will improve the fish farming management in aquaculture industry. [ABSTRACT FROM AUTHOR]

With the rapid development of blockchains, blockchain systems are moving on from a stand-alone manner to cross-chain interactions, and achieving interoperability is emerging as one of the essential features of blockchains. Unfortunately, existing mechanisms such as XCLAIM mostly focus on exchanging assets between two blockchains and it is slow and expensive to process each cross-chain trade among more than two blockchains as multiple transactions are required. In this paper, we present Interopera, a decentralized and efficient cross-chain trading protocol among two or more blockchains. Interopera atomically processes each cross-chain trade faster and more cheaply with fewer transactions by a two-phase lock/unlock process. Interopera also achieves efficient cross-chain communication by our presented Partitioned-FlyClient and Tx-FlyClient. Partitioned-FlyClient is based on FlyClient but more efficient with smaller proof size, reducing the storage and bandwidth overheads. Tx-FlyClient maintains efficiency even when cross-chain trades become frequent, instead of other mechanisms only being effective under low cross-chain trades volumes. We also develop a proof-of-concept implementation and the results demonstrate high efficiency of our protocol. [ABSTRACT FROM AUTHOR]

Training and validation of Neural Networks (NN) are very computationally intensive. In this paper, we propose a distributed system based NN infrastructure that achieves two goals: to accelerate model training, specifically for hyperparameter optimization, and to re-use some of these intermediate models to evaluate the uncertainty of the model. By accelerating model training, we can obtain a large set of potential models and compare them in a shorter amount of time. Automating this process reduces development time and provides an easy way to compare different models. Our application runs different models on distinct servers with a single training data set, each with tweaked hyperparameters. By adding uncertainty to our results, our framework provides not just a single prediction but a distribution over predictions. Adding uncertainty is essential to some NN applications since most models assume that the input data distributions are identical between test and validation. However, in reality, they are producing some catastrophic mistakes. Since our solution is a distributed system, we make our implementation robust to common distributed system failures (servers going down, loss of communication among some nodes, and others). Furthermore, we use a gossip-style heartbeat protocol for failure detection and recovery. Finally, some preliminary results using a black-box approach to generate the training models show that our infrastructure scales well in different hardware platforms. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of cross-chain payment whereby customers of different escrows—implemented by a bank or a blockchain smart contract—successfully transfer digital assets without trusting each other. Prior to this work, cross-chain payment problems did not require this success, or any form of progress. We introduce a new specification formalism called Asynchronous Networks of Timed Automata to formalise such protocols. We present the first cross-chain payment protocol that ensures termination in a bounded amount of time and works correctly in the presence of clock drift. We then demonstrate that it is impossible to solve this problem without assuming synchrony, in the sense that each message is guaranteed to arrive within a known amount of time. Yet, we solve an eventually terminating weaker variant of this problem, where success is conditional on the patience of the participants, without assuming synchrony, and in the presence of Byzantine failures. We also discuss the relation with the recently defined cross-chain deals. [ABSTRACT FROM AUTHOR]

The debate around the notions of a priori knowledge and a posteriori knowledge has proven crucial for the development of many fields in philosophy, such as metaphysics, epistemology, metametaphysics etc. We advocate that the recent debate on the two notions is also fruitful for man-made distributed computing systems and for the epistemic analysis thereof. Following a recently proposed modal and fallibilistic account of a priori knowledge, we elaborate the corresponding concept of a priori belief: We propose a rich taxonomy of types of a priori beliefs and their role for the different agents that participate in the system engineering process, which match the existing view exceedingly well and are particularly promising for explaining and dealing with unexpected behaviors in fault-tolerant distributed systems. Developing such a philosophical foundation will provide a sound basis for eventually implementing our ideas in a suitable epistemic reasoning and analysis framework and, hence, constitutes a mandatory first step for developing methods and tools to cope with the various challenges that emerge in such systems. [ABSTRACT FROM AUTHOR]

The construction of systems supporting spatial data has experienced great enthusiasm in the past, due to the richness of this type of data and their semantics, which can be used in the decision-making process in various fields. Thus, the problem of integrating spatial data into existing databases and information systems has been addressed by creating spatial extensions to relational tables or by creating spatial data warehouses, while arranging data structures and query languages by making them more spatially-aware. With the advent of Big Data, these conventional storage and spatial representation structures are becoming increasingly outdated, and required a new organization of spatial data. Approaches based on distributed storage and data lakes have been proposed, to integrate the complexity of spatial data, with operational and analytical systems which unfortunately quickly showed their limits. Recently the concept of lakehouse was introduced in order to integrate, among other things, the notion of reliability and ACID properties to the volume of data to be managed. This new data architecture is a combination of governed and reliable Data Warehouses and flexible, scalable and cost-effective Data Lakes. In this paper, we present how traditional approaches of spatial data management in the context of spatial big data have quickly shown their limits. We present a literature overview of these approaches, and how they led to the Data LakeHouse. We detail how the Lakehouse paradigm can be used and extended for managing spatial big data, by giving the different components and best practices for building a spatial data LakeHouse architecture optimized for the storage and computing over spatial big data. • Limitations of Data Warehouses and Data Lakes for Spatial Big Data. • Characteristics and Architecture of the Data LakeHouse. • Overview of Three Major Open Source LakeHouse Systems. • Challenges in Using Data LakeHouse for Spatial Big Data. • Optimized Data LakeHouse Architecture for Spatial Big Data. [ABSTRACT FROM AUTHOR]

Since the birth of Bitcoin, blockchain has shifted from a critical cryptocurrency technology to an enabling technology. Due to its immutability and trustworthiness, blockchain has revolutionized many fields requiring credibility and high-quality data for decision making. Particularly in business intelligence and business process management, users can use blockchain to build their blockchain-enabled collaboration and data-sharing ecosystem with their partners. In this paper, we present the development process of blockchain and consensus mechanisms, where important blockchain consensus mechanisms are introduced. The consensus mechanism is the kernel among various blockchain components to ensure security and performance. Again, we present a comparison of these consensus mechanisms from different perspectives. We take the blockchain-enabling business as an example and analyze the relationship between blockchain and business process characteristics and the ideas and principles for selecting consensus mechanisms. Finally, we describe the differences and connections among many consensus mechanisms while laying a foundation for selecting appropriate consensus mechanisms for different scenarios and fields of application. [ABSTRACT FROM AUTHOR]

Crude oil leakages and spills (OLS) are some of the problems attributed to pipeline failures in the oil and gas industry's midstream sector. Consequently, they are monitored via several leakage detection and localisation techniques (LDTs) comprising classical methods and, recently, Internet of Things (IoT)-based systems via wireless sensor networks (WSNs). Although the latter techniques are proven to be more efficient, they are susceptible to other types of failures such as high false alarms or single point of failure (SPOF) due to their centralised implementations. Therefore, in this work, we present a hybrid distributed leakage detection and localisation technique (HyDiLLEch), which combines multiple classical LDTs. The technique is implemented in two versions, a single-hop and a double-hop version. The evaluation of the results is based on the resilience to SPOFs, the accuracy of detection and localisation, and communication efficiency. The results obtained from the placement strategy and the distributed spatial data correlation include increased sensitivity to leakage detection and localisation and the elimination of the SPOF related to the centralised LDTs by increasing the number of node-detecting and localising (NDL) leakages to four and six in the single-hop and double-hop versions, respectively. In addition, the accuracy of leakages is improved from 0 to 32 m in nodes that were physically close to the leakage points while keeping the communication overhead minimal. [ABSTRACT FROM AUTHOR]

As technology continues to evolve, our society is becoming enriched with more intelligent devices that help us perform our daily activities more efficiently and effectively. One of the most significant technological advancements of our time is the Internet of Things (IoT), which interconnects various smart devices (such as smart mobiles, intelligent refrigerators, smartwatches, smart fire alarms, smart door locks, and many more) allowing them to communicate with each other and exchange data seamlessly. We now use IoT technology to carry out our daily activities, for example, transportation. In particular, the field of smart transportation has intrigued researchers due to its potential to revolutionize the way we move people and goods. IoT provides drivers in a smart city with many benefits, including traffic management, improved logistics, efficient parking systems, and enhanced safety measures. Smart transportation is the integration of all these benefits into applications for transportation systems. However, as a way of further improving the benefits provided by smart transportation, other technologies have been explored, such as machine learning, big data, and distributed ledgers. Some examples of their application are the optimization of routes, parking, street lighting, accident prevention, detection of abnormal traffic conditions, and maintenance of roads. In this paper, we aim to provide a detailed understanding of the developments in the applications mentioned earlier and examine current researches that base their applications on these sectors. We aim to conduct a self-contained review of the different technologies used in smart transportation today and their respective challenges. Our methodology encompassed identifying and screening articles on smart transportation technologies and its applications. To identify articles addressing our topic of review, we searched for articles in the four significant databases: IEEE Xplore, ACM Digital Library, Science Direct, and Springer. Consequently, we examined the communication mechanisms, architectures, and frameworks that enable these smart transportation applications and systems. We also explored the communication protocols enabling smart transportation, including Wi-Fi, Bluetooth, and cellular networks, and how they contribute to seamless data exchange. We delved into the different architectures and frameworks used in smart transportation, including cloud computing, edge computing, and fog computing. Lastly, we outlined current challenges in the smart transportation field and suggested potential future research directions. We will examine data privacy and security issues, network scalability, and interoperability between different IoT devices. [ABSTRACT FROM AUTHOR]

An Open Brain–Computer Interface (OpenBCI) provides unparalleled freedom and flexibility through open-source hardware and firmware at a low-cost implementation. It exploits robust hardware platforms and powerful software development kits to create customized drivers with advanced capabilities. Still, several restrictions may significantly reduce the performance of OpenBCI. These limitations include the need for more effective communication between computers and peripheral devices and more flexibility for fast settings under specific protocols for neurophysiological data. This paper describes a flexible and scalable OpenBCI framework for electroencephalographic (EEG) data experiments using the Cyton acquisition board with updated drivers to maximize the hardware benefits of ADS1299 platforms. The framework handles distributed computing tasks and supports multiple sampling rates, communication protocols, free electrode placement, and single marker synchronization. As a result, the OpenBCI system delivers real-time feedback and controlled execution of EEG-based clinical protocols for implementing the steps of neural recording, decoding, stimulation, and real-time analysis. In addition, the system incorporates automatic background configuration and user-friendly widgets for stimuli delivery. Motor imagery tests the closed-loop BCI designed to enable real-time streaming within the required latency and jitter ranges. Therefore, the presented framework offers a promising solution for tailored neurophysiological data processing. [ABSTRACT FROM AUTHOR]

Most of the existing sender-based message logging protocols cannot commonly handle simultaneous failures because, if both the sender and the receiver(s) of each message fail together, the receiver(s) cannot obtain the recovery information of the message. This unfortunate situation may happen due to their asymmetric logging behavior. This paper presents a novel sender-based message logging protocol for broadcast network based distributed systems to overcome the critical constraint of the previous ones with the following three features. First, when more than one process crashes at the same time, the protocol enables the system to ensure the always no rollback property by symmetrically replicating the recovery information at each process or group member connected on a network. Second, it can make the first feature persist even if the general form of communication for the system is a combination of point-to-point and group ones. Third, the communication overhead resulting from the replication can be highly lessened by making full use of the capability of the standard broadcast network in both communication modes. Experimental outcomes verify that, no matter which communication patterns are applied, it can reduce about 4.23∼9.96% of the total application execution time against the latest enabling the traditional ones to cope with simultaneous failures. [ABSTRACT FROM AUTHOR]

Location Fixing (LF) of Radar Emitters is carried out in an Electronic Warfare scenario by Sensors with Direction-of-Arrival and signal parameter measurement capabilities, geographically distributed over few tens of kilometres. The present paper brings out the Scenario Simulation and Synchronous generation of Radar Signal at Geographically distributed locations for comprehensive testing of the Emitter Location Systems (ELS). This test facility permits exhaustive testing of the ELS against various parameters since the performance of the ELS depends on factors such as Sensors' geometry with respect to the location of the Emitter of interest, signal parameter measurement accuracies, LF algorithms used, time synchronization accuracy achievable among the Sensors, etc. The Radar signals are synthesized synchronously, locally, at each of the distributed Sensors by replicating the Emitter Signal characteristics as received at the Sensors. The Simulator System uses a High Accuracy GPS Receiver as the Global Clock against which the signals are timed and a Field Programmable Gate Array for generating controls triggered on the absolute time of the Global Clock. [ABSTRACT FROM AUTHOR]

With the development of the Internet and communication technologies, the types of services provided by multitier Web systems are becoming more diverse and complex compared to those of the past. Ensuring a continuous availability of business services is crucial for multitier Web system providers, as service performance issues immediately affect customer experience and satisfaction. Large companies attempt to monitor the system performance indicator (SPI) that summarizes the status of multitier Web systems to detect performance anomalies at an early stage. However, the current anomaly detection methods are designed to monitor a single specific SPI. Moreover, the existing approaches consider performance anomaly detection and its root cause analysis separately, thereby aggravating the burden of resolving the performance anomaly. To support the system provider in diagnosing the performance anomaly, we propose an advanced causative metric analysis (ACMA) framework. First, we draw out 191 performance metrics (PMs) closely related to the target SPI. Among these PMs, the ACMA determines 62 vital PMs that have the most influence on the variance of the target SPI using several statistical methods. Then, we implement a performance anomaly detection model to identify the causative metrics (CMs) between the vital PMs using random forest regression. Even if the target SPI changes, our detection model does not require any change in its model structure and can derive closely related PMs of the target SPI. Based on our experiments, wherein we applied the ACMA to the business services in an enterprise system, we observed that the proposed ACMA could correctly detect various performance anomalies and their CMs. [ABSTRACT FROM AUTHOR]

Microgrids have a lot to offer, including helping smart grids operate on distribution grids or bringing electricity to some cities. The management system receives and transmits different states. This is because the elements adapt to the conditions of the network in the shortest possible time. The 5G communication technology has high transmission speed, owing to which it can improve equipment connectivity and reduce latency, allowing the real-time analysis and monitoring of electrical microgrids considerably better than earlier generations. In addition, it is estimated that, in the near future, many cities will be connected using communication systems that allow the interconnection of different systems safeguarding the connectivity, speed, and response time of these elements in an electrical system, smart grid, or microgrids with the growing development of the Internet of Things. For this reason, it is essential to analyze the integration of 5G technology to improve the management of microgrids. This literature review analyzes and presents the advantages of using 5G technologies in reducing communication latency and improving connectivity to enhance microgrids' control and management. The active implementation of 5G in the management and control of microgrids increases the transmission and reception of data and states, reduces latency, and allows for a greater density of information, collaborating positively with resilience to the various changes that microgrids can suffer in continuous working conditions. The implementation of 5G allows electrical microgrids to be more resilient in their management and control, directly and indirectly impacting the sustainable development goals. [ABSTRACT FROM AUTHOR]

Miniaturizing plastic recycling through distributed systems has been viewed as a way to manage waste closer to the source while minimizing logistics requirements. The environmental performance of distributed manufacturing and waste management systems has been evaluated, but few studies have measured the financial performance. This study combines life cycle costing and hybrid simulation modeling to compare the net present value of small-scale distributed versus large-scale centralized systems of sorting and recycling plastic bottles and takeaway containers disposed in Singapore over 7 years. The results showed that distributed systems face a net financial loss at existing prices of SGD80-120/tonne recycled pellets. This is because of the high operation costs, particularly the labor costs due to the reliance on manual sorting. Despite being closer to the waste sources, distributed scenarios have higher fuel costs due to the poorer fuel efficiency of commercial vans compared to the larger trucks in the centralized scenarios. To improve the financial performance of distributed small-scale plastic recycling systems, it is generally recommended that small-scale sorting facilities reduce the reliance on manual labor; the smaller trucks should have higher fuel economies than conventional large waste hauling trucks; the number of small-scale facilities set up should match the amount of waste to be converted; and the price of the recycled pellets should be high enough to recover the high operating costs of recycling. The findings of this study provide motivation for future research in evaluating the financial performance of distributed recycling of other waste streams. This article met the requirements for a gold-gold data badge JIE data openness badge described at. [ABSTRACT FROM AUTHOR]

One of the key issues with large distributed systems, such as IoT platforms, is gaining timely access to data objects. As a result, decreasing the operation time of reading and writing data in distributed communication systems become essential demands for asymmetric system. A common method is to replicate the data objects across multiple servers. Replica placement, which can be performed statically or dynamically, is critical to the effectiveness of distributed systems in general. Replication and placing them on the best available data servers in an optimal manner is an NP-complete optimization problem. As a result, several heuristic strategies for replica placement in distributed systems have been presented. The primary goals of this research are to reduce the cost of data access time, reduce the number of replicas, and increase the reliability of the algorithms for placing replicas. In this paper, a discretized heuristic algorithm with artificial individuals and a hybrid imitation method were developed. In the proposed method, particle and gray-wolf-based individuals use a local memory and velocity to search for optimal solutions. The proposed method includes symmetry in both local and global searches. Another contribution of this research is the development of the proposed optimization algorithm for solving the data object replication problem in distributed systems. Regarding the results of simulations on the standard benchmark, the suggested method gives a 35% reduction in data access time with about six replicates. Furthermore, the standard deviation among the results obtained by the proposed method is about 0.015 which is lower than the other methods in the same experiments; hence, the method is more stable than the previous methods during different executions. [ABSTRACT FROM AUTHOR]

The peculiarities of applying the finite-difference method (FDM) for solving nonlinear dynamic problems of distributed systems (DS) in a flow are considered. The main limitations for using the FDM for numerical modeling of wave propagation and reflection in DS are shown to be the special features of the constitutive quasilinear equations. They necessitate the simultaneous calculation of the variables corresponding to fast and slow wave processes. The term "singularly perturbed system of equations" is used for such systems. These perturbations are the result of a significant difference in the propagation velocities of longitudinal, configurational, bending, and torsional waves in the DS at the physical level. Therefore, it is necessary to use special step-by-step methods of regularization and filtering of the numerical results. It imposes certain contstraints on the ability to model real processes and the accuracy of the results and forces the use of implicit difference schemes and high-frequency filtering. When solving the system of linear algebraic equations, taking into account the poor conditioning of the matrix of convective terms, the method of regularization was chosen by experimental calculation. Calculations according to the Crank–Nicolson difference scheme, even using coarse grids, can give the results with the required degree of accuracy and minimal time cost. Another picture is observed when comparing the results on coarse and fine grids for the Euler difference scheme. Inherent errors brought in by errors in approximating the missing boundary conditions lead to greater differences. [ABSTRACT FROM AUTHOR]

Personal health records (PHR) represent health data managed by a specific individual. Traditional solutions rely on centralized architectures to store and distribute PHR, which are more vulnerable to security breaches. To address such problems, distributed network technologies, including blockchain and distributed hash tables (DHT) are used for processing, storing, and sharing health records. Furthermore, fully homomorphic encryption (FHE) is a set of techniques that allows the calculation of encrypted data, which can help to protect personal privacy in data sharing. In this context, we propose an architectural model that applies a DHT technique called the interplanetary protocol file system and blockchain networks to store and distribute data and metadata separately; two new elements, called data steward and shared data vault, are introduced in this regard. These new modules are responsible for segregating responsibilities from health institutions and promoting end-to-end encryption; therefore, a person can manage data encryption and requests for data sharing in addition to restricting access to data for a predefined period. In addition to supporting calculations on encrypted data, our contribution can be summarized as follows: (i) mitigation of risk to personal privacy by reducing the use of unencrypted data, and (ii) improvement of semantic interoperability among health institutions by using distributed networks for standardized PHR. We evaluated performance and storage occupation using a database with 1.3 million COVID-19 registries, which showed that combining FHE with distributed networks could redefine e-health paradigms. [ABSTRACT FROM AUTHOR]

Hadoop is an open source from Apache with a distributed file system and MapReduce distributed computing framework. The current Apache 2.0 license agreement supports on-demand payment by consumers for cloud platform services, helping users leverage their respective different hardware to provides cloud services. In cloud-based environment, there is a need to balance the resource requirements of workloads, optimize load performance, and the cloud compute costs to manage. When the processing power of clustered machines varies widely, such as when hardware is aging or overloaded, Hadoop offers a speculative execution (SE) optimization strategy, by monitoring task progress in real time, in the starting identical backup tasks on different nodes when multiple tasks under a job are not running at the same speed, providing the first to go. The completed calculations maintain the overall progress of the job. At present, the SE strategy's incorrect selection of backup nodes and resource constraints may result in poor Hadoop performance, and subsequent tasks cannot be completed execution and other problems. This paper proposes an SE optimization strategy based on near data prediction, which analyzes the prediction of real-time task execution information to predict the required running time, select backup nodes based on actual requirements and approximate data to make the SE strategy achieve the best performance. Experiments prove that in a heterogeneous Hadoop environment, the optimization strategy can effectively improve the effectiveness and accuracy of various tasks and enhance the performance of cloud computing. Platform performance can benefits consumers better than before. [ABSTRACT FROM AUTHOR]

Vulnerability reduction measures are taken during the early stage distributed ship system design process to ensure the availability of the required systems in a damaged state. Traditionally these vulnerability reduction measures are based on design rules or best practices resulting from past experiences. Therefore, the measures are not per definition applicable for future warships, as both the system concepts and operational environment changes. Recently developed vulnerability assessment methods are able to determine the vulnerability of a design early in the design process. With the integration of these methods in the early stage design process, the results of the analysis can be used to generate less vulnerable distributed ship system designs. This paper proposes an integral and holistic approach to optimisation of the design variables and distributed networks as these are becoming increasingly interdependent. The result of this approach is a model which generates distributed ship system designs consisting of component positions, a topology and routed connections based on a pre-defined system configuration and constraining physical architecture. Five testcases were conducted using this model, showing the necessity of the integral and holistic approach as the extent to which the contemporary design rules are implemented depends on the network complexity and operational environment. [ABSTRACT FROM AUTHOR]