Search

Although analyses of tephra-derived glass shards have been undertaken in New Zealand for nearly four decades (pioneered by Paul Froggatt), our study is the first to systematically develop a formal, comprehensive, open-access reference dataset of glass-shard compositions for New Zealand tephras. These data will provide an important reference tool for future studies to identify and correlate tephra deposits and for associated petrological and magma-related studies within New Zealand and beyond. Here we present the foundation dataset for TephraNZ, an open-access reference dataset for selected tephra deposits in New Zealand. Prominent, rhyolitic, tephra deposits from the Quaternary were identified, with sample collection targeting original type sites or reference locations where the tephra's identification is unequivocally known based on independent dating and/or mineralogical techniques. Glass shards were extracted from the tephra deposits, and major- and trace-element geochemical compositions were determined. We discuss in detail the data reduction process used to obtain the results and propose that future studies follow a similar protocol in order to gain comparable data. The dataset contains analyses of glass shards from 23 proximal and 27 distal tephra samples characterising 45 eruptive episodes ranging from Kaharoa (636 ± 12 cal yr BP) to the Hikuroa Pumice member (2.0 ± 0.6 Ma) from six or more caldera sources, most from the central Taupō Volcanic Zone. We report 1385 major-element analyses obtained by electron microprobe (EMPA), and 590 trace-element analyses obtained by laser ablation (LA)-ICP-MS, on individual glass shards. Using principal component analysis (PCA), Euclidean similarity coefficients, and geochemical investigation, we show that chemical compositions of glass shards from individual eruptions are commonly distinguished by major elements, especially CaO, TiO2, K2O, and FeOtt (Na2O+K2O and SiO2/K2O), but not always. For those tephras with similar glass major-element signatures, some can be distinguished using trace elements (e.g. HFSEs: Zr, Hf, Nb; LILE: Ba, Rb; REE: Eu, Tm, Dy, Y, Tb, Gd, Er, Ho, Yb, Sm) and trace-element ratios (e.g. LILE/HFSE: Ba/Th, Ba/Zr, Rb/Zr; HFSE/HREE: Zr/Y, Zr/Yb, Hf/Y; LREE/HREE: La/Yb, Ce/Yb). Geochemistry alone cannot be used to distinguish between glass shards from the following tephra groups: Taupō (Unit Y in the post-Ōruanui eruption sequence of Taupō volcano) and Waimihia (Unit S); Poronui (Unit C) and Karapiti (Unit B); Rotorua and Rerewhakaaitu; and Kawakawa/Ōruanui, and Okaia. Other characteristics, including stratigraphic relationships and age, can be used to separate and distinguish all of these otherwise-similar tephra deposits except Poronui and Karapiti. Bimodality caused by K2O variability is newly identified in Poihipi and Tahuna tephras. Using glass-shard compositions, tephra sourced from Taupō Volcanic Centre (TVC) and Mangakino Volcanic Centre (MgVC) can be separated using bivariate plots of SiO2/K2O vs. Na2O+K2O. Glass shards from tephras derived from Kapenga Volcanic Centre, Rotorua Volcanic Centre, and Whakamaru Volcanic Centre have similar major- and trace-element chemical compositions to those from the MgVC, but they can overlap with glass analyses from tephras from Taupō and Okataina volcanic centres. Specific trace elements and trace-element ratios have lower variability than the heterogeneous major-element and bimodal signatures, making them easier to fingerprint geochemically.

A 42-year-old woman who fell through a glass tabletop had her lower back pierced by a long shard of glass. She rapidly exsanguinated. At autopsy, a single penetrating wound was present in her left lower back, with complete transection of her left kidney and a 3.5 L haematoperitoneum. Death was due to exsanguination following accidental transection of the left kidney by a penetrating glass injury of the lower back. Glass-topped tables are a well-recognised source of injury in a domestic setting There are far more non-lethal than lethal injuries, many of which involve children. Quite long shards may remain undetected in wounds for considerable amounts of time. Non-tempered glass is a particular risk for breakage. Glass-topped tables should not be used as substitutes for chairs, particularly in overweight or obese individuals.

By supporting decentralized applications (DApps), modern blockchains have become the technology of choice for the Web3, a decentralized way for people to interact with each other. As the popularity of DApps is growing, the challenge is now to allocate shard or subnetwork resources to face the associated demand of individual DApps. Unfortunately, most sharding proposals are inherently static as they cannot be adjusted at runtime. Given that blockchains are expected to run for years without interruption, these proposals are insufficient to cope with the upcoming demand. In this paper, we present dynamic blockchain sharding, a new way to create and close shards on-demand, and adjust their size at runtime without requiring to hard fork (i.e., creating duplicated instances of the same blockchain). The novel idea is to reconfigure sharding through dedicated smart contract invocations: not only does it strengthen the security of the sharding reconfiguration, it also makes it inherently transparent as any other blockchain data. Similarly to classic sharding, our protocol relies on randomness to cope with shard-takeover attacks and on rotating nodes to cope with the bribery of a slowly-adaptive adversary. By contrast, however, our protocol is ideally suited for open networks as it does not require fully synchronous communications. To demonstrate its efficiency, we deploy it in 10 countries over 5 continents and demonstrate that its performance increases quasi-linearly with the number of shards as it reaches close to 14,000 TPS on only 8 shards., OASIcs, Vol. 101, 5th International Symposium on Foundations and Applications of Blockchain 2022 (FAB 2022), pages 6:1-6:17

Advances in blockchain technology have made a significant impact on a wide range of research areas due to the features such as transparency, decentralization and traceability. With the explosive growth of blockchain transactions, there has been a growing interest in improving the scalability of blockchain network. Sharding is one of the methods to solve this scalability problem by partitioning the network into several shards so that each shard can process the transactions in parallel. Ethereum places each transaction statically on a shard based on its account address without considering the complexity of the transaction or the load generated by the transaction. This causes the transaction utilization on each shard to be uneven, which makes the transaction throughput of the network decrease. This paper formulates this problem as a multi-dimensional knapsack problem (MKP) and proposes a heuristic algorithm called GARET. The GARET dynamically relocates the transaction load of each shard based on gas consumption to maximize the transaction throughput. Ethereum gas is a unit that represents the amount of computational effort needed to execute operations in a transaction. Benchmarking results show that GARET outperforms existing techniques by up to 12% in transaction throughput and decreases the makespan of transaction latency by about 74% under various conditions. It is also shown that the relocation overhead is minimal and does not affect the overall performance.

In distributed systems multiple computers interact with each other over a network to perform data processing operations parallel over a network. Distributed systems can be setup depending on the application and its data with different combination of replications and then the partitioning of the data can be performed using sharding technique to enhance the scalability, consistency and fault-tolerance issues. The gaps identified in existing sharding techniques are, it is difficult to determine which particular shard or partition need to be searched to retrieve the relevant document matching to the user query. And also to rank the documents which has highest relevance to the user query. The main goal is to develop a mechanism to handle large scale data processing and searching operations using efficient sharding technique. The proposed paper aims to enhance the performance of distributed processing systems by applying effective shard partitioning and efficient shard selection techniques and perform the comparative study analysis of shard selection techniques considering precision, MAP and cost measures. The results interpret that sharding technique provides efficient mechanism to handle large scale data. Effectiveness and efficiency of shard selection techniques interpret that Rank-S algorithm performance better compared to CORI and Redde algorithms by reducing the overall cost by 28%. Sharding technique is an efficient mechanism to process big data, and provides better scalability and fault-tolerance.

In various applications such as trajectory tracking in mobile social networks and online recommendation systems, the massive raw data are often incomplete due to various unpredictable or unavoidable reasons. Matrix completion algorithms are effective for reconstructing two-dimensional data, but sending raw data containing personal, sensitive information to cloud computing nodes for matrix completion may lead to privacy exposure issue. The homomorphic matrix completion is a promising approach to perform matrix completion while preserving privacy. However, CPU-based homomorphic matrix completion has low performance, making it impractical to process multiple or large-scale data completion tasks in real-time. In this paper, we propose a high-performance homomorphic matrix completion scheme by exploiting commodity GPUs (Graphics Processing Units) that are widely available in HPC servers and cloud computing nodes. First, we design and implement a baseline GPU-based homomorphic matrix completion, and propose techniques to optimize memory accesses, GPU utilization, and communications. Second, we propose a shard mode for large-scale matrices exceeding GPU memory capacity. Third, we propose a multi-GPU mode to fully utilize multiple GPUs in computing nodes. Experiment results show that the proposed scheme is both fast and accurate. On matrices of varying sizes, the proposed scheme running on a single Tesla V100 GPU achieves up to 116.23× speedups over the CPU MATLAB implementation running on dual Xeon CPUs. The multi-GPU mode achieves up to 1.84× speedups on two GPUs versus on a single GPU. For large-scale matrices, the shard mode achieves up to 174.92× speedups on a single GPU over the CPU MATLAB implementation on two CPUs, and further achieves up to 1.35× speedups when running on two GPUs using the multi-GPU mode.

Sharding technology can be used to improve the scalability of a blockchain system. The suitable placement of transactions into shards can reduce cross-shard ones, so as to improve system throughput and reduce delay. An optimization model for transaction placement to blockchain shards is proposed. Based on time analysis and a transaction flow graph, the fitness is computed to decide which shard is the most suitable one to place a transaction. The experimental results show that the proposed model can effectively reduce cross-shard transactions and reduce transaction delay.

With the rapid growth of the Web scale, large scale search engines have to set up a huge number of machines to place the index files of the Web contents. The index files are normally divided into smaller index shards which are often replicated so that queries can be processed in parallel. We observe from real systems that the index shard replication strategy could have a significant impact on the resource usage. In this paper, we investigate the index shard replication problem with the goal of minimizing the resource usage in search engine datacenters. We consider both the offline version and online version of the problem, and formulate the problems as non-linear integer programming problems. We propose several heuristic algorithms to approximate the optimal solution. The proposed algorithms are evaluated by extensive experiments using both synthetic data and real data from commercial search engines. The results demonstrate the effectiveness of the proposed algorithms. Our work also yields many insights about the impact of different input properties on the performance of each algorithm. We believe that this paper will provide valuable guidance to the design of the index shard replication strategy in practice.

Most of the existing smart-contract-based cryptocurrencies, such as Ethereum, use an account-based ledger. However, while the account-based model is advantageous for the efficient use of smart contracts and the increased exchangeability of cryptocurrencies, it is not well-suited to the parallel execution of smart contracts. However, unspent transaction output (UTXO)-based cryptocurrencies such as Bitcoin are advantageous for parallel cryptocurrency transfers but not well-suited to smart contracts. In this paper, we propose a hierarchical multi-blockchain system that uses multiple pairs of sidechain and dual-sidechains extended by independent block mining in their blockchain networks and a mainchain to control the branching and connection process of sidechains and dual sidechains. In the proposed method, one pair of a sidechain and dual sidechain forms one shard. The proposed method uses multiple shards to execute cryptocurrency transfers and smart contracts in parallel. In addition, the proposed model uses an accoutchain to record the resulting state changes generated by smart contract executions in each shard and securely share them with all other nodes. The proposed method uses a modifiable blockchain structure for the accountchain to obtain the database to record the smart contract execution results in each shard in as small and secure a manner as possible to ensure that all nodes trust the recorded results without executing smart contracts themselves. To examine the validity of the proposed method, we conducted a threat analysis of the proposed method by examining possible attacks in various scenarios as a thought experiment. This threat analysis concludes that the proposed blockchain system can execute smart contracts in parallel while keeping the concurrency in resulting state changes secure.

This work investigates the technology of pottery production at the Chalcolithic site of Radovanu-La Muscalu (first half of the fifth millennium BCE), in southern Romania. The excavation of this settlement yielded a rich and well-contextualised archaeological assemblage that represents the last phases of development of Boian material culture, a Chalcolithic phenomenon that spread throughout the lower Danube area at the end of the sixth millennium BCE and the first half of fifth millennium BCE. To carry out this investigation, a total of forty-nine ceramic samples, representative of the different pottery types and chronological horizons attested at this site, together with geological samples collected around Radovanu, were selected. These were analysed using a multi-pronged scientific approach including ceramic petrography, X-ray diffraction, and scanning electron microscopy. This interdisciplinary study elucidates different aspects of the local pottery production and indicates that ceramics at Radovanu were manufactured according to a conservative tradition that consistently made use of grog tempering throughout all phases in which this settlement was inhabited. The results also show that at Radovanu, regional technological traits coexisted with elements that were widely applied in the Balkan region at that time, such as graphite-painted decoration. This investigation together with the identification of potential non-local productions well illustrates the complex interrelationships between the Boian phenomenon and the neighbouring material cultures in the Balkans.

Sharding is widely used to scale an application. Despite a decade of effort to build generic sharding frameworks that can be reused across different applications, the extent of their success remains unclear. We attempt to answer a fundamental question: what barriers prevent a sharding framework from getting adopted by the majority of sharded applications? We analyze hundreds of sharded applications at Facebook and identify two major barriers: 1) lack of support for geo-distributed applications, which account for most of Facebook's applications, and 2) inability to maintain application availability during planned events such as software upgrades, which happen a1000 times more frequently than unplanned failures. A sharding framework that does not help applications to address these fundamental challenges is not sufficiently attractive for most applications to adopt it. Other adoption barriers include the burden of supporting many complex applications in a one-size-fit-all sharding framework and the difficulty in supporting sophisticated shard-placement requirements. Theoretically, a constraint solver can handle complex placement requirements, but in practice it is not scalable enough to perform near-realtime shard placement at a global scale. We have overcome these adoption barriers in Facebook's sharding framework called Shard Manager. Currently, Shard Manager is used by hundreds of applications running on over one million machines, which account for about 54% of all sharded applications at Facebook.

Blockchain has become a promising technology in distributed systems in recent years, but scalability remains a major problem. The traditional approach to scalability, namely sharding, does not solve the problem easily because the process of interleaving blocks stored in different shards to create a unified master ledger introduces overhead. This paper examines two techniques for interleaving the shards of permissioned blockchains, which we refer to as strong temporal coupling and weak temporal coupling. We implement these techniques in a prototype system with a Bitcoin-like transaction structure, using the EPaxos consensus protocol for transaction ordering. Our experimental results show that strong coupling can achieve lower latency as compared to weak coupling but same level of peak throughput. However, strong coupling requires all shards to grow at the same rate, and cannot tolerate any shard failure. In contrast, the higher latency of weak coupling is because of the consensus strategy it uses to order the blocks. However, if shard failure occurs, weak coupling can still make progress without stalling the whole system.

The design of sharding aims to solve the scalability challenge in a blockchain network. Typically, by splitting the whole blockchain network into smaller shards, the transaction throughput can be significantly improved. However, distributing fewer attesting nodes for transactions in a shard may cause higher security risks. This paper analyzes the security level of transaction verification in different types of shards and transactions. The analyzed result indicates that the size of shards and validating nodes number may influence the transaction security in shards. And the random distribution of attesting nodes can reduce such influence and improve the reliability of consensus in shards.

Sharding technology has been rapidly developed to improve blockchain scalability in recent years. However, in practical applications, a blockchain sharding system may face such issues as malicious node attacks and large cross-shard communications. An optimization method is thus demanded to improve their sharding performance. Based on the status and behavioral data of shard nodes stored in a trusted blockchain, can node trust be calculated and used to improve a sharding-based blockchain’s security? To answer this question, this work presents a sharding model to optimize fault tolerance ratio, the number of cross-shard transactions, and shard trust. Its application to a blockchain system can help it improve the credibility of its shard transaction processing and reduce its transaction delay, thereby enhancing the scalability of over the state of the art. Some open issues are discussed.

Contributing to mining algorithms using low-powered devices is one of the principal concerns in the growth of the Internet of Things (IoT) paradigm. To address this limitation, we introduce a new concept of allocating mining devices based on their computation power in sharding-based blockchain systems for the IoT. This study proposes two scenarios based on different ways of partitioning the transactions into the disjoint shards: Scenario 1 (Number of processors are equally distributed among shards); Scenario 2 (Number of processors are distributed among shards using computation power-based criterion). We also adopt an adversary model to obtain the fraction of participating processors that must be compromised by the adversaries to compromise our proposed method of sharding based blockchain networks. Our results demonstrate that the total computation cost of the entire system increases when the computing cost for verifying signatures and the number of shards increases. We observe that the total reward of each processor increases when the number of less powerful processors increases. The total number of mined processors of each shard decreases with a high number of less powerful processors. Thus, the total reward of the entire network is distributed among a few processors. We demonstrate that our new method of allocating mining devices based on their computation power (Scenario 2) provides fair reward distribution. Additionally, when the total number of processors in a network is higher than 500, each processor's total reward increment shows an insignificant change. However, when the total number of processors is lower than 500, each processor's total reward shows an upward linear trend and contradicts the assumption of Scenario 1. Thus, our results show that the proposed model (Scenario 2) is more suitable for allocating mining devices based on their computation power in sharding-based blockchain systems for IoT, especially devices with different computational capabilities or processors with selfish behavior. This concept motivates powerful processors to verify and validate new transactions to be added to the blockchain.

Sharding is one of the most promising and practical methods to achieve horizontal scalability of blockchain networks. However, the increasing number of cross-shard transactions in blockchain sharding protocols may degrade the system throughput. In this paper, we investigate how to distribute users properly in the shard-based blockchains to boost the system transaction performance. We first build an open Jackson queueing network model to capture users' transaction dynamics on shards. Then we cast users' interactions as a shard-based blockchain game, wherein each user aims to minimize its transaction confirmation time and transaction fee. We investigate the equilibrium of the game, and design a polynomial-time algorithm to find efficient equilibria with good system performance. We further design a novel sharding protocol with dynamic user distribution for the permissionless blockchain, and the protocol can maintain good performance in long-term dynamic environment. Extensive numerical results using realistic blockchain transaction data demonstrate that the proposed algorithm and the designed protocol can achieve superior performance for shard-based blockchains.

In order to demonstrate acceleration of electrons to relativistic scales by an on chipdielectric laser accelerator (DLA), a ponderomotive focusing scheme capable of capturingand transporting electrons through nanometer-scale apertures over extended interactionlengths has been proposed. We present a Matlab-based numerical code (SHarD) utilizinga spatial harmonic expansion of the fields within the dielectric structure to simulate theevolution of the beam phase space distribution in this scheme. The code can be usedto optimize key-parameters for the accelerator performance such as the final energy,transverse spot size evolution and total number of electrons accelerated through currentlyfabricated structures. Eventually, the simulation model will be applied to inform thephase mask profile to be added to a pulse front tilt drive laser pulse using a liquid crystalmask in the experimental setup being assembled at UCLA Pegasus Laboratory., 12 pages, 4 figures, 1 table, Manuscript Accepted at NIM-A

We propose Shard Scheduler, a system for object placement and migration in account-based sharded blockchains. Our system calculates optimal placement and decides on object migrations across shards. It supports complex multi-account transactions caused by smart contracts. Placement and migration decisions made by Shard Scheduler are fully deterministic, verifiable, and can be made part of the consensus protocol. Shard Scheduler reduces the number of costly cross-shard transactions, ensures balanced load distribution and maximizes the number of processed transactions for the blockchain as a whole. To this end, it leverages a novel incentive model motivating miners to maximize the global throughput of the entire blockchain rather than the throughput of a specific shard. In our simulations, Shard Scheduler can reduce the number of costly cross-shard transactions by half while ensuring equal load and increasing throughput more than 2 fold when using 60 shards. We also implement and evaluate Shard Scheduler on Chainspace, more than doubling its throughput and reducing user-perceived latency by 70% when using 10 shards.

As deep learning becomes more expensive, both in terms of time and compute, inefficiencies in machine learning (ML) training prevent practical usage of state-of-the-art models for most users. The newest model architectures are simply too large to be fit onto a single processor. To address the issue, many ML practitioners have turned to model parallelism as a method of distributing the computational requirements across several devices. Unfortunately, the sequential nature of neural networks causes very low efficiency and device utilization in model parallel training jobs. We propose a new form of "shard parallelism" combining task and model parallelism, then package it into a framework we name Hydra. Hydra recasts the problem of model parallelism in the multi-model context to produce a fine-grained parallel workload of independent model shards, rather than independent models. This new parallel design promises dramatic speedups relative to the traditional model parallelism paradigm., 2 pages, 3 figures. 1st place winner of ACM SIGMOD '21 Student Research Competition. Appeared in ACM SIGMOD/PODS '21 Proceedings

Scalability is one of the main roadblocks to business adoption of blockchain systems. Despite recent intensive research on using sharding techniques to enhance the scalability of blockchain systems, existing solutions do not efficiently address cross-shard transactions. In this paper, we introduce SharPer, a permissioned blockchain system that improves scalability by clustering (partitioning) the nodes and assigning different data shards to different clusters where each data shard is replicated on the nodes of a cluster. SharPer supports both intra-shard and cross-shard transactions and processes intra-shard transactions of different clusters as well as cross-shard transactions with non-overlapping clusters simultaneously. In SharPer, the blockchain ledger is formed as a directed acyclic graph where each cluster maintains only a view of the ledger. SharPer also incorporates a flattened protocol to establish consensus among clusters on the order of cross-shard transactions. The experimental results reveal the efficiency of SharPer in terms of performance and scalability especially in workloads with a low percentage of cross-shard transactions.

Zero-knowledge set membership (ZKSM) proof is widely used in blockchain to enable private membership attestation. However, existing mechanisms do not fully consider dynamic issues in the blockchain scenario. Particularly, frequent addition/removal of set elements, not only brings the significant cost to keep public parameters up to date to provers and verifiers but also affects mechanism efficiency (e.g., generation time of the proof and verification, etc.). In this paper, we propose DIV to shard elements on blockchain into independent subsets with the same cardinality to reduce the effect of dynamic issues. However, due to the diverse proof frequency, an improper element-set assignment can result in frequently used elements being easily inferred and corrupted. Thus, we formalize the assignment problem under both element addition and removal cases as two optimization problems and prove their NP-hardness. For each problem, we consider two cases if each element proof frequency is known in advance by the set maintainer or not, and propose solutions with theoretical guarantees. We implement DIV on both Merkle tree and RSA-based ZKSM mechanisms to evaluate its efficiency and effectiveness and apply DIV on a ZKSMbased application named zkSync to demonstrate its applicability. Results show that DIV can achieve O(1) time/space cost on ZKSM under dynamic situations while protecting the information about frequently used elements. It also notably reduces the system latency of zkSync.

Sharding is a way to solving the scalability problem in the blockchain. The sharding method uses traditional consensus algorithms, such as PBFT. In traditional consensus algorithms, network nodes can register blocks by working together. In these algorithms, an important issue is how to motivate group nodes to participate in the consensus algorithm so as not to cause collusion or free-riding. In this paper, we first examine the previous solutions that try to solve the incentive mechanism in sharding blockchain and show that these solutions do not consider the leader's role in involving other nodes in the consensus algorithm. By the game theory, we show that the leader for increasing his benefit prevents some nodes from participating in the consensus algorithm, which creates collusion. We propose a solution to motivate group nodes in the sharding method and by the game theory prove that this solution is incentive compatible and also does not causes any collusion.

The Miss Ratio Curve (MRC) describes the cache miss ratio as a function of the cache size. It has various shapes that represent the data access behaviors of workloads in the cache. MRC is an effective tool to guide cache partitioning, but its real-time construction is challenging. Miniature Simulation is a novel approach that constructs MRCs for non-stack algorithms in real time, via feeding a small number of sample references to multiple mini caches simultaneously to get the miss ratios. However, while using the Miniature Simulation, the size and number of mini-caches are difficult to set before the program runs. First, it may set too many mini-caches and cause repeated simulations. Second, it may miss some important cache sizes and consequently construct a less precise shape of MRC and result in incorrect cache partitioning. To address this problem, we propose DFShards, an adaptive cache shards (mini-caches) configuration approach based on program access patterns. The key idea is to dynamically adjust the configuration of the cache shards, including the number of the total cache shards and the size of each cache shard, based on the access behaviors to reflect changes in workload to build an precise MRC, thereby achieving better cache partitioning and overall performance. Our extensive experiments show that DFShards can construct precise MRCs in real-time during program running. Compared to the state-of-the-art approaches, it can save up to 47% of the cache space for MRC constructions while increasing the cache hit ratio by up to 17%.

Most present reconfiguration methods in sharding blockchains rely on a secure randomness, whose generation might be complicated. Besides, a reference committee is usually in charge of the reconfiguration, making the process not decentralized. To address the above issues, this paper proposes a secure and decentralized shard reconfiguration protocol, which allows each shard to complete the selection and confirmation of its own shard members in turn. The PoW mining puzzle is calculated using the public key hash value in the member list confirmed by the last shard. Through the mining and shard member list commitment process, each shard can update its members safely and efficiently once in a while. Furthermore, it is proved that our protocol satisfies the safety, consistency, liveness, and decentralization properties. The honest member proportion in each confirmed shard member list is guaranteed to exceed a certain safety threshold, and all honest nodes have an identical view on the list. The reconfiguration is ensured to make progress, and each node has the same right to participate in the process. Our secure and decentralized shard reconfiguration protocol could be applied to all committee-based sharding blockchains.

Blockchain performance cannot meet the requirement nowadays. One of the crucial ways to improve performance is sharding. However, most blockchain sharding research focuses on public blockchain. As for consortium blockchain, previous studies cannot support high cross-shard efficiency, cross-contract flexibility, shard availability, and strict transaction atomicity, which are the essential requirements but also the challenges in consortium blockchain systems. Facing these challenges, we propose Meepo, a systematic study on sharded consortium blockchain. Meepo enhances cross-shard efficiency via the cross-epoch and cross-call. Moreover, a partial cross-call merging strategy is designed to handle the multi-state dependency in contract calls, achieving cross-contract flexibility. Meepo employs a replay-epoch to ensure strict transaction atomicity, and it also uses a backup algorithm called shadow shard based recovery to improve the shard robustness. We implement Meepo on the AliCloud, using 32 shards in maximum, achieving more than 120,000 cross-shard TPS under the workload of 100,000,000 asset transactions.

Analytic DBMSs optimized for query interactivity commonly push the computation down to storage nodes, thus avoiding large network transfers and keeping query execution wall-time to a minimum. In these systems, data is sharded and stored locally by cluster nodes, which must all participate in query execution. As the system scales-out, hardware failures and other non-deterministic sources of tail latency start to dominate, to a point where query latency and success ratio increasingly violate the system’s SLA. We refer to this tipping point as the system’s scalability wall, when sharding data between more nodes only worsens the problem.This paper describes how an analytic DBMS optimized for low-latency queries can breach the scalability wall by sharding different tables to different subsets of cluster nodes — a strategy we call partial sharding — and reduce the query fan-out. Because partial sharding requires the DBMS to implement many tedious and complex shard management tasks, such as shard mapping, load balancing and fault tolerance, this paper describes how a database system can leverage an external general-purpose shard management service for such tasks. We present a case study based on Cubrick, an in-memory analytic DBMS developed at Facebook, highlighting the integration points with a shard management framework called Shard Manager. Finally, we describe the many design decisions, pitfalls and lessons learned during this process, which eventually allowed Cubrick to scale to thousands of nodes.

Diffusion-weighted MRI measures the direction and scale of the local diffusion process in every voxel through its spectrum in ${q}$ -space, typically acquired in one or more shells. Recent developments in microstructure imaging and multi-tissue decomposition have sparked renewed attention in the radial ${b}$ -value dependence of the signal. Applications in motion correction and outlier rejection, therefore, require a compact linear signal representation that extends over the radial as well as angular domain. Here, we introduce SHARD, a data-driven representation of the ${q}$ -space signal based on spherical harmonics and a radial decomposition into orthonormal components. This representation provides a complete, orthogonal signal basis, tailored to the spherical geometry of ${q}$ -space, and calibrated to the data at hand. We demonstrate that the rank-reduced decomposition outperforms model-based alternatives in human brain data, while faithfully capturing the micro- and meso-structural information in the signal. Furthermore, we validate the potential of joint radial-spherical as compared with single-shell representations. As such, SHARD is optimally suited for applications that require low-rank signal predictions, such as motion correction and outlier rejection. Finally, we illustrate its application for the latter using outlier robust regression.

Software defined networking controller platforms support clustering architectures to provide scalability and availability to large-scale carrier-grade networks. OpenDaylight, a well-known open source project, provides a clustering and distributed datastore architecture. The datastore is distributed into shards such that a subset of the shards can be located in any cluster member. To guarantee strong consistency in the datastore so all shard replicas have the same value, only the shard leader is responsible for accepting data updates. Even though OpenDaylight supports a distributed shard leader election algorithm, elected leaders are not distributed over the cluster members due to a lack of centralized control. Since update requests for the datastore are inherently concentrated in the leader, this results in unbalanced CPU usage of the controller cluster, as well as degradation of throughput in the distributed datastore. In this paper, we propose a shard leader distribution algorithm that maximizes the throughput of the distributed datastore by distributing the shard leaders to cluster members as evenly as possible. The shard leader distribution algorithm restricts the maximum number of leaders that a cluster member may have by monitoring the status of the shards within the cluster members. In the performance evaluation, we prove that shard leader distribution significantly improves the throughput of the datastore by load balancing data update requests to the cluster.

With the popularity of knowledge graphs growing rapidly, large amounts of RDF graphs have been released, which raises the need for addressing the challenge of distributed subgraph matching queries. In this paper, we propose an efficient distributed method to answer subgraph matching queries on big RDF graphs using MapReduce. In our method, query graphs are decomposed into a set of stars that utilize the semantic and structural information embedded RDF graphs as heuristics. Two optimization techniques are proposed to further improve the efficiency of our algorithms. One algorithm, called RDF property filtering, filters out invalid input data to reduce intermediate results; the other is to improve the query performance by postponing the Cartesian product operations. The extensive experiments on both synthetic and real-world datasets show that our method outperforms the close competitors S2X and SHARD by an order of magnitude on average.

Blockchains guarantee data integrity through consensus of distributed ledgers based on multiple validation nodes called miners. For this reason, any blockchain system can be critically disabled by a malicious attack from a majority of the nodes (e.g., 51% attack). These attacks are more likely to succeed as the number of nodes required for consensus is smaller. Recently, as blockchains are becoming too large (making them difficult to store, send, receive, and manage), sharding is being considered as a technology to help improve the transaction throughput and scalability of blockchains. Sharding distributes block validators to disjoint sets to process transactions in parallel. Therefore, the number of validators of each shard group is smaller, which makes shard-based blockchains more vulnerable to 51% attacks than blockchains that do not use sharding. To solve this problem, this paper proposes a trust-based shard distribution (TBSD) scheme that assigns potential malicious nodes in the network to different shards, preventing malicious nodes from gaining a dominating influence on the consensus of a single shard. TBSD uses a trust-based shard distribution scheme to prevent malicious miners from gathering in on one shard by integration of a trust management system and genetic algorithm (GA). First, the trust of all nodes is computed based on the previous consensus result. Then, a GA is used to compute the shard distribution set to prevent collusion of malicious miners. The performance evaluation shows that the proposed TBSD scheme results in a shard distribution with a higher level of fairness than existing schemes, which provides an improved level of protection against malicious attacks.

In the last few years, blockchain technologies have come to the forefront of the research and industrial communities as they bring potential benefits for many industries. Even though the blockchain is a safe, reliable, and innovative mechanism, current blockchain solutions have fatal drawbacks of non-supervision and huge computational overhead. Therefore, they cannot be directly applied for real world operations. To resolve this problem, a novel model is presented in this study where the key idea is to split the transactions among multiple shards while processing them in parallel. To achieve the maximum system efficiency through the shard mechanism, we focus on the cooperative game theory. Based on the egalitarian bargaining solution, total transactions per each time period are divided for each shard. According to the proportional bargaining solution, assigned transactions in each individual shard are validated by blockchain nodes in a distributed manner. The main advantage of our two-phase bargaining game model is to provide an axiom-based strategic solution for the shard-based consensus problem while dynamically responding to the current blockchain network conditions. The numerical simulation results show that the effectiveness and efficiency of our game based approach by comparing the existing state-of-the-art blockchain control schemes. In the conclusion, we present our conclusions and provide important future research directions based on the combination of blockchain with other technologies.

Despite the advantages of Federated Learning (FL), such as devolving model training to intelligent devices and preserving data privacy, FL still faces the risk of the single point of failure and attack from malicious participants. Recently, blockchain is considered a promising solution that can transform FL training into a decentralized manner and improve security during training. However, traditional consensus mechanisms and architecture for blockchain can hardly handle the large-scale FL task due to the huge resource consumption, limited throughput, and high communication complexity. To this end, this paper proposes a two-layer blockchain-driven FL framework, called as ChainsFL, which is composed of multiple Raft-based shard networks (layer-l) and a Direct Acyclic Graph (DAG)-based main chain (layer-2) where layer-l limits the scale of each shard for a small range of information exchange, and layer-2 allows each shard to update and share the model in parallel and asynchronously. Furthermore, FL procedure in a blockchain manner is designed, and the refined DAG consensus mechanism to mitigate the effect of stale models is proposed. In order to provide a proof-of-concept implementation and evaluation, the shard blockchain base on Hyperledger Fabric is deployed on the self-made gateway as layer-l, and the self-developed DAG-based main chain is deployed on the personal computer as layer-2. The experimental results show that ChainsFL provides acceptable and sometimes better training efficiency and stronger robustness comparing with the typical existing FL systems.

In this paper, we present a highly consistent, fault-tolerant distributed key-value store. It adapts the raft consensus algorithm broadly in the system and supports concurrent transactions across shards. Each operation on the store is handled by a set of co-ordinators as a raft group. The store’s keys are partitioned across shards, and each shard maintains multiple replicas forming their raft groups.

Sharding is considered to be the most promising solution to overcome and to improve the scalability limitations of blockchain networks. By doing this, the transaction throughput increases, at the same time compromises the security of blockchain networks. In this paper, a probability distribution model is proposed to analyze this trade-off between scalability and security of sharding-based blockchain networks. For this purpose hypergeometric distribution and Chebyshev's Inequality are mainly used. The upper bounds of hypergeometric distributed transaction processing and failure probabilities for shards are mainly evaluated. The model validation is accomplished with Class A (Omniledger, Elastico, Harmony, and Zilliqa), and Class B (RapidChain) sharding protocols. This validation shows that Class B protocols have a better performance compared to Class A protocols. The proposed model observes the transaction processing and failure probabilities are increased when shard size is reduced or the number of shards increased in sharding-based blockchain networks. This trade-off between the scalability and the security decides on the shard size of the blockchain network based on the real-world application and the blockchain platform. This explains the scalability trilemma in blockchain networks claiming that decentralization, scalability, and security cannot be met at primary grounds. In conclusion, this paper presents a comprehensive analysis providing essential directions to develop sharding protocols in the future to enhance the performance and the best-cost benefit of sharing-based blockchains by improving the scalability and the security at the same time.

Throughput performance is a critical issue in blockchain technology, especially in blockchain sharding systems. Although sharding proposals can improve transaction throughput by parallel processing, the essence of each shard is still a small blockchain. Using serial execution of smart contract transactions, performance has not significantly improved, and there is still room for improvement. A smart contract concurrent execution strategy based on concurrency degree optimization is proposed for performance optimization within a single shard. This strategy is applied to each shard. First, it characterizes the conflicting contract feature information by executing a smart contract, analyzing the factors that affect the concurrent execution of the smart contracts, and clustering the contract transaction. Second, in shards with high transaction frequency, considering the execution time, conflict rate, and available resources of contract transactions, finding a serializable schedule of contract transactions by redundant computation and a Variable Shadow Speculative Concurrency Control (SCC-VS) algorithm for smart contract scheduling is proposed. Finally, experimental results show that the strategy increases the concurrency of smart contract execution by 39% on average and the transaction throughput of the whole system by 21% on average.

Diffusion MRI offers a unique probe into neural microstructure and connectivity in the developing brain. However, analysis of neonatal brain imaging data is complicated by inevitable subject motion, leading to a series of scattered slices that need to be aligned within and across diffusion-weighted contrasts. Here, we develop a reconstruction method for scattered slice multi-shell high angular resolution diffusion imaging (HARDI) data, jointly estimating an uncorrupted data representation and motion parameters at the slice or multiband excitation level. The reconstruction relies on data-driven representation of multi-shell HARDI data using a bespoke spherical harmonics and radial decomposition (SHARD), which avoids imposing model assumptions, thus facilitating to compare various microstructure imaging methods in the reconstructed output. Furthermore, the proposed framework integrates slice-level outlier rejection, distortion correction, and slice profile correction. We evaluate the method in the neonatal cohort of the developing Human Connectome Project (650 scans). Validation experiments demonstrate accurate slice-level motion correction across the age range and across the range of motion in the population. Results in the neonatal data show successful reconstruction even in severely motion-corrupted subjects. In addition, we illustrate how local tissue modelling can extract advanced microstructure features such as orientation distribution functions from the motion-corrected reconstructions., revision. 15 pages, 10 figures, 2 tables

Volcanic ash (tephra) horizons represent powerful chronological and stratigraphic markers: rapid and widespread deposition allows for correlation of geological records in time and space. Recent analytical advances enable identification of invisible ash (cryptotephra) up to thousands of kilometers from its volcanic source. This momentum has greatly expanded the reach and potential of tephrochronology: some deposits can now be traced across continents and oceans. However, the laborious laboratory procedures required to identify tephra horizons in geological archives hold back the pace of progress. By allowing the rapid visualization of ash at micrometer (µm) scales, computed tomography (CT) holds great promise to overcome these restrictions. In this study, we further demonstrate the potential of this tool for the tephra community with experimental results and applications on conventionally analyzed archives. A custom-made scanner helps us strike a balance between the convenience of whole-core medical scanners and the µm-resolution of micro-CT systems. Using basic image processing tools that can be readily mastered by tephrochronologists, we identified invisible horizons down to ∼500 shards in synthetic cores. In addition, procedures for the removal of image artifacts can be used to visualize other paleoenvironmental indicators such as bioturbation burrows, ice rafted debris or mineral dust. When applied on segments of manually counted natural archives, our approach captures cryptic glass shard maxima down to ∼300 shards/cm3. We also highlight the value of CT to help optimize sampling strategies by identifying micrometer-scale ash horizons that were not detected in shard count profiles. In conclusion, this work helps broaden the applicability of CT as a promising frontier in tephrochronology that can advance the field by optimizing the efficiency and accuracy of isochron detection.