Search

This article explores and names some of the practical approaches used in the implementation of large-scale mission-critical applications in a world that rejects distributed transactions. Topics include the management of fine-grained pieces of application data that may be repartitioned over time as the application grows. Design patterns support sending messages between these repartitionable pieces of data.

Software Transactional Memory (STM) has recently emerged as a promising synchronization abstraction for multicore architectures. State-of-the-art STM algorithms, however, suffer from performance challenges due to contention and spinning on locks during the transaction commit phase. In this paper, we introduce Remote Transaction Commit (or RTC), a mechanism for executing commit phases of STM transactions. RTC dedicates server cores to execute transactional commit phases on behalf of application threads. This approach has two major benefits. First, it decreases the overheads of spinning on locks during commit, such as the number of cache misses, blocking of lock holders, and CAS operations. Second, it enables exploiting the benefits of coarse-grained locking algorithms (simple and fast lock acquisition, reduced false conflicts) and bloom filter-based algorithms (concurrent execution of independent transactions). Our experimental study on a 64-core machine with four sockets shows that RTC solves the problem of performance degradation due to spin locking on both micro-benchmarks (red-black trees), and macro-benchmarks (STAMP), especially when the commit phase is relatively long and when thread contention increases.

Distributed database is a technique in which copy of a database is replicated over the network. A distributed database appears to a client as a single database however in all actuality it is an arrangement of databases distributed on numerous computers or servers. Replication of data in a distributed database system is for enhancing data availability and making data more fault tolerant. Formal methods are used for insight knowledge and refinement of technique to be used and formal methods also helps in understanding how to accomplish those objectives. In our model a coordinator site finds a site with a largest replica number and then broadcast its updated replica to all other sites in a distributed environment.

Distributed Transactional Memory (DTM) is a recent but promising model for programming distributed systems. It aims to present programmers with a simple to use distributed concurrency control abstraction (transactions), while maintaining performance and scalability similar to distributed fine-grained locks. Any complications usually associated with such locks (e.g., distributed deadlocks) are avoided. In this article, we analyze the use of open nesting in the DTM setting. We extend two DTM algorithms, Transactional Forwarding Algorithm (TFA) and SCORe with support for open nested transactions and we implement them into two frameworks for running distributed transactions, such as Hyflow and Infinispan. We discuss the mechanisms and performance implications of such nesting, and identify the cases where using open nesting is warranted and the relevant parameters for such a decision. To the best of our knowledge, our work also contributes the first ever implementations of DTM systems with support for open-nested transactions.

Partition technology is the key step to realize the extensional architecture in the cloud and support the data placement on multiple nodes. This paper proposes a multi-tenant data partition model and algorithm for SaaS (Software as a Service) application. It solves the problem that data partitions would produce lots of distributed transactions caused by the existing cloud data management. The management is unconscious of SaaS tenants during the transformation from a single node to multiple nodes in the cloud to obtain the dynamic extension of the system’s scale. With the increase of tenants and data, the single node becomes the bottleneck of the whole system. Fortunately, the scale of the whole system can be expanded by data partition. This paper puts forward a multi-tenant data partition model with three-layer structure: Tenant layer, Relevance, Group layer and Tenant Partition layer. Furthermore, we propose the concepts of Relevance, Relevance Value and Relevance Matrix. The customized tables for one tenant accessed by the same transactions can form a minimum high-relevance granularity based on the Relevance Group algorithm. Then we construct an abstracted graph, where group is the basic unit and transaction accessing is weight. Through the Stoer–Wagner algorithm, the multi-tenant partition with group as granularity is obtained. The partition algorithm proposed in this paper enables the greatest reduction of distributed transactions between partitions while realizing the dynamic extension on multiple nodes for multi-tenant data based on shared storage. Experiments show that the number of distributed transactions is reduced dramatically compared with other data partition techniques. We also prove that the SaaS applications run at high efficiency.

This paper describes a remarkably simple deterministic (not probabilistic) contention-management algorithm for guaranteeing the forward progress of transactions - avoiding deadlocks, livelocks, and other anomalies. The transactions must be finite (no infinite loops), but on each restart, a transaction may access different shared-memory locations. The algorithm supports irrevocable transactions as long as the transaction satisfies a simple ordering constraint. In particular, a transaction that accesses only one shared-memory location is never aborted. The algorithm is suitable for both hardware and software transactional-memory systems. It also can be used in some contexts as a locking protocol for implementing transactions "by hand." HighlightsA remarkably simple algorithm can guarantee the forward progress of transactions.The algorithm supports irrevocable transactions.The algorithm is suitable for hardware or software transactional-memory systems.The algorithm can be used as a locking protocol.

As new applications of cloud data management system (CDMS) such as online games, cooperation edit, social network, and so on, are increasing, transaction processing capabilities for CDMS are required. Several transaction processing methods for cloud data management system (CDMS) have been proposed. However, existing transaction processing methods have some problems. Some of them provide limited transaction processing capabilities. Some of them are hard to be integrated with existing CDMSs. In this paper, we proposed a new concurrency control method to support transaction processing capability for CDMS to solve these problems. The proposed method was designed and implemented based on Spark, an in-memory distributed processing framework. It uses RDD (Resilient Distributed Dataset) model to provide fault tolerant to data in the main memory. In our proposed method, database stored in CDMS is loaded to main memory managed by Spark. The loaded data set is then transformed to RDD. In addition, we proposed a multi-version concurrency control method through immutable characteristics of RDD. Finally, we performed experiments to show the feasibility of the proposed method.

Transactional Memory (TM) is no longer just an academic interest as industry has started to adopt the idea in its commercial products. In this paper, we propose Dynamic Transaction Issue (DTI), a new scheme that can be easily implemented on top of existing Hardware TM (HTM) systems, provided additional messages. Instead of wasting power and energy in transaction aborts, Dynamic Transaction Issue puts a processor core into a low-power state when there is a reasonable suspicion that the current transaction running on it will be aborted soon in the future. We have implemented Dynamic Transaction Issue on a cycle-accurate simulator of a multicore processor system with out-of-order superscalar cores, augmented with a power package and a TM package which add accurate dynamic power estimates and a TM framework to the simulator. Our simulation results show that Dynamic Transaction Issue can achieve energy savings up to 37% from the energy consumption of a base machine with no mechanism to suppress useless aborts. We also compare Dynamic Transaction Issue with various alternative hardware TM mechanisms.

On-line Transaction Processing (OLTP) applications often rely on shared-nothing distributed databases that can sustain rapid growth in data volume. Distributed transactions (DTs) that involve data tuples from multiple geo-distributed servers can adversely impact the performance of such databases, especially when the transactions are short-lived and these require immediate responses. The k -way min-cut graph clustering based database repartitioning algorithms can be used to reduce the number of DTs with acceptable level of load balancing. Web applications, where DT profile changes over time due to dynamically varying workload patterns, frequent database repartitioning is needed to keep up with the change. This paper addresses this emerging challenge by introducing incremental repartitioning. In each repartitioning cycle, DT profile is learnt online and k -way min-cut clustering algorithm is applied on a special sub-graph representing all DTs as well as those non-DTs that have at least one tuple in a DT. The latter ensures that the min-cut algorithm minimally reintroduces new DTs from the non-DTs while maximally transforming existing DTs into non-DTs in the new partitioning. Potential load imbalance risk is mitigated by applying the graph clustering algorithm on the finer logical partitions instead of the servers and relying on random one-to-one cluster-to-partition mapping that naturally balances out loads. Inter-server data-migration due to repartitioning is kept in check with two special mappings favouring the current partition of majority tuples in a cluster-the many-to-one version minimising data migrations alone and the one-to-one version reducing data migration without affecting load balancing. A distributed data lookup process, inspired by the roaming protocol in mobile networks, is introduced to efficiently handle data migration without affecting scalability. The effectiveness of the proposed framework is evaluated on realistic TPC-C workloads comprehensively using graph, hypergraph, and compressed hypergraph representations used in the literature. To compare the performance of any incremental repartitioning framework without any bias of the external min-cut algorithm due to graph size variations, a transaction generation model is developed that can maintain a target number of unique transactions in any arbitrary observation window, irrespective of new transaction arrival rate. The overall impact of DTs at any instance is estimated from the exponential moving average of the recurrence period of unique transactions to avoid transient fluctuations. The effectiveness and adaptability of the proposed incremental repartitioning framework for transactional workloads ?have been established with extensive simulations on both range partitioned and consistent hash partitioned databases. We propose incremental repartitioning of distributed OLTP databases for high-scalability.We model two incremental repartitioning algorithm and lookup mechanism.We develop a unique transaction generation model for simulation.We derive novel impact metrics for distributed transactions.Simulation results indicate adaptability of the methods scalable OLTP applications.

The commit processing in a Distributed Real Time Database (DRTDBS) can significantly increase execution time of a transaction. Therefore, designing a good commit protocol is important for the DRTDBS; the main challenge is the adaptation of standard commit protocol into the real time database system and so, decreasing the number of missed transaction in the systems. In these papers we review the basic commit protocols and the other protocols depend on it, for enhancing the transaction performance in DRTDBS. We propose a new commit protocol for reducing the number of transaction that missing their deadline.

Modern transactional processing systems need to be fast and scalable, but this means many such systems settled for weak consistency models. It is however possible to achieve all of strong consistency, high scalability and high performance, by using fine-grained partitions and light-weight concurrency control that avoids superfluous synchronization and other overheads such as lock management. Independent transactions are one such mechanism, that rely on good partitions and appropriately defined transactions. On the downside, it is not usually straightforward to determine optimal partitioning schemes, especially when dealing with non-trivial amounts of data. Our work attempts to solve this problem by automating the partitioning process, choosing the correct transactional primitive, and routing transactions appropriately.

At present, to deal with a large amount of variety data in the database, various NoSQL databases have been proposed and put to practical use. However, since most of them support the transaction processing only on the single data, there is the problem that the plural data cannot be updated in a lump with maintaining the ACID properties. To solve this problem, in this paper, we propose a method to process plural data as a single transaction for MongoDB, which is a kind of document oriented NoSQL database. Concretely, each data has both of the before and after update fields, and the state of the transaction is managed. Then, in the case of before the commit, the former data is queried; after the commit, the latter data is queried. By this method, we show that the plural data can be updated as a transaction with the specified isolation level.

Locking is one of the predominant costs in transaction processing. While much work has focused on designing efficient concurrency control mechanisms, not much has been done on understanding how transaction applications issue queries and leveraging application semantics to improve application performance. This paper presents Q uro , a query-aware compiler that automatically reorders queries in transaction code to improve performance. Observing that certain queries within a transaction are more contentious than others as they require locking the same tuples as other concurrently executing transactions, Q uro automatically changes the application such that contentious queries are issued as late as possible. We have evaluated Q uro on various transaction benchmarks, and our results show that Q uro -generated implementations can increase transaction throughput by up to 6.53x, while reduce transaction latency by up to 85%.

Modern hardware is abundantly parallel and increasingly heterogeneous. The numerous processing cores have non-uniform access latencies to the main memory and processor caches, which causes variability in the communication costs. Unfortunately, database systems mostly assume that all processing cores are the same and that microarchitecture differences are not significant enough to appear in critical database execution paths. As we demonstrate in this paper, however, non-uniform core topology does appear in the critical path and conventional database architectures achieve suboptimal and even worse, unpredictable performance. We perform a detailed performance analysis of OLTP deployments in servers with multiple cores per CPU (multicore) and multiple CPUs per server (multisocket). We compare different database deployment strategies where we vary the number and size of independent database instances running on a single server, from a single shared-everything instance to fine-grained shared-nothing configurations. We quantify the impact of non-uniform hardware on various deployments by (a) examining how efficiently each deployment uses the available hardware resources and (b) measuring the impact of distributed transactions and skewed requests on different workloads. We show that no strategy is optimal for all cases and that the best choice depends on the combination of hardware topology and workload characteristics. Finally, we argue that transaction processing systems must be aware of the hardware topology in order to achieve predictably high performance.

The distributed trading mechanism has gained substantial popularity in protecting the privacy of the agents and reducing the computation burden. The existing studies focus on the distributed transaction mechanism applied in the power market, but the distributed multilateral transaction in the integrated energy system has not been explored. This paper proposes a novel power-heating multilateral contract transaction mechanism considering spot market prices. A multi-leader multi-follower (MLMF) Stackelberg game model is formulated to describe the multilateral contract transactions between the integrated energy service providers (IESPs) and the load aggregators (LAs). In this model, the IESPs play the role of leaders to select discriminate prices for different LAs while the LAs play the role of followers to determine the energy purchases. The competition among IESPs is formulated as a non-cooperative Nash game, where each stakeholder seeks to maximize its own profit. Then, the existence of the Nash equilibrium is proved. Besides, a distributed algorithm is developed to derive the equilibrium of the proposed hierarchical decision model, through which the optimal strategies for each player can be obtained. Two case studies with different scales demonstrate the effectiveness of the proposed contract transaction mechanism and the distributed solution algorithm.

The space of Internet-of-things has exploded, with billions of interconnected devices ranging from mainframes to refrigerators and thermostats. These devices offer the promise of greater automation and control, and the ability for micro-level transactions that did not exist before. The advent of blockchain offers an intriguing path to managing distributed transactions in this new ecosystem. The use of blockchain in IoT applications is relatively new, especially at the lower end of the computing spectrum. Consequently, the roadmap for the future is unclear, and there are several challenges and questions that need to be addressed, such as trust, security, and efficiency. In this paper, we survey some of the promising applications that are being implemented including supply chains, smarter energy, and healthcare. We outline strategies for overcoming many of the challenges, which should lead to successful adoptions of blockchain for IoT. Finally, we sound a cautionary note on the potential cybersecurity implications in using IoT devices, including increased attack surfaces and device vulnerabilities.

Replication protocol (RPCL) has been the major focused area for research in the replicated distributed real-time database system (RDRTDBS). The primary objective of RPCL is to improve the availability, scalability, and fault tolerance of the system. However, the most challenging task is to guarantee QoS for the data services in the RDRTDBS. The real-time transaction workload may not be balanced, and their access patterns may be time-varying and skewed. As a result, many RTTs miss their deadlines, or consistency constraints of the real-time data items may be violated. In the current paper, our objective is to propose an integrated version of a transaction manager that consists of the transaction sub-module and granule sub-module, thread to granule policy, and overload resolver (OLRE) to guarantee quality of service (QoS) during unpredictable workload. The experimental results show that our proposed algorithms reduce the overload condition and improve the processing of real-time transactions (RTTs) with maintaining mutual consistency.

In the process of enterprise informatization, enterprises have gradually established many “chimney” application systems so that the business data is hard to share, and because being built at different times, the systems are developed based on different framework versions. With the development of enterprise informatization, the enterprise information construction is developing towards business integration as vertical and horizontal integration of data sharing and process through. However, adopting traditional system architecture to realize the integration platform has problems such as high coupling, poor scalability, waste of resources and difficulty in developing and maintaining. Aiming at these problems, this paper proposed an integrated platform application architecture based on micro service, and the application model of integrated platform has been designed. It breaks through the key technical issues such as session sharing, service invocation and distributed transaction, and verifies the application architecture of the platform through experiments which proved the architecture and application model of the integrated platform is available. Through the architecture, the business functions are broken down to form the independent micro service so as to realize the decoupling business function component of the system. It effectively reduces the integration platform development and maintenance difficulty, and improve s run-time dynamic extension ability and system fault tolerance.

This paper introduces SagaMAS: a Multi-Agent based framework on development, dealing with distributed transactions in the microservices architecture. Microservices are an architectural style where the distributed system is decomposed in a series of highly cohesive and independent services. Each microservice can have different implementations and data persistence technologies, resulting in heterogeneous distributed systems. Given its heterogeneity and distributed nature, an open challenge in this architecture is the proper management of distributed transactions that expand through several microservices. Multi-Agent Systems are by definition distributed systems suited for complex coordination tasks, such as this. The proposed framework can be seen as a decoupled autonomous layer that coordinates the distributed transactions of the system, relieving the microservice developer from such tasks, and simplifying microservice interactions. Unlike existing approaches, our proposal is conceived and usable at an abstraction level appropriate to express reliability and robustness issues in terms of agent coordination.

There is a critical problem of metadata performance scalability in high-performance computing when accessing many small files from millions of clients. This paper proposes a design of PPMDS, a scalable distributed metadata server, to improve metadata performance and scalability. To efficiently manage hierarchical namespace of file system across multiple servers, it manages inode entries using a parent inode number and an entry name as a key. To update multiple inode entries atomically, it uses a nonblocking distributed transaction based on an algorithm of dynamic software transactional memory. It improves the metadata performance by introducing server-side transaction processing, multi-reader, and a shared lock mode to reduce the number of remote procedure calls and to prevent unnecessary blocking. A prototype implementation achieves 62,000 operations per second when creating files in a single directory using 8 metadata servers, which is 2.58x performance improvement compared to a single metadata performance.

Scheduling transactions in a real-time database requires an integrated approach in which the schedule does not only guarantee execution before the deadline, but also maintains data consistency. The problem has been studied under a common framework which considers both concurrency control issues and the real-time constraints in centralized and distributed transaction processing. A real-time transaction processing model has been defined for a centralized system. The proposed protocols use a unified approach to maximize concurrency while meeting real-time constraints at the same time. In order to test the behavior of the model and the proposed protocols, a real-time transaction processing testbed has been developed using discrete event simulation techniques. The results indicate that different protocols work better under different load scenarios and that the overall performance can be significantly enhanced by modifying the underlying system configuration. Among other system and transaction parameters, the effect of data partitioning, buffer management, preemption, disk contention, locking mode and multiprocessing has been studied. For the distributed environment, new concepts of real-time nested transactions and priority propagation have been proposed. Real-time nested transactions incorporate the deadline requirements in the hierarchical structure of nested transactions. Priority propagation addresses the issues related to transaction aborts in real-time nested transaction processing. The notion of priority ceiling has been used to avoid the priority inversion problem. The proposed protocols exhibit freedom from deadlock and have tightly bounded waiting period. Both of these properties make them very suitable for distributed real-time transaction processing environment.

Distributed Transactional systems need to be fast and scalable to increase the performance of the system. Many distributed databases achieve high throughput and scalability through data partitioning. The paper presents comprehensive survey of different techniques and parameters related to distributed transactions used in the Distributed databases. It specifies different ways of partitioning algorithms. The purpose of this review is to study the techniques of horizontal partitioning like range partitioning, schema level partitioning, graph level partitioning etc. giving the high performance and availability of data. The vital aim of these techniques is to reduce distributed transaction and increase scalability of distributed databases.

Chapter Objectives In this chapter, you will learn to: • grasp the basics of distributed systems and distributed databases; • discern key architectural implications of distributed databases; • understand the impact of fragmentation, allocation, and replication; • identify different types of transparency; • understand the steps in distributed query processing; • understand distributed transaction management and concurrency control; • grasp the impact of eventual consistency and BASE transactions. Opening Scenario As Sober envisions growing as part of its long-term strategy, it wants to have a careful understanding of the data implications involved. More specifically, the company wants to know if it would make sense to distribute its data across a network of offices and work with a distributed database. Sober wants to know the impact of data distribution on query processing and optimization, transaction management, and concurrency control. In this chapter, we focus on the specifics of distributed databases (i.e., systems in which the data and DBMS functionality are distributed over different nodes or locations on a network). First, we discuss the general properties of distributed systems and offer an overview of some architectural variants of distributed database systems. Then, we tackle the different ways of distributing data over nodes in a network, including the possibility of data replication. We also focus on the degree to which the data distribution can be made transparent to applications and users. Then, we discuss the complexity of query processing and query optimization in a distributed setting. A next section is dedicated to distributed transaction management and concurrency control, focusing on both tightly coupled and loosely coupled settings. The last section overviews the particularities of transaction management in Big Data and NoSQL databases, which are often distributed in a cluster set-up, presenting BASE transactions as an alternative to the traditional ACID transaction paradigms. Distributed Systems and Distributed Databases Ever since the early days of computing, which were dominated by monolithic mainframes, distributed systems have had their place in the ICT landscape. A distributed computing system consists of several processing units or nodes with a certain level of autonomy, which are interconnected by a network and which cooperatively perform complex tasks. These complex tasks can be divided into subtasks as performed by the individual nodes.

Distributed transactions, which access data items at multiple sites atomically, face well-known scalability challenges. To avoid the high overhead, in prior work Fan et al. proposed Epoch-based Concurrency Control (ECC), which makes transactions visible at epoch boundaries, and presented a system that supports high performance read-only and write-only transactions. However, this idea has a clear difficulty to overcome: the common case of a single transaction that does both reading and writing. This paper proposes ALOHA-DB, a scalable distributed transaction processing system. ALOHA-DB uses a novel paradigm of serializable transaction processing using functors, which conceptually resemble futures in modern programming languages. A functor is a placeholder for the value of a key, which can be computed asynchronously in the future in parallel with other functor computations of the same or other transactions. With multi-versioning in ECC, the functor computations only rely on accessing historical versions, and so the traditional locking mechanism is not needed for concurrency control. Functors elevate ECC to a new level: supporting serializable distributed read-write transactions. This combination of techniques never aborts transactions due to read-write or write-write conflicts, but allows transactions to fail due to logic errors or constraint violations. We used functor-enabled ECC to implement ALOHA-DB and evaluated it using TPC-C and YCSB read-write distributed transactions. Experimental results demonstrate that our system's performance on the TPC-C benchmark is nearly 2 million transactions per second over 20 eight-core virtual machines, which outperforms Calvin, a state-of-the-art transaction processing and replication layer, by one to two orders of magnitude.

The trend towards global applications and services has created an increasing demand for transaction processing on globally-distributed data. Many database systems, such as Spanner and CockroachDB, support distributed transactions but require a large number of wide-area network roundtrips to commit each transaction and ensure the transaction's state is durably replicated across multiple datacenters. This can significantly increase transaction completion time, resulting in developers replacing database-level transactions with their own error-prone application-level solutions. This paper introduces Carousel, a distributed database system that provides low-latency transaction processing for multi-partition globally-distributed transactions. Carousel shortens transaction processing time by reducing the number of sequential wide-area network round trips required to commit a transaction and replicate its results while maintaining serializability. This is possible in part by using information about a transaction's potential write set to enable transaction processing, including any necessary remote read operations, to overlap with 2PC and state replication. Carousel further reduces transaction completion time by introducing a consensus protocol that can perform state replication in parallel with 2PC. For a multi-partition 2-round Fixed-set Interactive (2FI) transaction, Carousel requires at most two wide-area network roundtrips to commit the transaction when there are no failures, and only one round trip in the common case if local replicas are available.

We propose EnclaveDB, a database engine that guarantees confidentiality, integrity, and freshness for data and queries. EnclaveDB guarantees these properties even when the database administrator is malicious, when an attacker has compromised the operating system or the hypervisor, and when the database runs in an untrusted host in the cloud. EnclaveDB achieves this by placing sensitive data (tables, indexes and other metadata) in enclaves protected by trusted hardware (such as Intel SGX). EnclaveDB has a small trusted computing base, which includes an in-memory storage and query engine, a transaction manager and pre-compiled stored procedures. A key component of EnclaveDB is an efficient protocol for checking integrity and freshness of the database log. The protocol supports concurrent, asynchronous appends and truncation, and requires minimal synchronization between threads. Our experiments using standard database benchmarks and a performance model that simulates large enclaves show that EnclaveDB achieves strong security with low overhead (up to 40% for TPC-C) compared to an industry strength in-memory database engine.

In distributed transactional systems deployed over some massively decentralized cloud servers, access policies are typically replicated. Interdependencies and inconsistencies among policy versions can affect performance, throughput, and accuracy, which can increase the transaction failure rate and cause long delays. Thus, policy and user credential inconsistencies need to be addressed. Several stringent levels of policy consistency constraints and enforcement approaches to guarantee the trustworthiness of transactions on cloud servers are proposed. However, there are performance issues associated with the policies proposed while retrieving the latest policy versions present in various cloud servers. In this paper, first, we define a look-up table in which the policy versions used for authorization is stored and updated on a regular basis and this information can be easily retrieved by the transaction manager. Next, we use the concept of "Tree-Based Consistency" approach to maintain a tree structure of the servers where a data item is replicated. By integrating look-up table for policy versions and the consistency tree based approach, finally, we propose an enhanced version of Two-Phase Validation Commit (2PVC) protocol integrated with the Paxos commit protocol to increase the number transaction commits with reduced or almost the same performance overhead (transaction execution time) without affecting accuracy and precision, but reducing the number of transaction aborts in comparison with a most recent work.

Supporting distributed transaction is a key function for large-scale database systems. Conventional database systems build distributed transactions on the top of replication storage to provide high availability. However, replication induces a large amount of storage overhead. In this paper, we make the first attempt to build highly available distributed transactions over erasure coding to achieve high storage efficiency. We propose Eunice, an Efficient and available distributed transaction protocol that Unifies Concurrency control and Erasure coding. In Eunice, we first design a single-layered coding update mechanism to optimize transaction latency. Then we propose a Paxos-based coding consensus protocol to provide fault-tolerance and strong consistency for coding update operation. Compare with conventional distributed transaction protocol with replication, Eunice can save up to 41.9% storage consumption, while achieving comparable throughput and latency performance.

Distributed online transaction processing (OLTP) database management systems (DBMS) are characterized with ACID properties, that is atomicity, consistency, isolation, durability. However, these OLTP DBMSs that handle massive data schema, need to be scalable that cannot give up strong transactional and consistency requirements. The large-scale schemas are deployed on distributed server and each server's memory is limited, in that case, how this system partition is the key factor to affect its performance. In this paper, we present a partition approach, that applies to general schema and taking care of load balancing and distributed transaction rates. Our method achieve this by periodically update the partition plan and migration hot data among the servers. OLTP workload are modeled as a graph and using a graph partition algorithm to partition set of data into same server that are often co-accessed by one transaction. To evaluate our method, our framework Assort is integrated into a distributed, main-memory DBMS and shows that it can partition schema in OLTP DBMS and enable the system to outperformance traditional method 3–4x throughput.

Designers of distributed database systems face the choice between stronger consistency guarantees and better performance. A number of applications only require read atomicity (RA) and prevention of lost updates (PLU). Existing distributed database systems that meet these requirements also provide additional stronger consistency guarantees (such as causal consistency), and therefore incur lower performance. In this paper we define a new distributed transaction protocol, ROLA, that targets applications where only RA and PLU are needed. We formally model ROLA in Maude. We then perform model checking to analyze both the correctness and the performance of ROLA. For correctness, we use standard model checking to analyze ROLA’s satisfaction of RA and PLU. To analyze performance we: (a) use statistical model checking to analyze key performance properties; and (b) compare these performance results with those obtained by analyzing in Maude the well-known protocol Walter. Our results show that ROLA outperforms Walter.

Microservice Architecture (MSA) is an architectural design pattern which was introduced to solve the challenges involved in achieving the horizontal scalability, high availability, modularity and infrastructure agility for the traditional monolithic applications. Though MSA comes with a large set of benefits, it is challenging to design isolated services using independent Database per Service pattern. We observed that with each micro service having its own database, when transactions span across multiple services, it becomes challenging to ensure data consistency across databases, particularly in case of roll backs. In case of monolithic applications using RDBMS databases, these distributed transactions and roll backs can be handled efficiently using 2 phase commit techniques. These techniques cannot be applied for isolated No-SQL databases in micro services. This research paper aims to address three things: 1) elucidate the challenges with distributed transactions and rollbacks in isolated No-SQL databases with dependent collections in MSA, 2) examine the application of event choreography and orchestration techniques for the Saga pattern implementation, and 3) present the fact-based recommendations on the saga pattern implementations for the use cases.

Challenges and issues of distributed database are well known. One of the challenges is obtaining database transparency. In distributed database, the physical database is distributed across heterogeneous database systems. There are several methods of data access from distributed database, still it involves complicated implementation of these methods. Oracle DBMS provides heterogeneous gateway service to connect oracle and non-oracle databases. The data access and distributed transaction execution is made very easy through this gateway. In this paper, the procedure to connect four heterogeneous databases namely PostgreSQL, MySQL, Oracle and MS Access is described. The successful implementation of data access from multiple heterogeneous databases provides very easy and efficient access of data in Oracle using its native language/commands. Oracle user will neither have to bother about architecture nor commands of the remote databases from where data is accessed. Even user need not have to worry about the location of database. Once all the databases are connected through heterogeneous gateway, user will be able to access data and process distributed transactions efficiently from Oracle. The paper covers the whole process.

Concurrency control faults may lead to unwanted interleavings, and breach data consistency in distributed transaction systems. However, due to the unpredictable delays between sites, detecting concurrency control faults in distributed transaction systems is difficult. In this paper, we propose a methodology, relying on model-based testing and mutation testing, for designing test cases in order to detect such faults. The generated test inputs are designated delays between distributed operations, while the outputs are the occurrence of unwanted interleavings that are consequences of the concurrency control faults. We mutate the distributed transaction specification with common concurrency control faults, and model them as UPPAAL timed automata, in which designated delays are encoded as stopwatches. Test cases are generated via reachability analysis using UPPAAL Model Checker, and are selected to form an effective test suite. Our methodology can reduce redundant test cases, and find the appropriate delays to detect concurrency control faults effectively.

Flash memory has accelerated the architectural evolution of storage systems with its unique characteristics compared to magnetic disks. The no-overwrite property of flash memory naturally supports transactions, a commonly used mechanism in systems to provide consistency. However, existing embedded transaction designs in flash-based Solid State Drives (SSDs) either limit the transaction concurrency or introduce high overhead in tracking transaction states. This leads to low or unstable SSD performance. In this paper, we propose a transactional SSD (TxSSD) architecture, LightTx, to enable better concurrency and low overhead. First, LightTx improves transaction concurrency arbitrarily by using a page-independent commit protocol. Second, LightTx tracks the recent updates by leveraging the near-log-structured update property of SSDs and periodically retires dead transactions to reduce the transaction state tracking cost. Experiments show that LightTx achieves nearly the lowest overhead in garbage collection, memory consumption and mapping persistence compared to existing embedded transaction designs. LightTx also provides up to 20.6 percent performance improvement due to improved transaction concurrency.

Large scale cloud applications are difficult to program due to the need to access data in a consistent manner. To lift this burden from programmers, Deferred Update Replication (DUR) protocols provide serializable transactions with both high availability and performance in read-dominated workloads. However, the inherently optimistic nature of DUR protocols makes them prone to thrashing in conflict-intensive scenarios: existing DUR schemes, in fact, avoid any synchronization during transaction execution; thus, these schemes end up aborting any update transaction whose reads are no longer up to date by the time it attempts to commit. To tackle this problem, we introduce Bumper, a set of innovative techniques to reduce aborts of transactions in high-contention scenarios. At its core, Bumper relies on two key ideas. First, we spare update transactions from spurious aborts (i.e., unnecessary aborts of serializable transactions), by attempting to serialize the transactions in the past. For this, we use a novel distributed concurrency control scheme that we call Distributed Time-Warping (DTW). And second, we avoid aborts due to contention hot spots (that cannot be tackled by DTW) via a programming abstraction that we call Delayed Actions. These, allow for efficiently serializing, in an abort-free fashion, the execution of conflict-prone data manipulations. By means of an extensive evaluation, on a distributed system of 160 nodes, we show that Bumper can boost performance up to 3 × in conflict-intensive workloads, while imposing negligible (about 2.5%) overheads in uncontended scenarios.

The concept of a digital ecosystem (DE) has been used to explore scenarios in which multiple online services and resources can be accessed by users without there being a single point of control, which can be used to effectively serialise their interactions. We argue in this paper that this weak coupling between services places additional demands on the modelling of compensation and recovery management in long-running transactions over traditional SOC related formalisms. We describe an adaptation of Shields’ vector languages, in that the synchronisation constraint is removed (no shared actions), as a formal semantics for a transaction in terms of the common ordering constraints on the underlying interactions between its participants. The notation afforded by the so-called transaction languages captures the invocations on each participant service (online resource), and at each point during execution, across the whole transaction. Concurrency is modelled explicitly through a notion of independence, which is lifted onto tuples of sequences (one for each participant of the transaction) rather than individual sequences, as in Mazurkiewicz trace languages or events, as in the event structures model. Participating subcomponents execute concurrently and failure of one or more causes the recovery of the whole transaction. Compensations are triggered immediately upon failure and concurrent forward actions are compensated concurrently. We highlight the benefits of our true-concurrent approach in the context of DEs and outline connections of transaction languages to other partial order models. Further, we discuss how our approach supports forward recovery in that recovering the whole transaction is avoided wherever possible.

User-facing online services utilize geo-distributed data stores to minimize latency and tolerate partial failures, with the intention of providing a fast, always-on experience. However, geo-distribution does not come for free; application developers have to contend with weak consistency behaviors, and the lack of abstractions to composably construct high-level replicated data types, necessitating the need for complex application logic and invariably exposing inconsistencies to the user. Some commercial distributed data stores and several academic proposals provide a lattice of consistency levels, with stronger consistency guarantees incurring increased latency and throughput costs. However, correctly assigning the right consistency level for an operation requires subtle reasoning and is often an error-prone task. In this paper, we present QUELEA, a declarative programming model for eventually consistent data stores (ECDS), equipped with a contract language, capable of specifying fine-grained application - level consistency properties. A contract enforcement system analyses contracts, and automatically generates the appropriate consistency protocol for the method protected by the contract. We describe an implementation of QUELEA on top of an off-the-shelf ECDS that provides support for coordination-free transactions. Several benchmarks including two large web applications, illustrate the effectiveness of our approach.

Supporting long transaction in HBase has always been a hot issue since NoSQL is widely approved. In this paper, we first divide the existing methods about long transaction of Key-Value databases into two categories: distributed and centralized, and analyze the distributed methods named Percolator and Haeinsa and the centralized methods named HBaseSI and Omid. Then we propose a non-intrusive long transaction approach for HBase, which bases on the idea of pre-conflict checking and timestamp comparison. We design three algorithms for the proposed approach, namely pre-submit, submit-process and snapshot obtain. We also introduce a timeout mechanism based on collision detection. Experiment results indicate that the proposed method has better performance than other long transaction schemes in the system with more-frequent transaction conflict.

Transaction privacy has attracted a lot of attention in the e-commerce. This study proposes an efficient and provable fair document exchange protocol with transaction privacy. Using the proposed protocol, any untrusted parties can fairly exchange documents without the assistance of online, trusted third parties. Moreover, a notary only notarizes each document once. The authorized document owner can exchange a notarized document with different parties repeatedly without disclosing the origin of the document or the identities of transaction participants. Security and performance analyses indicate that the proposed protocol not only provides strong fairness, non-repudiation of origin, non-repudiation of receipt, and message confidentiality, but also enhances forward secrecy, transaction privacy, and authorized exchange. The proposed protocol is more efficient than other works.

This paper presents Atomic RMI, a distributed transactional memory framework that supports the control flow model of execution. Atomic RMI extends Java RMI with distributed transactions that can run on many Java virtual machines located on different network nodes. Our system employs SVA, a fully-pessimistic concurrency control algorithm that provides exclusive access to shared objects and supports rollback and fault tolerance. SVA is capable of achieving a relatively high level of parallelism by interweaving transactions that access the same objects and by making transactions that do not share objects independent of one another. It also allows any operations within transactions, including irrevocable ones, like system calls, and provides an unobtrusive API. Our evaluation shows that in most cases Atomic RMI performs better than fine grained mutual-exclusion and read/write locking mechanisms. Atomic RMI also performs better than an optimistic transactional memory in environments with high contention and a high ratio of write operations, while being competitive otherwise.

There are various difficulties between power generators and power users during direct trade. The current direct trade rules and patterns cannot guarantee the interests of all parties directly related to the transaction, which is one of important reasons. Direct transaction space model of incentive compatible principles is proposed in this paper. How to guarantee the interests of grid, generators and big users during direct transaction is analyzed. For power generation companies, large users and grids, the change of interests are analyzed and marginal decision condition is studied. Based on the principle of incentive compatibility, direct transaction space model which not damage the interests of three transaction bodies is analyzed.

With the development of cloud computing and internet; e-Commerce, e-Business and corporate world revenue are increasing with high rate. These areas not only require scalable and consistent databases but also require inter database transaction support. In this paper, we present, a scalable three-tier architecture along with a distributed middle-ware protocol to support atomic transactions across heterogeneous NoSQL databases. Our methodology does not compromise on any assumption on the accuracy of failure modalities. Hence, it is suitable for a class of heterogeneous distributed systems. To achieve such a target, our architectural model exploits an innovative methodology to achieve distributed atomic transactions. We simulate this architectural setup with different latency tests under different environments to produce reliable impact and correctness. This study proposes a three-tier architecture to facilitate atomic transactions among heterogeneous NoSQL distributed database environment as HBase.The first tier called client tier, deals with user requests. The client tier wraps transaction request and deliver it to a corresponding global transaction manager (GTM), which is assigned by a manager node.The second tier called middle tier, contains a zookeeper, manager nodes, and labour nodes. Zookeeper assigns an active manager node to the client and the active manager node assigns a labour node to process the transaction.The third tier called distributed tier manages multiple heterogeneous NoSQL databases participating with different transactions.Experimental results show that the proposed architecture is scalable and fault tolerated.

Distributed database management systems (DBMSs)have limited capacity to manage overhead of distributed transactions due to certain failures. These types of failures don't guarantee the enforcement of transactional properties and which reduces the scalability and availability of these systems. Research in this area proved that scaling out whiling providing consistency of transactions and guaranteeing availability is the presence of different failures in distributed computing environment is not feasible. As a result, the database community have used vertical scaling rather than horizontal scaling. Moreover changes in data access patterns resulting from new generation of web applications with high scalability and availability guarantees weaker consistency. In this paper we have analyses that how different systems such as ElasTras, Mstore, Sinfonia, ecStore and Gstore provide scalable transactional data access in cloud computing environment. We have also discussed transactions management (TM) in Gstore in detail and compared with other techniques and protocols used in current cloud based systems.

The advancement in mobile technology and wireless network increase the using of mobile device in database driven application, these application require high reliability and availability due to nature inheritance of mobile environment, transaction is the center component in database systems, In this paper we present useful work done in mobile transaction, we show the mobile database environment and overview a lot of proposed model of mobile transaction and show many techniques used to enhance transaction execution.

Language-level transactions are said to provide “atomicity,” implying that the order of operations within a transaction should be invisible to concurrent transactions and thus that independent operations within a transaction should be safe to execute in any order. In this article, we present a mechanism for dynamically reordering memory operations within a transaction so that read-modify-write operations on highly contended locations can be delayed until the very end of the transaction. When integrated with traditional transactional conflict detection mechanisms, our approach reduces aborts on hot memory locations, such as statistics counters, thereby improving throughput and reducing wasted work. We present three algorithms for delaying highly contended read-modify-write operations within transactions, and we evaluate their impact on throughput for eager and lazy transactional systems across multiple workloads. We also discuss complications that arise from the interaction between our mechanism and the need for strong language-level semantics, and we propose algorithmic extensions that prevent errors from occurring when accesses are aggressively reordered in a transactional memory implementation with weak semantics.

We address the problem of building scalable transaction management mechanisms for multi-row transactions on key-value storage systems, which are commonly termed as NoSQL systems. We develop scalable techniques for transaction management utilizing the snapshot isolation (SI) model. Because the SI model can lead to non-serializable transaction executions, we investigate two conflict detection techniques for ensuring serializability. To support scalability, we investigate system architectures and mechanisms in which the transaction management functions are decoupled from the storage system and integrated with the application-level processes. We present two system architectures and demonstrate their scalability under the scale-out model of cloud computing platforms. In the first system architecture all transaction management functions are executed in a fully decentralized manner by the application processes. The second architecture is based on a hybrid approach in which the conflict detection functions are performed by a dedicated service. We perform a comparative evaluation of these architectures using the TPC-C benchmark and demonstrate their scalability.