Search

The rapid development of Internet of Things (IoT) technology and the widespread deployment of various sensors around the world have produced a large number of data streams. Thus, current computing systems face the challenge of quickly receiving and managing these large-scale streaming data. This study builds an efficient distributed database based on Greenplum (GP) and focuses on solving the problem of the low efficiency of structured data queries for observed ecological data collected from fragile areas in Northwest China's desert oasis. First, a distributed database is designed and deployed at the physical storage structure level. A database table structure is then established based on the characteristics of the streaming data. On this basis, the data storage strategy is optimized at the data table level. Additionally, the query efficiency of the distributed database is compared with the query efficiency of traditional standalone databases. The results show that the distributed database significantly improves the data query efficiency. The greater the amount of data stored, the better the improvement in efficiency. Finally, based on the optimized distributed database, we develop a data sharing system for streaming data from ecologically fragile areas in the desert oasis in Northwest China, which provides a new approach for the efficient sharing of massive amounts of IoT streaming data for ecological monitoring. Our storage system is still currently working normally, which is highly important to both data managers and users.

The problem with telecommunications companies today is that transactional data is more extensive than existing source tables. This makes business reporting less efficient and overwhelms query processing results in data warehouses so that they do not meet business requirements. The fast and complex evolution of the digital world must be scalable to the data warehouse process, so that the authors implement it in the data warehouse using massive parallel processing (MPP) with the Greenplum database, so that business users can get reports faster and more optimally. This case study explains how the MPP system implements and measures the performance of the Greenplum database by performing complex queries in the data warehouse with parallel processing. Therefore, this case study analyzes whether the use of MPP systems can measure the scalability of throughput and the response time in the data warehouse so that system performance in the Greenplum database remains stable for daily, weekly, and monthly operations. [ABSTRACT FROM AUTHOR]