Search

Although there are numerous data modeling tools for relational databases, data modeling for NoSQL databases has seen another perspective. These databases (a) do not define any explicit schema, (b) store data in a denormalized manner, and (c) give many structure alternatives. The decision to structure the data always relies on rules of thumb, which do not guarantee an optimal structural solution. Based on this motivation, this paper offers a workload-driven model for the logical schema design of a NoSQL document database. It consists of Model input, Intermediate transformation, and Final schema generation. The proposed model takes the conceptual schema (EER model) and application workload (queries and anticipated data volume) as input and describes a procedure to convert it into a logical model for NoSQL document stores. The conversion process initially converts the application queries into query graphs. The query graphs, along with the anticipated data volume, are used to generate the query labels. The resulting query labels are assigned on the schema graph designed from the EER model. The schema graph and labels are used to transform the EER model into the appropriate logical schema model based on the actions defined for each label. We evaluate the model using a case study in the eCommerce application domain. The experimental evaluation shows the proposed model outperforms the existing conventional, optimized, and query path graphs models in multiple aspects, including query performance, storage space efficiency, aggregate pipeline efficiency, read–write latency, collection-wise performance, scalability, throughput and latency. By effectively addressing the challenges associated with managing the variety and volume of big data through a well-designed schema, our proposed model significantly reduces the time, cost, and effort required for schema development and repair. [ABSTRACT FROM AUTHOR]

In order to solve the problem of data association and aggregation query between multiple measuring instruments during the online measurement process for parts, the distributed storage and sharing methods of measuring were studied. Firstly, non-relational database was used in multiple measuring instrument cluster, which realized distributed storage of measuring data and simplified parts data storage model. Then, tag aware sharding model was used to classify and store the measurement data of different processes. Finally, multiple Raspberry Pi 3 Model B+ development boards with embedded Linux were used as the system platforms for the measurement instruments, and MongoDB database was used as online measurement distributed data storage to verify the uniformity and stability of distributed data storage on each node using a large amount of test data. The cluster had been integrated with manufacturing execution system (MES), which could monitor and analyze the measuring data of parts in real time, and quickly summarize the working procedure data in all measuring nodes to generate the report forms. The research results show that the single-node query response speed is stable in the range of 125ms to 208ms when the data volume reaches 7.2×105 in the cluster, which is 88.15% higher than that of the hashed sharding model, at the same time, the response speed of multi-node cooperative query is 1308ms, which is 61.64% faster than the scheme of “ascending key + searching key”. Aggregating and summarizing 1×105 parts data within multiple measurement nodes takes approximately 5s. This storage cluster can monitor the production status of parts and plays an important role in improving manufacturing efficiency and quality. [ABSTRACT FROM AUTHOR]

Benchmarking serves as the foundation for selecting a database in any project. The available benchmarking tools evaluate system performance by subjecting it to random data and a set of arbitrary operations, without considering the specific characteristics of the application. The problem with these tools is that they reflect unrealistic benchmarks as they do not consider the nature, sequence, and type of queries the application will send to the database. In this paper, we introduced the approach of benchmarking the database based on the nature of interaction and queries between the application and database, and we built a benchmarking tool using Java to benchmark Redis and MongoDB as databases for a trip planning application with GTFS data of Budapest local transport data. Our study involved comparing the performance of both databases under ten different stress levels by simulating the number of querying clients. The results show that both database's performance is slightly decreased while increasing the number of clients (stress). However, Redis shows better performance compared to MongoDB. [ABSTRACT FROM AUTHOR]

During data modelling, NoSQL databases are usually considered schema-less databases. They do not support rigid schema as traditional databases. Instead, offer flexibility and support dynamic schema, which means schema design can be changed at run time to support analytical tasks. NoSQL databases allow storing similar information with multiple schemas, which facilitates data evolution by eliminating restrictions on the schema. It raises the question of whether the schema matters in NoSQL databases, given that explicit schema declaration is not required before data storage. Furthermore, if the schema does matter, what is its potential impact on application performance? This paper examines these questions by taking three types of NoSQL databases: Document store, Column store, and Key-Value store. All possible schema structure alternatives are designed using a sample Entity Relationship (ER) model. Then, a set of common queries are performed on various schema alternatives to analyse the impact of query performance in terms of query response time, speedup factor, read and write Latency, and database size. The results suggest that the scheme alternatives affect the database performance in different NoSQL models. [ABSTRACT FROM AUTHOR]

In applications whose databases have millions of rows, caching is essential to speed up the server response. The paper focuses on a new approach to testing the cache systems in the .NET 6 framework. In this paper, a system is created that uses Redis, MongoDB and SQL Server and an in-memory database as cache stores to evaluate their performance and scalability. A performance comparison of these cache systems provides detailed insights into the best practices for designing and implementing them in .NET 6 development environment. The results of this paper may be useful to developers working on similar projects to those who want to implement a caching system. [ABSTRACT FROM AUTHOR]

NoSQL document databases emerged as an alternative to relational databases for managing large volumes of data. NoSQL document databases ensure big data storage and good query performance and are essential when the data scheme does not fit into the scheme of relational databases. They store their data in the form of documents and can handle unstructured, semi-structured, and structured data. This work evaluates the top three open-source NoSQL document databases: Couchbase, CouchDB, and MongoDB with Yahoo! Cloud Serving Benchmark (YCSB), which has become a standard for NoSQL database evaluation. The performance and scale-up of document databases are assessed using YCSB workloads with a different number of records and threads, where the runtime is measured for each database. In the experimental evaluation, we concluded that MongoDB is the database with the best runtime, except for the workload composed by scan operations. In addition, we identified CouchDB as the database with the best scale-up when varying the number of threads. [ABSTRACT FROM AUTHOR]

Aim: The aim of the study is to estimate the high availability of Query processing using Novel Docker Container over kubernetes micro service architecture. Materials and Methods: Sample groups that are considered in the project can be classified into two each has 20 samples, one for Microservice architecture and other for Novel Docker container, which are tested using 0.80 for G-power to determine the sample size and for t-test analysis. 20 queries have been used in each group for estimating high availability. Results: The Novel Docker container with efficient accuracy of 91.10%, which by far seems to be better than the kubernetes based micro service architecture which gives around 87.50%. As a result, the study group has a statistically negligible difference. The significance is around 0.039 (p<0.05) and therefore there is a statistical insignificant difference among the study group. Conclusion: The query processing speed in the Novel Docker container is better compared to kubernetes based micro service architecture. [ABSTRACT FROM AUTHOR]

B-Tree based text indexes used in MongoDB are slow compared to different structures such as inverted indexes. In this study, it has been shown that the full-text search speed can be increased significantly by indexing a structure in which each different word in the text is included only once. The Multi-Stream Word-Based Compression Algorithm (MWCA), developed in our previous work, stores word dictionaries and data in different streams. While adding the documents to a MongoDB collection, they were encoded with MWCA and separated into six different streams. Each stream was stored in a different field, and three of them containing unique words were used when creating a text index. In this way, the index could be created in a shorter time and took up less space. It was also seen that Snappy and Zlib block compression methods used by MongoDB reached higher compression ratios on data encoded with MWCA. Search tests on text indexes created on collections using different compression options shows that our method provides 19 to 146 times speed increase and 34% to 40% less memory usage. Tests on regex searches that do not use the text index also shows that the MWCA model provides 7 to 13 times speed increase and 29% to 34% less memory usage. [ABSTRACT FROM AUTHOR]

The Automatic Identification System (AIS) device is mandatory for ships that comply with the International Convention for the Safety of Life at Sea (SOLAS). AIS is intended for vessel traffic monitoring to improve shipping safety. In the examined area, the base station received 22 128 345 messages in April 2019. Approximately 80% of these messages included position reports, which were subjected to geospatial analysis. One possible utilization of AIS messages is used in an intelligent maritime transport statistics production system called TranStat in the Gospostrateg project. This specific study compares the speed of executing geospatial queries in a relational PostgreSQL database engine and a non-relational MongoDB database engine. For the purpose of this research, we have defined four AIS datasets, four test polygons of varied number of vertices, and a reference point on a fairway. The tests were used to assess the execution of the queries in a database that returns the number of ships located in a predefined area and the number of ships located at a preset distance from the defined point. It has been determined from the test results that test queries are performed faster and data stored in the database occupy less disk space in MongoDB than in PostgreSQL. Faster geospatial analysis of AIS messages may improve the navigation safety by earlier detection of dangerous situations. [ABSTRACT FROM AUTHOR]

Databases are an integral part of almost every application nowadays. For example, applications using Internet of Things (IoT) sensory data, such as in Industry 4.0, are a classic example of an organized storage system. Due to its enormous size, it may be stored in the cloud. This paper presents the authors' proposition for cloudcentric sensory measurements and measurements acquisition. Then, it focuses on evaluating industrial cloud storage engines for sensory functions, experimenting with three open-source types of distributed Database Management Systems (DBMS); MongoDB and PostgreSQL, with two forms of PostgreSQL schemes (Javascript Object Notation (JSON)-based and relational), against their respective horizontal scaling strategies. Several experimental cases have been performed to measure database queries' response time, achieved throughput, and corresponding failures. Three distinct scenarios have been thoroughly tested, the most common but widely used: (i) data insertions, (ii) select/find queries, and (iii) queries related to aggregate correlation functions. The experimental results concluded that PostgreSQL with JSON achieves a 5–57% better response than MongoDB for the insert queries (cases of native, two, and four shards implementations), while, on the contrary, MongoDB achieved 56–91% higher throughput than PostgreSQL for the same set up. Furthermore, for the data insertion experimental cases of six and eight shards, MongoDB performed 13–20% more than Postgres in response time, achieving × 2 times higher throughput. Relational PostgreSQL was × 2 times faster than MongoDB in its standalone implementation for selection queries. At the same time, MongoDB achieved 19–31% faster responses and 44–63% higher throughput than PostgreSQL in the four tested sharding subcases (two, four, six, eight shards), accordingly. Finally, the relational PostgreSQL outperformed MongoDB and PostgreSQL JSON significantly in all correlation function experiments, with performance improvements from MongoDB, closing the gap with PostgreSQL towards minimizing response time to 26% and 3% for six and eight shards, respectively, and achieving significant gains towards average achieved throughput. [ABSTRACT FROM AUTHOR]

The objective of our study was to provide practical directions on the storage of genomic information and novel phenotypes (treated here as unstructured data) using a non‐relational database. The MongoDB technology was assessed for this purpose, enabling frequent data transactions involving numerous individuals under genetic evaluation. Our study investigated different genomic (Illumina Final Report, PLINK, 0125, FASTQ, and VCF formats) and phenotypic (including media files) information, using both real and simulated datasets. Advantages of our centralized database concept include the sublinear running time for queries after increasing the number of samples/markers exponentially, in addition to the comprehensive management of distinct data formats while searching for specific genomic regions. A comparison of our non‐relational and generic solution, with an existing relational approach (developed for tabular data types using 2 bits to store genotypes), showed reduced importing time to handle 50M SNPs (PLINK format) achieved by the relational schema. Our experimental results also reinforce that data conversion is a costly step required to manage genomic data into both relational and non‐relational database systems, and therefore, must be carefully treated for large applications. [ABSTRACT FROM AUTHOR]

A locality-sensitive hashing (LSH) method in the document-based NoSQL database is proposed for enhancing the ability of arbitrary reads over the previous methodologies. The proposed hash index improves efficiency by reducing the amount of accessing data for search queries by creating buckets based on hyperplanes. The LSH hashes the input data where similar items with high probability maps to the same bucket. They attempt to decrease the volume of candidate data objects matched when reducing the missed nearest neighbors. The data space is divided with randomly chosen hyperplanes to decrease the volume of candidate objects. The values which are nearer to the boundaries (adjacent to the two sides of the hyperplane) are considered. The bucket label's string length is equivalent to the amount of used hyperplanes. The effect of LSH for bucket size balancing and analysis of the non-indexed, hash index, and global-indexed dataset on MongoDB depicts the pre-eminence of the presented hash index. [ABSTRACT FROM AUTHOR]

Technology has developed rapidly in the last 20 years. Along with the developing technology, there has been a significant increase in the size of the data in the digital environment. Traditional relational databases have not been able to efficiently store and process the amount of data in the digital environment. Relational databases can associate all the data in the database with each other and can generate detailed queries and reports for their users through the advanced query language. However, relational databases lag far behind non-relational databases in performance and financially. The fact that relational databases are not economically efficient and that the database performance decreases due to the increasing amount of digital data has seriously increased the use of non-relational databases. Non-relational databases are specially prepared structures and have gained popularity with their performance, easy maintainability, and cost-effectiveness. Although non-relational databases cannot be used relationally due to their structure, it is possible to use relationally with today's advanced programming languages and advanced database architecture. In this study, it is emphasized how MongoDB, one of the most advanced and most popular non-relational databases of today, can be used relationally through the C# programming language and what benefits this will provide for us. [ABSTRACT FROM AUTHOR]

Wireless sensor networks (WSN) Environment is a collection of nodes that sensing and sending data to other nodes in a large number of nodes. One of the WSN concepts is continuous or periodic data collection, then the data obtained can be further processed. From this process, it is possible to produce large data, so the process of sending and storing data becomes a high load. Data obtained from WSN nodes vary greatly, which affects on sending and storing of data. Therefore, a study is needed to analyze WSN's ability to send and store large amounts of data. Message query telemetry protocol (MQTT) is used because it could save resources on communication and MongoDB used because it does not have the concept of tables and rows, this is very suitable for variations in data generated by WSN. In this study, it can be concluded that the performance of MongoDB on high data amount is acceptable, so MongoDB is highly recommended for WSN implementation. [ABSTRACT FROM AUTHOR]

Natural products (NPs) are small molecules produced by living organisms with potential applications in pharmacology and other industries as many of them are bioactive. This potential raised great interest in NP research around the world and in different application fields, therefore, over the years a multiplication of generalistic and thematic NP databases has been observed. However, there is, at this moment, no online resource regrouping all known NPs in just one place, which would greatly simplify NPs research and allow computational screening and other in silico applications. In this manuscript we present the online version of the COlleCtion of Open Natural prodUcTs (COCONUT): an aggregated dataset of elucidated and predicted NPs collected from open sources and a web interface to browse, search and easily and quickly download NPs. COCONUT web is freely available at https://coconut.naturalproducts.net. [ABSTRACT FROM AUTHOR]

EAST experiment device uses MDSPlus database to store a large number of experimental data. The users access the data through the API provided by MDSPlus. However, due to the large amount of internal data and the lack of unified resource directory, it is difficult to browse experimental data quickly. This paper proposes a metadata management system based on MongoDB to construct resource directory for data. Firstly, the system uses C++ language to scan the entire MDSPlus database to extract and build metadata. Secondly, based on the basic document in BSON format, the system adopts the form of document nested document array, collects all the metadata of each shot in the same document, constructs single metadata using stream document, and establishes metadata database. Thirdly, encapsulates the interface of typical query and cross cut statistics, and performs performance optimization based on index. Finally, a Web project is constructed based on "Spring Boot+Bootstrap" to provide users with database resource directory navigation services. The realization of Metadata database and metadata management system helps users quickly understand the database resource overview, and supports efficient and convenient access to experimental data. [ABSTRACT FROM AUTHOR]

The high abundance of IoT devices have caused an unprecedented accumulation of avalanches of geo-referenced IoT spatial data that if could be analyzed correctly would unleash important information. This can feed decision support systems for better decision making and strategic planning regarding important aspects of our lives that depend heavily on location-based services. Several spatial data management systems for IoT data in Cloud has recently gained momentum. However, the literature is still missing a comprehensive survey that conceptualize a convenient framework that classify those frameworks under appropriate categories. In this survey paper, we focus on the management of big geospatial data that are generated by IoT data sources. We also define a conceptual framework and match the works of the recent literature with it. We then identify future research frontiers in the field depending on the surveyed works. [ABSTRACT FROM AUTHOR]

The purpose of this research is to present knowledge in the field of work of databases that contain semi-structured data, also known as NoSQL, in applications using big data. The characteristics of the main categories of the non-relational databases and some of their typical representatives (MongoDB, Redis) have been reviewed from the point of view of using them for storage and analysis of data in the social networks. The article presents a study, conducted on the already existing research in regards to the implementation of the non-relational databases in the social networks. Some of the perspectives of the future research are outlined as well. [ABSTRACT FROM AUTHOR]

In the current era of the Internet of Things (IoT), massive number of sensors are used in our daily lives. Sensors are everywhere around us. They exist in our homes, work places, streets, cars, and even ourselves. Examples include home appliances, wearable devices, and medical sensors. These sensors generate huge amount of dynamic, heterogeneous, and unstructured data that need special handling beyond the capabilities of conventional relational databases. Thus, identification of suitable data management platform to store and query this data is necessary. Despite of its popularity and efficiency in processing various types of big data, there is no single-guided study of how NoSQL data stores will behave with the Internet of Things (IoT) datasets. IoT data have its own characteristics that make it special. IoT data come from various sensors, with a wide range of formats, high velocity, and require high throughput processing with low latency. NoSQL data stores are commonly used to provide flexibility and availability for big data handling. However, there is a lack of comprehensive studies about which NoSQL data store performs the best from the two scalability aspects (scale-up and scale-out) in a distributed and parallel processing environment. This paper benchmarks the commonly used NoSQL data stores (MongoDB, Cassandra, and HBase), and compares their performance with real industrial IoT dataset. In addition, we focus on comparing the throughput, latency, and run time of the evaluated NoSQL data stores. [ABSTRACT FROM AUTHOR]

Nowadays, many NoSQL systems are developed to deal with data elasticity in distributed environments. This is very useful for Data mining such as association rules technique which generates a huge number of rules. To avoid any manual post-processing for selecting the interesting rules, many researchers suggest integrating expert users' knowledge by using ontology and rule patterns. Nevertheless, with NoSQL Big Data that contain very large data, the number of generated rules is so huge that any post-processing becomes complicated especially in industrial areas. Also, any solution and results have to be tested and checked with a real Big Data context. In order to deal with this issue, we use an adjusted approach with ontology and rule patterns to reduce database NoSQL context before generating any rule. After that, we conduct a real experiment on distributed industrial MongoDB database to calculate execution time and generated rules. This work proves the gain in performance for using association rules with ontology in the NoSQL systems. [ABSTRACT FROM AUTHOR]

Abstract This paper presents the big data architecture and work flow used to download georeferenced tweets, store them in a NoSQL database, analyse them using the Apache Spark framework, and visualize the results. The study covers a complete year (from December 10, 2016 to December 10, 2017) in the city of Valencia (Eastern Spain), which is considered to be the third most important in Spain, having a population of nearly 800,000 inhabitants and a size of 135 km2. The concepts of a specific event map and a specific event map with positive or negative sentiment are developed to highlight the location of an event. This approach is undertaken by subtracting the heat map of a specific day from the mean daily heat map, which is obtained by taking into account the 365 days of the studied period. This paper demonstrates how the proposed analysis from tweets can be used to depict city events and discover their spatiotemporal characteristics. Finally, the combination of all daily specific events maps in a single map, leads to the conclusion that the city of Valencia city has appropriate urban infrastructures to support these events. Highlights • Big Data tools are used for harvesting, storing and analysing Twitter data. • Specific events are highlighted automatically in maps. • Urban infrastructure adequacy to city events is studied. • Background knowledge of the dynamics of the city is obtained. [ABSTRACT FROM AUTHOR]

The database (DB) landscape has been significantly diversified during the last decade, resulting in the emergence of a variety of non-relational (also called NoSQL) DBs, e.g., xml and json-document DBs, key-value stores, and graph DBs. To enable access to such data, we generalize the well-known ontology-based data access (OBDA) framework so as to allow for querying arbitrary data sources using sparql. We propose an architecture for a generalized OBDA system implementing the virtual approach. Then, to investigate feasibility of OBDA over non-relational DBs, we compare an implementation of an OBDA system over MongoDB, a popular json-document DB, with a triple store. This article is an extended and revised version of an article that appeared in the proceedings of the 17th International Conference of the Italian Association for Artificial Intelligence (AI*IA) [4]. [ABSTRACT FROM AUTHOR]

One database commonly used by clinicians for searching the medical literature and practicing evidence-based medicine is PubMed. As the literature grows, it has become challenging for users to find the relevant material quickly because most of the time the relevant results are not on the top. In this article, we propose a search and ranking pipeline to improve the search results based on relevancy. We first propose an ensemble model consisting of three classifiers: bidirectional long-short-term memory conditional random field (bi-LSTM-CRF), support vector machine and naive Bayes to recognise PICO (patient, intervention, comparison, outcome) elements from abstracts. The ensemble was trained on an annotated corpus consisting of 5000 abstracts split into 4000 and 1000 training and testing data, respectively. The ensemble recorded an accuracy of 93%. We then retrieved around 927,000 articles from PubMed for the years 2017–2021 (access date 16 April 2021). For every abstract, we extracted and grouped all P, I and O terms, and stored these groups along with the article ID in a separate database. During the search, every P, I and O term of the query is searched only in its corresponding group of every abstract. The scoring method simply counts the number of matches between the query’s P, I and O elements and the words in P, I and O groups, respectively. The abstracts are sorted by the number of matches and the top five abstracts are listed using their pre-stored abstract IDs. A comprehensive user study was conducted where 60 different queries were formulated and used to generate ranked search results using both PubMed and our proposed model. Five medical professionals assessed the ranked search results and marked every item as relevant or non-relevant. Both models were compared using precision@K and mean-average-precision@K metrics where K is 5. For most of the queries, our model produced higher precision@K values than PubMed. The mean-average-precision@K value of our model is also higher than PubMed (0.83 versus 0.67). [ABSTRACT FROM AUTHOR]

The amount of data being produced is increasing constantly, as the number and variety of connected devices are growing and the advances in data storage and mining are supporting this evolution. However, storing and handling high quantities of data is challenging the current Relational Database Management Systems. Big Data and its related products came to help in this matter, and the NoSQL databases arise with the purpose to offer better solutions and features to handle massive amounts of data with higher performance, sometimes near real-time. The present study presents the NoSQL databases scenario and background, and elaborates a detailed study with the characteristics, a features comparison and a performance evaluation of three different NoSQL databases extensively used in the market nowadays: Couchbase, MongoDB and RethinkDB. Tests were performed in two different scenarios: single thread and multiple threads. The results reveal that Couchbase had a better performance at most of the operations, except for retrieving multiple documents and inserting documents with multiple threads, operations in which MongoDB scored better. [ABSTRACT FROM AUTHOR]

As the Internet and big data become more widespread, relational database management systems (RDBMSs) become increasingly inadequate for web applications. To provide what RDBMSs cannot, during the past 15 years distributed database systems (DDBSs) have thus emerged. However, given the complicated structure of these systems, the methods used to validate the efficiency of databases, all towards ensuring quality and security, have become diversified. In response, this paper demonstrates a system for performing automated testing with DDBSs, given that testing is significant in software verification and that accredited systems are more productive in business environments. The proposed system applies several tests to MongoDB-based NoSQL databases to evaluate their instantaneous conditions, such as average query response times and fragment utilizations and, if necessary, suggest improvements for indexes and fragment deployment. Within this context, autogenerated data, replica, meta, system, fragment and index tests are applied. Clearly, the system's most important feature is its fuzzy logic-enabled fragment reallocation module, which allows the creation and application of reallocation strategies that account for data changes in query frequency. [ABSTRACT FROM AUTHOR]

Most of today's world web applications are designed using a 'stack' of various technologies. MEAN stack and MERN stack are two of the most popular and extremely powerful stacks that are used for the development of modern web applications. MERN stands for MongoDB, ExpressJS, ReactJS, and NodeJS and MEAN being for MongoDB, ExpressJS, AngularJS, and NodeJS. The key factor is that both the stacks ushers under a single umbrella named JavaScript. This ensures faster development and helps developers quickly get products to market as the need of understanding different languages to contribute to both the front-end and back-end gets eliminated, and eventually leads to a huge cut in development costs and improves efficiency. This paper revolves around analyzing features, advantages and disadvantages of the two stacks and also throws light upon the key factors that play a major role during the selection of one out of the former two. [ABSTRACT FROM AUTHOR]

In order to solve the bottleneck problem encountered in the large data processing of relational database and satisfy the demand of large data processing for enterprises. this paper proposed a method that migrated the relational database to No-SQL document database. It studied the set model transformation method in MongoDB for the relational model in RDBMS, and proposed a directed graph representation model that expressed the referential integrity of the relationship, and automatically transformed the MongoDB document model based on the relational data model. It realized the insertion algorithm that the RDBMS migrated data into MongoDB’s. Aiming at the above scheme and algorithm, combined with the typical open source RDBMS-MySQL example. this paper validated the generation of directed graph model, the transformation algorithm based on directed graph model and the application of data migration algorithm. The experimental results show that the RDBMS can be smoothly migrated to MongoDB according to a certain data structure. [ABSTRACT FROM AUTHOR]

The 3D cadastre concept brings data models, which are mo-re complex than traditional 2D cadastral data models and could be followed by a large amount of data. The 3D cadastral data should be stored in database management systems, since the cadastral data integrity and consistency have to be satisfied. Relational database management system requires a tabular structure where data are stored within predefined columns and data types, and this could be uncomfortable for 3D cadastre until relational provides full 3D support. This study examines the possibility of using NoSQL databases in the 3D cadastre domain. NoSQL database stores unstructured data, which means that it is not required to define in advance what data types and categories will be used. From the 3D cadastre point of view, the NoSQL approach provides flexibility in data types and allows easier implementation of the 3D cadastral models. The implementation is conducted by using MongoDB and 3D Cadastral Data Model, where 3D cadastral data, including both alphanumerical and geometry part of data, are prepared for importing and then stored, managed, queried, and updated within NoSQL database. Furthermore, data stored in MongoDB are visualized and queried inside a web browser by using Cesium library. [ABSTRACT FROM AUTHOR]

Integration of data stored in heterogeneous database systems is a very challenging task and it may hide several difficulties. As NoSQL databases are growing in popularity, integration of different NoSQL systems and interoperability of NoSQL systems with SQL databases become an increasingly important issue. In this paper, we propose a novel data integration methodology to query data individually from different relational and NoSQL database systems. The suggested solution does not support joins and aggregates across data sources; it only collects data from different separated database management systems according to the filtering options and migrates them. The proposed method is based on a metamodel approach and it covers the structural, semantic and syntactic heterogeneities of source systems. To introduce the applicability of the proposed methodology, we developed a web-based application, which convincingly confirms the usefulness of the novel method. [ABSTRACT FROM AUTHOR]

This paper analyzes the information content of metadata and full texts in German Medical Science (GMS) articles in English language. The object of the study is to compare semantic entities that are used to enrich GMS metadata (titles and abstracts) and GMS full texts. The aim of the study is to test whether using full texts increases the value added information. The comparison and evaluation of semantic entities was done statistically. Measures of descriptive statistics were gathered for this purpose. In addition to the ratio of central tendencies and scatterings, we computed the overlaps and complements of the values. The results show a distinct increase of information when full texts are added. On average, metadata contain 25 differententities and full texts 215. 89% of the concepts in the metadata are also represented in the full texts. Hence, 11% of the metadata concepts are found in the metadata only. In summary, the results show that the addition of full texts increases the informational value, e.g. for information retrieval processes. [ABSTRACT FROM AUTHOR]

The article considers how to speed up data processing in the construction route in DTN network, in particular by applying multi-thread processing using the PHP library "pthreads", selecting the noSQL MongoDB database and applying additional mathematical methods (fuzzy logic) to reduce the amount of data. The results of testing the PHP library "pthreads" on the obtained data set are presented. And also, it is shown the results of comparing the execution time of queries in the DBMS MySQL and MongoDB. [ABSTRACT FROM AUTHOR]

Background: Exploring the structure of genomes and analyzing their evolution is essential to understanding the ecological adaptation of organisms. However, with the large amounts of data being produced by next-generation sequencing, computational challenges arise in terms of storage, search, sharing, analysis and visualization. This is particularly true with regards to studies of genomic variation, which are currently lacking scalable and user-friendly data exploration solutions. Description: Here we present Gigwa, a web-based tool that provides an easy and intuitive way to explore large amounts of genotyping data by filtering it not only on the basis of variant features, including functional annotations, but also on genotype patterns. The data storage relies on MongoDB, which offers good scalability properties. Gigwa can handle multiple databases and may be deployed in either single- or multi-user mode. In addition, it provides a wide range of popular export formats. Conclusions: The Gigwa application is suitable for managing large amounts of genomic variation data. Its user-friendly web interface makes such processing widely accessible. It can either be simply deployed on a workstation or be used to provide a shared data portal for a given community of researchers. [ABSTRACT FROM AUTHOR]

Scientific data storage plays a significant role in research facility. The explosion of data in recent years was always going to make data access, acquiring and management more difficult especially in fusion research field. For future long-pulse experiment like ITER, the extremely large data will be generated continuously for a long time, putting much pressure on both the write performance and the scalability. And traditional database has some defects such as inconvenience of management, hard to scale architecture. Hence a new data storage system is very essential. J-TEXTDB is a data storage and management system based on an application cluster and a storage cluster. J-TEXTDB is designed for big data storage and access, aiming at improving read–write speed, optimizing data system structure. The application cluster of J-TEXTDB is used to provide data manage functions and handles data read and write operations from the users. The storage cluster is used to provide the storage services. Both clusters are composed with general servers. By simply adding server to the cluster can improve the read–write performance, the storage space and redundancy, making whole data system highly scalable and available. In this paper, we propose a data system architecture and data model to manage data more efficient. Benchmarks of J-TEXTDB performance including read and write operations are given. [ABSTRACT FROM AUTHOR]

The big data is currently a growing topic in the world of information technology. Based on the literature mentioned that manage of big data can create significant value for the world economy, improving productivity and competitiveness of enterprises and the public sector as well as creating a large economic surplus for consumers. However, based on the information obtained, the big data is still not widely applied in the company or organization. This study aimed to explore more information about the big data and proceed with making an application prototype big data management. This experiment established with the big data storage that is database, this research use NoSQL database technology that can map the needs of both structured and unstructured. And this research will be carried out migration of Relational Database (RDBMS) into the database MongoDB. Prototype will be create with the object of study is structured and unstructured data. The expected result of this research is a model or prototype of big data management that can help organizations and companies (especially education) to make decisions based on various types of data. [ABSTRACT FROM AUTHOR]

Background: Mass spectrometry-based shotgun proteomics experiments rely on accurate matching of experimental spectra against a database of protein sequences. Existing computational analysis methods are limited in the size of their sequence databases, which severely restricts the proteomic sequencing depth and functional analysis of highly complex samples. The growing amount of public high-throughput sequencing data will only exacerbate this problem. We designed a broadly applicable metaproteomic analysis method (ComPIL) that addresses protein database size limitations. Results: Our approach to overcome this significant limitation in metaproteomics was to design a scalable set of sequence databases assembled for optimal library querying speeds. ComPIL was integrated with a modified version of the search engine ProLuCID (termed "Blazmass") to permit rapid matching of experimental spectra. Proof-of-principle analysis of human HEK293 lysate with a ComPIL database derived from high-quality genomic libraries was able to detect nearly all of the same peptides as a search with a human database (~500x fewer peptides in the database), with a small reduction in sensitivity. We were also able to detect proteins from the adenovirus used to immortalize these cells. We applied our method to a set of healthy human gut microbiome proteomic samples and showed a substantial increase in the number of identified peptides and proteins compared to previous metaproteomic analyses, while retaining a high degree of protein identification accuracy and allowing for a more in-depth characterization of the functional landscape of the samples. Conclusions: The combination of ComPIL with Blazmass allows proteomic searches to be performed with database sizes much larger than previously possible. These large database searches can be applied to complex meta-samples with unknown composition or proteomic samples where unexpected proteins may be identified. The protein database, proteomic search engine, and the proteomic data files for the 5 microbiome samples characterized and discussed herein are open source and available for use and additional analysis. [ABSTRACT FROM AUTHOR]

This paper addresses how to manage planar spatial data using MongoDB, a popular NoSQL database characterized as a document-oriented, rich query language and high availability. The core idea is to flatten a hierarchical R-tree structure into a tabular MongoDB collection, during which R-tree nodes are represented as collection documents and R-tree pointers are expressed as document identifiers. By following this strategy, a storage schema to support R-tree-based create, read, update, and delete (CRUD) operations is designed and a module to manage planar spatial data by consuming and maintaining flattened R-tree structure is developed. The R-tree module is then seamlessly integrated into MongoDB, so that users could manipulate planar spatial data with existing command interfaces oriented to geodetic spatial data. The experimental evaluation, using real-world datasets with diverse coverage, types, and sizes, shows that planar spatial data can be effectively managed by MongoDB with our flattened R-tree and, therefore, the application extent of MongoDB will be greatly enlarged. Our work resulted in a MongoDB branch with R-tree support, which has been released on GitHub for open access. [ABSTRACT FROM AUTHOR]

Indoor thermal comfort has been the subject of several studies in the last years, aiming to create a satisfying work environment for the occupants inside a building. However, thermal comfort is strongly related to the occupants. Accordingly, it is necessary to take into consideration the user's thermal preferences when adjusting the thermal conditions in buildings. This work aims to develop an innovative and smart solution, based on BIM, IoT, Machine Learning, and the user experience, to build a thermal comfort model. A BIM model was designed to ingest data streams from the IoT devices together with the user's thermal preferences specified via a mobile application. The study area was divided into a three-dimensional grid where, for each voxel, the Predicted Mean Vote (PMV) was calculated. The occupant indicates his thermal sensation, noted as Personal Vote (PV), associated with his profile and his position. These measures were used to train a logistic regression model to find the correlation between the PMV and PV. Ultimately, an LSTM Deep Learning model was trained to forecast the optimal temperature of the study area in the next working hours. The results were visualized both on the mobile application and in the BIM environment. [ABSTRACT FROM AUTHOR]

Background: Exploring the structure of genomes and analyzing their evolution is essential to understanding the ecological adaptation of organisms. However, with the large amounts of data being produced by next-generation sequencing, computational challenges arise in terms of storage, search, sharing, analysis and visualization. This is particularly true with regards to studies of genomic variation, which are currently lacking scalable and user-friendly data exploration solutions. Description: Here we present Gigwa, a web-based tool that provides an easy and intuitive way to explore large amounts of genotyping data by filtering it not only on the basis of variant features, including functional annotations, but also on genotype patterns. The data storage relies on MongoDB, which offers good scalability properties. Gigwa can handle multiple databases and may be deployed in either single- or multi-user mode. In addition, it provides a wide range of popular export formats. Conclusions: The Gigwa application is suitable for managing large amounts of genomic variation data. Its user-friendly web interface makes such processing widely accessible. It can either be simply deployed on a workstation or be used to provide a shared data portal for a given community of researchers. [ABSTRACT FROM AUTHOR]

A Tokamak device consists of numerous control systems, which need to be integrated. CODAC (Control, Data Access and Communication) system requires the configuration settings of these control systems to carry out the integration smoothly. SDD (Self-description data) is designed to describe the static configuration of control systems. ITER CODAC group has released an SDD software package for control system designers to manage the static configuration, but it is specific for ITER plant control systems. Following the idea of ITER SDD, we developed a flexible and scalable SDD framework to develop SDD software for J-TEXT and other sophisticated devices. The SDD framework describes the configuration settings of various control systems, including physical and logical elements and their relation information, in SDD models which are classified into Components and Connections. The framework is composed of three layers: the MongoDB database, an open-source, dynamic schema, NoSQL (Not Only SQL) database; the SDD service, which maps SDD models to MongoDB and handles the transaction and business logic; the SDD applications, which can be used to create and maintain SDD information, and generate various kinds of output using the stored SDD information. [ABSTRACT FROM AUTHOR]

Because healthcare data comes from multiple, vastly different sources, databases must adopt a range of models to process and store it. A polyglot-persistent framework combines relational, graph, and document data models to accommodate information variety. [ABSTRACT FROM PUBLISHER]

The Regional Climate Model Evaluation System (RCMES) facilitates the rapid, flexible inclusion of NASA observations into climate model evaluations. RCMES provides two fundamental components. A database (RCMED) is a scalable point-oriented cloud database used to elastically store remote sensing observations and to make them available using a space time query interface. The analysis toolkit (RCMET) is a Python-based toolkit that can be delivered as a cloud virtual machine, or as an installer package deployed using Python Buildout to users in order to allow for temporal and spatial regridding, metrics calculation (RMSE, bias, PDFs, etc.) and end-user visualization. RCMET is available to users in an 'offline', lone scientist mode based on a virtual machine dynamically constructed with model outputs and observations to evaluate; or on an institution's computational cluster seated close to the observations and model outputs. We have leveraged RCMES within the content of the Coordinated Regional Downscaling Experiment (CORDEX) project, working with the University of Cape Town and other institutions to compare the model output to NASA remote sensing data; in addition we are also working with the North American Regional Climate Change Assessment Program (NARCCAP). In this paper we explain the contribution of cloud computing to RCMES's specifically describing studies of various cloud databases we evaluated for RCMED, and virtualization toolkits for RCMET, and their potential strengths in delivering user-created dynamic regional climate model evaluation virtual machines for our users. [ABSTRACT FROM AUTHOR]

This project describes the development of an application using a search engine full-text inverted index system, on information contained in a NoSQL database. With reference previous work entitled "Integration of full text searches NoSQL Databases" in the Journal Links, Volume 8, Number 1, January-June 2011, pages 81- 92. [1]. [ABSTRACT FROM AUTHOR]

• In clinical trials, data collection/management together with data analysis minimises bias. • The use of NoSQL databases in clinical trials can be a user-friendly way of managing multidimensional clinical research data. • The use of relational databases introduces certain limitations when dealing with clinical data. Background and objectives : In the last decade, clinical trial management systems have become an essential support tool for data management and analysis in clinical research. However, these clinical tools have design limitations, since they are currently not able to cover the needs of adaptation to the continuous changes in the practice of the trials due to the heterogeneous and dynamic nature of the clinical research data. These systems are usually proprietary solutions provided by vendors for specific tasks. In this work, we propose FIMED, a software solution for the flexible management of clinical data from multiple trials, moving towards personalized medicine, which can contribute positively by improving clinical researchers quality and ease in clinical trials. Methods : This tool allows a dynamic and incremental design of patients' profiles in the context of clinical trials, providing a flexible user interface that hides the complexity of using databases. Clinical researchers will be able to define personalized data schemas according to their needs and clinical study specifications. Thus, FIMED allows the incorporation of separate clinical data analysis from multiple trials. Results : The efficiency of the software has been demonstrated by a real-world use case for a clinical assay in Melanoma disease, which has been indeed anonymized to provide a user demonstration. FIMED currently provides three data analysis and visualization components, guaranteeing a clinical exploration for gene expression data: heatmap visualization, clusterheatmap visualization, as well as gene regulatory network inference and visualization. An instance of this tool is freely available on the web at https://khaos.uma.es/fimed. It can be accessed with a demo user account, "researcher" , using the password "demo". Conclusion : This paper shows FIMED as a flexible and user-friendly way of managing multidimensional clinical research data. Hence, without loss of generality, FIMED is flexible enough to be used in the context of any other disease where clinical data and assays are involved. [ABSTRACT FROM AUTHOR]

Recent developments in massive machine-type communication (mMTC) scenarios have given rise to never-seen requirements, which triggered the Industry 4.0 revolution. The new scenarios bring even more pressure to comply with the reliability and communication security and enable flawless functionality of the critical infrastructure, e.g., smart grid infrastructure. We discuss typical network grid architecture, communication strategies, and methods for building scalable and high-speed data processing and storage platform. This paper focuses on the data transmissions using the sets of standards IEC 60870-6 (ICCP/TASE.2). The main goal is to introduce the TASE.2 traffic generator and the data collection back-end with the implemented load balancing functionality to understand the limits of current protocols used in the smart grids. To this end, the assessment framework enabling generating and collecting TASE.2 communication with long-term data storage providing high availability and load balancing capabilities was developed. The designed proof-of-concept supports complete cryptographic security and allows users to perform the complex testing and verification of the TASE.2 network nodes configuration. Implemented components were tested in a cloud-based Microsoft Azure environment in four geographically separated locations. The findings from the testing indicate the high performance and scalability of the proposed platform, allowing the proposed generator to be also used for high-speed load testing purposes. The load-balancing performance shows the CPU usage of the load-balancer below 15% while processing 5000 messages per second. This makes it possible to achieve up to a 7-fold improvement of performance resulting in processing up to 35,000 messages per second. [ABSTRACT FROM AUTHOR]

Reliability of foreign keys, which is natural in relational databases, requires additional efforts when working with non-relational databases, as non-relational database management systems generally don't support foreign key constraints due to their distributed nature. Referential integrity is an important property whenever documents need to refer to each other, which is the common case. This work discusses an implementation of a verification approach which makes use of the MapReduce programming model, in order to detect incorrect references in document oriented databases that may be caused by errors in the program code or incomplete transactions. Furthermore, the method can be applied for the verification of more complex dependencies between documents, such that bind aggregated values from certain sets of documents with the values of documents referred by them. [ABSTRACT FROM AUTHOR]

Document-oriented bases allow high flexibility in data representation which facilitates a rapid development of applications and enables many possibilities for data structuring. Unfortunately, in many cases, due to this flexibility and the absence of data modelling, the choice of a data representation is neglected by developers leading to many issues on several aspects of the document base and application quality; e.g., memory print, data redundancy, readability and maintainability. We aim at facilitating the study of data structuring alternatives and providing objective metrics to better reveal the advantages and disadvantages of a structure with respect to user needs. The main contributions of our approach are twofold. First of all, the semi-automatic generation of many suitable alternatives for data structuring given an initial UML model. Second, the automatic computation of structural metrics, allowing a comparison of the alternatives for JSON-compatible schema abstraction. These metrics reflect the complexity of the structure and are intended to be used in decision criteria for schema analysis and design process. This work capitalises on experiences with MongoDB, XML and software complexity metrics. The paper presents the schema generation and the metrics together with a validation scenario where we discuss how to use the results in a schema recommendation perspective. [ABSTRACT FROM AUTHOR]

• A database built with disruption related information. • This database makes it easy to develop disruption predictors. • The disruption database can be applied to different Tokamak devices. • The process of generating disruption related information can be automated. Machine learning based disruption prediction methods have exhibited good prediction performance with higher success rate, lower false alarm rate and earlier warning time than physical based methods. One important thing pushed recent advances in machine learning field is high-quality training data. So a database with rich set of accurate disruption related information is crucial to the development of a high performance disruption predictor. In order to develop machine learning based disruption predictors fast and iteratively on J-TEXT, a database dedicated for machine learning disruption prediction has been built. This database provides not only disruption related labels, interfaces for querying the data based on user proved filter and auto generation of training and test data, but also interface for benchmark the predictors. Its modular design allows us to plug-in various shot analysis modules which process diagnostic signals and generate different labels automatically. These modules can be scheduled and run parallel on a cluster which will speed up the shot analysis process. The generated labels are inserted into a MongoDB NoSQL database for later querying. But one major hurdle for machine learning disruption prediction is they perform un-acceptably poor on devices other than they are trained on. It requires data from many different tokamaks to possibly develop a cross machine predictor, so this database also has a data import module which reads diagnostic data from different data sources such as MDSplus and store them as HDF5 files with unified data structure on a parallel file system. It is easy to import data from different machines and provide a unified data access interface. [ABSTRACT FROM AUTHOR]

This work is focused on creating an open-source software-based solution for monitoring traffic transmitted through gigabit passive optical network. In this case, the data are captured by the field-programmable gate array (FPGA) card and reassembled using parsing software from a passive optical network built on the International Telecommunication Unit telecommunication section (ITU-T) G.984 gigabit-capable passive optical network GPON recommendation. Then, the captured frames are converted by suitable software into GPON frames, which will be further processed for analysis. Due to the high transfer rate of GPON recommendations, the work describes the issue of writing to the Mongo database system. In order to achieve the best possible results and minimal loss of transmitted frames, a series of tests were performed. The proposed test scenarios are based on different database writing approaches and are implemented in the Python and C# programming languages. Based on our results, it has been shown that the high processing speed is too high for Python processing. Critical operations must be implemented in the C# programming language. Due to rapid application development, Python can only be used for noncritical time-consuming data processing operations. [ABSTRACT FROM AUTHOR]