Search

In this paper we describe a parallel algorithm for generating all non-isomorphic rank $3$ simple matroids with a given multiplicity vector. We apply our implementation in the HPC version of GAP to generate all rank $3$ simple matroids with at most $14$ atoms and a splitting characteristic polynomial. We have stored the resulting matroids alongside with various useful invariants in a publicly available, ArangoDB-powered database. As a byproduct we show that the smallest divisionally free rank $3$ arrangement which is not inductively free has $14$ hyperplanes and exists in all characteristics distinct from $2$ and $5$. Another database query proves that Terao's freeness conjecture is true for rank $3$ arrangements with $14$ hyperplanes in any characteristic., Improved exposition

International audience; Next generation sequencing technologies enabled high-density genotyping for large numbers of samples. Nowadays SNP calling pipelines produce up to millions of such markers, but which need to be filtered in various ways according to the type of analyses. One of the main challenges still lies in the management of an increasing volume of genotyping files that are difficult to handle for many applications. Here, we provide a practical guide for efficiently managing large genomic variation data using Gigwa, a user-friendly, scalable and versatile application that may be deployed either remotely on web servers or on a local machine.

Este documento describe FIMED 2.0, un servicio para la gestión flexible y análisis de datos clínicos heterogéneos. Esta herramienta de software permite la gestión flexible de datos clínicos de múltiples ensayos, lo que puede ayudar a mejorar la calidad de los datos clínicos y facilitar los ensayos clínicos. El servicio propuesto se ha desarrollado sobre una base de datos NoSQL (MongoDB) que permite recoger e integrar los datos clínicos en esquemas dinámicos e incrementales en función de sus necesidades y de los requisitos de la investigación clínica. requisitos de la investigación clínica. Basándonos en nuestras experiencias con la Gestión Flexible de Datos Biomédicos (FIMED), hemos desarrollado esta nueva versión de la herramienta con el objetivo no sólo de replicar la anterior, sino también de incluir más análisis de redes reguladoras de genes y visualización de datos orientados a anotar la funcionalidad de los genes e identificar los genes centrales. Esta versión permite Esta versión permite al profesional utilizar cuatro métodos diferentes de construcción de redes, como como la asimilación de datos, la interpolación lineal, el conjunto basado en árboles o la regresión Boosting Machine. Puede encontrar una versión gratuita de esta herramienta en la web https://khaos.uma.es/fimedV2. Se ha creado una cuenta de usuario de demostración para proporcionar una demostración de usuario, "iwbbio", utilizando la contraseña "demo". Un caso de uso real para un ensayo clínico en la enfermedad del melanoma también se incluye en esta demostración, que sí ha sido anonimizada. Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Scientific data come from a domestic rooftop PV plant of 4 kW, located at University of La Réunion (-20.90126, 55.48422), in the South-West of Indian Ocean. The electrical data (voltage and current at AC and DC side, produced power and energy, grid frequency) are collected via the PV system's own operating and management module. Solar irradiance data was measured by a pyranometer in the vicinity of the solar panels and at the same tilt (26°). Ambient and PV module temperature data were measured with two thermocouples. The present dataset includes healthy behaviour and different types of shading faults, identified and labelled by means of a numeric variable. The data streams from each connected device require proper techniques to ensure their persistence. To be able to provide both efficient ingestion and retrieval of these time series collections, the NoSQL database management system InfluxDB has been implemented. The whole dataset can be used for instance for PV modelling, PV plant behaviour analysis, PV production forecasting and PV Fault Detection and Diagnosis (FDD) tool development., This work was supported by the ERDF ("European Regional Development Fund") and the Region Council of La Réunion.

Tiled maps are one of the key GIS technologies used in the development and construction of WebGIS in the era of big data; there is an urgent need for high-performance tile map services hosted on big data GIS platforms. To address the current inefficiency of massive tile map data management and access, this paper proposes a massive tile map data access model that utilizes the jump consistent hash algorithm. Via the uniformity and consistency of a certain seed of a pseudo-random function, the algorithm can generate a storage slot for each tile data efficiently. By recording the slot information in the head of a row key, a uniform distribution of the tiles on the physical cluster nodes is achieved. This effectively solves the problem of hotspotting caused by the monotonicity of tile row keys in the data access process, thereby maximizing the random-access performance of a big data platform and greatly improving concurrent database access. Experiments show that this model can significantly improve the efficiency of tile map data access by more than 39% compared to a direct storage method, thereby confirming the model’s advantages in accessing massive tile map data on a big data GIS platform.

The analysis of relational and NoSQL databases leads to the conclusion that these data processing systems are to some extent complementary. In the current Big Data applications, especially where extensive analyses (so-called Big Analytics) are needed, it turns out that it is nontrivial to design an infrastructure involving data and software of both types. Unfortunately, the complementarity negatively influences integration possibilities of these data stores both at the data model and data processing levels. In terms of performance, it may be beneficial to use a polyglot persistence, a multimodel approach or multilevel modeling, or even to transform the SQL database schema into NoSQL and to perform data migration between the relational and NoSQL databases. Another possibility is to integrate a NoSQL database and relational database with the help of a third data model. The aim of the paper is to show these possibilities and present some new methods of designing such integrated database architectures.

Modern NoSQL database systems use log-structured merge (LSM) storage architectures to support high write throughput. LSM architectures aggregate writes in a mutable MemTable (stored in memory), which is regularly flushed to disk, creating a new immutable file called an SSTable. Some of the SSTables are chosen to be periodically merged—replaced with a single SSTable containing their union. A mergepolicy (a.k.a. compaction policy) specifies when to do merges and which SSTables to combine. A bounded depth merge policy is one that guarantees that the number of SSTables never exceeds a given parameter k, typically in the range 3–10. Bounded depth policies are useful in applications where low read latency is crucial, but they and their underlying combinatorics are not yet well understood. This paper compares several bounded depth policies, including representative policies from industrial NoSQL databases and two new ones based on recent theoretical modeling, as well as the standard Tiered policy and Leveled policy. The results validate the proposed theoretical model and show that, compared to the existing policies, the newly proposed policies can have substantially lower write amplification with comparable read amplification.

The current trend of information systems and informatization moving forward in big steps also influences database systems. This direction of a data transfer from the databases with a rigid structure to the systems with a free data structure created a vast amount of challenges, including problems with backward compatibility, the inconsistency of data types or possibly of data editing during the transformation transfer. As a solution to this problem, we created MigrationLayer which can manage the structural and data compatibility between a relational and nonrelational database which is divided into three modules: The first is module of transformation, respectively, of the data transfer from the relational database to the nonrelational database. The second module is based on the catching of the data during its' modification which could influence the data entering the system, and the third module is a module of backward compatibility, which, based on the data change, evokes a consolidation edit of the structure and of course, the data. Experiments show that MigrationLayer can catch the data incoming to the transformation module in real-time with sufficient effectiveness. This leads to reduction of the time needed for the modifications after the whole migration ends. The module of the backward compatibility effectively helps us to maintain the backward compatibility between the structures of relational and nonrelational database.

Background and objectives: In the last decade, clinical trial management systems have become an essential support tool for data management and analysis in clinical research. However, these clinical tools have design limitations, since they are currently not able to cover the needs of adaptation to the continuous changes in the practice of the trials due to the heterogeneous and dynamic nature of the clinical research data. These systems are usually proprietary solutions provided by vendors for specific tasks. In this work, we propose FIMED, a software solution for the flexible management of clinical data from multiple trials, moving towards personalized medicine, which can contribute positively by improving clinical researchers quality and ease in clinical trials. Methods: This tool allows a dynamic and incremental design of patients’ profiles in the context of clinical trials, providing a flexible user interface that hides the complexity of using databases. Clinical researchers will be able to define personalized data schemas according to their needs and clinical study specifications. Thus, FIMED allows the incorporation of separate clinical data analysis from multiple trials. Results: The efficiency of the software has been demonstrated by a real-world use case for a clinical assay in Melanoma disease, which has been indeed anonymized to provide a user demonstration. FIMED currently provides three data analysis and visualization components, guaranteeing a clinical exploration for gene expression data: heatmap visualization, clusterheatmap visualization, as well as gene regulatory network inference and visualization. An instance of this tool is freely available on the web at https://khaos.uma.es/fimed . It can be accessed with a demo user account, “researcher”, using the password “demo”. Conclusion: This paper shows FIMED as a flexible and user-friendly way of managing multidimensional clinical research data. Hence, without loss of generality, FIMED is flexible enough to be used in the context of any other disease where clinical data and assays are involved.

Model-Driven Engineering is largely recognized as the most powerful method for the design of complex software. This study deals with the automated archival of metadata about the content of UML class diagrams (a particularly relevant category of models) into a pre-existing repository. To define the structure of the repository, we started from the definition of a UML metamodel. From the latter, we derived the schema of the metadata repository. Then, a parser was developed that is responsible for extracting the useful information from the XMI file about class diagrams and enters it as metadata into the repository. The parser has been implemented as a Java web interface, while the metadata repository has been implemented as a PostgreSQL database based on the JSONB data type. The metadata repository is thought to support modelers in the initial phase of the process of the development of new models when looking for artifacts to start from. The schema of the metadata repository and the Java code of the parser are available from the authors.

The 3D cadastre concept brings data models, which are more complex than traditional 2D cadastral data models and could be followed by a large amount of data. The 3D cadastral data should be stored in database management systems, since the cadastral data integrity and consistency have to be satisfied. Relational database management system requires a tabular structure where data are stored within predefined columns and data types, and this could be uncomfortable for 3D cadastre until relational provides full 3D support. This study examines the possibility of using NoSQL databases in the 3D cadastre domain. NoSQL database stores unstructured data, which means that it is not required to define in advance what data types and categories will be used. From the 3D cadastre point of view, the NoSQL approach provides flexibility in data types and allows easier implementation of the 3D cadastral models. The implementation is conducted by using MongoDB and 3D Cadastral Data Model, where 3D cadastral data, including both alphanumerical and geometry part of data, are prepared for importing and then stored, managed, queried, and updated within NoSQL database. Furthermore, data stored in MongoDB are visualized and queried inside a web browser by using Cesium library.

Today, one of the main challenges of big data research is the processing of big time-series data. Moreover, time data analysis is of considerable importance, because previous trends are useful for predicting the future. Due to the considerable delay when the volume of the data increases, the presence of redundancy, and the innate lack of time-series structures, the traditional relational data model does not seem to be adequately able to analyze time data. Moreover, many traditional data structures do not support time operators, which results in an inefficient access to time data. Therefore, relational database management systems have difficulty in dealing with big data-it may require massively parallel software that runs on many servers. This has led us to implement Chronos Software, an in-memory background-based time database for key-value pairs; this software was implemented using C++ language. An independent design has been suggested through appropriately using temporal algorithms, parallelism algorithms, and methods of data storage in RAM. Our results indicate that the employment of RAM for storing the data and of the Timeline Index algorithm for getting access to the time background of the keys in Chronos translate into an increase of about 40%-90% in the efficiency as compared with other databases, such as MySQL and MongoDB.

NoSQL databases are flexible and efficient for many data intensive applications, and the key-value store is one of them. In recent years, a new Ethernet accessed disk drive called the &ldquo, Kinetic Drive&rdquo, was developed by Seagate. This new Kinetic Drive is specially designed for key-value stores. Users can directly access data with a Kinetic Drive via its IP address without going through a storage server/layer. With this new innovation, the storage stack and architectures of key-value store systems have been greatly changed. In this paper, we propose a novel global key-value store system based on Kinetic Drives. We explore data management issues including data access, key indexing, data backup, and recovery. We offer scalable solutions with small storage overhead. The performance evaluation shows that our location-aware design and backup approach can reduce the average distance traveled for data access requests.

In this article the model of strong replicas agreement was analyzed. The process of preparing and conducting the nature experiment in the cloud in order to proof the model adequacy was described. Specifications of the program for implementation of database access to the NoSQL system and the program for handling journals were presented. One part of obtained experiments results was used for model adaptation, another part - for adequacy evaluating. The adequacy analysis is presented.

BACKGROUND: The objective of this research is to compare the relational and non-relational (NoSQL) database systems approaches in order to store, recover, query and persist standardized medical information in the form of ISO/EN 13606 normalized Electronic Health Record XML extracts, both in isolation and concurrently. NoSQL database systems have recently attracted much attention, but few studies in the literature address their direct comparison with relational databases when applied to build the persistence layer of a standardized medical information system. METHODS: One relational and two NoSQL databases (one document-based and one native XML database) of three different sizes have been created in order to evaluate and compare the response times (algorithmic complexity) of six different complexity growing queries, which have been performed on them. Similar appropriate results available in the literature have also been considered. RESULTS: Relational and non-relational NoSQL database systems show almost linear algorithmic complexity query execution. However, they show very different linear slopes, the former being much steeper than the two latter. Document-based NoSQL databases perform better in concurrency than in isolation, and also better than relational databases in concurrency. CONCLUSION: Non-relational NoSQL databases seem to be more appropriate than standard relational SQL databases when database size is extremely high (secondary use, research applications). Document-based NoSQL databases perform in general better than native XML NoSQL databases. EHR extracts visualization and edition are also document-based tasks more appropriate to NoSQL database systems. However, the appropriate database solution much depends on each particular situation and specific problem. This research has been partially supported by projects PI12/00508 “Plataforma de innovación en nuevos servicios de Telemedicina y e-Salud: Definición, diseño y desarrollo de herramientas para interoperabilidad, seguridad del paciente y ayuda a la decisión (Innovation platform in new services based on Telemedicine and e-Health: definition, design and development of tools for interoperability, patient security and support to decision) – PITES-ISA”, PI15CIII/00003 “Plataforma de innovación en Telemedicina y e-Salud: TIC para los retos de I+i en servicios de salud (Platform for Innovation in Telemedicine and e-Health: ICT for the challenges of I + i in health services) – PITES-TIiSS”, PI12/01476, PI12/01558, PI12/01399 “Continuidad de la asistencia basada en estándares de tecnología y conocimiento: arquitectura tecnológica para usos primario y secundario de la información (Continuity of care based on technology and knowledge standards: technological architecture for primary and secondary information uses.) – CAMAMA2” and PI15CIII/00010 - PI15/00321 - PI15/00831 “Modelo normalizado de historia clínica electrónica compartida para la asistencia sanitaria y social integrada. Factibilidad y utilidad de un repositorio de uso secundario en cáncer de mama (Standard model of shared electronic health record for integrated helath and social care. Feasibility and usefulness of a repository for secundary use for breast cancer patients.) – CAMAMA 3” from Fondo de Investigación Sanitaria (FIS) Plan Nacional de I + D + i. Sí

The popularization of big data makes the enterprise need to store more and more data. The data in the enterprise&rsquo, s database must be accessed as fast as possible, but the Relational Database (RDB) has the speed limitation due to the join operation. Many enterprises have changed to use a NoSQL database, which can meet the requirement of fast data access. However, there are more than hundreds of NoSQL databases. It is important to select a suitable NoSQL database for a certain enterprise because this decision will affect the performance of the enterprise operations. In this paper, fifteen categories of NoSQL databases will be introduced to find out the characteristics of every category. Some principles and examples are proposed to choose an appropriate NoSQL database for different industries.

NoSQL databases such as Apache Cassandra have attracted large interest in recent years thanks to their high availability, scalability, flexibility and low latency. Still, there is limited research work on performance engineering methods for NoSQL databases, which yet are needed since these systems are highly distributed and thus can incur significant cost/performance trade-offs. To address this need, we propose a novel queueing network model for the Cassandra NoSQL database aimed at supporting resource provisioning. The model defines explicitly key configuration parameters of Cassandra such as consistency levels and replication factor, allowing engineers to compare alternative system setups. Experimental results based on the YCSB benchmark indicate that, with a small amount of training for the estimation of its input parameters, the proposed model achieves good predictive accuracy across different loads and consistency levels. The average performance errors of the model compared to the real results are between 6% and 10%. We also demonstrate the applicability of our model to other NoSQL databases and we show other possible utilization of it.

With the proliferation of high-throughput technologies, genome-level data analysis has become common in molecular biology. Bioinformaticians are developing extensive resources to annotate and mine biological features from high-throughput data. The underlying database management systems for most bioinformatics software are based on a relational model. Modern non-relational databases offer an alternative that has flexibility, scalability, and a non-rigid design schema. Moreover, with an accelerated development pace, non-relational databases like CouchDB can be ideal tools to construct bioinformatics utilities. We describe CouchDB by presenting three new bioinformatics resources: (a) geneSmash, which collates data from bioinformatics resources and provides automated gene-centric annotations, (b) drugBase, a database of drug–target interactions with a web interface powered by geneSmash, and (c) HapMap-CN, which provides a web interface to query copy number variations from three SNP-chip HapMap datasets. In addition to the web sites, all three systems can be accessed programmatically via web services.

The web applications and websites of the enterprises are accessed by a huge number of users with the expectation of reliability and high availability. Social networking sites are generating the data exponentially large amount of data. It is a challenging task to store data efficiently. SQL and NoSQL are mostly used to store data. As RDBMS cannot handle the unstructured data and huge volume of data, so NoSQL is better choice for web applications. Graph database is one of the efficient ways to store data in NoSQL. Graph database allows us to store data in the form of relation. In Graph representation each tuple is represented by node and the relationship is represented by edge. But, to handle the exponentially growth of data into a single server might decrease the performance and increases the response time. Data partitioning is a good choice to maintain a moderate performance even the workload increases. There are many data partitioning techniques like Range, Hash and Round robin but they are not efficient for the small transactions that access a less number of tuples. NoSQL data stores provide scalability and availability by using various partitioning methods. To access the Scalability, Graph partitioning is an efficient way that can be easily represent and process that data. To balance the load data are partitioned horizontally and allocate data across the geographical available data stores. If the partitions are not formed properly result becomes expensive distributed transactions in terms of response time. So the partitioning of the tuple should be based on relation. In proposed system, Schism technique is used for partitioning the Graph. Schism is a workload aware graph partitioning technique. After partitioning the related tuples should come into a single partition. The individual node from the graph is mapped to the unique partition. The overall aim of Graph partitioning is to maintain nodes onto different distributed partition so that related data come onto the same cluster.

Although NoSQL databases are claimed to be schemaless, several NoSQL database vendors have chosen JSON as agile data representation format and provide a JSON-based API or query facility to simplify the life of application developers. Whereas many applications require the management of temporal data, the JSON Schema language lacks explicit support for time-varying data. In this paper, for a systematic approach to the management of temporal data in NoSQL databases, we propose a framework called Temporal JSON Schema (τJSchema), inspired by the τXSchema framework defined for XML data. τJSchema allows defining a temporal JSON schema from a conventional JSON schema and a set of temporal logical and physical characteristics. Our framework guarantees logical and physical data independence for temporal schemas and provides a low-impact solution since it requires neither modifications of existing JSON documents, nor extensions to the JSON format, the JSON Schema language, and all related tools and languages.

Big Data has recently gained popularity and has strongly questioned relational databases as universal storage systems, especially in the presence of analytical workloads. As result, co-relational alternatives, commonly known as NOSQL (Not Only SQL) databases, are extensively used for Big Data. As the primary focus of NOSQL is on performance, NOSQL databases are directly designed at the physical level, and consequently the resulting schema is tailored to the dataset and access patterns of the problem in hand. However, we believe that NOSQL design can also benefit from traditional design approaches. In this paper we present a method to design databases for analytical workloads. Starting from the conceptual model and adopting the classical 3-phase design used for relational databases, we propose a novel design method considering the new features brought by NOSQL and encompassing relational and co-relational design altogether.

The article investigates a model of matching record versions. The goal of this work is to analyse the model adequacy. This model allows estimating a user’s processing time distribution of the record versions and a distribution of the record versions count. The second option of the model was used, according to which, for a client the time to process record versions depends explicitly on the number of updates, performed by the other users between the sequential updates performed by a current client. In order to prove the model adequacy the real experiment was conducted in the cloud cluster. The cluster contains 10 virtual nodes, provided by DigitalOcean Company. The Ubuntu Server 14.04 was used as an operating system (OS). The NoSQL system Riak was chosen for experiments. In the Riak 2.0 version and later provide “dotted vector versions” (DVV) option, which is an extension of the classic vector clock. Their use guarantees, that the versions count, simultaneously stored in DB, will not exceed the count of clients, operating in parallel with a record. This is very important while conducting experiments. For developing the application the java library, provided by Riak, was used. The processes run directly on the nodes. In experiment two records were used. They are: Z – the record, versions of which are handled by clients; RZ – service record, which contains record update counters. The application algorithm can be briefly described as follows: every client reads versions of the record Z, processes its updates using the RZ record counters, and saves treated record in database while old versions are deleted form DB. Then, a client rereads the RZ record and increments counters of updates for the other clients. After that, a client rereads the Z record, saves necessary statistics, and deliberates the results of processing. In the case of emerging conflict because of simultaneous updates of the RZ record, the client obtains all versions of that record and merges their information using a special algorithm. There were conducted 4 series of experiments with 5 ones in each. From experiment to experiment the following parameters were changed: clients count and k – an attitude of the average deliberation time to the average processing time of one update. An average relative error of the average record versions count was 7.5%; dispersion was 5.5%. An average processing time error of the record versions was estimated very small, i.e. 0. 22%.

Relational databases are providing storage for several decades now. However for today's interactive web and mobile applications the importance of flexibility and scalability in data model can not be over-stated. The term NoSQL broadly covers all non-relational databases that provide schema-less and scalable model. NoSQL databases which are also termed as Internetage databases are currently being used by Google, Amazon, Facebook and many other major organizations operating in the era of Web 2.0. Different classes of NoSQL databases namely key-value pair, document, column-oriented and graph databases enable programmers to model the data closer to the format as used in their application. In this paper, data modeling and query syntax of relational and some classes of NoSQL databases have been explained with the help of an case study of a news website like Slashdot. Refereed/Peer-reviewed