Search

A Data Warehouse (DW) is a centralized database that stores large volumes of historical data for analysis and reporting. In a world where enterprise data grows exponentially, new architectures are being investigated to overcome the deficiencies of traditional Database Management Systems (DBMSs), driving a shift towards more modern, cloud-based solutions that provide resources such as distributed processing, columnar storage, and horizontal scalability without the overhead of physical hardware management, i.e., a Database as a Service (DBaaS). Choosing the appropriate class of DBMS is a critical decision for organizations, and there are important differences that impact data volume and query performance (e.g., architecture, data models, and storage) to support analytics in a distributed cloud environment efficiently. In this sense, we carry out an experimental evaluation to analyze the performance of several DBaaS and the impact of data modeling, specifically the usage of a partially normalized Star Schema and a fully denormalized Flat Table Schema, to further comprehend their behavior in different configurations and designs in terms of data schema, storage form, memory availability, and cluster size. The analysis is done in two volumes of data generated by a well-established benchmark, comparing the performance of the DW in terms of average execution time, memory usage, data volume, and loading time. Our results provide guidelines for efficient DW design, showing, for example, that the denormalization of the schema does not guarantee improved performance, as solutions performed differently depending on its architecture. We also show that a Hybrid Processing (HTAP) NewSQL solution can outperform solutions that support only Online Analytical Processing (OLAP) in terms of overall execution time, but that the performance of each query is deeply influenced by its selectivity and by the number of join functions. [ABSTRACT FROM AUTHOR]

The growing importance of data analytics is leading to a shift in data management strategy at many companies, moving away from simple data storage towards adopting Online Analytical Processing (OLAP) query analysis. Concurrently, NoSQL databases are gaining ground as the preferred choice for storing and querying analytical data. This article presents a comprehensive, systematic mapping, aiming to consolidate research efforts related to the integration of OLAP with NoSQL databases in Big Data environments. After identifying 1646 initial research studies from scientific digital repositories, a thorough examination of their content resulted in the acceptance of 22 studies. Utilizing the snowballing technique, an additional three studies were selected, culminating in a final corpus of twenty-five relevant articles. This review addresses the growing importance of leveraging NoSQL databases for OLAP query analysis in response to increasing data analytics demands. By identifying the most commonly used NoSQL databases with OLAP, such as column-oriented and document-oriented, prevalent OLAP modeling methods, such as Relational Online Analytical Processing (ROLAP) and Multidimensional Online Analytical Processing (MOLAP), and suggested models for batch and real-time processing, among other results, this research provides a roadmap for organizations navigating the integration of OLAP with NoSQL. Additionally, exploring computational resource requirements and performance benchmarks facilitates informed decision making and promotes advancements in Big Data analytics. The main findings of this review provide valuable insights and updated information regarding the integration of OLAP cubes with NoSQL databases to benefit future research, industry practitioners, and academia alike. This consolidation of research efforts not only promotes innovative solutions but also promises reduced operational costs compared to traditional database systems. [ABSTRACT FROM AUTHOR]

The main objective of this article is to implement business intelligence (BI) technology to improve incident management in the control center of a security company; making use of the Ralph Kimball method for the development of the data mart, contemplating the phases: planning of projects, definition of requirements, design of technological architecture, performing a dimensional modeling a star with 9 dimensions and 1 table made, also used the structured query language (MySQL) database, extract, transform, and load (ETL). Development the pentaho data integration, which allowed the migration, transformation and cleaning of the data from the online transaction processing (OLTP) and online analytical processing (OLAP) database, obtaining as a result that the BI managed to increase the level of the service by 28.42% and reducing the frequency index by 25%. [ABSTRACT FROM AUTHOR]

Updating the most recent set of point cloud data is critical in autonomous driving environments. However, existing systems for point cloud data management often fail to ensure real-time updates or encounter situations in which data cannot be effectively refreshed. To address these challenges, this study proposes a distributed hybrid transactional/analytical processing architecture designed for the efficient management and real-time processing of point cloud data. The proposed architecture leverages both columnar and row-based tables, enabling it to handle the substantial workloads associated with its hybrid architecture. The construction of this architecture as a distributed database cluster ensures real-time online analytical process query performance through query parallelization. A dissimilarity analysis algorithm for point cloud data, built by utilizing the capabilities of the spatial database, updates the point cloud data for the relevant area whenever the online analytical process query results indicate high dissimilarity. This research contributes to ensuring real-time hybrid transactional/analytical processing workload processing in dynamic road environments, helping autonomous vehicles generate safe, optimized routes. [ABSTRACT FROM AUTHOR]

The exponential growth of business data coming from heterogeneous sources imposes the use of new generations of database management systems and new data storage architectures. The major players in the big data market have turned to NoSQL (Not only SQL) technology, which provides a flexible data model and high scalability. In this paper, we investigate OLAP (Online Analytical Processing) implementation using columnar databases (a type of NoSQL system). We provide a set of formal transformation rules in order to map the multidimensional conceptual model to a target model that is suitable for the column-oriented model. Then, we propose two OLAP cube operators called MRC-Cube and SC-Cube, which allow to build the OLAP cube using the MapReduce paradigm and Spark respectively. We conduct an experimental comparison of their performance to analogous relational implementation using Oracle OLAP, we focus particularly on read latency metric under different experimental configurations. The obtained results show a clear difference when performing the OLAP cube building between the relational implementation and the columnar one. [ABSTRACT FROM AUTHOR]

Due to unstructured and large amounts of data, relational databases are no longer suitable for data management. As a result, new databases known as NOSQL have been introduced. The issue is that such a database is difficult to analyze. Online analytical processing (OLAP) is the foundational technology for data analysis in business intelligence. Because these technologies were designed primarily for relational database systems, performing OLAP in NOSQL is difficult. We present a model for extracting OLAP cubes from a document-oriented NOSQL database in this article. A scalable Naïve Bayes classifier method was used for this purpose. The proposed solution is divided into three stages of preparation, Naïve Bayes, and NBMR. Our proposed algorithm, NBMR, is based on the Naïve Bayes classifier (NBC) and the MapReduce (MR) programming model. Each NOSQL database document with nearly the same attribute will belong to the same class, and as a result, OLAP cubes can be used to perform data analysis. Because the proposed model allows for distributed and parallel Naïve Bayes Classifier computing, it is appropriate and suitable for large-scale data sets. Our proposed model is a proper and efficient approach when considering the speed and reduced the number of required comparisons. [ABSTRACT FROM AUTHOR]

Decision makers in the educational field always seek new technologies and tools, which provide solid, fast answers that can support decision-making process. They need a platform that utilize the students’ academic data and turn them into knowledge to make the right strategic decisions. In this paper, a roadmap for implementing a data driven decision support system (DSS) is presented based on an educational data mart. The independent data mart is implemented on the students’ degrees in 8 subjects in a private school (Al-Iskandaria Primary School in Basrah province, Iraq). The DSS implementation roadmap is started from pre-processing paper-based data source and ended with providing three categories of online analytical processing (OLAP) queries (multidimensional OLAP, desktop OLAP and web OLAP). Key performance indicator (KPI) is implemented as an essential part of educational DSS to measure school performance. The static evaluation method shows that the proposed DSS follows the privacy, security and performance aspects with no errors after inspecting the DSS knowledge base. The evaluation shows that the data driven DSS based on independent data mart with KPI, OLAP is one of the best platforms to support short-tolong term academic decisions. [ABSTRACT FROM AUTHOR]

Copyright of Informatica (03505596) is the property of Slovene Society Informatika and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

A database management system is a constant application of science that provides a platfonn for the creation, movement, and use of voluminous data. The area has witnessed a series of developments and technological advancements from its conventional structured database to the recent buzzword, bigdata. This paper aims to provide a complete model of a relational database that is still being widely used because of its well known ACID properties namely, atomicity, consistency, integrity and durability. Specifically, the objective of this paper is to highlight the adoption of relational model approaches by bigdata techniques. Towards addressing the reason for this in corporation, this paper qualitatively studied the advancements done over a while on the relational data model. First, the variations in the data storage layout are illustrated based on the needs of the application. Second, quick data retrieval teclmiques like indexing, que1y processing and concu!l'ency control methods are revealed. The paper provides vital insights to appraise the efficiency of the structured database in the unstructure.d enviromnent, pa1ticularly when both consistency and scalability become an issue in the working of the hybrid transactional and analytical database management system. [ABSTRACT FROM AUTHOR]

This article sets in context Data Warehouses (DWs) and Online Analytical Processing (OLAP) against the backdrop of databases and Big Data and shows how data warehouses and OLAP were incorporated into the digital archiving specifications work in the E-ARK project. Both theoretical and practical aspects are thoroughly considered, and the article ends with a detailed use case of the steps needed to produce a Dissemination Information Package (DIP) for an OLAP representation for a data warehouse. [ABSTRACT FROM AUTHOR]

Summary: The advent of big data technologies has changed the way many companies manage their data. Several companies moved their data to the cloud using the concept of database‐as‐a‐service (DBaaS). Moving databases to the cloud presents several challenges related to flexible and scalable management of data. Although some of these companies migrated to NoSQL databases, most still rely on relational databases in the cloud to manage data, especially data that is critical to the decision making process. Online analytical processing (OLAP) queries take a long time to be processed, thus demanding high‐performance capabilities from their associated database systems to get results in a feasible time. In this article, we propose a middleware solution that can be deployed in any cloud provider, named C‐ParGRES, which explores database replication and interquery and intraquery parallelism to efficiently support OLAP queries in the cloud. C‐ParGRES is an extension of ParGRES, an open‐source database cluster middleware for high‐performance OLAP query processing in clusters. C‐ParGRES exploits cloud capabilities such as on‐demand resource provisioning and elasticity. In addition, C‐ParGRES can create multiple and independent virtual clusters for different database and users. We evaluate C‐ParGRES with two real‐world OLAP applications, both from the Brazilian Institute of Geography and Statistics. Results show that C‐ParGRES is a cost‐effective solution for OLAP query processing in the cloud. [ABSTRACT FROM AUTHOR]

Intra-query fault tolerance has increasingly been a concern for online analytical processing, as more and more enterprises migrate data analytical systems from mainframes to commodity computers. Most massive parallel processing (MPP) databases do not support intra-query fault tolerance. They may suffer from prolonged query latency when running on unreliable commodity clusters. While SQL-on-Hadoop systems can utilize the fault tolerance support of low-level frameworks, such as MapReduce and Spark, their cost-effectiveness is not always acceptable. In this paper, we propose a smart intra-query fault tolerance (SIFT) mechanism for MPP databases. SIFT achieves fault tolerance by performing checkpointing, i.e., materializing intermediate results of selected operators. Different from existing approaches, SIFT aims at promoting query success rate within a given time. To achieve its goal, it needs to: (1) minimize query rerunning time after encountering failures and (2) introduce as less checkpointing overhead as possible. To evaluate SIFT in real-world MPP database systems, we implemented it in Greenplum. The experimental results indicate that it can improve success rate of query processing effectively, especially when working with unreliable hardware. [ABSTRACT FROM AUTHOR]

In the paper "The Construction of Sports Culture Industry Growth Forecast Model Based on Big Data", the feasibility of sports culture industry growth prediction based on big data theory from the perspective of data mining is analyzed, and a sport culture industry growth prediction model based on genetic neural network is proposed. In the paper "Design of Sports Culture Data Fusion System based on Data Mining Algorithm", K-means spatial clustering, fusion decision tree, Naive Bayes and other data mining algorithms and data warehouse technologies are combined to the sports stationery industry. In the paper "Semantic Change Analysis of Korean Verbs based on Massive Culture Corpus Data", database technology, big data mining and analysis, on-line analytical processing technology and verb feature analysis technology are combined to construct the basic model. [Extracted from the article]

The explosion of data over the last decades puts significant strain on the computational capacity of the central processing unit (CPU), challenging online analytical processing (OLAP). While previous studies have shown the potential of using Field Programmable Gate Arrays (FPGAs) in database systems, integrating FPGA-based hardware acceleration with relational databases remains challenging because of the complex nature of relational database operations and the need for specialized FPGA programming skills. Additionally, there are significant challenges related to optimizing FPGA-based acceleration for specific database workloads, ensuring data consistency and reliability, and integrating FPGA-based hardware acceleration with existing database infrastructure. In this study, we proposed a novel end-to-end FPGA-based acceleration system that supports native SQL statements and storage engine. We defined a callback process to reload the database query logic and customize the scanning method for database queries. Through middleware process development, we optimized offloading efficiency on PCIe bus by scheduling data transmission and computation in a pipeline workflow. Additionally, we designed a novel five-stage FPGA microarchitecture module that achieves optimal clock frequency, further enhancing offloading efficiency. Results from systematic evaluations indicate that our solution allows a single FPGA card to perform as well as 8 CPU query processes, while reducing CPU load by 34%. Compared to using 4 CPU cores, our FPGA-based acceleration system reduces query latency by 1.7 times without increasing CPU load. Furthermore, our proposed solution achieves 2.1 times computation speedup for data filtering compared with the software baseline in a single core environment. Overall, our work presents a valuable end-to-end hardware acceleration system for OLAP databases. [ABSTRACT FROM AUTHOR]

This paper reports on conceptual development in the areas of database mining and knowledge discovery in databases (KDD). The authors' efforts have also led to a prototype implementation, called MOTC, for exploring hypothesis space in large and complex data sets. Their KDD conceptual development rests on two main principles. First, they use the crosstab representation for working with qualitative data. This is by now standard in on-line analytical processing (OLAP) applications, and the authors reaffirm it with additional reasons. Second, and innovatively, they use prediction analysis as a measure of goodness for hypotheses. Prediction analysis is an established statistical technique for analysis of associations among qualitative variables. It generalizes and subsumes a large number of other such measures of association, depending on specific assumptions the user is willing to make. As such, it provides a very useful framework for exploring hypothesis space in a KDD context. The paper illustrates these points with an extensive discussion of MOTC. [ABSTRACT FROM AUTHOR]

This paper presents CirroData, a high-performance SQL-on-Hadoop system designed for Big Data analytics workloads. As a home-grown enterprise-level online analytical processing (OLAP) system with more than seven-year research and development (R&D) experiences, we share our design details to the community about how to achieve high performance in CirroData. Multiple optimization techniques have been discussed in the paper. The effectiveness and the efficiency of all these techniques have been proved by our customers' daily usage. Benchmark-level studies, as well as several real application case studies of CirroData, have been presented in this paper. Our evaluations show that CirroData can outperform various types of counterpart database systems in the community, such as "Spark+Hive", "Spark+HBase", Impala, DB-X/Y, Greenplum, HAWQ, and others. CirroData can achieve up to 4.99x speedup compared with Greenplum, HAWQ, and Spark in the standard TPC-H queries. Application-level evaluations demonstrate that CirroData outperforms "Spark+Hive" and "Spark+HBase" by up to 8.4x and 38.8x, respectively. In the meantime, CirroData achieves the performance speedups for some application workloads by up to 20x, 100x, 182.5x, 92.6x, and 55.5x as compared with Greenplum, DB-X, Impala, DB-Y, and HAWQ, respectively. [ABSTRACT FROM AUTHOR]

This paper studies the problem of efficiently utilizing hybrid memory systems, consisting of both Dynamic Random Access Memory (DRAM) and novel Non-Volatile Memory (NVM) in database management systems (DBMS) for online analytical processing (OLAP) workloads. We present a methodology to determine the database operators that are responsible for most main memory accesses. Our analysis uses both cost models and empirical measurements. We develop heuristic decision procedures to allocate data in hybrid memory at the time that the data buffers are allocated, depending on the expected memory access frequency. We implement these heuristics in the MonetDB column-oriented database and demonstrate performance improvement and energy-efficiency as compared to state-of-the-art application-agnostic hybrid memory management techniques. [ABSTRACT FROM AUTHOR]

Classical databases represent the traditional DBMS's and the most widely used DBMS in the world of databases and information systems; they have been regarded as the best systems for managing data. Today with the growth of data for both applications and consumers, and its openness to the public, traditional databases are not able to meet the needs of a large number of applications, including Online Analytical Processing models: OLAP data processing and Business Intelligence analysis. As a result, many dedicated DBMS have emerged like: Column Store, In Memory and NoSQL databases. They meet users' expectations and fit well with current needs. Consequently, the scope of classical databases has become increasingly restricted to handle Online Transaction Processing OLTP models and small problems. However, those dedicated solutions hardly cope with rich features of DBMS like simplicity, flexibility or scalability. They remain limited to mainly process single database models, and users cannot deal with both OLTP and OLAP queries in a single environment. To deal with this problem, vertical fragmentation is the best way to effectively handle the OLAP model, but this technique fails to handle some analytical queries with low selectivity, presenting poor results in some cases. In this perspective, we propose a new vertical fragmentation design T-Plotter which makes it possible to deal effectively with the whole of analytical queries and improve the performance of DBMS's to process the OLAP data models. [ABSTRACT FROM AUTHOR]

We presentsmart drill-down, an operator for interactively exploring a relational table to discover and summarize “interesting” groups of tuples. Each group of tuples is described by arule. For instance, the rule $(a, b, \star, 1000)$ tells us that there are 1,000 tuples with value $a$ in the first column and $b$ in the second column (and any value in the third column). Smart drill-down presents an analyst with a list of rules that together describe interesting aspects of the table. The analyst can tailor the definition of interesting, and can interactively apply smart drill-down on an existing rule to explore that part of the table. We demonstrate that the underlying optimization problems areNP-Hard, and describe an algorithm for finding the approximately optimal list of rules to display when the user uses a smart drill-down, and a dynamic sampling scheme for efficiently interacting with large tables. Finally, we perform experiments on real datasets on our experimental prototype to demonstrate the usefulness of smart drill-down and study the performance of our algorithms. [ABSTRACT FROM AUTHOR]

In this article, I studied Business Intelligence and Olap, that is, I have emphasized the basics of OLAP, how to deploy OLAP, metadata, the multidimensional model, and the cube. In terms of OLAP deployment, we have studied the most important server implementations that are available (MOLAP, ROLAP, HOLAP), while also emphasizing DOLAP. In addition to OLAP, we have also studied the most commonly used ETL metadata and tools. In the last part we studied the multidimensional model and cube model by realizing a vision of the DB2 Cube and MQT. [ABSTRACT FROM AUTHOR]

This paper presents VelocityOLAP (VOLAP), a distributed real-time OLAP system for high-velocity data. VOLAP makes use of dimension hierarchies, is highly scalable, exploits both multi-core and multi-processor parallelism, and can guarantee serializable execution of insert and query operations. In contrast to other high performance OLAP systems such as SAP HANA or IBM Netezza that rely on vertical scaling or special purpose hardware, VOLAP supports cost-efficient horizontal scaling on commodity hardware or modest cloud instances. Experiments on 20 Amazon EC2 nodes with TPC-DS data show that VOLAP is capable of bulk ingesting data at over 600 thousand items per second, and processing streams of interspersed insertions and aggregate queries at a rate of approximately 50 thousand insertions and 20 thousand aggregate queries per second with a database of 1 billion items. VOLAP is designed to support applications that perform large aggregate queries, and provides similar high performance for aggregations ranging from a few items to nearly the entire database. [ABSTRACT FROM PUBLISHER]

Copyright of Przegląd Elektrotechniczny is the property of Przeglad Elektrotechniczny and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Data Warehouses (DWs) are historical databases on business events, organized as multi-dimensional hypercubes that support analytical decision making. Extract, Transform, and Load (ETL) processes apply measurement functions to compute parameterized scores from the business events, such as sales figures, customer reliability scores, churn likelihood, or sentiment indices. These scores, saved as measures in the DW, serve as basis for analytical reports and corporate performance analysis. Dimensions model subject-oriented data used as analysis perspectives when interpreting the measures. While measures and measurement functions are traditionally regarded as stable within the DW schema, its dimension values commonly change over time. In reality, measures are also subject to change if DW designers (i) tune a parameter of the underlying measurement function, or (ii) update the scoring algorithm as a whole. In both scenarios, the changes must be obvious to the business analysts. Otherwise the changed measure semantics leads to incomparable measure values, and thus unsound and worthless analysis results. To handle measure evolution properly, this paper proposes Slowly Changing Measures (SCMs) as an additional DW modeling concept. Its core idea is a valid time for measurement functions, analogous to dimensions. The paper proposes four increasingly rich SCM types, each specifying a set of options for the change history management of measure definitions. Most of the SCM types provide for probable changes of measurement functions at design time, reducing manual interference to the necessary minimum upon the actual change. Each SCM type is explained in detail, and illustrated using a practical scenario. Furthermore, the paper presents a proof-of-concept prototype based on the TPC-H business model, and implemented in a simple relational database system. The pros and cons of every SCM type are discussed, and recommendations given for their implementation in a practical DW system. [ABSTRACT FROM AUTHOR]

In contrast to queries for on-line transaction processing (OLTP) systems that typically access only a small portion of a database, OLAP queries may need to aggregate large portions of a database which often leads to performance issues. In this paper we introduce CR-OLAP , a scalable C loud based R eal-time OLAP system based on a new distributed index structure for OLAP, the distributed PDCR tree . CR-OLAP utilizes a scalable cloud infrastructure consisting of multiple commodity servers (processors). That is, with increasing database size, CR-OLAP dynamically increases the number of processors to maintain performance. Our distributed PDCR tree data structure supports multiple dimension hierarchies and efficient query processing on the elaborate dimension hierarchies which are so central to OLAP systems. It is particularly efficient for complex OLAP queries that need to aggregate large portions of the data warehouse, such as “report the total sales in all stores located in California and New York during the months February–May of all years”. We evaluated CR-OLAP on the Amazon EC2 cloud, using the TPC-DS benchmark data set. The tests demonstrate that CR-OLAP scales well with increasing number of processors, even for complex queries. For example, for an Amazon EC2 cloud instance with 16 processors, a data warehouse with 160 million tuples, and a TPC-DS OLAP query stream where each query aggregates between 60% and 95% of the database, CR-OLAP achieved a query latency of below 0.3 s which can be considered a real time response. [ABSTRACT FROM AUTHOR]

Traceability has always been acknowledged as a relevant topic in Software Engineering. However, keeping track of the relationships between the different assets involved in a development process is a complex and tedious task. The fact that the main assets handled in any model-driven engineering project are models and model transformations eases the task. In order to take advantage of this scenario, which has not been appropriately capitalized on by the most widely adopted model transformation languages before, this work presents MeTAGeM-Trace, a methodological and technical proposal with which to support the model-driven development of model transformations that include trace generation. The underlying idea is to start from a high-level specification of the transformation which is subsequently refined into lower-level transformation models in terms of a set of DSLs until the source code that implements the transformation can be generated. Running this transformation produces not only the corresponding target models, but also a trace model between the elements of the source and target models. As part of the proposal, an EMF-based toolkit has been developed to support the development of ATL and ETL model transformations. This toolkit has been empirically validated by conducting a set of case studies following a systematic research methodology. [ABSTRACT FROM PUBLISHER]

Improving OLAP (Online Analytical Processing) query performance in a distributed system on top of Hadoop is a challenging task. An OLAP Cube query comprises several relational operations, such as selection, join, and group-by aggregation. It is well-known that star join and group-by aggregation are the most costly operations in a Hadoop database system. These operations indeed increase network traffic and may overflow memory; to overcome these difficulties, numerous partitioning and data load balancing techniques have been proposed in the literature. However, some issues remain questionable, such as decreasing the Spark stages and the network I/O for an OLAP query being executed on a distributed system. In a precedent work, we proposed a novel data placement strategy for a big data warehouse over a Hadoop cluster. This data warehouse schema enhances the projection, selection, and star-join operations of an OLAP query, such that the system's query-optimizer can perform a star join process locally, in only one spark stage without a shuffle phase. Also, the system can skip loading unnecessary data blocks when executing the predicates. In this paper, we extend our previous work with further technical details and experiments, and we propose a new dynamic approach to improve the group-by aggregation. To evaluate our approach, we conduct some experiments on a cluster with 15 nodes. Experimental results show that our method outperforms existing approaches in terms of OLAP query evaluation time. • The Group-by aggregation may involve high communication cost during the shuffle phase. • We propose a dynamic technique for the Partitioning and Load Balancing (PLB) of data. • Our approach Enhances OLAP query execution time over Hadoop Clusters compared to existing approaches. • Our approach combines data and workload-driven models. • The star join operation is performed in only one Spark stage, without a shuffle phase. [ABSTRACT FROM AUTHOR]

The paper discusses the methods to apply OLAP techniques for multidimensional databases that leverage the existing, performance-enhancing technique, known as practical pre-aggregation, by making this technique relevant to a much wider range of medical applications, as a logistic support to the data warehousing techniques. The transformations have practically low computational complexity and they may be implemented using standard relational database technology. The paper also describes how to integrate the transformed hierarchies in current OLAP systems, transparently to the user and proposes a flexible, "multimodel" federated system for extending OLAP querying to external object databases. [ABSTRACT FROM AUTHOR]

The article reports on the integration of in-memory technology in the forecasting and planning functions of organizations. Topics discussed include the impact of databases and OLAP technology on the speed, flexibility, and timeliness of data analysis, the creation and use of data cubes, and data synchronization. Also provided is information regarding the benefits of distributed computing.

This paper describes a CASE shell of a new type, namely, the instrumental system SWS constructed in accordance with a framework data model. A scheme of the instrumental system is proposed. The list of main user functions of the SWS shows that the typification and formalization of requests to a database with a well-formalized data model and unification and minimization of the interface allow modelling the totality of user requests. The results of a numerical experiment on the real-time formation of OLAP data elements are presented. The real-time approach is shown to be most advanced. [ABSTRACT FROM AUTHOR]

With the popularity of column-store databases, modern multi-core CPUs, and general-purpose computing on graphics processing units (GPGPUs), there will be radical changes in how processing is done in the online analytical processing (OLAP) and data warehousing fields. Cube computation is a core and time-consuming problem which has been researched extensively. However, most of the algorithms have been proposed without considering the prevalent multi-core architectures and column storage. This paper presents a new parallel cube algorithm that takes advantage of multi-core architectures. We first propose a cache-conscious bottom-up computation (BUC) algorithm called CC-BUC that adopts an integrated bottom-up and breadth-first partitioning order. Each dimension is separately stored and processed. In processing each dimension, breadth-first data scanning and results outputting reduce memory I/O and enhance cache locality. Cache misses are limited in a dimension scope, and translation lookaside buffer (TLB) misses are reduced. Based on CC-BUC, we give a multi-core architecture-based cube algorithm called MC-Cubing. Multiple partitions are processed simultaneously and multiple threads undergo parallel execution inside each partition. MC-Cubing is consistent with multi-core architectures and high parallelism. The layout and associated algorithms take advantage of single instruction, multiple data (SIMD) instructions and thread-level parallelism (TLP). We implement and demonstrate the effectiveness of MC-Cubing on two multi-core architectures: multi-core CPUs and GPUs. Experimental results show that the MC-Cubing algorithm can speed up nearly six times faster than BUC in real datasets. [ABSTRACT FROM AUTHOR]

In this paper, we propose a methodology for providing linguistic answers to queries involving the comparison of time series obtained from data cubes with time dimension. Time series related to events which are interesting for the user are obtained by querying data cubes using OnLine Analytical Processing (OLAP) operations on the time dimension. The comparison of these query results can be summarized so that an appropriate short linguistic description of the series is provided to the user. Our approach is based on linguistically quantified statements and pointwise definitions of the degree and sign of local change. Our linguistic summaries are well suited to be included in an interface layer of a data warehouse system, improving the quality of human-machine interaction and the understandability of the results. © 2011 Wiley Periodicals, Inc. [ABSTRACT FROM AUTHOR]

Chapter 4 of the book "Foundations and Trends in Databases," by Rada Chirkova and Jun Yang is presented. It explores the ideas of data-intensive computing and data warehousing in a standard database system. It highlights the use of view selection and index selection to speed up queries in database management and also focuses on online analytical processing (OLAP) of data.

We propose content-based methods for construction of recommendations for reports in the OLAP reporting tool. Recommendations are generated based on preference information in user profile, which is updated implicitly by collecting and analyzing user activity in the reporting tool. Taking advantage of data about user preferences for data warehouse schema elements, existing reports that potentially may be interesting to the user are distinguished and recommended. The approach used for recommending reports is composed of two methods - cold-start and hot-start. [ABSTRACT FROM AUTHOR]

Optimizing the two computing resources of any computing system - time and space - has always been one of the priority objectives of any database. A current and effective solution in this respect is the computer database. Optimizing computer applications by means of database machines has been a steady preoccupation of researchers since the late seventies. Several information technologies have revolutionized the present information framework. Out of these, those which have brought a major contribution to the optimization of the databases are: efficient handling of large volumes of data (Data Warehouse, Data Mining, OLAP - On Line Analytical Processing), the improvement of DBMS - Database Management Systems facilities through the integration of the new technologies, the dramatic increase in computing power and the efficient use of it (computer networks, massive parallel computing, Grid Computing and so on). All these information technologies, and others, have favored the resumption of the research on database machines and the obtaining in the last few years of some very good practical results, as far as the optimization of the computing resources is concerned. [ABSTRACT FROM AUTHOR]

Several classes of scientific and commercial applications require the execution of a large number of independent tasks. One highly successful and low-cost mechanism for acquiring the necessary computing power for these applications is the ‘public-resource computing’, or ‘desktop Grid’ paradigm, which exploits the computational power of private computers. So far, this paradigm has not been applied to data mining applications for two main reasons. First, it is not straightforward to decompose a data mining algorithm into truly independent sub-tasks. Second, the large volume of the involved data makes it difficult to handle the communication costs of a parallel paradigm. This paper introduces a general framework for distributed data mining applications called Mining@home. In particular, we focus on one of the main data mining problems: the extraction of closed frequent itemsets from transactional databases. We show that it is possible to decompose this problem into independent tasks, which however need to share a large volume of the data. We thus introduce a data-intensive computing network, which adopts a P2P topology based on super peers with caching capabilities, aiming to support the dissemination of large amounts of information. Finally, we evaluate the execution of a pattern extraction task on such network. Copyright © 2009 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Database machines research was initiated at the end of the seventies, but after that, a long period of time, it was stopped. Several new information technologies revolutionized the modern informatics context. Mainly, it is about: efficient handling of large volumes of data (Data Warehouse, Data Mining, OLAP -- On Line Analytical Processing), the improvement of DBMS -- Database Management Systems facilities through the integration of the new technologies, the dramatic increase in computing power and the efficient use of it (computer networks, massive parallel computing, Grid Computing and so on). All these information technologies, and others, have favored the resumption of the research on database machines and the obtaining in the last few years of some very good practical results. [ABSTRACT FROM AUTHOR]

In recent years, a number of indirect data collection methodologies have led to the proliferation of uncertain data. Such databases are much more complex because of the additional challenges of representing the probabilistic information. In this paper, we provide a survey of uncertain data mining and management applications. We will explore the various models utilized for uncertain data representation. In the field of uncertain data management, we will examine traditional database management methods such as join processing, query processing, selectivity estimation, OLAP queries, and indexing. In the field of uncertain data mining, we will examine traditional mining problems such as frequent pattern mining, outlier detection, classification, and clustering. We discuss different methodologies to process and mine uncertain data in a variety of forms. [ABSTRACT FROM AUTHOR]

We consider the problem of improving the performance of OLAP applications in a database cluster (DBC), which is a low cost and effective parallel solution for query processing. Current DBC solutions for OLAP query processing provide for intra-query parallelism only, at the cost of full replication of the database. In this paper, we proposemore efficient distributed database design alternatives which combine physical/virtual partitioning with partial replication.We also propose a new load balancing strategy that takes advantage of an adaptive virtual partitioning to redistribute the load to the replicas. Our experimental validation is based on the implementation of our solution on the SmaQSS DBC middleware prototype. Our experimental results using the TPC-H benchmark and a 32-node cluster show very good speedup. [ABSTRACT FROM AUTHOR]

OLAP query processing is critical for enterprise grids. Capitalizing on our experience with the ParGRES database cluster, we propose a middleware solution, GParGRES, which exploits database replication and inter- and intra-query parallelism to efficiently support OLAP queries in a grid. GParGRES is designed as a wrapper that enables the use of ParGRES in PC clusters of a grid (in our case, Grid5000). Our approach has two levels of query splitting: grid-level splitting, implemented by GParGRES, and node-level splitting, implemented by ParGRES. GParGRES has been partially implemented as database grid services compatible with existing grid solutions such as the open grid service architecture and the Web services resource framework. We give preliminary experimental results obtained with two clusters of Grid5000 using queries of the TPC-H Benchmark. The results show linear or almost linear speedup in query execution, as more nodes are added in all tested configurations. Copyright © 2008 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Abstract: The discovery of dependencies between attributes in databases is an important problem in data mining, and can be applied to facilitate future decision-making. In the present paper some properties of the branching dependencies are examined. We define a minimal branching dependency and we propose an algorithm for finding all minimal branching dependencies between a given set of attributes and a given attribute in a relation of a database. Our examination of the branching dependencies is motivated by their application in a database storing realized sales of products. For example, finding out that arbitrary p products have totally attracted at most q new users can prove to be crucial in supporting the decision making. In addition, we also consider the fractional and the fractional branching dependencies. Some properties of these dependencies are examined. An algorithm for finding all fractional dependencies between a given set of attributes and a given attribute in a database relation is proposed. We examine the general case of an arbitrary relation, as well as a particular case where the problem of discovering the fractional dependencies is considerably simplified. [Copyright &y& Elsevier]

The article describes a system that employs wavelets to support exact, approximate, and progressive online analytical processing (OLAP) queries on large multidimensional datasets, while keeping update costs relatively low. OLAP tools support a range of complex analytical queries, unlike traditional databases. Meanwhile, progressive data analysis system (ProDA) supports a wide range of analytical queries while also being able to handle massive datasets. The authors used ProDA to analyze real-world data applications.

Due to the ability of graphs to represent more generic and more complicated relationships among different objects, graph mining has played a significant role in data mining, attracting increasing attention in the data mining community. In addition, frequent coherent subgraphs can provide valuable knowledge about the underlying internal structure of a graph database, and mining frequently occurring coherent subgraphs from large dense graph databases has witnessed several applications and received considerable attention in the graph mining community recently. In this article, we study how to efficiently mine the complete set of coherent closed quasi-cliques from large dense graph databases, which is an especially challenging task due to the fact that the downward-closure property no longer holds. By fully exploring some properties of quasi-cliques, we propose several novel optimization techniques which can prune the unpromising and redundant subsearch spaces effectively. Meanwhile, we devise an efficient closure checking scheme to facilitate the discovery of closed quasi-cliques only. Since large databases-cannot be held in main memory, we also design an out-of-core solution with efficient index structures for mining coherent closed quasi-cliques from large dense databases. We call this Cocain. Thorough performance study shows that Cocain is very efficient and scalable for large dense graph databases. [ABSTRACT FROM AUTHOR]

The ability to approximately answer aggregation queries accurately and efficiently is of great benefit for decision support and data mining tools. In contrast to previous sampling-based studies, we treat the problem as an optimization problem where, given a workload of queries, we select a stratified random sample of the original data such that the error in answering the workload queries using the sample is minimized. A key novelty of our approach is that we can tailor the choice of samples to be robust, even for workloads that are "similar" but not necessarily identical to the given workload. Finally, our techniques recognize the importance of taking into account the variance in the data distribution in a principled manner. We show how our solution can be implemented on a database system, and present results of extensive experiments on Microsoft SQL Server that demonstrate the superior quality of our method compared to previous work. [ABSTRACT FROM AUTHOR]

Several studies have demonstrated the effectiveness of the Haar wavelet decomposition as a tool for reducing large amounts of data down to compact wavelet synopses that can be used to obtain fast, accurate approximate answers to user queries. Although originally designed for minimizing the overall mean-squared (i.e., L2-norm) error in the data approximation, recently proposed methods also enable the use of Haar wavelets in minimizing other error metrics, such as the relative error in data value reconstruction, which is arguably the most important for approximate query answers. Relatively little attention, however, has been paid to the problem of using wavelet synopses as an approximate query answering tool over complex tabular datasets containing multiple measures, such as those typically found in real-life OLAP applications. Existing decomposition approaches will either operate on each measure individually, or treat all measures as a vector of values and process them simultaneously. As we demonstrate in this article, these existing individual or combined storage approaches for the wavelet coefficients of different measures can easily lead to suboptimal storage utilization, resulting in drastically reduced accuracy for approximate query answers. To address this problem, in this work, we introduce the notion of an extended wavelet coefficient as a flexible, efficient storage method for wavelet coefficients over multimeasure data. We also propose novel algorithms for constructing effective (optimal or near-optimal) extended wavelet-coefficient synopses under a given storage constraint, for both sum-squared error and relative-error norms. Experimental results with both real-life and synthetic datasets validate our approach, demonstrating that our techniques consistently obtain significant gains in approximation accuracy compared to existing solutions. [ABSTRACT FROM AUTHOR]

Given an OLAP query expressed over multiple source OLAP databases, we study the problem of estimating the resulting OLAP target database. The problem arises when it is not possible to derive the result from a single database. The method we use is linear indirect estimation, commonly used for statistical estimation. We examine two obvious computational methods for computing such a target database, called the full cross-product (F) and preaggregation (P) methods. We study the accuracy and computational cost of these methods. While the F method provides a more accurate estimate, it is more expensive computationally than P. Our contribution is in proposing a third, new method, called the partial preaggregation method (PP), which is significantly less expensive than F, but just as accurate. We prove formally that the PP method yields the same results as the F method, and provide analytical and experimental results on the accuracy and computational benefits of the PP method. [ABSTRACT FROM AUTHOR]

The Internet now offers more than just simple information to the users. Decision makers can now issue analytical as opposed to transactional queries that involve massive data (such as, aggregations of millions of rows in a relational database) in order to identify useful trends and patterns. Such queries are often referred to as On-Line-Analytical Processing (OLAP). Typically, pages carrying query results do not exhibit temporal locality and, therefore, are not considered for caching at Internet proxies. In OLAP processing, this is a major problem as the cost of these queries is significantly larger than that of the transactional queries. This paper proposes a technique to reduce the response time for OLAP queries originating from geographically distributed private LANs and issued through the Web toward a central data warehouse (DW) of an enterprise. An active caching scheme is introduced that enables the LAN proxies to cache some parts of the data, together with the semantics of the DW, in order to process queries and construct the resulting pages. OLAP queries arriving at the proxy are either satisfied locally or from the DW, depending on the relative access costs. We formulate a cost model for characterizing the respective latencies, taking into consideration the combined effects of both common Web access and query processing. We propose a cache admittance and replacement algorithm that operates on a hybrid Web-OLAP input, outperforming both pure-Web and pure-OLAP caching schemes. [ABSTRACT FROM AUTHOR]