Search

Big Data (i.e., social big data, vehicular big data, healthcare big data etc) points to massive and complex data, that require special technologies and approaches for storage, processing, and analysis. Similarly, big data applications are software and systems utilizing large and complex datasets to extract insights, support decision-making, and address diverse business and societal challenges. Recently, the significance of big data applications has grown immensely for organizations across diverse sectors as they increasingly rely on insights derived from data. The increasing reliance on data insights has rendered traditional technologies and platforms inefficient due to scalability limitations and performance issues. This study contributes by identifying key domains impacted by big data, examining its effect on decision-making, addressing inherent complexities and opportunities, exploring core technologies, and offering solutions for potential concerns. Additionally, it conducts a comparative analysis to demonstrate the superiority of this research. These contributions provide valuable insights into the evolving landscape shaped by big data applications. [ABSTRACT FROM AUTHOR]

Big data involves processing vast amounts of data using advanced techniques. Its potential is harnessed for predictive analytics, a sophisticated branch that anticipates unknown future events by discerning patterns observed in historical data. Various techniques obtained from modeling, data mining, statistics, artificial intelligence, and machine learning are employed to analyze available history to extract discriminative patterns for predictors. This study aims to analyze the main research approaches on Big Data Predictive Analytics (BDPA) based on very up-to-date published articles from 2014 to 2023. In this article, we fully concentrate on predictive analytics using big data mining techniques, where we perform a Systematic Literature Review (SLR) by reviewing 109 articles. Based on the application and content of current studies, we introduce taxonomy including seven major categories of industrial, e-commerce, smart healthcare, smart agriculture, smart city, Information and Communications Technologies (ICT), and weather. The benefits and weaknesses of each approach, potentially important changes, and open issues, in addition to future paths, are discussed. The compiled SLR not only extends on BDPA’s strengths, open issues, and future works but also detects the need for optimizing the insufficient metrics in big data applications, such as timeliness, accuracy, and scalability, which would enable organizations to apply big data to shift from retrospective analytics to prospective predictive if fulfilled. [ABSTRACT FROM AUTHOR]

Cloud computing systems are widely used to deploy big data-based applications because of their high storage and computation capacity. The key component for storage in cloud computing environment is distributed file system which can store and process data produced by big data-based applications effectively. The users of such big data-based applications issue read requests more frequently when compared to write requests. So, most of these cloud-based applications demand optimal performance from the distributed file system, especially for read operations. Numerous caching and prefetching techniques have been proposed in the existing literature to enhance the performance of distributed file system. However, these techniques typically adopt a synchronous approach, focusing on either application data prefetching or user data prefetching, when the user application starts executing and this may result in an extended read access time. Furthermore, the data is prefetched either based on access frequency or reuse distance with out considering the access recency of data which may result in less cache hit ratio. In this paper, we have proposed application-specific and user-specific data prefetching algorithms for prefetching the data from the distributed file system and storing the same in the multi-level caches present in the distributed file system based on the combination of access frequency and recency ranking of file blocks that were previously accessed by client application programs. Additionally, we have divided the cache into two partitions namely user and application caches to store the prefetched data as per the popularity value calculated by considering user and application level accesses. We have also introduced a parallel read algorithm to read data simultaneously from the multiple caches present in the distributed file system environment. The simulation results demonstrate that, the proposed algorithms improved the distributed file systems performance by minimum of 8 to maximum of 92 percent in terms of average read access time when compared with different existing approaches. [ABSTRACT FROM AUTHOR]

Traditionally, urban traveling patterns have been obtained through origin-destination surveys. This method presents drawbacks such as high costs, reduced representativeness of the surveyed population, and low spatial and temporal resolution of the results obtained. This study proposes using historical data collected massively and passively from GPS-enabled cell phones to describe population mobility patterns with high spatial and temporal resolution levels. The solutions formulated to tackle the challenges posed by this method are described, as well as the algorithms used to obtain dynamic origin-destination matrices, quantify the average number of daily trips and kilometers traveled per inhabitant, population density per hour, the tracks of roads more frequently used, and to identify the destinations attractor of most trips. As an illustrative example, the traveling patterns were derived for a megacity in Latin America (the metropolitan area of Monterrey, Mexico) with a database of 0.7 million users monitored with high temporal resolution ( $10^{10}$ data points.

In order for company’s promotions to continue to have a beneficial impact on sales, it is important for companies to identify which of the interested buyers can be converted into repeat buyers. By targeting these potential loyal customers, companies can significantly reduce promotional costs and increase return on investment. The existing studies related to repurchase prediction in the e-commerce area have focused on the statistical techniques and more common binary classification models. In this paper, we propose a survival analysis-based machine learning/deep learning model to predict TV repurchase time of customers using home appliance company’s CRM data. The prediction model is verified based on actual operational data such as customer profile, purchase, counseling, and repair history for approximately 1.45 million customers in electronics company’s CRM. As a deep learning method, Algo 6–1 (DeepHit with the feature set selected from Cox regression and preprocessed with multiple imputation) achieved the best performance (c-index 0.828). Algo3 (Random Survival Forest with the feature set selected from Cox regression and preprocessed with multiple imputation), a machine learning method, not only showed similar performance to deep learning (c-index 0.823), but also provided insights in key features that influenced repurchase. In addition, we provided a utility function that provides TV repurchase probability over time so that marketers can cost-effectively determine the timing to provide promotional events or benefits to customers.

Achieving sustainable development goals (SDGs) necessitates the adoption of sustainable policies, which entails a crucial task of policy formulation. Policymakers must consider multiple factors, such as the present development status, which can be assessed using diverse data points, as well as the policy’s impact and an action plan outlining its implementation. These data points typically originate from various sources, primarily governmental bodies encompassing statistics, budgets, legislation, public services, and geospatial data (maps, satellite imagery), along with domain users comprising individuals, organizations, and governments. The SDGs are intricately crafted, taking into account a multitude of factors and pinpointing key indicators that influence these factors. These factors typically span across diverse data points sourced from various sectors. For instance, in assessing the student dropout rate, it is imperative to consider factors such as the availability of transportation facilities, basic hygiene amenities, access to drinking water, and the effectiveness of government-organized schemes in utilization and impact. Employing an open data framework coupled with advanced artificial intelligence (AI) models has the potential to facilitate in-depth exploration and analysis, providing valuable insights into the complex interplay of these factors and contributing to a more comprehensive understanding of SDG-related challenges and opportunities. [ABSTRACT FROM AUTHOR]

Manual labeling and categorization are extremely time-consuming and, thus, costly. AI and ML-supported information systems can bridge this gap and support labor-intensive digital activities. Since it requires categorization, coding-based analysis, such as qualitative content analysis, reaches its limits with large amounts of data and could benefit from AI and ML-based support. Empirical social research, its application domain, benefits from Big Data's ability to create more extensive human behavior and development models. A range of applications are available for statistical analysis to serve this purpose. This paper aims to implement an information system that supports researchers in empirical social research in performing AI-supported qualitative content analysis. AI2VIS4BigData is a reference model that standardizes use cases and artifacts for Big Data information systems that integrate AI and ML for user empowerment. Thus, this work's concepts and implementations try to achieve an AI2VIS4BigData-compliant information system that supports social researchers in categorizing text data and creating insightful dashboards. Thereby, the text categorization is based on an existing ML component. Furthermore, it presents two evaluations that were conducted for these concepts and implementations: a qualitative cognitive walkthrough assessing the system's usability and a quantitative user study with 18 participants revealed that though the users perceive AI support as more efficient, they need more time to reflect on the recommendations. The research revealed that AI support increased the correctness of the users' categorizations but also slowed down their decision-making. The assumption that this is due to the UI design and additional information for processing requires follow-up research. [ABSTRACT FROM AUTHOR]

Copyright of Arab International Journal of Information Technology & Data is the property of Arab Institution of Knowledge Management and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Detecting and anticipating global proliferation expertise and capability evolution from unstructured, noisy, and incomplete public data streams is a highly desired, but extremely challenging task. In this article, we present our pioneering data-driven approach to support the non-proliferation mission to detect and explain the evolution of proliferation expertise and capability development globally from terabytes of publicly available information (PAI), focusing on our knowledge extraction pipeline and descriptive analytics. We first discuss how we fuse nine open-source data streams, including multilingual data, to convert 4 TB of unstructured data to structured knowledge and encode dynamically evolving proliferation expertise representations - content and context graphs. For this, we rely on natural language processing (NLP) and deep learning (DL) models to perform information extraction, topic modeling, and distributed text representation (aka embedding) learning. We then present interactive, usable, and explainable descriptive analytics to refine domain knowledge and present it in a human-understandable form. Finally, we introduce future work avenues that will leverage our dynamic knowledge representations and descriptive analytics to enable predictive and prescriptive inferences to achieve real-time domain understanding and contextual reasoning about global proliferation expertise and capability evolution.

This paper proposes a data fusion approach to automatically detect activity patterns in a GPS dataset based on travel diaries and correct misclassification errors. The Activity Patterns Detection consists of a Supervised Learning framework, thanks to which the activity purposes in the travel diaries are learned and then predicted in the GPS dataset. Furthermore, we deploy Unsupervised Learning to identify similar spatial and temporal activities in the GPS dataset and, based on travel diaries, to correct the misclassification errors. This work shows that, based on a few observations in the travel diaries and a set of features such as the resting time before the activity takes place, the number of occurrences of the same trip and the percentage of the trip made during daytime and the speed, it is possible to detect activities with an overall accuracy of 90%. Since the GPS dataset does not have information on the activity performed by the user, in reality, the aggregated results are validated based on the Kolmogorov-Smirnov test. The experiment shows that, with a confidence level of 99%, the majority of spatial and temporal feature distributions of activities in the travel diaries dataset are similar to those in the GPS dataset. Thanks to this approach, planners and transport operators can automatically obtain spatial and temporal patterns of frequent activities in urban areas. [ABSTRACT FROM AUTHOR]

Abstract Customer churn is a major issue for large enterprises. In particular, in the rental business sector, companies are looking for ways to retain their customers because they are their main source of revenue. The main contribution of our work is to analyze the customer behavior information of actual water purifier rental company, where customer churn occurs very frequently, and to develop and verify the churn prediction model. A machine learning algorithm was applied to a large-capacity operating dataset of rental care service in an electronics company in Korea, to learn meaningful features. To measure the performance of the model, the F-measure and area under curve (AUC) were adopted whereby an F1 value of 93% and an AUC of 88% were achieved. The dataset containing approximately 84,000 customers was used for training and testing. Another contribution was to evaluate the inference performance of the predictive model using the contract status of about 250,000 customer data currently in operation, confirming a hit rate of about 80%. Finally, this study identified and calculated the influence of key variables on individual customer churn to enable a business person (rental care customer management staff) to carry out customer-tailored marketing to address the cause of the churn.

In customer relationship management (CRM), high-quality customer data is at the heart of reliable data analysis and is the foundation for data-driven decisions that impact business goals. To find performance indicators of data quality to maximize the effectiveness of CRM, we need to devise an approach to identifying and managing “business-relevant” information quality metrics. Therefore, this paper deals with the discovery and validation of the Data Quality Dimension (DQD) in terms of meaning and utilization value of data values other than the aspects such as syntax criteria or data format. We design the quality index and scoring logic of the customer integration profile and prove its usefulness by applying it to actual CRM data. A sample of real business operations data of approximately 1 million CRM customers was used to analyze the relevance between the DQDs and business performance indicators. As business performance indicators, we used both the company’s purchasing loyalty index and the performance of past promotional campaigns. We analyzed the significant impact of each DQD on purchase loyalty and promotional campaign success rate. Next, we confirmed the effectiveness of DQDs in terms of providing analytic ease for predictive analysis such as target marketing in CRM. In addition, we showed some possibilities to consider improving data quality by analyzing the granularity of a specific attribute based on a certain DQD. Through these verification results, the validity of the DQDs of the customer profile was confirmed in the context of ‘suitability for use’ of customer data that affects business activities critical to the company in the CRM system.

Depth data and the digital bottom model created from it are very important in the inland and coastal water zones studies and research. The paper undertakes the subject of bathymetric data processing using reduction methods and examines the impact of data reduction according to the resulting representations of the bottom surface in the form of numerical bottom models. Data reduction is an approach that is meant to reduce the size of the input dataset to make it easier and more efficient for analysis, transmission, storage and similar. For the purposes of this article, test datasets were created by discretizing a selected polynomial function. The real dataset, which was used to verify the analyzes, was acquired using an interferometric echosounder mounted on a HydroDron-1 autonomous survey vessel. The data were collected in the ribbon of Lake Klodno, Zawory. Data reduction was conducted in two commercial programs. Three equal reduction parameters were adopted for each algorithm. The research part of the paper presents the results of the conducted analyzes of the reduced bathymetric datasets based on the visual comparison of numerical bottom models, isobaths, and statistical parameters. The article contains tabular results with statistics, as well as the spatial visualization of the studied fragments of numerical bottom models and isobaths. This research is being used in the course of work on an innovative project that aims to develop a prototype of a multi-dimensional and multi-temporal coastal zone monitoring system using autonomous, unmanned floating platforms at a single survey pass. [ABSTRACT FROM AUTHOR]

Global warming effects are seen around the world and Latin American countries are not an exception, especially for expanding drought areas. Therefore, underground water resources used in the region are incrementing exponentially. However, temporal and spatial underground water information concerning availability and quality is scarce, disabling proper decision making. In order to close that breach, we propose and embedded edge-processing Internet of Things (IoT)-based water quality monitoring system. This letter introduces the design and implementation of this solution, specifically targeted to monitor irrigation and drinking water extracted from water wells. The system is designed to be deployed in central Chile, considering the topographic conditions, which severely affect power availability and communication resources. The captured data are stored in a data lake, for further processing according to water quality models. [ABSTRACT FROM AUTHOR]

Big data collection means collecting large volumes of data to have insight into better business decisions and greater customer satisfaction. Securing big data is difficult not just because of the large amount of data it handles, but also because of the continuous streaming of data, multiple types of data, and cloud-based data storage. Additionally, traditional security and privacy methods are insufficient due to large data volume, speed, and data from different sources. In this study, we focused on the security and privacy issues of big data. To this end, many related articles published in the literature have been extensively examined. Moreover, within the scope of big data, data security and privacy practices, current technologies, and methods are presented. Furthermore, we present an analysis of the security requirements of big data and the elimination of security vulnerabilities against cyber attacks. [ABSTRACT FROM AUTHOR]

Customer churn is a major issue for large enterprises. In particular, in the rental business sector, companies are looking for ways to retain their customers because they are their main source of revenue. The main contribution of our work is to analyze the customer behavior information of actual water purifier rental company, where customer churn occurs very frequently, and to develop and verify the churn prediction model. A machine learning algorithm was applied to a large-capacity operating dataset of rental care service in an electronics company in Korea, to learn meaningful features. To measure the performance of the model, the F-measure and area under curve (AUC) were adopted whereby an F1 value of 93% and an AUC of 88% were achieved. The dataset containing approximately 84,000 customers was used for training and testing. Another contribution was to evaluate the inference performance of the predictive model using the contract status of about 250,000 customer data currently in operation, confirming a hit rate of about 80%. Finally, this study identified and calculated the influence of key variables on individual customer churn to enable a business person (rental care customer management staff) to carry out customer-tailored marketing to address the cause of the churn. [ABSTRACT FROM AUTHOR]

Manual labeling and categorization are extremely time-consuming and, thus, costly. AI and ML-supported information systems can bridge this gap and support labor-intensive digital activities. Since it requires categorization, coding-based analysis, such as qualitative content analysis, reaches its limits with large amounts of data and could benefit from AI and ML-based support. Empirical social research, its application domain, benefits from Big Data’s ability to create more extensive human behavior and development models. A range of applications are available for statistical analysis to serve this purpose. This paper aims to implement an information system that supports researchers in empirical social research in performing AI-supported qualitative content analysis. AI2VIS4BigData is a reference model that standardizes use cases and artifacts for Big Data information systems that integrate AI and ML for user empowerment. Thus, this work’s concepts and implementations try to achieve an AI2VIS4BigData-compliant information system that supports social researchers in categorizing text data and creating insightful dashboards. Thereby, the text categorization is based on an existing ML component. Furthermore, it presents two evaluations that were conducted for these concepts and implementations: a qualitative cognitive walkthrough assessing the system’s usability and a quantitative user study with 18 participants revealed that though the users perceive AI support as more efficient, they need more time to reflect on the recommendations. The research revealed that AI support increased the correctness of the users’ categorizations but also slowed down their decision-making. The assumption that this is due to the UI design and additional information for processing requires follow-up research.

The increasing availability of vast quantities of data from various sources significantly impacts the insurance industry, although this industry has always been data driven. It accelerates manual processes and enables new products or business models. On the other hand, it also burdens insurance analysts and other users that need to cope with this development parallel to other global changes. A novel information system (IS) for artificial intelligence (AI)-supported big data analysis, introduced within this paper, shall help to overcome user overload and to empower human data analysts in the insurance industry. The IS research's focus lies neither in novel algorithms nor datasets but in concepts that combine AI and big data analysis for synergies, such as usability enhancements. For this purpose, this paper systematically designs and implements an AI2VIS4BigData reference model to help information systems conform to automatically detect anomalies and increase its users' confidence and efficiency. Practical relevance is assured by an interview with an insurance analyst to verify the demand for the developed system and derive all requirements from two insurance industry user stories. A core contribution is the introduction of the IS. Another significant contribution is an extension of the AI2VIS4BigData service-based architecture and user interface (UI) concept on AI and machine learning (ML)-based user empowerment and data transformation. The implemented prototype was applied to synthetic data to enable the evaluation of the system. The quantitative and qualitative evaluations confirm the system's usability and applicability to the insurance domain yet reveal the need for improvements toward bigger quantities of data and further evaluations with a more extensive user group. [ABSTRACT FROM AUTHOR]

Graph convolutional networks (GCNs) have shown great potential in recommender systems. GCN models contain multiple layers of graph convolutions to exploit signals from higher-order neighbors. In each graph convolution, the embedding of a user or item is influenced by its directly connected neighbors. Some of the main problems with this approach are as follows. First, too many graph convolutional layers make different users or items have similar embeddings. Second, the obtained interaction data have some unfavorable characteristics, such as the sparsity of the data, the noise inside the data, and the distribution skewness of the data, that may impair the model’s performance. This paper proposes an interest-aware contrastive-learning-based GCN (IC-GCN) model. IC-GCN applies an interest-aware mechanism, divides users into different subgraphs according to their interests, and performs multilayer graph convolution on the subgraphs, where all collaborative signals received from multi-hop neighbors are positive. Furthermore, IC-GCN takes the contrastive learning task as an auxiliary task, where the interest-aware encoder receives two modified graphs generated by applying the node dropout operator on the full interaction graph. These two graphs generate two sets of embeddings as two additional views of the nodes. The contrastive learning loss function compares these two sets of embeddings. Extensive experiments are conducted to demonstrate the effectiveness of our model.

Topic models are often useful in categorization of related documents in information retrieval and knowledge discovery systems, especially for large datasets. Interpreting the output of these models remains an ongoing challenge for the research community. The typical practice in the application of topic models is to tune the parameters of a chosen model for a target dataset and select the model with the best output based on a given metric. We present a novel perspective on topic analysis by presenting a process for combining output from multiple models with different theoretical underpinnings. We show that this results in our ability to tackle novel tasks such as semantic characterization of content that cannot be carried out by using single models. One example task is to characterize the differences between topics or documents in terms of their purpose and also importance with respect to the underlying output of the discovery algorithm. To show the potential benefit of leveraging multiple models we present an algorithm to map the term-space of Latent Dirichlet Allocation (LDA) to the neural document-embedding space of doc2vec. We also show that by utilizing both models in parallel and analyzing the resulting document distributions using the Normalized Pointwise Mutual Information (NPMI) metric we can gain insight into the purpose and importance of topics across models. This approach moves beyond topic identification to a richer characterization of the information and provides a better understanding of the complex relationships between these typically competing techniques.

Big data applications and analytics are vital in proposing ultimate strategic decisions. The existing literature emphasizes that big data applications and analytics can empower those who apply Big Data Analytics during the COVID-19 pandemic. This paper reviews the existing literature specializing in big data applications pre and peri-COVID-19. A comparison between Pre and Peri of the pandemic for using Big Data applications is presented. The comparison is expanded to four highly recognized industry fields: Healthcare, Education, Transportation, and Banking. A discussion on the effectiveness of the four major types of data analytics across the mentioned industries is highlighted. Hence, this paper provides an illustrative description of the importance of big data applications in the era of COVID-19, as well as aligning the applications to their relevant big data analytics models. This review paper concludes that applying the ultimate big data applications and their associated data analytics models can harness the significant limitations faced by organizations during one of the most fateful pandemics worldwide. Future work will conduct a systematic literature review and a comparative analysis of the existing Big Data Systems and models. Moreover, future work will investigate the critical challenges of Big Data Analytics and applications during the COVID-19 pandemic. [ABSTRACT FROM AUTHOR]

Climate change means coping directly or indirectly with extreme weather conditions for everybody. Therefore, analyzing meteorological data to create precise models is gaining more importance and might become inevitable. Meteorologists have extensive domain knowledge about meteorological data yet lack practical data analysis skills. This paper presents a method to bridge this gap by empowering the data knowledge carriers to analyze the data. The proposed system utilizes symbolic AI, a knowledge base created by experts, and a recommendation expert system to offer suiting data analysis methods or data pre-processing to meteorologists. This paper systematically analyzes the target user group of meteorologists and practical use cases to arrive at a conceptual and technical system design implemented in the CAMeRI prototype. The concepts in this paper are aligned with the AI2VIS4BigData Reference Model and comprise a novel first-order logic knowledge base that represents analysis methods and related pre-processings. The prototype implementation was qualitatively and quantitatively evaluated. This evaluation included recommendation validation for real-world data, a cognitive walkthrough, and measuring computation timings of the different system components. [ABSTRACT FROM AUTHOR]

Abstract Whole Slide Image (WSI) datasets are giga-pixel resolution, unstructured histopathology datasets that consist of extremely big files (each can be as large as multiple GBs in compressed format). These datasets have utility in a wide range of diagnostic and investigative pathology applications. However, the datasets present unique challenges: The size of the files, propriety data formats, and lack of efficient parallel data access libraries limit the scalability of these applications. Commercial clouds provide dynamic, cost-effective, scalable infrastructure to process these datasets, however, we lack the tools and algorithms that will transfer/transform them onto the cloud seamlessly, providing faster speeds and scalable formats. In this study, we present novel algorithms that transfer these datasets onto the cloud while at the same time transforming them into symmetric scalable formats. Our algorithms use intelligent file size distribution, and pipelining transfer and transformation tasks without introducing extra overhead to the underlying system. The algorithms, tested in the Amazon Web Services (AWS) cloud, outperform the widely used transfer tools and algorithms, and also outperform our previous work. The data access to the transformed datasets provides better performance compared to the related work. The transformed symmetric datasets are fed into three different analytics applications: a distributed implementation of a content-based image retrieval (CBIR) application for prostate carcinoma datasets, a deep convolutional neural network application for classification of breast cancer datasets, and to show that the algorithms can work with any spatial dataset, a Canny Edge Detection application on satellite image datasets. Although different in nature, all of the applications can easily work with our new symmetric data format and performance results show near-linear speed-ups as the number of processors increases.

An end-to-end supervised learning method is proposed for fault detection in the electric grid using Big Data from multiple Phasor Measurement Units (PMUs). The approach consists of preprocessing steps aimed at reducing data noise and dimensionality, followed by utilization of six classification models considered for detecting faults. Three of the models were variants of Convolutional Neural Network (CNN) architectures that consider a single type of measurement (voltage, current or frequency) at all PMUs or all types together also at all PMUs. CNN based models were compared to traditional methods of Logistic Regression (LR), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Evaluation was conducted on two-year data measured by PMUs at 37 locations in a large electric grid. The response variable for classification were extracted from the grid-wide outage event log. Experiments show that CNN-based models outperformed traditional methods on one year out-of-sample outage detection over the entire grid. [ABSTRACT FROM AUTHOR]

Reducing emissions caused by urban freight transportation is an increasingly important policy objective for transportation planners around the world. New and innovative ways of data collection provide new possibilities to analyze these issues. In this paper we present MASS-GT, a new multi-agent simulation system for urban goods transport. The empirical basis is provided by an exceptionally large dataset of truck trip travel diaries for The Netherlands that was collected from transportation management systems using an automated data collection interface. The dataset is very dense and includes information on vehicles, routes, and shipments carried. The strategic part of the model simulates the formation of individual shipments based on logistic processes at a strategic level, such as sourcing, distribution channel choice and shipment size choice. At tactical level disaggregate choices are simulated for tour formation, vehicle type- and time of day choice, based on observed distributions. The multi-agent approach allows to implement heterogeneous preferences and thus differentiated responses to new policies. We present an application of the model to study the impacts of urban consolidation centers (UCC) and zero emission zones. The freight transportation volumes transported to these UCC and their impact on logistic indicators are analyzed. Simulation results show that vehicle kilometers travelled within the wider region increase with the introduction of UCC, and at the same time the efficiency of deliveries increases as well. Thus the model allows to study trade-offs between regional and local systems that emerge from different behavioural responses to policies. • Simulation models should take into account the heterogeneity and logistic decision making behind urban freight demand. • MASS-GT is an empirical multi-agent simulation system for the analysis of urban freight transportation demand. • The empirical basis is provided with an exceptionally large and dense dataset of truck trip travel diaries. • To manage complexity the model is developed following an incremental development path. • Possibilities for strategic policy studies are illustrated with a zero emission zone case study. [ABSTRACT FROM AUTHOR]

The identification of data that is in various sources of information and its consolidation to deliver it as useful is achieved with Big Data. The overall objective of this work is to develop an information consolidation architecture design for health insurance using Big Data. For this research proposal, the analytical empirical method is used, of a quasi-experimental type with a quantitative approach, through the analysis of relevant references and specification of the architecture components. The results of this research allow categorizing different computational architectures for health insurance through a review of relevant literature, developing an architectural model of a computational system for an Ecuadorian health insurance company oriented to the consolidation of information, and evaluating the study methodology used to establish feasible factors of the model. The contribution of this work allows us to determine the applicability of the model to national or foreign health insurance companies by contrasting feasible factors in a specific company of the environment. It is concluded that the different sources of information or types of data used in the field of health insurance allow to know several edges of data analysis through a Big Data architecture, in addition to obtaining indicators to improve decision making; 73% of the established factors are viable in an Ecuadorian health insurance company. [ABSTRACT FROM AUTHOR]

Depth data and the digital bottom model created from it are very important in the inland and coastal water zones studies and research. The paper undertakes the subject of bathymetric data processing using reduction methods and examines the impact of data reduction according to the resulting representations of the bottom surface in the form of numerical bottom models. Data reduction is an approach that is meant to reduce the size of the input dataset to make it easier and more efficient for analysis, transmission, storage and similar. For the purposes of this article, test datasets were created by discretizing a selected polynomial function. The real dataset, which was used to verify the analyzes, was acquired using an interferometric echosounder mounted on a HydroDron-1 autonomous survey vessel. The data were collected in the ribbon of Lake Klodno, Zawory. Data reduction was conducted in two commercial programs. Three equal reduction parameters were adopted for each algorithm. The research part of the paper presents the results of the conducted analyzes of the reduced bathymetric datasets based on the visual comparison of numerical bottom models, isobaths, and statistical parameters. The article contains tabular results with statistics, as well as the spatial visualization of the studied fragments of numerical bottom models and isobaths. This research is being used in the course of work on an innovative project that aims to develop a prototype of a multi-dimensional and multi-temporal coastal zone monitoring system using autonomous, unmanned floating platforms at a single survey pass.

Log-structured merge (LSM) tree key-value (KV) stores have been widely deployed in many NoSQL and SQL systems, serving online big data applications such as social networking, graph processing, machine learning, etc. The batch processing of sorted data merging (i.e., compaction) in LSM-tree key-value stores improves the write efficiency, and some lazy compaction methods have been proposed to accumulate more data within a batch. However, these batched writing methods lead to significant tail latency, which is unacceptable for online processing. Aiming to optimize both latency and throughput, we propose a novel Lower-level Driven Compaction (LDC) method which breaks the limitations of the traditional upper-level driven compaction manner and triggers practical compaction actions bottom-up, with the benefits of both decreasing the compaction granularity for smaller latency and reducing write amplification for higher throughput. Furthermore, we extend LDC to Adaptive LDC (ALDC) by adding an adaptive policy to adjust the key compaction threshold to fit the changes of workloads’ features. The experimental results indicate that ALDC reduces the tail latency significantly and meanwhile achieves a much higher and stable throughput compared with existing approaches. [ABSTRACT FROM AUTHOR]

In the context of the heavy demands of Big Data, software developers have also begun to consider NoSQL data storage solutions. One of the important criteria when choosing a NoSQL database for an application is its performance in terms of speed of data accessing and processing, including response times to the most important CRUD operations (CREATE, READ, UPDATE, DELETE). In this paper, the behavior of two of the major document-based NoSQL databases, MongoDB and document-based MySQL, was analyzed in terms of the complexity and performance of CRUD operations, especially in query operations. The main objective of the paper is to make a comparative analysis of the impact that each specific database has on application performance when realizing CRUD requests. To perform this analysis, a case-study application was developed using the two document-based MongoDB and MySQL databases, which aim to model and streamline the activity of service providers that use a lot of data. The results obtained demonstrate the performance of both databases for different volumes of data; based on these, a detailed analysis and several conclusions were presented to support a decision for choosing an appropriate solution that could be used in a big-data application. [ABSTRACT FROM AUTHOR]

Big data frameworks such as Apache Spark is becoming prominent to perform large-scale data analytics jobs in various domains. However, due to limited resource availability, the local or on-premise computing resources are often not sufficient to run these jobs. Therefore, public cloud resources can be hired on a pay-per-use basis from the cloud service providers to deploy a Spark cluster entirely on the cloud. Nevertheless, using only cloud resources can be costly. Hence, both local and cloud resources nowadays are used together to deploy a hybrid cloud computing cluster. However, scheduling jobs in a cluster deployed on hybrid clouds is challenging in the presence of various Service-Level Agreement (SLA) demands such as cost minimization and job deadline guarantee. Most of the existing works either consider a public or a locally deployed cluster and mainly focus on improving job performance in the cluster. In this article, we propose efficient scheduling algorithms that leverage from different VM instance pricing in a hybrid cloud deployed cluster to optimize the Virtual Machine (VM) usage cost for both local and cloud resources and maximize the job deadline met percentage. We have conducted extensive simulation-based experiments to compare our proposed algorithms with the baseline approaches. In addition, we have developed a prototype system on top of Apache Mesos cluster manager and performed real experiments to evaluate the applicability of our proposed approaches in a real platform with benchmark applications. The results show that our proposed algorithms are highly scalable and reduce the cost of VM usage of a hybrid cluster for up to 20 percent. [ABSTRACT FROM AUTHOR]