Search

This paper presents the design, implementation, and practical use of a specialized online data repository and an interactive visualization of a transportation dashboard. Specifically tailored to handle and interpret large-scale transportation data within the Qatari context, the combined platform serves as a comprehensive solution for managing extensive datasets, including GPS traces from taxis and e-scooters, which are examined as primary use-cases in this paper. The online data repository provides a centralized hub for efficient data storage and management of public transport including Mobility-as-a-Service (MaaS) data. Concurrently, the visualization dashboard fosters an intuitive, user-friendly interface for data exploration and analysis. Through real-world applications within Qatar's transportation ecosystem, we elucidate how these innovative developments can inform data-driven policy decisions in crucial areas such as infrastructure development, resource allocation, and safety measures. Ultimately, this study underscores the pivotal role of effective data management and advanced visualization in maximizing the potential of big data, providing valuable insights for urban mobility planning and enhancing the landscape of policy-making in Qatar and worldwide. [ABSTRACT FROM AUTHOR]

Small data and big data are distinct approaches to data analysis and utilization in various applications. While big data has been the focus of many research and business efforts for more than ten years, small data is increasingly being recognized as having potential value in certain settings. We systematically review literature and conclude that small data can be indeed valuable in certain scenarios. This paper incorporates the data value perspective of small data within various application areas. For this, we apply the data-information-knowledge-wisdom (DIKW) hierarchy to categorize papers and findings, and discuss the papers from the view point of "data value". Our review identifies various contexts where small data can be used to create value, such as data pre-processing, classification tasks, anomaly detection, forecasting and decision support. We also highlight industries that may be particularly promising areas for practitioners and researchers focused on small data. In addition, we provide an overview of methods and tools for small data analysis, including statistical techniques, visualization, and machine learning algorithms. Finally, based on our results, we suggest, that further research should focus on small data analysis. [ABSTRACT FROM AUTHOR]

Abstract Severe weather events pose a significant threat to transportation networks. This research analyzes and discusses the impact of precipitation, temperature, visibility and wind speed on hourly weekday traffic flow volume in Atlanta, Georgia. The study involves the following: determine weather variables that affect traffic volume, develop a machine learning technique to derive decision rules based on weather and traffic volume, and create a web-based decision support visualization tool using the analyzed results. The relationship between extreme weather events and traffic volume was investigated by comparing traffic volume between a base case scenario and an extreme weather scenario. Data from 48 Automatic Traffic Recorder (ATR) sites around Atlanta, GA, USA and hourly precipitation data from 4 climate measurement stations were used to conduct this study. The spatiotemporal relationships between traffic volume and weather variables were analyzed individually and evaluated using a non-parametric statistical test. A machine learning technique is applied to derive decision rules that result in reduction in traffic volume. Results show significant impacts on traffic volume from visibility, precipitation and temperature and helps in isolating hours in a typical weekday when such impacts are felt. A decision support tool was also developed to visualize traffic volume and weather interactions. The data-driven insights from this analysis is applicable to transportation planners, centralized traffic control rooms and urban infrastructure decision makers. Highlights • Paper analyzes correlations between 4 weather hazards and traffic volume by combining Big Data repositories from climate science and transportation. • The paper using extensive statistical analysis to derive hours in a typical day where traffic at a particular location is impacted due to a weather hazard. • Machine learning models were also applied to derive decision rules for traffic planners and transportation personnel to optimize and plan for weather hazards. • A decision support tool was also derived to help display the data analysis, derive insights and optimize transportation decision making. [ABSTRACT FROM AUTHOR]

Abstract The amount of data created by people, machines and corporations around the world is growing every year. Thanks to innovations such as the Internet of Things, this trend will continue, giving rise to the creation of Big Data. Data visualization leverages principles of visual psychology to help stakeholders identify patterns, trends and correlations that might go undetected in text-based or spreadsheet data. The return on investment (ROI) of big data visualization is well-documented in numerous studies and use cases. However, to achieve ROI from analytics investments, key insights must be uncovered, understood and communicated. Synthesizing huge quantities of data into key insights grows more challenging as data volumes and varieties increase. To address visualization challenges posed by big and high-dimensional data, this paper explores algorithms and techniques that compress the amount of data and/or reduce the number of attributes to be analyzed and visualized. Specifically, this paper examines applying dimensionality reduction and data compression algorithms to reduce attributes, tuples and data points returned to the visualization. By reducing data returned to the visualization, trends, patterns and correlations are easier to view and visualization tool performance is optimized. [ABSTRACT FROM AUTHOR]

This paper presents an important concept of Internet of People, a social network analysis approach to perform Big Data Analytics. The paper describes the development, management implications and analysis. To illustrate six points in the management issues, an in-house development of a SocialNetwork API with six functions has been demonstrated. The proposed method is focused on processing the contacts who click like or comment on the author's posts, as well as the queries and visualization. Results can be extracted and presented in data visualization. Six functions in the SocialNetwork API have evaluation tests, including a large scale of 50,000 simulations completed within 60,000 s. Results support our case of Big Data processing for social network analysis can be equivalent to CRM, ERP and MIS. Additionally, there are no costs involved. Related topics have been discussed in details. Our research contributions have been consolidated since our work have met research challenges for social network analysis and six management implications. [ABSTRACT FROM AUTHOR]

Social media platforms, search engine transaction logs, Web portal transaction logs, large text corpora, and other data sources offer users a variety of invaluable data sources. Data generated based on social media platforms and other data sources have become important sources of big data for researchers in information science. Big data provide not only challenges but also opportunities for information science. Emerging big data trends inevitably have an impact on research methods in information science. The authors of this paper discuss the impact of big data on research methods in information science. This paper addresses these challenges and opportunities through the lens of research methods, ranging from data processing, to sampling, to information visualization, to temporal analysis, to sentiment analysis, to correlation, to cause-effect relationship, to data accessibility, to data privacy, and data ethics issues. The discussions on related aspects of research methods provoke a healthy reflection and debate for the information science research community, which can help researchers in the field produce sound research designs for big data-oriented research studies and assist information practitioners in solving problems they face in the big data age. Research methods are fundamental and essential for any research studies. Big data analysis has a natural relationship with research methods. The paper discusses an emerging and important research topic big data from the unique angle of research methods. [ABSTRACT FROM AUTHOR]

We propose a new framework for research synthesis of both evidence and influence, named research weaving. It summarizes and visualizes information content, history, and networks among a collection of documents on any given topic. Research weaving achieves this feat by combining the power of two methods: systematic mapping and bibliometrics. Systematic mapping provides a snapshot of the current state of knowledge, identifying areas needing more research attention and those ready for full synthesis. Bibliometrics enables researchers to see how pieces of evidence are connected, revealing the structure and development of a field. We explain how researchers can use some or all of these tools to gain a deeper, more nuanced understanding of the scientific literature. Highlights An exponential increase in scientific publications requires informative and integrative reviews to provide a detailed synthesis of a particular research field, and this has resulted in the emergence of novel methods for synthesizing heterogeneous research. Research weaving provides a novel framework that combines bibliometrics and systematic mapping to inform the development of a field, the influence of research papers and their interconnections, and to visualize content across and within publications. Research weaving has the potential to provide a more efficient, in-depth, and broad synthesis of a research field, to identify research biases, gaps, and limitations. Such insights have the potential to inform ecological and environmental policy and communicate research findings to the general public in more effective ways then are typically done in current research syntheses. [ABSTRACT FROM AUTHOR]

In this paper, a formal method is presented to: (1) establish a system to visualize huge quantities of related information (2) graphically show the degree of similarity between these related documents (3) navigate between related documents so that the views are reconfigured in a dynamic and gradual way. This method uses, as input, a set of documents that have been related using any existing algorithm (relationships based on contents, on votes to contents, on social relations, on demographic information, etc.). The proposed method includes two different visualization and navigational versions related to: (a) visualization by explicit graph and (b) visualization by implicit graph. As a summary, the following characteristics are presented: (1) this method maximizes the information that is displayed at each moment, so that a balance is found between: (a) the number of relations shown, (b) the number of documents viewed, (c) the amount of information presented by each document and (d) how to show the degree of similarity between documents, (2) the method includes specific rules of interaction with the user to allow navigation in a simple, gradual and intuitive way. [ABSTRACT FROM AUTHOR]

The distribution and management of energetic resources are the source of big amount of data in structured or unstructured form. These data are collected in a targeted or a random fashion manner for the purpose of efficient distribution and management in the required timeframe. Urban as well as industrial complexes are typical environments where are generated these data. In this process, there are data that are deliberately collected in order to generate the required reporting and outputs for management decision. Input data are the basis of reporting which can have simple or detailed format. In this work we are collecting random input data with intricate structure that cannot simply interpreted and explain in relation with others. The topic of our paper is focused on the interpretation of collected data that is oriented to energetic resources in regional city. We are describing the process of data collection, data cleaning, analysis, identification of relations, informational links, visualization, and interpretation. By analysis of these data we propose information value, which can be exploited in management decision to reduce the cost of electricity and the gas there. [ABSTRACT FROM AUTHOR]

Currently dashboards are the preferred tool across organizations to monitor business performance. Dashboards are often composed of different data visualization techniques, amongst which are Key Performance Indicators (KPIs) which play a crucial role in quickly providing accurate information by comparing current performance against a target required to fulfil business objectives. However, KPIs are not always well known and sometimes it is difficult to find an appropriate KPI to associate with each business objective. In addition, Data Mining techniques are often used when forecasting trends and visualizing data correlations. In this paper we present a new approach to combining these two aspects in order to drive Data Mining techniques to obtain specific KPIs for business objectives in a semi-automated way. The main benefit of our approach is that organizations do not need to rely on existing KPI lists or test KPIs over a cycle as they can analyze their behavior using existing data. In order to show the applicability of our approach, we apply our proposal to the fields of Massive Open Online Courses (MOOCs) and Open Data extracted from the University of Alicante in order to identify the KPIs. [ABSTRACT FROM AUTHOR]

In the scenario of big data, incorporating multiple attribute descriptions of the same subject (i.e., so-called multi-view data analysis) and the developing effective data visualization techniques have been one of the most important topics in the computer vision and machine learning community. In this paper, a multiple view source attributes embedding framework is proposed that can assemble various object attributes together and embed them into a low dimensional space to realize visual data structure demonstration. Classical sole attribute data embedding approaches such as MDS and SHE can be utilized under this framework. The core idea of this framework is to achieve a uniform contribution merging, which means every source attribute will contribute equally to the final embedded output. The underlying reason for this idea is that now that an attribute is input as a source and no prior weight is given beforehand, it should be treated equally. The most exciting advantage of this strategy is that it can avoid prejudice caused by major attributes (exerting due to their evident quantity advantage), which may lead the minor attributes be ignored. In addition, we propose a simple iterating algorithm to implement this framework, and plenty of experiments are conducted under diverse source attribute configurations to validate its effectiveness. [ABSTRACT FROM AUTHOR]

Although there are many systems that have implemented customer behavior analytics, it's still an upcoming and unexplored market that has greater potential for better advancements. Big data is one of the most rising technology trends that have the capability for significantly changing the way business organizations use customer behavior to analyze and transform it into valuable insights. Even decision trees can be used efficiently for analyzing data. At the end of this paper, a proposed Map Reduce implementation of well-known statistical classifier, C4.5 decision tree algorithm has been proposed. Apart from this, the system aims to implement Customer data visualization using Data Driven Documents (d3.js) which allows us to build well customized graphics. [ABSTRACT FROM AUTHOR]

The self-organizing map (SOM) methodology does vector quantization and clustering on the dataset, and then projects the obtained clusters to a lower dimensional space, such as a 2D map, by positioning similar clusters in locations that are spatially closer in the lower dimension space. This makes the SOM methodology an effective tool for data visualization. However, in a world where mined information from big data have to be available immediately, SOM becomes an unattractive tool because of its time complexity. In this paper, we propose an alternative visualization methodology for large datasets that emulates SOM methodology without the speed constraints inherent to SOM. To demonstrate the efficiency and the potential of the proposed scheme as a fast visualization tool, the methodology is used to cluster and project the 3,823 image samples of handwritten digits of the Optical Recognition of Handwritten Digits dataset. Although the dataset is not, by any means large, it is sufficient to demonstrate the speed-up that can be achieved by using this proposed SOM emulation procedure. [ABSTRACT FROM AUTHOR]

The twenty first century sees the tremendous advancement of computer and machine technologies that are able to produce ginormous amout of data. Current software architecture, management and analysis approaches are unable to cope with the flood of data. The challenge of understanding large and complex data includes issues such as clutter, performance, information loss and limited cognition. Medical field involves analyzing the body system which includes many different scientists and medical professionals. The datasets are a hybrid of many different medical areas databases to understand and answer the many questions of the human body. This paper explores the capability of interactive star coordinate visualization technique to identify clusters correlation between selected attributes using interactive star coordinate for multi-dimensional datasets An interactive Star coordinates is designed consists of four stages that includes Information Objects Transformation; Dimension Mapping; Interactive Features design and Coloring . Finally the performance of the interactive star coordinates is compared to histograms of the data of interest. Interactive star coordinate is found as a promising method of visualizing information clusters pattern which provides one of the means for fast decision making. [ABSTRACT FROM AUTHOR]

This paper proposes a multidimensional cluster scaling (MDCS) which obtains scaling of multidimensional clus- ters (or groups). Conventional multidimensional scaling is well known as a method to obtain the scaling of multidi- mensional objects. However, there is no method to obtain the scaling of the clusters. The merit of this scaling method is its applicability for the analysis of large amounts of data such as big data. Since the purpose of the scaling method is to obtain the latent structure of a given data in a lower dimensional space in order to summarize the data features and the visualization of the data structure, for large amounts of data, a loss of the data information through the reduction of the dimensions has been a main problem with the use of the scaling method. The proposed method can solve this problem by the use of clusters of objects. Several numerical examples show the better performance of the proposed method. [ABSTRACT FROM AUTHOR]

• Develop ARIMA based time-series anomaly detection approach to handle vehicle operating status identification. • Use immersive Virtual Reality to interpret obtained results. • Use large scale vehicle operating time series dataset to validate proposed approach. Nowadays, with the increasing complexity of vehicle operating environment, vehicle operating reliability becomes growing essential and has attracted significant attention in recent decades. The majority of vehicle operating activities are organized in the format of time series data. Many conventional approaches are proposed to deal with time-series data. Autoregressive Integrated Moving Average (ARIMA) approach is one of the most widely used methods to analyze time-series data. In this paper, an ARIMA-based anomaly detection framework is developed to identify abnormal states of the vehicles based on the multiple-channel operating time series data. The state anomaly is captured by the deviation of real-time values at different channels from the predictions. After abnormal operating states are identified, a novel immersive Virtual Reality (VR) tool is applied to visualize the difference between normal and abnormal status. The combination of ARIMA anomaly detection and VR visualization for vehicle multiple-channel operating time-series data would provide a robust way to present the results of vehicle operating status. A large scale data set of 14 days operational performance channels of a specific vehicle is utilized to validate the performance of the proposed approach. [ABSTRACT FROM AUTHOR]

Data exploration and visual analytics systems are of great importance in Open Science scenarios, where less tech-savvy researchers wish to access and visually explore big raw data files (e.g., json, csv) generated by scientific experiments using commodity hardware and without being overwhelmed in the tedious processes of data loading, indexing and query optimization. In this paper, we present our work for enabling efficient query processing on large raw data files for interactive visual exploration scenarios and analytics. We introduce a framework, named RawVis, built on top of a lightweight in-memory tile-based index, VALINOR, that is constructed on-the-fly given the first user query over a raw file and progressively adapted based on the user interaction. We evaluate the performance of a prototype implementation compared to three other alternatives and show that our method outperforms in terms of response time, disk accesses and memory consumption. Particularly during an exploration scenario, the proposed method in most cases is about 5-10 × faster compared to existing solutions, and requires significantly less memory resources. • Progressive and Adaptive processing for in-situ visualization and analytics. • Visual user interactions over raw data as data-access operations. • A main-memory index, constructed on-the-fly based on the first user interaction. • User-driven techniques that progressively adapt index structure during exploration. • Improvement in terms of execution time, I/O operations, and memory consumption. [ABSTRACT FROM AUTHOR]