Search

The exponential growth of urban data presents significant challenges in efficiently analyzing and gaining actionable insights for urban planning and design. This paper proposes a big data analytics framework using MapReduce-based parallel FP-growth (MP-PFP) algorithm leveraging tools like Hadoop, MapReduce, and distributed crawlers to uncover patterns and trends from large-scale, heterogeneous urban datasets. A key contribution is the integration of diverse data types, from socio-economic datasets to environmental parameters, into a consistent analysis framework. The methodology employs frequent pattern mining algorithms on a scalable analytics platform to process behavior data and derive planning directives. Additionally, data visualization and parametric analysis techniques transform raw statistics into interactive 3D landscape representations that expose environmental site attributes. Specifically, the MapReduce capabilities enable distributed parallel processing of vast urban data volumes, ensuring speed and efficiency. The data visualization module creates immersive VR representations of urban landscapes, allowing interactive modifications. Advanced simulation techniques are incorporated to model the impact of planning directives on multiple landscape attributes. The framework is designed as a scalable, customizable solution that can integrate diverse urban data sources with customizable analytics, modeling and visualization modules through APIs. Comparative evaluations demonstrate a classification accuracy improvement from 68 to 93% over prevailing approaches. The framework has proven superior in data integration, real-time responsiveness, and accurately modeling the dynamic complexities of urban landscapes. The quantifiable simulations empower designers to make more informed planning decisions aligned with community needs. Despite ongoing data accuracy and privacy concerns, the methodology shows promising capabilities in harnessing urban big data to drive intelligent, sustainable urban development through its integration of data-driven insights, computational analysis, and interactive visualization. It brings impactful innovations to the future of urban informatics and planning. [ABSTRACT FROM AUTHOR]

Twitter produces a massive amount of data due to its popularity that is one of the reasons underlying big data problems. One of those problems is the classification of tweets due to use of sophisticated and complex language, which makes the current tools insufficient. We present our framework HTwitt, built on top of the Hadoop ecosystem, which consists of a MapReduce algorithm and a set of machine learning techniques embedded within a big data analytics platform to efficiently address the following problems: (1) traditional data processing techniques are inadequate to handle big data; (2) data preprocessing needs substantial manual effort; (3) domain knowledge is required before the classification; (4) semantic explanation is ignored. In this work, these challenges are overcome by using different algorithms combined with a Naïve Bayes classifier to ensure reliability and highly precise recommendations in virtualization and cloud environments. These features make HTwitt different from others in terms of having an effective and practical design for text classification in big data analytics. The main contribution of the paper is to propose a framework for building landslide early warning systems by pinpointing useful tweets and visualizing them along with the processed information. We demonstrate the results of the experiments which quantify the levels of overfitting in the training stage of the model using different sizes of real-world datasets in machine learning phases. Our results demonstrate that the proposed system provides high-quality results with a score of nearly 95% and meets the requirement of a Hadoop-based classification system. [ABSTRACT FROM AUTHOR]

Electricity data could generate a large number of records from smart meter day by day. The traditional architecture might not properly handle the increasingly dynamic data that need flexibility. For effective storing and analytics, efficient architecture is needed to provide much greater data volumes and varieties. In this paper, we proposed the architecture of data storage and analytic in the big data lake of electricity usage using Spark. Apache Sqoop was used to migrate historical data to Apache Hive for processing from an existing system. Apache Kafka was used as the input source for Spark to stream data to Apache HBase to ensure the integrity of the streaming data. In order to integrate the data, we use the Hive and HBase principle of Data Lake as search engines for Hive and HBase. Apache Impala and Apache Phoenix are used separately. This work also analyzes electricity usage and power failure with Apache Spark. All of the visualizations of this project are presented in Apache Superset. Moreover, the usage prediction comparison is presented using HoltWinters algorithm. [ABSTRACT FROM AUTHOR]

In order to improve the quality of user experience of the wireless mobile multimedia communication and the mobile multimedia communication control efficiency, based on the little data and big data visualization, this paper proposed a mobile multimedia cooperative control algorithm. Firstly, the scale and content of the data generated by the wireless intelligent mobile terminal were considered in heterogeneous network environment. The paper established the consistency guarantee scheme of multimedia data. The control scheme of bandwidth allocation, small data distortion and channel state scheduling are presented. Then, based on the multimedia data frame sequence description of the entity mobile multimedia, according to user needs and three-dimensional space to reorganize the multimedia data. This kind of multimedia data can be used to control the quality of the user by multi dimension cross control of the multimedia stream and the entity captured by the intelligent mobile terminal. NS simulation results from the media playback quality, the quality of user experience and communication control efficiency compared with the proposed control algorithm with the adaptive cooperative FEC based on combination of network coding and channel Coding. The results show that the proposed control algorithm has high real-time performance, high efficiency, high reliability and high user satisfaction. [ABSTRACT FROM AUTHOR]

With the continuous deepening of study of data mining, the application area of data mining gradually expanded, its influence also spread to the media industry. Data visualization technology has changed the traditional narrative mode, make the news becomes a product that be produced. This paper analyzes the history of computer aided reporting to data news, the main models of data news visualization, and the process of data news production through data mining. The study found that data news focuses on the way of data processing in the entire news workflow, it involves not only classical computer graphics technology, image-processing technology and computer audio technology, but also more data analysis and visual processing technologies based on new media and cloud computing involved in. Research data mining and visualization of data-driven journalism can help journalists use big data to do news work better, deepen people's cognition of news events, find the logic which cannot be reflected in traditional news, and maximize the connotation of news report. [ABSTRACT FROM AUTHOR]

With the advent of the Internet and the flourishing of connected technology, electronic commerce has become a new business model that disrupts the traditional transactional model and is transforming the consumer’s lifestyle. Electronic commerce leads to constantly changing customer needs, therefore quick action and collaboration between production and the market is essential. Meanwhile, the abundant transactional data generated by electronic commerce allows us to explore browsing behaviors, habits, preferences and even characteristics of customers, which can help companies to understand their customer’s needs more clearly. Traditional supply chain management (SCM) simply cannot keep up with electronic commerce because demand forecasts are constantly changing. Customer demands create and affect the whole supply chain. The purpose of SCM is to satisfy the customers who support the company by paying for the products; so meeting changing customer needs should be incorporated into SCM by developing demand chain management (DCM). In this paper, we explore how DCM can perform better in the electronic commerce environment based on studying website behavior data and using data analytics tools. The results show that DCM performs much better when paired with the benefits of electronic commerce and Big Data than traditional SCM methods. [ABSTRACT FROM AUTHOR]

As web technology and (big) data continue to transform how we organize ourselves, scholarly research increasingly zooms in on the socio-material conditions of citizen participation and public engagement, the objects and devices that organize publics. Where social issues may often be the driver of such public engagement, increasingly the city and, more specifically, the neighborhood itself have become a central objects connecting their inhabitants through online networks and neighborhood events. Tools and apps for citizen participation then weave together neighborhood stakeholders (e.g. inhabitants, municipal parties and entrepreneurs). This paper zooms in on a sample of 40 such tools that enable and organize bottom-up citizen participation in the city of Amsterdam. Combining a theoretical framework with content analysis, digital methods and data visualization, this paper marks the starting point of a longitudinal analysis of online tools for the urban bottom-up movement. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of visual instance mining, which is to automatically discover frequently appearing visual instances from a large collection of images. We propose a scalable mining method by leveraging the graph structure with images as vertices. Different from most existing approaches that focus on either instance-level similarities or image-level context properties, our method captures both information. In the proposed framework, the instance-level information is integrated during the construction of a sparse instance graph based on the similarity between augmented local features, while the image-level context is explored with a greedy breadth-first search algorithm to discover clusters of visual instances from the graph. This framework can tackle the challenges brought by small visual instances, diverse intra-class variations, as well as noise in large-scale image databases. To further improve the robustness, we integrate two techniques into the basic framework. First, to better cope with the increasing noise of large databases, weak geometric consistency is adopted to efficiently combine the geometric information of local matches into the construction of the instance graph. Second, we propose the layout embedding algorithm, which leverages the algorithm originally designed for graph visualization to fully explore the image database structure. The proposed method was evaluated on four annotated data sets with different characteristics, and experimental results showed the superiority over state-of-the-art algorithms on all data sets. We also applied our framework on a one-million Flickr data set and proved its scalability. [ABSTRACT FROM AUTHOR]

Exploring and analyzing data is becoming increasingly difficult due to the growth of data. Visual analytics tools can be an attractive solution to support the process to derive insights from data. Currently, there are many visual representation methods to visualize the diversity in multivariate data sets. However, most of these applications focus on visual representation problems, and these solutions support limited interactive components for users to effectively explore and analyze data on screen. In this paper, the adaptive diversity table (ADT) is proposed to solve the visual representation problems (occlusion and technique interference). Furthermore, it integrates the mantra techniques to support users to accomplish seven important tasks (i.e. overview, zoom, filter, details-on-demand, relate, history, and extract) that are useful for high dimensional data exploration and data analysis. Experimental results show that the proposed ADT is a better visual representation as compared to other prior techniques. Majority of the respondents prefers to use the proposed ADT over the other visual representation methods. User studies also show that the proposed ADT is more useful as it enables the respondents to be more efficient in analyzing the data sets provided. Graphical Abstract: [ABSTRACT FROM AUTHOR]

This paper reports our research in developing a cyberinfrastructure platform to support multivariate visualization of data collected from distributed sensor network. Three new techniques were introduced in this platform: (1) a hybrid data caching strategy that takes advantages of a scalable and distributed time series database, OpenTSDB, to realize efficient data retrieval; (2) a hyper-dimensional data cube is established to map and translate multivariate and heterogeneous sensor data into a common data structure to support location-aware visual analysis; and (3) a data-driven visualization module is implemented to support interactive and dynamic visualization on a simulated virtual globe. A series of experiments were conducted to demonstrate the good runtime performance of the proposed system. We expect this work to make a major contribution to both the visualization building block development in cyberinfrastructure research and the advancement of visual presentation and analysis of sensor data in domain sciences. [ABSTRACT FROM AUTHOR]