Search

Developing and executing large-scale data analysis applications in parallel and distributed environments can be a complex and time-consuming task. Developers often find themselves diverted from their application logic to handle technical details about the underlying runtime and related issues. To simplify this process, ParSoDA, a Java library, has been proposed to facilitate the development of parallel data mining applications executed on HPC systems. It simplifies the process by providing built-in scalability mechanisms relying on the Hadoop and Spark frameworks. This paper presents ParSoDA-Py, the Python version of the ParSoDA library, which allows for further support of commonly used runtimes and libraries for big data analysis. After a complete library redesign, ParSoDA can be now easily integrated with other Python-based distributed runtimes for HPC systems, such as COMPSs and Apache Spark, and with the large ecosystem of Python-based data processing libraries. The paper discusses the adaptation process, which takes into consideration the new technical requirements, and evaluates both usability and scalability through some case study applications. [ABSTRACT FROM AUTHOR]

At present, there is little research on the application of wireless sensor networks in the agricultural field. In order to improve the operating performance and practical effects of the intelligent agricultural system, based on data mining technology, this paper uses ZigBee wireless sensor network as the networking technology to cover all aspects of crops under the guidance of the concept of sustainable agricultural development, and realizes the data collection and remote-control process of the agricultural production process, and conducts data analysis and processing through data mining. Moreover, in order to improve the performance of the model, this paper proposes to use the single-point crossover multiple-generation genetic algorithm to optimize the weights and thresholds of the BP neural network to establish the Multi-Generation Genetic Algorithm Back Propagation MGABP model. In addition, in application, the analytic hierarchy process is introduced as the guidance mechanism of neural networks. Finally, this paper designs experiments to analyze the performance of the system constructed in this paper, and uses mathematical statistics to perform statistics on experimental results. The experimental analysis and statistical diagrams of various parameters shows the outcome of this study. The research results show that the intelligent agricultural system model constructed in this paper has certain practical effects. [ABSTRACT FROM AUTHOR]

A distributed firewall is a security application that monitors and controls traffic on an organization's network. While centralized firewalls are used against attacks coming from outside a network, distributed firewalls are considered for inside attacks from internal networks such as wireless access and VPN tunnel. Distributed firewalls use policies, which are stated by rules, to find anomalous packets. However, such static rules may be incomplete. In this case, by monitoring firewall logs, the anomalies can be detected. Such logs become big when networks have high traffic, but their hidden knowledge contains valuable information about existing anomalies. In this paper, to detect the anomalies, we extract patterns from big data logs of distributed firewalls using data mining and machine learning. The proposed method is applied to big logs from distributed firewalls in a real security environment, and results are analyzed. [ABSTRACT FROM AUTHOR]

The mining of Tweets to develop datasets on recent issues, global challenges, pandemics, virus outbreaks, emerging technologies, and trending matters has been of significant interest to the scientific community in the recent past, as such datasets serve as a rich data resource for the investigation of different research questions. Furthermore, the virus outbreaks of the past, such as COVID-19, Ebola, Zika virus, and flu, just to name a few, were associated with various works related to the analysis of the multimodal components of Tweets to infer the different characteristics of conversations on Twitter related to these respective outbreaks. The ongoing outbreak of the monkeypox virus, declared a Global Public Health Emergency (GPHE) by the World Health Organization (WHO), has resulted in a surge of conversations about this outbreak on Twitter, which is resulting in the generation of tremendous amounts of Big Data. There has been no prior work in this field thus far that has focused on mining such conversations to develop a Twitter dataset. Furthermore, no prior work has focused on performing a comprehensive analysis of Tweets about this ongoing outbreak. To address these challenges, this work makes three scientific contributions to this field. First, it presents an open-access dataset of 556,427 Tweets about monkeypox that have been posted on Twitter since the first detected case of this outbreak. A comparative study is also presented that compares this dataset with 36 prior works in this field that focused on the development of Twitter datasets to further uphold the novelty, relevance, and usefulness of this dataset. Second, the paper reports the results of a comprehensive analysis of the Tweets of this dataset. This analysis presents several novel findings; for instance, out of all the 34 languages supported by Twitter, English has been the most used language to post Tweets about monkeypox, about 40,000 Tweets related to monkeypox were posted on the day WHO declared monkeypox as a GPHE, a total of 5470 distinct hashtags have been used on Twitter about this outbreak out of which #monkeypox is the most used hashtag, and Twitter for iPhone has been the leading source of Tweets about the outbreak. The sentiment analysis of the Tweets was also performed, and the results show that despite a lot of discussions, debate, opinions, information, and misinformation, on Twitter on various topics in this regard, such as monkeypox and the LGBTQI+ community, monkeypox and COVID-19, vaccines for monkeypox, etc., "neutral" sentiment was present in most of the Tweets. It was followed by "negative" and "positive" sentiments, respectively. Finally, to support research and development in this field, the paper presents a list of 50 open research questions related to the outbreak in the areas of Big Data, Data Mining, Natural Language Processing, and Machine Learning that may be investigated based on this dataset. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) is an emerging paradigm that offers remarkable opportunities for data mining and analysis. IoT envisions a world where all smartphones, vehicles, public services facilities, and home appliances that can be connected to the internet act as data sources. Even today, a significant portion of electronic devices, including watches, emergency alarms, parking doors, and many appliances can be linked to IoT systems and remotely controlled. Big data analysis and data mining methods can be utilized to improve the performance of IoT systems and address their challenges in the area of data storage, processing, and analysis. Extensive studies on IoT with big data can make it possible to accumulate tremendous data and transform it into valuable knowledge using data mining techniques. With this background, this paper provides a systematic survey of the literature on the use of big data analytics and data mining methods in IoT. This review aims to identify the lines of research that should receive more attention in future works. To achieve this goal, the articles published between 2010 and 2021 on the subjects of IoT-based big data and IoT-based data mining (60 articles) have been reviewed. These articles fall into four general categories in terms of focus: architecture/platform, framework, applications, and security. The paper provides a summary of the methods used in IoT-based big data analysis and IoT-based data mining in these four categories to highlight the promising avenues of research for future works. [ABSTRACT FROM AUTHOR]

Recently, the rapid development of experimental methods in the field of catalytic research allows for large amounts of data to be obtained. The use of new statistical and computational processing methods, including the extraction of information from experimental data and their unbiased interpretation, is important for accelerating the development and implementation of catalytic technologies. Necessary information can be extracted using statistical approaches such as PCA, MCR, and ALS. At the same time, machine learning algorithms are beginning to be actively used to interpret and build descriptive models. This paper discusses the main methods of machine learning and examples of their successful application to the analysis of infrared and X-ray absorption spectroscopic data. [ABSTRACT FROM AUTHOR]

In order to improve the effect of precision intelligent agricultural decision support system, this paper combines big data technology to carry out precision mining of agricultural data, and combines decision tree algorithm to carry out data classification processing. Moreover, this paper obtains the most effective agricultural decision reference data through data mining, combines the agricultural decision support process to set the functional modules of the decision system, and analyzes the implementation process of each functional module. In addition, this paper studies the theoretical basis and key technologies of the agricultural production structure optimisation decision support system, and builds a precision and intelligent agricultural decision support system based on big data analysis. The system mainly performs accurate processing of agricultural data and makes effective predictions, and then makes scientific decision results. Finally, this paper verifies the structure of the model in this paper combined with experimental analysis. From the comparison of experiments, it can be seen that the precision and intelligent agricultural decision support system constructed in this paper has significant effects. [ABSTRACT FROM AUTHOR]

With the emergence of social media and the Internet, bulks of data are generated every day in different fields. However, filtering out useful information, which can be turned into knowledge, requires significant effort. This can be achieved using various Big Data Technologies, statistics, data mining and a variety of machine learning algorithms that corporates and businesses are using to improve their performance every day for a long time now. In this paper, we analyze YouTube trending video data that analyses the target that grabs the user's attention over a relatively short time. We present our analysis by measuring, mining, analyzing and comparing key aspects of time-series YouTube data with respect to its view and audience response statistics from 40 000 trending YouTube videos collected over 205 days. We have performed an exploratory data analysis (EDA) on all of it's aspect to get data insights and used statistics to find similarities between them to understand viewing pattern of different video categories. We also compare and observe the variation of activity over time with the nature of the event that affects the quality of our analysis. We make the viewership pattern clear belonging to each category separately by direct analysis. The results and findings can help us predict the category of the video that could probably be trending next in the upcoming days and help individuals get boosted views and performance when they upload video as per the predictions and analysis. [ABSTRACT FROM AUTHOR]

As a very common and classic big data (BD) mining algorithm, the association rule data mining (DM) algorithm is often used to determine the internal correlation between different items and set a certain threshold to determine the size of the correlation. However, the traditional association rule algorithm is more suitable for establishing Boolean association rules between different items of different types of data, and hardening the sharp boundaries of the data causes the performance of the association rules to decrease. In order to overcome this shortcoming of classic DM, this article introduces association rules, support and confidence, the Apriori algorithm and fuzzy association rules based on the neutrosophic fuzzy association rule (NFAR). This paper is based on the data set of the supermarket purchase goods database, by drawing a radar chart to describe the correlation between different goods and different item sets support, and confidence calculation based on association rules support. Finally, the association rules are generated. Compared to the results produced by NFAR and ordinary association rules, the accuracy of the NFAR association rules algorithm in small data sets is 88.48%, while the accuracy of traditional association rules algorithm is only 80.87%, nearly 8 percentage points higher. On large data sets, the prediction accuracy of the neutral fuzzy association rules algorithm is 95.68%, while that of the traditional method is only 89.63%. Therefore, the NFAR algorithm can improve the accuracy and effectiveness of DM. This algorithm has great application prospects and development space in big DM and analysis. [ABSTRACT FROM AUTHOR]

B2C online marketing mode is the development trend of future marketing. The key to improve the efficiency of such a mode is to make efficient use of the current large-scale data and mining the corresponding potential value. To control the cost of B2C online marketing mode, this paper analyzes the UJRP model and proposes a hybrid bat difference algorithm (BADE). Experimental results are verified the effectiveness of the proposed method in diversity and cost control. Furthermore, we utilize multimodel fusion strategy (linear weighted fusion) to achieve better performances in B2C online marketing on cost than BADE method. Finally, in the random and diverse online marketing environment, such an improved method can provide the decision-makers of B2C online marketing mode with more flexible choices. [ABSTRACT FROM AUTHOR]

We provide a brief overview of the connotation and characteristics of data mining technology in the era of big data, analyze the feasibility of data mining technology in business management from the economic and technical perspectives, and propose specific application suggestions according to the content and requirements of business management. This paper describes in detail the principles and steps of using the weighted plain Bayesian algorithm and the decision tree algorithm to analyze students' performance; firstly, we need to obtain the plain Bayesian analysis model of college students' learning literacy in physical education and the C4.5 graduation literacy analysis model, and then use certain rules to combine the weighted plain Bayesian algorithm and the decision tree algorithm to obtain the WNB-C4.5 college students' learning literacy analysis model. In addition, in the prediction of financial risks, the classification scheme can be used in the judgment of violation of regulations, but the most used classification scheme is the decision tree. Experiments show that the effectiveness of this scheme in data mining for financial companies is increased by 2% compared to the benchmark method. [ABSTRACT FROM AUTHOR]

Traditional motion data mining models have some problems, such as poor dynamic data capture effect, low information classification effect rate, poor quantitative representation effect, and so on. Based on this, this paper studies the mining method of dynamic motion data based on neural network, constructs a data mining model based on discrete dynamic modeling technology, and realizes the collection of data information from the aspects of motion characteristics and types combined with multilayer sensors. Neural network algorithm is used for comprehensive analysis to realize multivariate analysis and objective evaluation of all data of dynamic motion process and accurate analysis and evaluation according to different data characteristics of different types of motion data. The results show that the data mining model based on discrete dynamic modeling technology and wearable sensor technology has the advantages of high feasibility, high intelligence, and wide application range. [ABSTRACT FROM AUTHOR]

This paper presents a big data analysis and prediction system based on convolutional neural networks. Continuous template matching technology is used to analyze the distributed data structure of big data, and the information fusion processing of cloud service combination big data is combined with matching related detection methods, frequent item detection, and association rule feature extraction of high-dimensional fusion data. A clustering method is adopted to realize the classification and mining of cloud service portfolio big data. The hardware equipment of the car to detect the surrounding environment is complicated, and the combination of the convolutional neural network and the camera to detect the surrounding environment has become a research hotspot. However, simply using the convolutional neural network to process the camera data to control the turning angle of the car has the problems of long training time and low accuracy. An improved convolutional neural network is proposed. The experimental results show that the accuracy of data mining by this method is 12.43% and 21.76% higher than that of traditional methods, and the number of iteration steps is shorter, indicating that the timeliness of mining is higher. This network structure can effectively improve the training speed of the network and improve the accuracy of the network. It is proven that the convolutional neural network has faster training speed and higher accuracy. [ABSTRACT FROM AUTHOR]

Location and navigation services based on global navigation satellite systems (GNSS) are needed for real-time high-precision positioning applications in relevant economic sectors, such as precision agriculture, transport, civil engineering or mapping. Real-time navigation users of GNSS networks have significantly increased all around the world, since the 1990s, and usage has exceeded initial expectations. Therefore, if the evolution of GNSS network users is monitored, the dynamics of market segments can be studied. The implementation of this hypothesis requires the treatment of big volumes of navigation data over several years and the continuous monitoring of customers. This paper is focused on the management of massive connection of GNSS users in an efficient way, in order to obtain analysis and statistics. Big data architecture and data analyses based on data mining algorithms have been implemented as the best way to approach the hypothesis. Results demonstrate the dynamic of users of different market segments, the increasing demand over the years and, specifically, conclusions are obtained about the trends, year-on-year correlation and business volume recovering after economic crisis periods. [ABSTRACT FROM AUTHOR]

In the big data environment, the visualization technique has been increasingly adopted to mine the data on library and information (L&I), with the diversification of data sources and the growth of data volume. The previous research into the information association of L&I visualization network rarely tries to construct such a network or explore the information association of the network. To overcome these defects, this paper explores the visualization of L&I from the perspective of big data analysis and fusion. Firstly, the authors analyzed the topology of the L&I visualization network and calculated the metrics for the construction of L&I visualization topology map. Next, the importance of meta-paths of the L&I visualization network was calculated. Finally, a complex big data L&I visualization network was established, and the associations between information nodes were analyzed in detail. Experimental results verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In recent years, the application of the gradient boosting-back propagation (GB-BP) neural network algorithm in many industries has brought huge benefits, so how to combine the GB-BP neural network algorithm with sports has become a research hotspot. Based on this, this paper studies the application of the GB-BP neural network algorithm in wrestling, designs the sports athletes action recognition and classification model based on the GB-BP neural network algorithm, first analyzes the research status of wrestling action recognition, and then optimizes and improves the shortcomings of action recognition and big data analysis technology. The GB-BP neural network algorithm can realize the accurate recognition and classification of wrestlers' training actions and carry out big data mining analysis with known action recognition, so as to achieve accurate classification. The experimental results show that the model can play a good role in wrestling and effectively improve the efficiency of wrestlers in training. [ABSTRACT FROM AUTHOR]