Search

We consider the two-alternative processing of big data in the framework of the two-armed bandit problem. We assume that there are two processing methods with different, fixed but a priori unknown efficiencies which are due to different reasons including those caused by legislation. Results of data processing are interpreted as random incomes. During control process, one has to determine the most efficient method and to provide its primary usage. The difficulty of the problem is caused by the fact that its solution essentially depends on distributions of one-step incomes corresponding to results of data processing. However, in case of big data we show that there are universal processing strategies for a wide class of distributions of one-step incomes. To this end, we consider Gaussian two-armed bandit which naturally arises when batch data processing is analyzed. Minimax risk and minimax strategy are searched for as Bayesian ones corresponding to the worst-case prior distribution. We present recursive integro-difference equation for computing Bayesian risk and Bayesian strategy with respect to the worst-case prior distribution and a second order partial differential equation into which integro-difference equation turns in the limiting case as the control horizon goes to infinity. We also show that, in case of big data, processing of data one-by-one is not more efficient than optimal batch data processing for some types of distributions of one-step incomes, e.g. for Bernoulli and Poissonian distributions. Numerical experiments are presented and show that proposed universal strategies provide high performance of two-alternative big data processing.

In connection with the wide spreading of various intelligent sensors, IoT devices, smartphones, autonomous transport systems, various industrial and home automation systems, an unprecedented amount of data is generated, including those intelligently linked to each other. Linked data allows you to build complex and varied relationships between objects and subjects of the real world. Unfortunately, modern big data processing systems and machine learning models are extremely poorly suited for working with such dynamically linked data, especially in the case of real-time systems. We discuss current and future-proof approaches to working with such data using graph analysis models.

There are many power transformer parameters, and their changes are closely related to factors such as grid operation and meteorological environment. It is urgent to use effective big data technology to mine and analyze a large amount of relevant data and extract information to improve the timeliness and accuracy of power transformer experiments. First, based on a large number of fault and defect samples, the corresponding relationship between the key performance of the power transformer and the state quantity is mined through the confidence of the association rule, and then the time series of the state quantity of the power transformer is characterized by big data through the big data processing algorithm. The characteristic root spectrum distribution and circle rate of big data containing time series models are studied, the history and current state information of state quantities are analyzed, and the key performance experiments and abnormal detection of power transformers are realized. Taking a 500kV substation as an example, the transformer load, online monitoring and environmental meteorological data fusion constitute the key performance big data, and the big data processing algorithm is used to compare the spectral properties of the historical and current time period matrices to realize the key performance experiment of the transformer And anomaly detection. The research results show that the big data processing algorithm is effective for analyzing the operating status of power transformers, and provides a new way of thinking for the application of big data technology in power transformer status experiments.

Improving the capability of geohazard data online analysis and evaluation for geohazard prevention and mitigation is the major developing trend of geohazard information service and relevant research. Making use of the information technologies to solve the technical challenges and difficulties encountered in geohazard big data processing and analysis is important to effectively improve the capacity of geohazard information service. By using the micro-service, one of the advanced information technology, this study proposed an technical solution for the construction of geohazard data online analysis and processing service, which was designed and constructed based on the working mechanism and procedure of the analysis. As an example, the geohazard online susceptibility analysis service was designed and developed. In the proposed technical solution, the online susceptibility analysis service was splitted into multiple operational-independent and also interconnected micro-services for the completion of the analytical process. The developments of each individual micro-service provide support for further revision, upgrade and expansion of the data analysis ultimization or model optimization. The propose of the technical solution further deepens the integration of geohazard research and information technology to improve the level and practical efficiency of geohazard information service leading by the advancement of information technology.

The article considers the basic methods of machine learning applied by individual entrepreneurs within the framework of transition to digital production to improve the efficiency of data processing, classification of existing and would-be customers and their subsequent work with them. The main attention is paid to the problem of increasing the effectiveness of methods of machine learning applied for solving the current questions. Areas of application of technology are shown. The peculiarities of machine learning are briefly analyzed. The main features and prospects of the development of Machine Learning services are shown on the basis of the concept of a step-by-step combination of the methods under consideration. One of the main algorithms for working with data is analyzed; its main features, scope and procedure are described. Recommendations are given for the further use of machine learning algorithms. The role of machine learning in the development of modern science and industry is analyzed, the main tendencies of the industry development are determined, and the practical application of big data is shown. As part of the transition to Industry 4.0, the main areas of application of machine learning, big data, Artificial Intelligence and their relations with the corresponding fields of science and production are described. The article also offers a review of the application of Artificial Intelligence and machine learning in particular in the context of the transition to digitalization and the issues of individual entrepreneurship.