Search

Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems. However, with the constant evolution of algorithms, data, and computing power, the increasing size and complexity of these models have led to increased training costs and reduced efficiency. This study aims to minimize the inference time of such models while maintaining computational performance. It also proposes a novel Distillation model for PAL-BERT (DPAL-BERT), specifically, employs knowledge distillation, using the PAL-BERT model as the teacher model to train two student models: DPAL-BERT-Bi and DPAL-BERT-C. This research enhances the dataset through techniques such as masking, replacement, and n-gram sampling to optimize knowledge transfer. The experimental results showed that the distilled models greatly outperform models trained from scratch. In addition, although the distilled models exhibit a slight decrease in performance compared to PAL-BERT, they significantly reduce inference time to just 0.25% of the original. This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency. [ABSTRACT FROM AUTHOR]

This study addresses the limitations of Transformer models in image feature extraction, particularly their lack of inductive bias for visual structures. Compared to Convolutional Neural Networks (CNNs), the Transformers are more sensitive to different hyperparameters of optimizers, which leads to a lack of stability and slow convergence. To tackle these challenges, we propose the Convolution-based Efficient Transformer Image Feature Extraction Network (CEFormer) as an enhancement of the Transformer architecture. Our model incorporates E-Attention, depthwise separable convolution, and dilated convolution to introduce crucial inductive biases, such as translation invariance, locality, and scale invariance, into the Transformer framework. Additionally, we implement a lightweight convolution module to process the input images, resulting in faster convergence and improved stability. This results in an efficient convolution combined Transformer image feature extraction network. Experimental results on the ImageNet1k Top-1 dataset demonstrate that the proposed network achieves better accuracy while maintaining high computational speed. It achieves up to 85.0% accuracy across various model sizes on image classification, outperforming various baseline models. When integrated into the Mask Region-Convolutional Neural Network (R-CNN) framework as a backbone network, CEFormer outperforms other models and achieves the highest mean Average Precision (mAP) scores. This research presents a significant advancement in Transformer-based image feature extraction, balancing performance and computational efficiency. [ABSTRACT FROM AUTHOR]

At present, super-resolution algorithms are employed to tackle the challenge of low image resolution, but it is difficult to extract differentiated feature details based on various inputs, resulting in poor generalization ability. Given this situation, this study first analyzes the features of some feature extraction modules of the current super-resolution algorithm and then proposes an adaptive feature fusion block (AFB) for feature extraction. This module mainly comprises dynamic convolution, attention mechanism, and pixel-based gating mechanism. Combined with dynamic convolution with scale information, the network can extract more differentiated feature information. The introduction of a channel spatial attention mechanism combined with multi-feature fusion further enables the network to retain more important feature information. Dynamic convolution and pixel-based gating mechanisms enhance the module's adaptability. Finally, a comparative experiment of a super-resolution algorithm based on the AFB module is designed to substantiate the efficiency of the AFB module. The results revealed that the network combined with the AFB module has stronger generalization ability and expression ability. [ABSTRACT FROM AUTHOR]

Typically, the current and speed loop closure of servo motor of the parallel platform is accomplished with incremental PI regulation. The control method has strong robustness, but the parameter tuning process is cumbersome, and it is difficult to achieve the optimal control state. In order to further optimize the performance, this paper proposes a double-loop control structure based on fuzzy integral and neuron proportional integral (FI-NPI). The structure makes full use of the control advantages of the fuzzy controller and integrator to improve the performance of speed closed-loop control. And through the feedforward branch, the speed error is used as the teacher signal for neuron supervised learning, which improves the effect of current closed-loop control. Through comparative simulation experiments, this paper verifies that the FI-NPI controller has a faster dynamic response speed than the traditional PI controller. Finally, in this paper, the FI-NPI controller is implemented in C language in the servo-driven lower computer, and the speed closed-loop test of the BLDC motor is carried out. The experimental results show that the FI-NPI double-loop controller is better than the traditional double-PI controller in performance indicators such as convergence rate and RMSE, which confirms that the FI-NPI double-loop controller is more suitable for BLDC servo control. [ABSTRACT FROM AUTHOR]

With the rapid economic development, the serious air pollution in Beijing attracts increasing attention in the last decade. Seen as one whole complex and grey system, the causal relationship between the social development and the air pollution in Beijing has been quantitatively analyzed in this paper. By using the grey relational model, the aim of this study is to explore how the socio-economic and human activities affect on the air pollution in the city of Beijing, China. Four air pollutants, as the particulate matter with size 2.5 micrometers or less (PM2.5), particulate matter with size 10 micrometers or less (PM10), sulfur dioxide (SO2) and nitrogen dioxide (NOx), are selected as the indicators of air pollution. Additionally, fifteen socio-economic indicators are selected to account for the regional socio-economic characteristics (economy variables, energy consumption variables, pollution emissions variables, environment and construction activity variables). The results highlight that all variables are associated with the concentrations of the four selected air pollutants, but with notable differences between the air pollutants. Most of the socio-economic indicators, such as industrial output, total energy consumption are highly correlated with PM2.5, while PM10, SO2, and NOx present in general moderate correlations with most of the socio-economic variables. Contrary to other studies and reports this study reveals that vehicles and life energy do not have the strongest effect on air pollution in Beijing. This study provides useful information to reduce air pollution and support decision-making for sustainable development.

With the development of recommendation technology, it is of great significance to analyze users' digital footprints on social networking sites, extract user behavior rules, and make a relatively accurate assessment of each user's needs, to provide personalized services for users. It has been found that the users' behavior on social networking sites has a great correlation with the user's personalities. The OCEAN model's five characteristics can cover all aspects of user personality. There are some shortcomings in the current micro-directional propagation method. This paper has improved the traditional collaborative filtering method and proposed a collaborative filtering method based on user clustering. The model first clusters the users according to their OCEAN model, and then it filters the users collaboratively in the cluster to which the user belongs with the collaborative filtering method based on an optimized singular value decomposition (SVD) recommendation algorithm, called the BiasSVD recommendation algorithm -- to reduce the dimensionality of the data. Then it generates recommendations. Experiments show that clustering users' OCEAN models can improve the accuracy of recommendations. Compared with the entire user space, it greatly reduces the recommendation time, and effectively solves the cold start problem in micro directional propagation. [ABSTRACT FROM AUTHOR]

How to eliminate useless information in the vast network information and retain effective information is a problem that needs to be continuously explored in the field of deep learning. This paper conducts text classification on the network evaluation frequently encountered in daily life, mainly to screen out the meaningless comments published by Internet users, to have access to more useful information. In this paper, a text filtering model was constructed based on word vector and Long Short-Term Memory (LSTM) and improved by adding Deep Averaging Networks (DAN) and convolutional neural network (CNN). The major improvement of the LSTM&DAN model was to retain the original word vector information and to improve the accuracy of the text classification model without increasing hyperparameter and model structure complexity. The LSTM&CNN model mainly combines the advantages of convolutional neural network in exploring the deep information of text, which was an improvement over the original LSTM. It was proved by experiments that this improvement is meaningful. Compared with the shallow neural network, the accuracy has been greatly improved. [ABSTRACT FROM AUTHOR]

In recent years, with economic development, urbanization has been accelerating. In the past 100 years, the global average temperature has increased by 0.5°C, and according to the predictions of most global climate models, the temperature will continue to increase by 1.5°C to 3.0°C in the next 100 years. Under the influence of global warming, the urban heat island effect problem is becoming more and more serious, bringing much harm. Studying the thermal effect and its influencing factors is of great significance for sustainable urban development. In this study, the seasonal and interannual surface temperature changes of the study area are inverted based on the SW-TES algorithm, and the LST inversion of the algorithm is simply verified. At the same time, a cell dichotomous model was established to explore the influencing factors of the urban heat island effect and the influence of different land use types and normalized vegetation indices in the Hangzhou metropolitan area was analyzed. The results showed that the overall heating trend of the study area showed a heating trend, the expansion rate of regional construction land was relatively fast, and the scale of urban land was increasing. Based on the results of surface temperature changes of areas with different land types, we obtained the temperature trends of each land type [from 2005 to 2018] and found that there was a negative correlation between surface temperature and normalized vegetation index. This study provides a theoretical basis to evaluate the urban heat island effect and analyze factors that impact it and would promote sustainable urban development. [ABSTRACT FROM AUTHOR]

Text classification has been highlighted as the key process to organize online texts for better communication in the Digital Media Age. Text classification establishes classification rules based on text features, so the accuracy of feature selection is the basis of text classification. Facing fast-increasing Chinese electronic documents in the digital environment, scholars have accumulated quite a few algorithms for the feature selection for the automatic classification of Chinese texts in recent years. However, discussion about how to adapt existing feature selection algorithms for various types of Chinese texts is still inadequate. To address this, this study proposes three improved feature selection algorithms and tests their performance on different types of Chinese texts. These include an enhanced CHI square with mutual information (MI) algorithm, which simultaneously introduces word frequency and term adjustment (CHMI); a term frequency–CHI square (TF–CHI) algorithm, which enhances weight calculation; and a term frequency–inverse document frequency (TF–IDF) algorithm enhanced with the extreme gradient boosting (XGBoost) algorithm, which improves the algorithm's ability of word filtering (TF–XGBoost). This study randomly chooses 3000 texts from six different categories of the Sogou news corpus to obtain the confusion matrix and evaluate the performance of the new algorithms with precision and the  F 1 -score. Experimental comparisons are conducted on support vector machine (SVM) and naive Bayes (NB) classifiers. The experimental results demonstrate that the feature selection algorithms proposed in this paper improve performance across various news corpora, although the best feature selection schemes for each type of corpus are different. Further studies of the application of the improved feature selection methods in other languages and the improvement in classifiers are suggested. [ABSTRACT FROM AUTHOR]

The identification of small land targets in remote sensing imagery has emerged as a significant research objective. Despite significant advancements in object detection strategies based on deep learning for visible remote sensing images, the performance of detecting a small and densely distributed number of small targets remains suboptimal. To address this issue, this study introduces an improved model named YOLOV4_CPSBi, based on the YOLOV4 architecture, specifically designed to enhance the detection capability of small land targets in remote sensing imagery. The proposed model enhances the traditional CSPNet by redefining its channel partitioning and integrating this enhanced structure into the neck part of the YOLO network model. Additionally, the conventional pyramid fusion structure used in the traditional BiFPN is removed. By integrating a weight-based bidirectional multi-scale mechanism for feature fusion, the model is capable of effectively reasoning about objects of various sizes, with a particular focus on detecting small land targets, without introducing a significant increase in computational costs. Using the DOTA dataset as research data, this study quantifies the object detection performance of the proposed model. Compared with various baseline models, for the detection of small targets, its AP performance has been improved by nearly 8% compared with YOLOV4. By combining these modifications, the proposed model demonstrates promising results in identifying small land targets in visible remote sensing images. [ABSTRACT FROM AUTHOR]

The process of computationally identifying and categorizing opinions expressed in a piece of text is of great importance to support better understanding and services to online users in the digital environment. However, accurate and fast multi-label automatic classification is still insufficient. By considering not only individual in-sentence features but also the features in the adjacent sentences and the full text of the tweet, this study adjusted the Multi-label K-Nearest Neighbors (MLkNN) classifier to allow iterative corrections of the multi-label emotion classification. It applies the new method to improve both the accuracy and speed of emotion classification for short texts on Twitter. By carrying out three groups of experiments on the Twitter corpus, this study compares the performance of the base classifier of MLkNN, the sample-based MLkNN (S-MLkNN), and the label-based MLkNN (L-MLkNN). The results show that the improved MLkNN algorithm can effectively improve the accuracy of emotion classification of short texts, especially when the value of K in the MLkNN base classifier is 8, and the value of α is 0.7, and the improved L-MLkNN algorithm outperforms the other methods in the overall performance and the recall rate reaches 0.8019. This study attempts to obtain an efficient classifier with smaller training samples and lower training costs for sentiment analysis. It is suggested that future studies should pay more attention to balancing the efficiency of the model with smaller training sample sizes and the completeness of the model to cover various scenarios. [ABSTRACT FROM AUTHOR]

Context: Numerous dams have been built in China to develop hydropower, a sustainable and clean energy source. In recent years, the impact of dam construction on the regional climate has gradually attracted the attention of researchers. Aims: This study has evaluated the impact of large-scale dam construction on regional precipitation. Methods: This paper used the precipitation data of more than 2400 national stations of the China National Meteorological Information Center from 1990 to 2012. The regional precipitation data before and after the construction of the Xiaolangdi Dam and the Three Gorges Dam on the Yangtze River were analysed using geostatistical tools. Wavelet transform and Yamamoto signal-to-noise ratio analysis were further adopted. Key results: Analysis of the variation points of precipitation characteristics confirmed the correlation between dam construction and regional precipitation; the precipitation values in the two dam study areas had an increasing trend after the completion of the dams, and the fitting trend line showed an obvious increasing trend. Conclusions: According to the analysis of precipitation variation points, it can be concluded that the establishment of the dam affected the precipitation in the area of ∼200 km upstream and increased the precipitation value in this area. The study showed that dam construction has a strong correlation with regional precipitation. Implications: It is speculated that the construction of super dams will have a greater impact on precipitation. The impact of dam construction on the regional climate has attracted great attention of researchers. This study evaluated the impact of a large-scale dam construction on regional precipitation. The regional precipitation data before and after the construction of the Xiaolangdi Dam and the Three Gorges Dam were analysed using geostatistical tools. It can be concluded that the establishment of the dam increased the precipitation value in this area. It is speculated that the larger dams will have greater precipitation. [ABSTRACT FROM AUTHOR]