逐层 Transformer 在类别不均衡数据的应用.

In order to solve the problem that class-imbalance data of clinical medical tables tend to have an impact on the model and that the performance of deep learning framework is difficult to match that of traditional machine learning methods when processing scale data tasks, this paper proposed a layer by layer Transformer (LLT) network model based on cascaded under-sampling. LLT deleted the most types of data layer by layer by cascade under-sampling method to achieve the balance of data categories and reduced the impact of class-imbalance data on the classifier. Moreover, LLT used attention mechanism to carry out correlation evaluation on the features of the input data to achieve feature selection, refined the feature extraction abi-lity and improved the model performance. This paper used RA (rheumatoid arthritis) data as test samples. Experimental results show that, on the premise of not changing the sample distribution, the recognition rate of a few categories is increased by 6.1% by the proposed cascade under-sampling method, which is 1.4% and 10.4% higher than that of the commonly used NEARMISS and ADASYN respectively. The accuracy of the RA tabular data and the F1-score index of LLT reach 72.6% and 71.5%, the AUC value is 0.89, the mAP value is 0.79, and the performance exceeds the current mainstream tabular data classification models such as RF, XGBoost and GBDT. This paper also visualized the model process and analyzed the characteristics affecting RA. It has a good guiding significance for the clinical diagnosis of RA. [ABSTRACT FROM AUTHOR]