Search

To address the issue of calculation errors in neural network matrix-vector multiplication caused by manufacturing processes of RRAM cells, this paper modeled the characteristics of multiple faults in RRAM crossbar arrays and proposed a multi-fault tolerant algorithm. Firstly, it modeled the impacts of common transition fault and stuck at fault in RRAM crossbar arrays on the accuracy of neural network computations. Secondly, it partitioned the neural network and conducted partitioned training based on an improved knowledge distillation method. Lastly, it further optimized the algorithm by selecting an appro priate loss function and incorporating normalization layers. Experimental results on the MNIST and Cifar-10 datasets demonstrate that the proposed method can achieve a recovery rate of over 98% across multiple neural networks, indicating its effeetiveness in mitigating the impact of multiple faults in RRAM crossbar arrays on the accuracy of neural network computations. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the deep reinforcement learning method, the intrinsic curiosity model (ICM) guides the agent to obtain the opportunity to learn unknown strategies in the sparse reward environment, but the curiosity reward is a state difference value, which will make the agent pay too much attention to the exploration of new states, then could be the problem of blind exploration arises. To solve the above problem, this paper proposed an intrinsic curiosity model algorithm based on knowledge distillation (KD-ICM). Firstly, it introduced the method of knowledge distillation to make the agent acquire more abundant environmental information and strategy knowledge in a short time and accelerate the learning process. Secondly, by pre-training teachers' neural network model to guide the forward network to obtain a forward network model with higher accuracy and performance, reduced the blind exploration of agents. It designed two different simulation experiments on the Unity simulation platform for comparison. The experiments show that in the complex simulation task environment, the average reward of KDICM algorithm is 136% higher than that of ICM, and the optimal action probability is 13.47% higher than that of ICM. Both the exploration performance of the agent and the exploration quality can be improved, and it verifies the feasibility of the algorithm. [ABSTRACT FROM AUTHOR]

The reproductive performance of sows can play a critical role in animal breeding, particularly in the efficiency and effectiveness of selection. However, manual recordings of piglet births and their survival rates cannot fully meet the large-scale production in recent years. The high precision is often required to capture more nuanced data, such as the intervals between births. The advanced technologies can be expected to enhance both the accuracy and efficiency of animal breeding programs. In this study, a lightweight network was developed to rapidly and accurately monitor the sow birthing in real time. Specifically, essential birthing metrics were engineered to analyze, such as the number of piglets born and the precise intervals between each birth. The lightweight network was tailored for the real-time monitoring of sow birthing activities. The critical birthing parameters were obtained to significantly enhance the efficiency and accuracy of breeding programs. Initially, the efficacy of different monitoring views—specifically, single versus double-column views—were evaluated on the accuracy of the improved model. A single-column view was significantly improved to accurately monitor the birthing events. The real-time decisionmaking and direct implications were obtained from the breeding outcomes. Advanced video processing techniques were incorporated, such as horizontal and vertical flipping. Some challenges were remained on the dynamic changes in the sow posture and varying camera perspectives during monitoring. Moreover, different lighting conditions were adapted to capture the inherent motion blur of active piglets during birth. Color jittering and Gaussian blur were then employed to significantly enhance the robustness of the model. The reliable performance was obtained under diverse operational conditions. Further advancements were achieved through a comparative analysis of classification networks. The results revealed that ResNet50 was greatly contributed to the recognition accuracy. MobileNetV3-S was performed the best with the compact model size and superior processing speed of 505.14 frames per second, indicating the optimal operational efficiency. Furthermore, MobileNetV3-S was refined to apply with the masked generative distillation—a sophisticated technique that was effectively enhanced the network's ability to capture and interpret essential birthing features. ResNet50 was utilized as the teacher model in the practical application, while MobileNetV3-S as the student model. The training was conducted using masked generative distillation followed by dependency graph pruning. The tests were carried out on a DELL OptiPlex microcomputer. An impressive detection speed of 83.10 frames per second was achieved with a test accuracy in a single-column field of view of 91.48%. Although there was a slight decrease in the accuracy of 0.98 percentage points, the detection speed was improved by 67.13 frames per second. This improved model was then deployed at the edge for testing. The better performance was achieved in the managed farrowing intervals with a detection error of just 0.31 seconds and the duration of piglet birth events with a mere 0.02-second error. Highly efficient and exceptionally precise real-time monitoring was obtained to promote the management practices of breeding activities in complex farm environments. In conclusion, the advanced computational techniques were integrated for the transformative potential to the monitoring of sow birthing. Real-time data was acquired to combine the image processing and machine learning. Some standards can be offered for the accuracy and efficiency in livestock management. The reproductive dynamics can greatly contribute to the sustainable and scientifically-informed animal husbandry. [ABSTRACT FROM AUTHOR]

In order to efficiently construct a multimodal subject knowledge graph in the field of education, a textbook text entity relationship extraction algorithm based on large model knowledge distillation and multi-model collaborative reasoning is proposed. During the model training phase, this paper uses a closed source model with 100 billion parameters to annotate text data and achieve implicit knowledge distillation. Then, this paper fine-tunes the domain data instructions for the open-source billion scale parameter model to enhance the instruction compliance ability of the entity relationship extraction task of the open-source model. In the model inference stage, the closed source model serves as the guiding model, and the open-source billion scale parameter model serves as the execution model. Experimental results show that knowledge distillation, multi-model collaboration, and domain data instruction fine-tuning are effective, significantly improving the effectiveness of textbook text entity relationship extraction tasks based on instruction prompts. A multimodal named entity recognition algorithm for textbook diagrams with explicit and implicit knowledge enhancement has been proposed. Firstly, this paper uses techniques such as image OCR (optical character recognition) and visual language modeling to extract textual information and global content description information from textbook diagrams. Then, by using explicit knowledge base retrieval and implicit LLM hint enhancement methods, auxiliary knowledge that may be associated with image title pairs is obtained. The knowledge obtained from explicit knowledge base and implicit LLM is further fused to form the final auxiliary knowledge. Finally, the auxiliary knowledge of the schematic diagram is combined with the schematic diagram title to achieve multimodal named entity recognition of the textbook schematic diagram title. Experimental results show that the algorithm is advanced and the interpretability of the algorithm is enhanced.

Purposes In order to solve the computational cost problem of complex model in time and hardware, and improve the performance of speech enhancement algorithm, a speech enhancement algorithm using multi-task adaptive knowledge distillation is proposed. Methods First, the idea of knowledge distillation is adopted to solve the problems that the existing speech enhancement model is too large, has many parameters, and has high calculation cost. Second, the differences between different time-frequency units are fully considered, and the weighting factor is introduced to optimize the traditional loss function to improve the network performance of students. In order to avoid the uncertainty of teacher network prediction affecting the performance of student network, the knowledge distillation network of multi-task adaptive learning is built to better utilize the correlation between different tasks to optimize the model. Findings The simulation results show that the proposed algorithm can effectively improve the performance of speech enhancement model while reducing the number of parameters and shortening the calculation time.

Machine translation (MT) is the process of using a computer to convert one language into another language with the same semantics. With the introduction of neural network, neural machine translation (NMT), as a powerful machine translation technology, has achieved remarkable success in the field of automatic translation and artificial intelligence. Due to the problem of redundant parameters and structure in traditional neural translation models, knowledge distillation (KD) technology is proposed to compress the model and accelerate the inference of neural machine translation, which has attracted wide attention in the field of machine learning and natural language processing. This paper systematically investigates and compares various translation models with introduction of know-ledge distillation from the perspectives of evaluation indicators and technical innovations. Firstly, this paper briefly reviews the development process, mainstream frameworks and evaluation indicators of machine translation. Secondly, the knowledge distillation technology is introduced in detail. Thirdly, the development direction of neural machine translation based on knowledge distillation is detailed from four perspectives: multi-language model, multi-modal translation, low-resource language, autoregressive and non-autoregressive, and the research status of other fields is briefly introduced. Finally, the problems of existing large language models, zero-resource languages and multi-modal machine translation are analyzed, and the development trend of neural machine translation is prospected.

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In marine biology research and fishing industries, ships using underwater robots to capture and identify marine organisms may face limited communication bandwidth and computing resources, making lightweight network models better suited for such conditions. To address this issue, a modified YOLOv5n model for marine organism detection was proposed, incorporating improvements in the neck network, model pruning, and knowledge distillation techniques. The model size was effectively reduced by leveraging the Gsconv lightweight convolution module to replace standard convolutions in the YOLOv5n Neck section. A novel α- giou loss function was adopted to enhance bounding box regression accuracy. Based on weight coefficients, L1-norm regularization pruning was applied to eliminate unnecessary channels and associated convolutional kernels. Finally, retraining and L2 knowledge distillation were employed to fine-tune the model accuracy close to pre-pruning levels. Experimental results demonstrated a 53% reduction in computational load and a 51% decrease in parameters compared to the original YOLOv5n baseline network. The proposed algorithm ensures the effectiveness [ABSTRACT FROM AUTHOR]

Knowledge distillation algorithms have a great effect on the streamlining of deep neural networks. The current feature-based knowledge distillation algorithms either focus on a single part for improvement and ignore other beneficial parts, or provides effective guidance for the part that a small model should focus on, which makes the distillation effect insufficient. In order to make full use of the beneficial information of the large model and process it to improve the knowledge conversion rate of the small model, this paper proposed a new distillation algorithm. Firstly, it used the conditional probability distribution to fit the feature spatial distribution of the large model's middle layer, and then extracted the spatial attention maps that tended to be similar after fitting together with other beneficial information. Finally, it used the small convolutional layer, narrowed the gap between models, transmitted the transformed information to the small model to achieve distillation. Experimental results show that the algorithm has the applicability of multiple teacher-student combinations and the generality of multiple data sets, and compared with the current more advanced distillation algorithms, the performance is improved by about 1.19% and the time is shortened by 0.16 h. It has important engineering significance and wide application prospects for large networks' optimization and the application of deep learning on low-resource devices. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Shanghai University / Shanghai Daxue Xuebao is the property of Journal of Shanghai University (Natural Sciences) Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Knowledge distillation is a general approach to improve the performance of the named entity recognition (NER) models. However, the classical knowledge distillation loss functions are coupled, which leads to poor logit distillation. In order to decouple and effectively improve the performance of logit distillation, this paper proposes an approach, k-best Viterbi decoupling knowledge distillation (kvDKD), which combines k-best Viterbi decoding to improve the computational efficiency, effectively improving the model performance. Additionally, the NER based on deep learning is easy to introduce noise in data augmentation. Therefore, a data augmentation method combining data filtering and entity rebalancing algorithm is proposed, aiming to reduce noise introduced by the original dataset and to enhance the problem of mislabeled data, which can improve the quality of data and reduce overfitting. Based on the above method, a novel named entity recognition model NER-kvDKD (named entity recognition model based on k-best Viterbi decoupling knowledge distillation) is proposed. The comparative experimental results on the datasets of MSRA, Resume, Weibo, CLUENER and CoNLL-2003 show that the proposed method can improve the generalization ability of the model and also effectively improves the student model performance.

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Data Acquisition & Processing / Shu Ju Cai Ji Yu Chu Li is the property of Editorial Department of Journal of Nanjing University of Aeronautics & Astronautics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The lack of large parallel corpora is one of the key issues in low-resource neural machine translation. This paper proposes a neural machine translation method based on language model distillation, which regularizes neural machine translation training by using a monolingual language model. This method introduces prior knowledge contained in the language model to improve translation results. Specifically, we draw on the idea of knowledge distillation, and use the target-side language model (teacher model) trained on rich monolingual data to construct the regularization factor of the low-resource neural machine translation model (student model), allowing the translation model to learn highly generalized prior knowledge from the language model. Unlike traditional monolingual language models that participate in the decoding process, the language model in this method is only used during training and does not participate in the inference stage, so it can effectively improve decoding speed. Experimental results on two low-resource translation datasets of Uyghur-Chinese and Tibetan-Chinese from the 17th national machine translation conference (CCMT2021) show that compared with the current state-of-the-art language model fusion baseline system, BLEU can be improved by 1. 42 points (Tibetan-Chinese) to 2.11 points (Chinese-Uyghu). [ABSTRACT FROM AUTHOR]

In traditional knowledge distillation, if there is a significant difference in the parameter scale between the teacher and student models, there will be a negative impact that the student model cannot learn from the larger teacher model. In order to achieve better performance of student models on different tasks in knowledge distillation for BERT, a new multi-teacher distillation scheme for BERT model is proposed by making intensive research on existing model distillation methods and analysis on the advantages and disadvantages of different teacher models, that is, BERT, RoBERTA and XLNET are used to distill the teacher model with BERT structure. Meanwhile, the distillation scheme for the knowledge representation of the middle layer of the teacher model is modified, and the distillation on Transformer layer is added. Finally, experiments on several datasets in GLUE show that the final distillation results are ideal and can retain 95.1% accuracy of the teacher model. [ABSTRACT FROM AUTHOR]

Copyright of Computer Measurement & Control is the property of Magazine Agency of Computer Measurement & Control and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Control Theory & Applications / Kongzhi Lilun Yu Yinyong is the property of Editorial Department of Control Theory & Applications and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Knowledge distillation combined with pre-trained language models is one of the primary methods for constructing question-answering models. However, these methods suffer from inefficiencies in knowledge transfer, time-consuming teacher model training, and mismatched capabilities between teacher and student models. To address these issues, this paper proposed a question-answering model based on self-distillation and self-ensemble, named SD-SE-BERT. The self-ensemble mechanism was designed based on a sliding window; the student model used BERT; the teacher model was derived from a weighted average combination of several student models during the training process, based on their performance on the validation set. The loss function used the output of the ensemble and the true labels to guide the training of the student model in the current round. Experimental results on the SQuAD1.1 dataset show that the EM and F1 scores of SD-SE-BERT are respectively 7.5 and 4.9 higher than those of the BERT model, and the model's performance surpasses other representative single models and distillation models. Compared to the fine-tuning results of the large-scale language model ChatGLM-6B, the EM score was improved by 4.5, and the F1 score by 2.5. It proves that SD-SE-BERT can leverage the model's supervision information to enhance the model's capacity to combine different text data features, eliminating the need for complex teacher-model training and avoiding the problem of mismatch between teacher and student models. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Jilin University (Science Edition) / Jilin Daxue Xuebao (Lixue Ban) is the property of Zhongguo Xue shu qi Kan (Guang Pan Ban) Dian zi Za zhi She and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Remote Sensing is the property of Editorial Office of Journal of Remote Sensing & Science Publishing Co. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Fuzhou University is the property of Journal of Fuzhou University, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The large-scale deployment and rapid development of the new generation mobile communication system underpin the widespread application of a massive and diverse range of Internet of things (IoT) devices.However, the distributed application of IoT devices results to significant disparities in private data and substantial heterogeneity in local processing models, which severely limits the aggregation capability of global intelligent model.Therefore, to tackle the challenges of data heterogeneity, model heterogeneity, and insufficient labeling faced by intelligent modulation recognition in cognitive IoT, an algorithm was proposed for heterogeneous federated bidirectional semi-supervised modulation recognition, which incorporated bidirectional knowledge distillation.In the proposed algorithm, a public pseudo dataset was generated by variational autoencoder in the cloud for supporting uplink global knowledge distillation, and adaptively sharing to the local devices for downlink heterogeneous knowledge distillation, while integrating a semi-supervised algorithm within the distillation process.The simulation results indicate that the proposed algorithm outperforms current federated learning algorithms in terms of effectiveness and applicability in the field of communication signal processing.

Knowledge distillation is an effective method for model compression with access to training data. However, due to privacy, confidentiality, or transmission limitations, people cannot get the support of data. Existing data-free knowledge distillation methods only use biased feature statistics contained in one model and run into pro-blems with low generalizability and diversity in synthetic images and unsatisfactory student model performance. To address these problems, this paper proposes a multi-teacher contrastive knowledge inversion (MTCKI) method that extracts and fuses model-specific knowledge from the available teacher models into a student model to eliminate model bias. Further, this paper improves the diversity of synthesized images using contrastive learning, which encourages the synthetic images to be distinguishable from the previously stored images. Meanwhile, this paper proposes the strategy of contrastive loss based on multi-teacher and student to improve the feature representation ability of student network. Experiments demonstrate that MTCKI not only can generate visually satisfactory images but also outperforms existing state-of-the-art approaches. The resulting synthesized images are much closer to the distribution of the original dataset and can be generated only once to provide comprehensive guidance for various networks rather than a specific one.

In order to solve the problems of low recognition rate and large amount of calculation in lip reading, this paper proposes a lightweight model for lip reading named HSKDLR （homogeneous self-knowledge distillation for lip reading）. Firstly, the S-SE （spatial SE）attention module is designed to pay attention to the spatial features of the lip image, which can construct the i-Ghost Bottleneck （improved Ghost Bottleneck） module to extract the channel features and spatial features of the lip image, thereby improving the accuracy of the lip language recognition model. Secondly, a lip reading model is built based on i-Ghost Bottleneck, which reduces the model computation by optimizing the combination of bottleneck structures to a certain extent. Then, in order to improve the accuracy of the model and reduce time consumption, a model optimization method of the homogeneous self-knowledge distillation （HSKD） is proposed. Finally, this paper employs the HSKD to train the lip reading model and verify its recognition performance. And the experimental results show that HSKDLR has higher recognition accuracy and lower computational complexity than the compared methods. The accuracy of the proposed method on LRW dataset is 87.3%, the floating-point number computation is as low as 2.564 GFLOPs, and the parameter quantity is as low as 3.8723×107. Moreover, HSKD can be applied to most lip reading models to improve recognition accuracy effectively and reduce training time.

In the task of knowledge distillation, a dual-source adaptive knowledge distillation (DSAKD) method is proposed to address the issues of feature information loss during the feature alignment process and the lack of consideration for the differences in samples in the soft label distillation method. The DSAKD method extracts more discriminative knowledge from both the feature layer and soft labels of the teacher network, which enhances the performance of the lightweight student network. An attention-based feature adaptive fusion module was proposed to integrate the intermediate layer features of the teacher network and the student network, and then the feature embedding contrastive distillation strategy was used to optimize the features of the student network. An adaptive temperature distillation strategy was also proposed, which assigned different temperature coefficients to all training samples adaptively based on the prediction confidence of the teacher network. The experimental results show that our proposed method achieves the optimal distillation effect on three benchmark datasets, significantly improving the classification performance of lightweight student networks. Specifically, compared with the best-performing method, the proposed method improves the average top-1 validation accuracy on CIFAR10, CIFAR100, and ImageNet datasets by 0.46%, 0.41%, and 0.59%, respectively.

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Frontiers of Computer Science & Technology is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Dalian Polytechnic University is the property of Journal of Dalian Polytechnic University Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal on Communication / Tongxin Xuebao is the property of Journal on Communications Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of National University of Defense Technology / Guofang Keji Daxue Xuebao is the property of NUDT Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Chinese Journal of Liquid Crystal & Displays is the property of Chinese Journal of Liquid Crystal & Displays and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Fine-grained visual classification(FGVC) is extremely challenging due to the subtle inter-class differences and the large intra-class differences. In order to learn the embedded features of fine-grained images efficiently, this paper attempts to introduce the idea of knowledge distillation to FGVC, and proposed TRS-DeiT, which is equipped with the common advantages of CNN models and Transformer models simultaneously. Besides, a novel target regions selection module was proposed in TRS-DeiT to obtain the most discriminative regions. A contrastive loss function that measures the similarity of images was employed to distinguish the confusable classes in the task. Finally, the effectiveness of the proposed TRS-DeiT model was demonstrated through three datasets, CUB-200-2011, Stanford Cars and Stanford Dogs, which achieved the accuracy of 90.8%, 95.0% and 95.1% respectively. The experimental results shows that the proposed model outperforms the traditional models. Furthermore, the visualization results further illustrate that the attention learned by the proposed model mainly focuses on recognizing objects, thus contributes to fine-grained image classification tasks. [ABSTRACT FROM AUTHOR]

Copyright of Acta Agriculturae Zhejiangensis is the property of Acta Agriculturae Zhejiangensis Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Fuzhou University is the property of Journal of Fuzhou University, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the task of knowledge distillation, a dual-source adaptive knowledge distillation(DSAKD)method is proposed to address the issues of feature information loss during the feature alignment process and the lack of consideration for the differences in samples in the soft label distillation method. The DSAKD method extracts more discriminative knowledge from both the feature layer and soft labels of the teacher network, which enhances the performance of the lightweight student network. An attention-based feature adaptive fusion module was proposed to integrate the intermediate layer features of the teacher network and the student network, and then the feature embedding contrastive distillation strategy was used to optimize the features of the student network. An adaptive temperature distillation strategy was also proposed, which assigned different temperature coefficients to all training samples adaptively based on the prediction confidence of the teacher network. The experimental results show that our proposed method achieves the optimal distillation effect on three benchmark datasets, significantly improving the classification performance of lightweight student networks. Specifically, compared with the best-performing method, the proposed method improves the average top-1 validation accuracy on CIFAR10, CIFAR100, and ImageNet datasets by 0.46%, 0.41%, and 0.59%, respectively. [ABSTRACT FROM AUTHOR]

To address the low efficiency and high false alarm rate in detection of DDoS (Distributed Denial of Service) flood attacks, this paper proposes a DWT (Discrete Wavelet Transform) and AKD (Adaptive Knowledge Distillation) self-encoder neural network based approach to detect DDoS attacks.The approach uses the DWT to extract frequency features, the auto-encoder neural network to encode and classify the features, and the AKD to compress the model in order to achieve efficient detection of DDoS attacks.The results show that the approach has high detection efficiency for proxy server attacks, database vulnerabilities & TCP flood attacks, and UDP flood attacks, with low false alarm rate. [ABSTRACT FROM AUTHOR]

In recent years, the powerful representation and modeling capabilities of graph neural networks for graph data have made them widely used in many fields and made breakthroughs. However, the existing models tend to optimize the graph convolution aggregation strategy and network structure and lack the exploration of the prior knowledge of the graph data itself. In response to the above problems, this paper designed a multi-teacher learning graph neural network based on feature information and structural information enhancement through the method of knowledge distillation, which broke the limitations of existing models for data prior knowledge extraction. Given the rich features and structural information behind the graph data, this paper designed the data enhancement methods of node feature and edge respectively. On this basis, knowledge embedding was performed on the original data and the enhanced data through the multi-teacher learning module, so that the student model could learn more prior knowledge about the data. On the Cora, Citeseer, and PubMed datasets, the node classification accuracy increased by 1%, 1.3% and 1.1%, respectively. Experimental results demonstrate that the information-augmented multi-teacher learning model proposed in this paper can effectively capture prior knowledge. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of East China University of Science & Technology is the property of Journal of East China University of Science & Technology Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Guangdong University of Technology is the property of Journal of Guangdong University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)