Search

• A digital twin-assisted dual transfer method was proposed with information and model adaptation for bearing fault diagnosis. • DTa-DT aims to simultaneously synthesize information transfer and model transfer for domain transfer error minimization. • DTd-IT method was proposed to information transfer with low distribution difference of twin data adapted. • DAd-MT method, composed of DBTN, was proposed to model transfer with feature distribution difference of domain-model adapted. • Experiments and ablation study verified that the proposed DTa-DT method can be well applied to small sample learning. Rolling bearing fault diagnosis is of great importance to the safety management of mechanical equipment. The scarcity of labelled fault data makes it difficult to adequately perform the training process of intelligent diagnosis models, and this will result in these intelligent models not being effectively and widely used in practice. Although some recent studies have verified that the addition of dynamic model response to the training process will greatly improve the ability of the model with low cost and high efficiency, it is still stuck in poor effect caused by large information distribution difference between dynamic model response and real measured data. Focusing on this issue, a digital twin-assisted dual transfer (DTa-DT) method with information and model adaptation was proposed for rolling bearing fault diagnosis. Different from the traditional digital-analogue driven transfer methods, the proposed DTa-DT aims to simultaneously synthesize data information transfer and feature model transfer together with domain transfer error minimization. In particular, it should be noted that the DTa-DT architecture consists of a dual transfer learning process, including digital twin-driven information transfer (DTd-IT) and digital-analogue-driven model transfer (DAd-MT), where the information is collaborated with the model to improve the integrated transfer diagnosis effect under sampling. On one aspect, with the employment of bearing dynamic model responses, DTd-IT is innovatively designed to establish the transfer of dynamic information and measured information. The information distribution difference between these twin data and real measured data is effetely adjusted with the introduced actual inference components, where the twin data with low information distribution difference can be well fusion generated by the information transfer digital twin (ITDT) model. On the other aspect, considering the truth that there are still small sample cases of real measured data and information distribution differences will affect the quality of the twin data, a digital-analogue driven model transfer (DAd-MT) method is further proposed, where the deep branch transfer network (DBTN) model with improved convolutional neural network (CNN) is used to achieve an accurate fault diagnosis effect with the help of digital twin data. Experiments and wear analysis verified that the proposed DTa-DT can significantly reduce the distribution difference between the dynamic model response and the real measured data, thus achieving low-cost and efficient rolling bearing transfer diagnosis compared to other ten state-of-the-art deep learning models. It can be predicted that the proposed dual transfer architecture provides more opportunities for the practical application of intelligent fault diagnosis under small sample sizes. [ABSTRACT FROM AUTHOR]

• A KiwiDetector Android APP was developed to detect kiwifruits for yield estimation. • Kiwifruits were detected by two lightweight backbones MobileNetV2 and InceptionV3. • An 8-bit quantization was applied to compress model size and improve detection speed. • MobileNetV2 outperformed InceptionV3 in both true detected rate and detection speed. • Quantized MobileNetV2 performed fastest and can be improved by more powerful hardware. Fast and accurate detection of kiwifruit in orchard under natural environment is the primary technology for yield estimation. Deep learning has become a prevalent way of fruit detection and achieved outstanding results. Besides, easy-carry smartphones are getting popular and powerful. In this paper, single shot multibox detector (SSD) with two lightweight backbones MobileNetV2 and InceptionV3 were employed to develop an Android APP named KiwiDetector for field kiwifruit detection. An 8-bit quantization method was used to compress model size and improve detection speed by quantizing weight tensor and activation function data of convolutional neural networks from 32-bit floating point to 8-bit integer. Detection test was performed on 100 selected kiwifruit field images with resolution of 3,968 × 2,976 pixels using the four models on a HUAWEI P20 smartphone. Results showed that MobileNetV2, quantized MobileNetV2, InceptionV3, and quantized InceptionV3 obtained true detected rate (TDR) of 90.8%, 89.7%, 87.6%, and 72.8%, respectively. The TDR of MobileNetV2 and quantized MobileNetV2 was generally consistent and higher than InceptionV3 and quantized InceptionV3. For processing an image on the smartphone, MobileNetV2, quantized MobileNetV2, InceptionV3, and quantized InceptionV3 took about 163 ms, 103 ms, 1085 ms, and 685 ms with model sizes of 17.5 MB, 4.5 MB, 96.1 MB, and 24.1 MB, respectively. Quantized MobileNetV2 reached a significant TDR with the fastest detection speed and the smallest model size. The results indicated that the proposed Android APP is promising for yield estimation. [ABSTRACT FROM AUTHOR]

• A method to avoid detecting branch/wire occluded kiwifruits as pickable targets. • Fruits were divided as four/five classes by robot picking strategy/field occlusion. • YOLOv3 and YOLOv4 were compared by mAP and detection speed. • Fruits classified into more classes was beneficial to achieving higher mAP. • YOLOv4 in the five-classes achieved the highest mAP and met real-time detection. Deep learning has achieved kiwifruit detection with high accuracy and fast speed. However, all the kiwifruits have been labeled and detected as only one class in most researches for robotic fruit picking, where fruits occluded by branches or wires have been detected as pickable targets. End-effectors or robots may be damaged by the branches or wires when they are forced to pick those fruits. Therefore, kiwifruits are labeled, trained, and detected in multi-classes based on their occlusions to avoid detecting fruits occluded by branches or wires as pickable targets. Fruits are classified into four classes and five classes according to robotic picking strategy and field occlusions, respectively. Well-known YOLOv3 and recently released YOLOv4 are employed to do transfer learning for multi-classes kiwifruit detection. Results show that mAP (mean average precision) of fruits in the five-classes is higher than that in the four-classes, while mAP of YOLOv4 is higher than YOLOv3. The mAP of YOLOv4 and YOLOv3 in the five-classes and four-classes are 91.9%, 91.5%, 91.1%, and 89.5%, respectively. The results demonstrate that fruits labeled and trained in more classes can achieve higher mAP. There are significant differences in average detection speed in YOLOv3 and YOLOv4, but no in the four-classes and five-classes. Overall, the highest mAP of 91.9% was achieved by YOLOv4 in the five-classes, which cost 25.5 ms on average to process a 2352 × 1568 image. The results illustrate that multi-classes kiwifruit detection is helpful for avoiding damage to the end-effectors or robots. [ABSTRACT FROM AUTHOR]