Search

Deep learning-based approaches have demonstrated impressive performance in single-image super-resolution (SISR). Efficient super-resolution compromises the reconstructed image’s quality to have fewer parameters and Flops. Ensured efficiency in image reconstruction and improved reconstruction quality of the model are significant challenges. This paper proposes a trio branch module (TBM) based on structural reparameterization. TBM achieves equivalence transformation through structural reparameterization operations, which use a complex network structure in the training phase and convert it to a more lightweight structure in the inference, achieving efficient inference while maintaining accuracy. Based on the TBM, we further design a lightweight version of the enhanced spatial attention mini (ESA-mini) and the residual trio feature block (RTFB). Moreover, the multiple RTFBs are combined to construct the residual trio network (RTFN). Finally, we introduce a localized contrast loss for better applicability to the super-resolution task, which enhances the reconstruction quality of the super-resolution model. Experiments show that the RTFN framework proposed in this paper outperforms other state-of-the-art efficient super-resolution methods in terms of inference speed and reconstruction quality. [ABSTRACT FROM AUTHOR]

Lightweight object detection algorithms play a paramount role in unmanned aerial vehicles (UAVs) remote sensing. However, UAV remote sensing requires target detection algorithms to have higher inference speeds and greater accuracy in detection. At present, most lightweight object detection algorithms have achieved fast inference speed, but their detection precision is not satisfactory. Consequently, this paper presents a refined iteration of the lightweight object detection algorithm to address the above issues. The MobileNetV3 based on the efficient channel attention (ECA) module is used as the backbone network of the model. In addition, the focal and efficient intersection over union (FocalEIoU) is used to improve the regression performance of the algorithm and reduce the false-negative rate. Furthermore, the entire model is pruned using the convolution kernel pruning method. After pruning, model parameters and floating-point operations (FLOPs) on VisDrone and DIOR datasets are reduced to 1.2 M and 1.5 M and 6.2 G and 6.5 G, respectively. The pruned model achieves 49 frames per second (FPS) and 44 FPS inference speeds on Jetson AGX Xavier for VisDrone and DIOR datasets, respectively. To fully exploit the performance of the pruned model, a plug-and-play structural re-parameterization fine-tuning method is proposed. The experimental results show that this fine-tuned method improves mAP@0.5 and mAP@0.5:0.95 by 0.4% on the VisDrone dataset and increases mAP@0.5:0.95 by 0.5% on the DIOR dataset. The proposed algorithm outperforms other mainstream lightweight object detection algorithms (except for FLOPs higher than SSDLite and mAP@0.5 Below YOLOv7 Tiny) in terms of parameters, FLOPs, mAP@0.5, and mAP@0.5:0.95. Furthermore, practical validation tests have also demonstrated that the proposed algorithm significantly reduces instances of missed detection and duplicate detection. [ABSTRACT FROM AUTHOR]

Abstract Deep learning-based approaches have demonstrated impressive performance in single-image super-resolution (SISR). Efficient super-resolution compromises the reconstructed image’s quality to have fewer parameters and Flops. Ensured efficiency in image reconstruction and improved reconstruction quality of the model are significant challenges. This paper proposes a trio branch module (TBM) based on structural reparameterization. TBM achieves equivalence transformation through structural reparameterization operations, which use a complex network structure in the training phase and convert it to a more lightweight structure in the inference, achieving efficient inference while maintaining accuracy. Based on the TBM, we further design a lightweight version of the enhanced spatial attention mini (ESA-mini) and the residual trio feature block (RTFB). Moreover, the multiple RTFBs are combined to construct the residual trio network (RTFN). Finally, we introduce a localized contrast loss for better applicability to the super-resolution task, which enhances the reconstruction quality of the super-resolution model. Experiments show that the RTFN framework proposed in this paper outperforms other state-of-the-art efficient super-resolution methods in terms of inference speed and reconstruction quality.

Recently, super-resolution (SR) has achieved superior performance with the development of deep learning. However, previous methods usually require considerable computational resources with a large model size, which hinders practical applications. To achieve real-time inference and high quality for SR, this paper presents an edge-priority-extraction network, which is constructed with our proposed edge-priority blocks (EPB). The EPB utilizes multiple branches with edge information to further improve the network representation. Moreover, it can be re-parameterized for efficient inference. For more effective utilization of edge information, this paper also proposes the mix-priority filter with edge extraction of horizontal and vertical priorities to improve the network performance. The filters can adaptively extract the edge information with multi-direction derivatives. The experimental results show that our models can use less computational cost to meet the real-time demand and have a better SR performance than the recent real-time SR models. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In this paper, we aim to improve super-resolution (SR) imaging quality on 4K/8K images with a negligible increase in computational cost and alleviate the accuracy drop after quantization. Experiments have discovered two phenomena: (1) the re-parameterization (Rep) technique has no apparent advantages in regions with smooth textures, and (2) the accuracy drop after quantization compared with SR methods based on No-Rep because the structure information of the image will be weakened caused by the multi-branch fusion in Rep technique. Inspired by the above phenomenon, we innovatively combine texture classification and Rep techniques to propose a generic TARepSR framework (consisting of TA-Module and VarRepSR-Module) to adjust the branching of Rep blocks texture-awarely. Specifically, the TA-Module is a lightweight classification network to classify textures in different regions. An existing SR network using texture-aware Rep techniques can be used as the VarRepSR-Module to super-resolute images with higher imaging quality without additional computational costs. Moreover, we propose a TC loss to avoid over-fitting caused by an unbalanced degree of the tendency of classification results to classify different textures better. Experiments show that our TARepSR can not only improve the imaging quality of most existing methods (e.g., FSRCNN, CARN, EDSR, XLSR) on 4K/8K images with negligible increase in computational cost but also improve the accuracy after quantization compared with the state-of-the-art Rep methods. [ABSTRACT FROM AUTHOR]

Focusing on the issues of complex models, high computational cost, and low identification speed of existing coal gangue image identification object detection algorithms, an optimized YOLOv5s lightweight detection model for coal gangue is proposed. Using ShuffleNetV2 as the backbone network, a convolution pooling module is used at the input end instead of the original convolution module. Combining the re-parameterization idea of RepVGG and introducing depthwise separable convolution, a neck feature fusion network is constructed. And using the WIoU function as the loss function. The experimental findings indicate that the improved model maintains the same accuracy, the number of parameters is only 5.1% of the original, the computational effort is reduced to 6.3 % of the original, and the identification speed is improved by 30.9% on GPU and 4 times on CPU. This method significantly reduces model complexity and improves detection speed while maintaining detection accuracy. [ABSTRACT FROM AUTHOR]

Due to the considerable computational demands of physics-based numerical weather prediction, especially when modeling fine-grained spatio-temporal atmospheric phenomena, deep learning methods offer an advantageous approach by leveraging specialized computing devices to accelerate training and significantly reduce computational costs. Consequently, the application of deep learning methods has presented a novel solution in the field of weather forecasting. In this context, we introduce a groundbreaking deep learning-based weather prediction architecture known as Hierarchical U-Net (HU-Net) with re-parameterization techniques. The HU-Net comprises two essential components: a feature extraction module and a U-Net module with re-parameterization techniques. The feature extraction module consists of two branches. First, the global pattern extraction employs adaptive Fourier neural operators and self-attention, well-known for capturing long-term dependencies in the data. Second, the local pattern extraction utilizes convolution operations as fundamental building blocks, highly proficient in modeling local correlations. Moreover, a feature fusion block dynamically combines dual-scale information. The U-Net module adopts RepBlock with re-parameterization techniques as the fundamental building block, enabling efficient and rapid inference. In extensive experiments carried out on the large-scale weather benchmark dataset WeatherBench at a resolution of 1.40625 ∘ , the results demonstrate that our proposed HU-Net outperforms other baseline models in both prediction accuracy and inference time. [ABSTRACT FROM AUTHOR]

Side-scan sonar (SSS) is now a prevalent instrument for large-scale seafloor topography measurements, deployable on an autonomous underwater vehicle (AUV) to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory. However, SSS images often suffer from speckle noise caused by mutual interference between echoes, and limited AUV computational resources further hinder noise suppression. Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge. To address the problem, RepDNet, a novel and effective despeckling convolutional neural network is proposed. RepDNet introduces two re-parameterized blocks: the Pixel Smoothing Block (PSB) and Edge Enhancement Block (EEB), preserving edge information while attenuating speckle noise. During training, PSB and EEB manifest as double-layered multi-branch structures, integrating first-order and second-order derivatives and smoothing functions. During inference, the branches are re-parameterized into a 3 × 3 convolution, enabling efficient inference without sacrificing accuracy. RepDNet comprises three computational operations: 3 × 3 convolution, element-wise summation and Rectified Linear Unit activation. Evaluations on benchmark datasets, a real SSS dataset and Data collected at Lake Mulan aestablish RepDNet as a well-balanced network, meeting the AUV computational constraints in terms of performance and latency.

Abstract Stitched images can offer a broader field of view, but their boundaries can be irregular and unpleasant. To address this issue, current methods for rectangling images start by distorting local grids multiple times to obtain rectangular images with regular boundaries. However, these methods can result in content distortion and missing boundary information. We have developed an image rectangling solution using the reparameterized transformer structure, focusing on single distortion. Additionally, we have designed an assisted learning network to aid in the process of the image rectangling network. To improve the network’s parallel efficiency, we have introduced a local thin-plate spline Transform strategy to achieve efficient local deformation. Ultimately, the proposed method achieves state-of-the-art performance in stitched image rectangling with a low number of parameters while maintaining high content fidelity. The code is available at https://github.com/MelodYanglc/TransRectangling .

Aiming at the two problems that most existing deep network-based fusion approaches have complex network architecture with high computation cost, and they fail to adequately consider the intrinsic characteristics of multi-modal images which results in insufficient information interaction of cross-modality features, a re-parameterized infrared and visible image fusion network based on coordinate attention is proposed. In this network, re-parameterization technique is introduced and combined with residual learning to perform feature extraction for computing efficiency and satisfying fusion quality. Moreover, to improve the interactivity of cross-modality features and fully utilize multi-modal image information, a coordinate attention-based fusion module is devised to yield fused feature. Considering the information loss during extracting process, a fused feature enhance module which leverages the preceding cross-modality features to implement feature compensation is further developed. Extensive experiments demonstrate that the proposed method not only has lower computational cost, but also achieves the improvement of multiple objective evaluation metrics while ensuring good visual effects. [ABSTRACT FROM AUTHOR]

Stitched images can offer a broader field of view, but their boundaries can be irregular and unpleasant. To address this issue, current methods for rectangling images start by distorting local grids multiple times to obtain rectangular images with regular boundaries. However, these methods can result in content distortion and missing boundary information. We have developed an image rectangling solution using the reparameterized transformer structure, focusing on single distortion. Additionally, we have designed an assisted learning network to aid in the process of the image rectangling network. To improve the network's parallel efficiency, we have introduced a local thin-plate spline Transform strategy to achieve efficient local deformation. Ultimately, the proposed method achieves state-of-the-art performance in stitched image rectangling with a low number of parameters while maintaining high content fidelity. The code is available at https://github.com/MelodYanglc/TransRectangling. [ABSTRACT FROM AUTHOR]

Forest is the largest carbon reservoir and carbon absorber on earth. Thus, mapping forest cover change accurately is of great significance to achieving the global carbon neutrality goal. Accurate forest change information could be acquired by deep learning methods using high-resolution remote sensing images. However, deforestation detection based on deep learning on a large-scale region with high-resolution images required huge computational resources. Therefore, there was an urgent need for a fast and accurate deforestation detection model. In this study, we proposed an interesting but effective re-parameterization deforestation detection model, named RepDDNet. Unlike other existing models designed for deforestation detection, the main feature of RepDDNet was its decoupling feature, which means that it allowed the multi-branch structure in the training stages to be converted into a plain structure in the inference stage, thus the computation efficiency can be significantly improved in the inference stage while maintaining the accuracy unchanged. A large-scale experiment was carried out in Ankang city with 2-meter high-resolution remote sensing images (the total area of it was over 20,000 square kilometers), and the result indicated that the model computation efficiency could be improved by nearly 30% compared with the model without re-parameterization. Additionally, compared with other lightweight models, RepDDNet also displayed a trade-off between accuracy and computation efficiency.

In this paper, we aim to improve the imaging quality of super-resolution (SR) without increasing the inference time to address the difficulty of trading off between quality and inference time in many existing methods and design a deployment-friendly, lightweight model for mobile devices. Specifically, we propose a general RepDFSR framework to enhance the textures of SR images while avoiding additional inference time overhead, which can be applied to existing SR networks. It incorporates innovative convolutional block design, loss function design, and the re-parameterizable technique. In RepDFSR, we propose a re-parameterizable texture-enhanced convolution based on differential filters. It extracts texture information more advantageous and efficient in training than regular convolution. Secondly, we propose a DF loss function to compel the model to super-resolve the gradient mappings with high variance, thus reconstructing images with sharper textures. Moreover, we propose a TELNet network for mobile devices based on the RepDFSR framework to validate the effectiveness of RepDFSR and our thoughts on practical applications of SR on mobile devices. The experimental results demonstrate the successful integration of the RepDFSR framework with the existing SR methods. Additionally, the TELNet meets the requirements of mobile devices for hardware and quantization limitations, showcasing superior SR performance compared to classic and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The speaker encoder is an important front-end module that explores discriminative speaker features for many speech applications requiring speaker information. Current speaker encoders aggregate multi-scale features from utterances using multi-branch network architectures. However, naively adding many branches through a fully convolutional operation cannot efficiently improve its capability to capture multi-scale features due to the problem of rapid increase of model parameters and computational complexity. Therefore, in current network architectures, only a few branches corresponding to a limited number of temporal scales are designed for capturing speaker features. To address this problem, this paper proposes an effective temporal multi-scale (TMS) model where multi-scale branches could be efficiently designed in a speaker encoder while negligibly increasing computational costs. The TMS model is based on a time-delay neural network (TDNN), where the network architecture is separated into channel-modeling and temporal multi-branch modeling operators. In the TMS model, adding temporal multi-scale elements in the temporal multi-branch operator only slightly increases the model's parameters, thus saving more of the computational budget to add branches with large temporal scales. After model training, we further develop a systemic re-parameterization method to convert the multi-branch network topology into a single-path-based topology to increase the inference speed.We conducted automatic speaker verification (ASV) experiments under in-domain (VoxCeleb) and out-of-domain (CNCeleb) conditions to investigate the proposed TMS model's performance.Experimental results show that the TMS-method-based model outperformed state-of-the-art ASV models (e.g., ECAPA-TDNN) and improved robustness. Moreover, the proposed model achieved a 29%–46% increase in the inference speed compared to ECAPA-TDNN. [ABSTRACT FROM AUTHOR]

Lane detection is a vital component of intelligent driving systems, offering indispensable functionality to keep the vehicle within its designated lane, thereby reducing the risk of lane departure. However, the complexity of the traffic environment, coupled with the rapid movement of vehicles, creates many challenges for detection tasks. Current lane detection methods suffer from issues such as low feature extraction capability, poor real-time detection, and inadequate robustness. Addressing these issues, this paper proposes a lane detection algorithm that combines an online re-parameterization ResNet with a hybrid attention mechanism. Firstly, we replaced standard convolution with online re-parameterization convolution, simplifying the convolutional operations during the inference phase and subsequently reducing the detection time. In an effort to enhance the performance of the model, a hybrid attention module is incorporated to enhance the ability to focus on elongated targets. Finally, a row anchor lane detection method is introduced to analyze the existence and location of lane lines row by row in the image and output the predicted lane positions. The experimental outcomes illustrate that the model achieves F1 scores of 96.84% and 75.60% on the publicly available TuSimple and CULane lane datasets, respectively. Moreover, the inference speed reaches a notable 304 frames per second (FPS). The overall performance outperforms other detection models and fulfills the requirements of real-time responsiveness and robustness for lane detection tasks. [ABSTRACT FROM AUTHOR]

Apple leaf diseases without timely control will affect fruit quality and yield, intelligent detection of apple leaf diseases was especially important. So this paper mainly focuses on apple leaf disease detection problem, proposes a machine vision algorithm model for fast apple leaf disease detection called LALNet (High-speed apple leaf network). First, an efficient sacked module for apple leaf detection, known as EALD (efficient apple leaf detection stacking module), was designed by utilizing the multi-branch structure and depthseparable modules. In the backbone network of LALNet, (High-speed apple leaf network) four layers of EALD modules were superimposed and an SE (Squeeze-and-Excitation) module was added in the last layer of the model to improve the attention of the model to important features. A structural reparameterization technique was used to combine the outputs of two layers of deeply separable convolutions in branch during the inference phase to improve the model’s operational speed. The results show that in the test set, the detection accuracy of the model was 96.07%. The total precision was 95.79%, the total recall was 96.05%, the total F1 was 96.06%, the model size was 6.61 MB, and the detection speed of a single image was 6.68 ms. Therefore, the model ensures both high detection accuracy and fast execution speed, making it suitable for deployment on embedded devices. It supports precision spraying for the prevention and control of apple leaf disease. [ABSTRACT FROM AUTHOR]

Various types of crop diseases have posed a serious threat to the high-quality development of crops in agricultural production. A timely identification of disease types can greatly contribute to the prevention and control of crop diseases. However, the convolutional neural networks (CNN) recognition of crop leaf diseases cannot fully meet the large-scale production in recent years, due to the numerous parameters, high computational complexity, and poor real-time performance. There is a high demand to improve the accuracy of lightweight CNN models under the relatively complex background of crop leaf images in natural environments. In this study, a lightweight re parameterized leaf diseases identification network (RLDNet) was proposed to identify crop leaf diseases. Firstly, reparameterization was introduced into the MobileNetV2. The inverted residual blocks were reparameterized to construct the reparameterized inverse residual (RIR) block for the high inference speed. A multi-branch architecture was adopted during the training phase. The diversity of the feature space was enriched to improve the representation learning of the network. The reparameterization was used to equivalently convert the multi-branch architecture into a single path structure for the inference after training. The output of the transformed inference model was the same as the original multi-branch. The parameter identity transformation was achieved to improve the inference speed of the model with high recognition accuracy. There was a decrease in the number of output channels and the stacking times of core modules. A shallow and narrow network structure was obtained to enhance the extraction of shallow features, such as the disease colors and textures, indicating the reduced number of model parameters. Secondly, a lightweight ultra lightweight subspace attention module attention mechanism (ULSAM) was added to the last two RIR blocks of the model. Leaf disease characteristics were combined to more effectively distinguish between backgrounds and prospects for better target classification. The interference was reduced in the disease areas from the complex backgrounds. Finally, the DepthShrinker pruning was utilized to learn the importance of the activation function in the model. The redundant parameters were removed to further reduce the space occupation. The lossless lightweight and high accuracy of the model were achieved using knowledge distillation. The recognition accuracy of 99.53% was achieved in the RLDNet on the PlantVillage dataset, whereas 98.49% on the self-built leaf disease dataset with a parameter size of 0.65 M. The inference time for a single-leaf disease image was 2.51 ms. RLDNet shared a performance similar to the Transformer-based models (such as MobileVit-S) and CNN-based models (such as ResNet18) on both datasets, with a significant reduction in parameter, respectively, compared with MobileVitS and ResNet18. The higher recognition accuracy and lighter weight were obtained in the RLDNet, compared with the lightweight models, such as MobileNetV3 and ShuffleNetV2. The improved model can be expected to effectively identify the leaf diseases in the complex backgrounds, with less parameter memory and faster inference speed. The findings can also be applied in the practical production and application of smart agriculture. [ABSTRACT FROM AUTHOR]

Simple Summary: Achieving automatic detection of plant diseases in real agricultural scenarios where low-computing-power platforms are deployed is a significant research topic. As fine-grained agriculture continues to expand and farming methods deepen, traditional manual detection methods demand a high labor intensity. In recent years, the rapid advancement of computer network vision has greatly enhanced the computer-processing capabilities for pattern recognition problems across various industries. Consequently, a deep neural network based on an automatic pruning mechanism is proposed to enable high-accuracy plant disease detection even under limited computational power. Furthermore, an application is developed based on this method to expedite the translation of theoretical results into practical application scenarios. Timely and accurate detection of plant diseases is a crucial research topic. A dynamic-pruning-based method for automatic detection of plant diseases in low-computing situations is proposed. The main contributions of this research work include the following: (1) the collection of datasets for four crops with a total of 12 diseases over a three-year history; (2) the proposition of a re-parameterization method to improve the boosting accuracy of convolutional neural networks; (3) the introduction of a dynamic pruning gate to dynamically control the network structure, enabling operation on hardware platforms with widely varying computational power; (4) the implementation of the theoretical model based on this paper and the development of the associated application. Experimental results demonstrate that the model can run on various computing platforms, including high-performance GPU platforms and low-power mobile terminal platforms, with an inference speed of 58 FPS, outperforming other mainstream models. In terms of model accuracy, subclasses with a low detection accuracy are enhanced through data augmentation and validated by ablation experiments. The model ultimately achieves an accuracy of 0.94. [ABSTRACT FROM AUTHOR]

With the supervised deep learning algorithms, it is prone to generate artifacts when restoring the blurred solar speckle images taken by Yunnan Observatories, and it has a long training time and over-reliance on reference images, this paper proposed a weakly supervised method based on structural reparameterization combined with multi-branch module to reconstruct solar speckle images. First, deblurring model combined single-scale and multi-scale network to design, with constructing multi-branch modules to extract features of different scales, enhance detailed information, and reduce the generation of artifacts; second, each branch structure re-parameterized to make the reuse of structure parameters runs through the entire feature extraction process; after that, the deblurring model embedded in the weakly supervised training, the blurred image assorted firstly, then the degradation model used to learn different levels of degradation. Constituted paired dataset of corresponding levels, and the deblurring model used to inversely degenerate the dataset to reconstruct solar speckle images. Experimental results show that compared with the existing deblurring method, the proposed method has higher model training efficiency and less dependence on reference images, which can meet the high-resolution reconstruction requirements of solar speckle images. [ABSTRACT FROM AUTHOR]

Explainability in medical images analysis plays an important role in the accurate diagnosis and treatment of tumors, which can help medical professionals better understand the images analysis results based on deep models. This paper proposes an explainable brain tumor detection framework that can complete the tasks of segmentation, classification, and explainability. The re-parameterization method is applied to our classification network, and the effect of explainable heatmaps is improved by modifying the network architecture. Our classification model also has the advantage of post-hoc explainability. We used the BraTS-2018 dataset for training and verification. Experimental results show that our simplified framework has excellent performance and high calculation speed. The comparison of results by segmentation and explainable neural networks helps researchers better understand the process of the black box method, increase the trust of the deep model output, and make more accurate judgments in disease identification and diagnosis. [ABSTRACT FROM AUTHOR]

In the field of steel production, the detection of steel surface defects is one of the most important guarantees for the quality of steel production. In the process of defect detection, there are problems regarding the noise of the acquisition background, the scale of defects, and the detection speed. At present, in the face of complex steel surface defects, realizing efficient real-time steel surface defect detection has become a difficult problem. In this paper, we propose a lightweight and efficient real-time defect detection method, LDE-YOLO, based on YOLOv8. First, we propose a lightweight multi-scale feature extraction module, LighterMSMC, which not only achieves a lightweight backbone network, but also effectively guarantees the long range dependence of the features, so as to realize multi-scale feature extraction more efficiently. Secondly, we propose lightweight re-parameterized feature pyramid, DE-FPN, in which the sparse patterns of the overall features and the detailed features of the local features are efficiently captured by the DE-Block, and then efficiently fused by the PAN feature fusion structure. Finally, we propose Efficient Head, which lightens the model by group convolution while its improves the diagonal correlation of the feature maps on some specific datasets, thus enhancing the detection performance. Our proposed LDE-YOLO obtains 80.8 mAP and 75.5 FPS on NEU-DET , 80.5 mAP and 75.5 FPS on GC10-DET. It obtains 2.5 mAP and 4.7 mAP enhancement compared to the baseline model, and the detection speed is also improved by 10.4 FPS, while in terms of the number of floating point operations and parameters of the model reduced by 60.2% and 49.1%, which is sufficient to illustrate its lightweight effectiveness and realize an efficient real-time steel surface defect detection model. • A lightweight multi-scale feature extraction module LighterMSMC was proposed, for the backbone network. Lightweight convolution PConv and multi-scale feature extraction module LMSMC were applied. While the model is more lightweight, so as to achieve multi-scale feature extraction more efficiently and improve the generalization performance of the defect detection model in complex situations. • Propose a lightweight re-parameterized module, DE-Block, and apply it to the feature fusion part of YOLOv8 to obtain the DE-FPN. DE-Block can better extract sparse patterns from overall features, while supplement detailed features. And its structural design can minimize the parameters increase. During the inference process, convolutions in DE-Block can be re-parameterized to reduce inference costs. Thus, a lightweight feature pyramid structure is constructed. • Propose an efficient decoupled detection head, Efficient Head, to lightweight detection heads through group convolution. Applying the aforementioned methods to YOLOv8, this paper proposes LDE-YOLO, which not only has the ability to effectively detect defects in complex situations. Meanwhile, LDE-YOLO significantly reduces the floating-point operations and parameters of the model, thereby improving its defect detection speed. [ABSTRACT FROM AUTHOR]

Vehicle and drone-mounted surveillance equipment face severe computational constraints, posing significant challenges for real-time, accurate crack segmentation. This paper introduces the crack location segmentation transformer (CLST) to address these issues. Images are processed to better resemble patches associated with cracks, enabling precise segmentation while significantly reducing the model's computational load. To handle varying segmentation challenges, a range of models with different computational demands has been designed to suit diverse needs. The most lightweight model can be deployed for real-time use on edge devices. A module in the neck of the pipeline encodes crack coordinate information, and end-to-end training has resulted in state-of-the-art performance across multiple datasets. • Crack location segmentation transformer (CLST) is introduced. • Focusing the field of view on the cracks reduces the amount of computation. • Rep-Crackformer can select modes to cope with different scenarios. • CLocation can encode crack spatial information with fewer parametric quantities. [ABSTRACT FROM AUTHOR]

In multi-view video systems, the decoded texture video and its corresponding depth video are utilized to synthesize virtual views from different perspectives using the depth-image-based rendering (DIBR) technology in 3D-high efficiency video coding (3D-HEVC). However, the distortion of the compressed multi-view video and the disocclusion problem in DIBR can easily cause obvious holes and cracks in the synthesized views, degrading the visual quality of the synthesized views. To address this problem, a novel two-stream re-parameterized refocusing hybrid attention (TRRHA) network is proposed to significantly improve the quality of synthesized views. Firstly, a global multi-scale residual information stream is applied to extract the global context information by using refocusing attention module (RAM), and the RAM can detect the contextual feature and adaptively learn channel and spatial attention feature to selectively focus on different areas. Secondly, a local feature pyramid attention information stream is used to fully capture complex local texture details by using re-parameterized refocusing attention module (RRAM). The RRAM can effectively capture multi-scale texture details with different receptive fields, and adaptively adjust channel and spatial weights to adapt to information transformation at different sizes and levels. Finally, an efficient feature fusion module is proposed to effectively fuse the extracted global and local information streams. Extensive experimental results show that the proposed TRRHA achieves significantly better performance than the state-of-the-art methods. The source code will be available at https://github.com/647-bei/TRRHA. • A two-stream re-parameterization refocusing hybrid attention network (TRRHA) for SVQE. • Design includes global multi-scale residual (GMR) and local feature pyramid attention (LFPA). • Proposed re-parameterized refocusing attention module (RRAM) for local multi-scale texture. • Captures multi-scale features with re-parameterized convolution (RC) branches. • Efficient feature fusion module (EFFM) significantly enhances SVQE performance. [ABSTRACT FROM AUTHOR]

The modeling and simulation of biological tissue is the core part of a virtual surgery system. In this study, the geometric and physical methods related to soft tissue modeling were investigated. Regarding geometric modeling, the problem of repeated inverse calculations of control points in the Bezier method was solved via re-parameterization, which improved the calculation speed. The base surface superposition method based on prior information was proposed to make the deformation model not only have the advantages of the Bezier method but also have the ability to fit local irregular deformation surfaces. Regarding physical modeling, the fitting ability of the particle spring model to the anisotropy of soft tissue was improved by optimizing the topological structure of the particle spring model. Then, the particle spring model had a more extensive nonlinear fitting ability through the dynamic elastic coefficient parameter. Finally, the secondary modeling of the elastic coefficient based on the virtual body spring enabled the model to fit the creep and relaxation characteristics of biological tissue according to the elongation of the virtual body spring. [ABSTRACT FROM AUTHOR]

Forest is the largest carbon reservoir and carbon absorber on earth. Thus, mapping forest cover change accurately is of great significance to achieving the global carbon neutrality goal. Accurate forest change information could be acquired by deep learning methods using high-resolution remote sensing images. However, deforestation detection based on deep learning on a large-scale region with high-resolution images required huge computational resources. Therefore, there was an urgent need for a fast and accurate deforestation detection model. In this study, we proposed an interesting but effective re-parameterization deforestation detection model, named RepDDNet. Unlike other existing models designed for deforestation detection, the main feature of RepDDNet was its decoupling feature, which means that it allowed the multi-branch structure in the training stages to be converted into a plain structure in the inference stage, thus the computation efficiency can be significantly improved in the inference stage while maintaining the accuracy unchanged. A large-scale experiment was carried out in Ankang city with 2-meter high-resolution remote sensing images (the total area of it was over 20,000 square kilometers), and the result indicated that the model computation efficiency could be improved by nearly 30% compared with the model without re-parameterization. Additionally, compared with other lightweight models, RepDDNet also displayed a trade-off between accuracy and computation efficiency. [ABSTRACT FROM AUTHOR]

Copyright of Forest Research is the property of Forest Research Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

With the continuous increase in human–robot integration, battlefield formation is experiencing a revolutionary change. Unmanned aerial vehicles, unmanned surface vessels, combat robots, and other new intelligent weapons and equipment will play an essential role on future battlefields by performing various tasks, including situational reconnaissance, monitoring, attack, and communication relay. Real-time monitoring of maritime scenes is the basis of battle-situation and threat estimation in naval battlegrounds. However, images of maritime scenes are usually accompanied by haze, clouds, and other disturbances, which blur the images and diminish the validity of their contents. This will have a severe adverse impact on many downstream tasks. A novel large kernel encoder–decoder network with multihead pyramids (LKEDN-MHP) is proposed to address some maritime image dehazing-related issues. The LKEDN-MHP adopts a multihead pyramid approach to form a hybrid representation space comprising reflection, shading, and semanteme. Unlike standard convolutional neural networks (CNNs), the LKEDN-MHP uses many kernels with a 7 × 7 or larger scale to extract features. To reduce the computational burden, depthwise (DW) convolution combined with re-parameterization is adopted to form a hybrid model stacked by a large number of different receptive fields, further enhancing the hybrid receptive fields. To restore the natural hazy maritime scenes as much as possible, we apply digital twin technology to build a simulation system in virtual space. The final experimental results based on the evaluation metrics of the peak signal-to-noise ratio, structural similarity index measure, Jaccard index, and Dice coefficient show that our LKEDN-MHP significantly enhances dehazing and real-time performance compared with those of state-of-the-art approaches based on vision transformers (ViTs) and generative adversarial networks (GANs). [ABSTRACT FROM AUTHOR]

Although detectors currently perform well in well-light conditions, their accuracy decreases due to insufficient object information. In addressing this issue, we propose the Re-parameterization Forward Semantic Compensation Network (RFSC-Net). We propose the Reparameterization Residual Efficient Layer Aggregation Networks (RSELAN) for feature extraction, which integrates the concepts of re-parameterization and the Efficient Layer Aggregation Networks (ELAN). While focusing on the fusion of feature maps of the same dimension, it also incorporates upward fusion of lower-level feature maps, enhancing the detailed texture information in higher-level features. Our proposed Forward Semantic Compensation Feature Fusion (FSCFF) network reduces interference from high-level to low-level semantic information, retaining finer details to improve detection accuracy in low-light conditions. Experiments on the low-light ExDark and DarkFace datasets show that RFSC-Net improves mAP by 2% on ExDark and 0.5% on DarkFace over the YOLOv8n baseline, without an increase in parameter counts. Additionally, AP50 is enhanced by 2.1% on ExDark and 1.1% on DarkFace, with a mere 3.7 ms detection latency on ExDark. • Retaining more detailed information improves detection performance in low-light environments. • Fusing lower-level feature maps with higher-level feature can retain more detailed information. • Designed an information compensation detection network that retains more effective detailed information. • The new model outperforms YOLOv8n by 2% in mAP on the low-light ExDark dataset, according to experimental results. [ABSTRACT FROM AUTHOR]

Single image super-resolution is a well-established low-level vision task that aims to reconstruct high-resolution images from low-resolution images. Methods based on Transformer have shown remarkable success and achieved outstanding performance in SISR tasks. While Transformer effectively models global information, it is less effective at capturing high frequencies such as stripes that primarily provide local information. Additionally, it has the potential to further enhance the capture of global information. To tackle this, we propose a novel Large Kernel Hybrid Attention Transformer using re-parameterization. It combines different kernel sizes and different steps re-parameterized convolution layers with Transformer to effectively capture global and local information to learn comprehensive features with low-frequency and high-frequency information. Moreover, in order to solve the problem of using batch normalization layer to introduce artifacts in SISR, we propose a new training strategy which is fusing convolution layer and batch normalization layer after certain training epochs. This strategy can enjoy the acceleration convergence effect of batch normalization layer in training and effectively eliminate the problem of artifacts in the inference stage. For re-parameterization of multiple parallel branch convolution layers, adopting this strategy can further reduce the amount of calculation of training. By coupling these core improvements, our LKHAT achieves state-of-the-art performance for single image super-resolution task. [Display omitted] • The proposed module takes into account both high-frequency and low-frequency information of the image • The proposed module is conducive to generating clear texture details • The training strategy we proposed helps to reduce artifacts [ABSTRACT FROM AUTHOR]

Gastric cancer is the third most common cause of cancer-related death in the world. Human epidermal growth factor receptor 2 (HER2) positive is an important subtype of gastric cancer, which can provide significant diagnostic information for gastric cancer pathologists. However, pathologists usually use a semi-quantitative assessment method to assign HER2 scores for gastric cancer by repeatedly comparing hematoxylin and eosin (H&E) whole slide images (WSIs) with their HER2 immunohistochemical WSIs one by one under the microscope. It is a repetitive, tedious, and highly subjective process. Additionally, WSIs have billions of pixels in an image, which poses computational challenges to Computer-Aided Diagnosis (CAD) systems. This study proposed a deep learning algorithm for HER2 quantification evaluation of gastric cancer. Different from other studies that use convolutional neural networks for extracting feature maps or pre-processing on WSIs, we proposed a novel automatic HER2 scoring framework in this study. In order to accelerate the computational process, we proposed to use the re-parameterization scheme to separate the training model from the deployment model, which significantly speedup the inference process. To the best of our knowledge, this is the first study to provide a deep learning quantification algorithm for HER2 scoring of gastric cancer to assist the pathologist's diagnosis. Experiment results have demonstrated the effectiveness of our proposed method with an accuracy of 0.94 for the HER2 scoring prediction. [ABSTRACT FROM AUTHOR]

Deep saliency models can effectively imitate the attention mechanism of human vision, and they perform considerably better than classical models that rely on handcrafted features. However, deep models also require higher-level information, such as context or emotional content, to further approach human performance. Therefore, this study proposes a multilevel saliency prediction network that aims to use a combination of spatial and channel information to find possible high-level features, further improving the performance of a saliency model. Firstly, we use a VGG style network with an identity block as the primary network architecture. With the help of re-parameterization, we can obtain rich features similar to multiscale networks and effectively reduce computational cost. Secondly, a subnetwork with a channel attention mechanism is designed to find potential saliency regions and possible high-level semantic information in an image. Finally, image spatial features and a channel enhancement vector are combined after quantization to improve the overall performance of the model. Compared with classical models and other deep models, our model exhibits superior overall performance. [ABSTRACT FROM AUTHOR]

Gastric cancer is the third most common cause of cancer-related death in the world. Human epidermal growth factor receptor 2 (HER2) positive is an important subtype of gastric cancer, which can provide significant diagnostic information for gastric cancer pathologists. However, pathologists usually use a semi-quantitative assessment method to assign HER2 scores for gastric cancer by repeatedly comparing hematoxylin and eosin (H&E) whole slide images (WSIs) with their HER2 immunohistochemical WSIs one by one under the microscope. It is a repetitive, tedious, and highly subjective process. Additionally, WSIs have billions of pixels in an image, which poses computational challenges to Computer-Aided Diagnosis (CAD) systems. This study proposed a deep learning algorithm for HER2 quantification evaluation of gastric cancer. Different from other studies that use convolutional neural networks for extracting feature maps or pre-processing on WSIs, we proposed a novel automatic HER2 scoring framework in this study. In order to accelerate the computational process, we proposed to use the re-parameterization scheme to separate the training model from the deployment model, which significantly speedup the inference process. To the best of our knowledge, this is the first study to provide a deep learning quantification algorithm for HER2 scoring of gastric cancer to assist the pathologist's diagnosis. Experiment results have demonstrated the effectiveness of our proposed method with an accuracy of 0.94 for the HER2 scoring prediction.

Timely and accurate detection of plant diseases is a crucial research topic. A dynamic-pruning-based method for automatic detection of plant diseases in low-computing situations is proposed. The main contributions of this research work include the following: (1) the collection of datasets for four crops with a total of 12 diseases over a three-year history; (2) the proposition of a re-parameterization method to improve the boosting accuracy of convolutional neural networks; (3) the introduction of a dynamic pruning gate to dynamically control the network structure, enabling operation on hardware platforms with widely varying computational power; (4) the implementation of the theoretical model based on this paper and the development of the associated application. Experimental results demonstrate that the model can run on various computing platforms, including high-performance GPU platforms and low-power mobile terminal platforms, with an inference speed of 58 FPS, outperforming other mainstream models. In terms of model accuracy, subclasses with a low detection accuracy are enhanced through data augmentation and validated by ablation experiments. The model ultimately achieves an accuracy of 0.94.

Explainability in medical images analysis plays an important role in the accurate diagnosis and treatment of tumors, which can help medical professionals better understand the images analysis results based on deep models. This paper proposes an explainable brain tumor detection framework that can complete the tasks of segmentation, classification, and explainability. The re-parameterization method is applied to our classification network, and the effect of explainable heatmaps is improved by modifying the network architecture. Our classification model also has the advantage of post-hoc explainability. We used the BraTS-2018 dataset for training and verification. Experimental results show that our simplified framework has excellent performance and high calculation speed. The comparison of results by segmentation and explainable neural networks helps researchers better understand the process of the black box method, increase the trust of the deep model output, and make more accurate judgments in disease identification and diagnosis.

The modeling and simulation of biological tissue is the core part of a virtual surgery system. In this study, the geometric and physical methods related to soft tissue modeling were investigated. Regarding geometric modeling, the problem of repeated inverse calculations of control points in the Bezier method was solved via re-parameterization, which improved the calculation speed. The base surface superposition method based on prior information was proposed to make the deformation model not only have the advantages of the Bezier method but also have the ability to fit local irregular deformation surfaces. Regarding physical modeling, the fitting ability of the particle spring model to the anisotropy of soft tissue was improved by optimizing the topological structure of the particle spring model. Then, the particle spring model had a more extensive nonlinear fitting ability through the dynamic elastic coefficient parameter. Finally, the secondary modeling of the elastic coefficient based on the virtual body spring enabled the model to fit the creep and relaxation characteristics of biological tissue according to the elongation of the virtual body spring.

Combining knowledge distillation (KD) and network pruning (NP) shows promise in learning a light network to accelerate object re-identification. However, KD requires an untrained student network to establish more critical connections in early epochs, but NP demands a well-trained student network to avoid destroying critical connections. This presents a dilemma, potentially leading to a collapse of the student network and harming object Re-ID performance. For that, we propose a distillation embedded absorbable pruning (DEAP) method. We design a pruner-convolution-pruner (PCP) unit to resolve the dilemma by loading NP's sparse regularization on extra untrained pruners. Additionally, we propose an asymmetric relation knowledge distillation method. It readily transfers feature representation knowledge and asymmetric pairwise similarity knowledge without using additional adaptation modules. Finally, we apply re-parameterization to absorb pruners of PCP units to simplify student networks. Experiments demonstrate the superiority of DEAP, such as on the VeRi-776 dataset, with ResNet-101 as a teacher, DEAP saves 73.24% of model parameters and 71.98% of floating-point operations without sacrificing accuracy. • We design a distillation embedded absorbable pruning method to readily transfer knowledge a teacher network from to a student network. • We propose an asymmetric relation knowledge distillation method to transfer both feature representation knowledge and pairwise similarity knowledge. • Our method achieves the state-of-the-art performance in terms of both accuracy and speed. [ABSTRACT FROM AUTHOR]

Recent advancements in deep learning have notably advanced the field of image denoising. Yet, blindly increasing the depth or width of convolutional neural networks (CNNs) cannot ameliorate the network effectively, and even leads to training difficulties and sophisticated training tricks. In this paper, a lightweight CNN with heterogeneous kernels (HKCNN) is designed for efficient noise removal. HKCNN comprises four modules: a multiscale block (MB), an attention enhancement block (AEB), an elimination block (EB), and a construct block (CB). Specifically, the MB leverages heterogeneous kernels alongside re-parameterization to capture diverse complementary structure information, bolstering discriminative ability and the denoising robustness of the denoiser. The AEB incorporates an attention mechanism that prioritizes salient features, expediting the training stage and boosting denoising efficacy. The EB and CB are designed to further suppress noise and reconstruct latent clean images. Besides, the HKCNN integrates perceptual loss for both retaining semantic details and improving image perceptual quality, so as to refine the denoising output. Comprehensive qualitative and quantitative evaluations highlight the superior performance of HKCNN over state-of-the-art denoising methods, validating its efficacy in practical image denoising scenarios. [ABSTRACT FROM AUTHOR]

In many speaker verification tasks, the quality of speaker embedding is an important factor in affecting speaker verification systems. Advanced speaker embedding extraction networks aim to capture richer speaker features through the multi-branch network architecture. Recently, speaker verification systems based on transformer encoders have received much attention, and many satisfactory results have been achieved because transformer encoders can efficiently extract the global features of the speaker (e.g., MFA-Conformer). However, the large number of model parameters and computational latency are common problems faced by the above approaches, which make them difficult to apply to resource-constrained edge terminals. To address this issue, this paper proposes an effective, lightweight transformer model (MCA-former) with multi-scale convolutional self-attention (MCA), which can perform multi-scale modeling and channel modeling in the temporal direction of the input with low computational cost. In addition, in the inference phase of the model, we further develop a systematic re-parameterization method to convert the multi-branch network structure into the single-path topology, effectively improving the inference speed. We investigate the performance of the MCA-former for speaker verification under the VoxCeleb1 test set. The results show that the MCA-based transformer model is more advantageous in terms of the number of parameters and inference efficiency. By applying the re-parameterization, the inference speed of the model is increased by about 30%, and the memory consumption is significantly improved. • Designing a lightweight multi-scale convolutional self-attention module. • An efficient transformer encoder for speaker verification. • Using the re-parameterization method improves the model's inference efficiency. [ABSTRACT FROM AUTHOR]