Search

Deep learning-based approaches have demonstrated impressive performance in single-image super-resolution (SISR). Efficient super-resolution compromises the reconstructed image’s quality to have fewer parameters and Flops. Ensured efficiency in image reconstruction and improved reconstruction quality of the model are significant challenges. This paper proposes a trio branch module (TBM) based on structural reparameterization. TBM achieves equivalence transformation through structural reparameterization operations, which use a complex network structure in the training phase and convert it to a more lightweight structure in the inference, achieving efficient inference while maintaining accuracy. Based on the TBM, we further design a lightweight version of the enhanced spatial attention mini (ESA-mini) and the residual trio feature block (RTFB). Moreover, the multiple RTFBs are combined to construct the residual trio network (RTFN). Finally, we introduce a localized contrast loss for better applicability to the super-resolution task, which enhances the reconstruction quality of the super-resolution model. Experiments show that the RTFN framework proposed in this paper outperforms other state-of-the-art efficient super-resolution methods in terms of inference speed and reconstruction quality. [ABSTRACT FROM AUTHOR]

Recently, super-resolution (SR) has achieved superior performance with the development of deep learning. However, previous methods usually require considerable computational resources with a large model size, which hinders practical applications. To achieve real-time inference and high quality for SR, this paper presents an edge-priority-extraction network, which is constructed with our proposed edge-priority blocks (EPB). The EPB utilizes multiple branches with edge information to further improve the network representation. Moreover, it can be re-parameterized for efficient inference. For more effective utilization of edge information, this paper also proposes the mix-priority filter with edge extraction of horizontal and vertical priorities to improve the network performance. The filters can adaptively extract the edge information with multi-direction derivatives. The experimental results show that our models can use less computational cost to meet the real-time demand and have a better SR performance than the recent real-time SR models. [ABSTRACT FROM AUTHOR]

In this paper, we aim to improve super-resolution (SR) imaging quality on 4K/8K images with a negligible increase in computational cost and alleviate the accuracy drop after quantization. Experiments have discovered two phenomena: (1) the re-parameterization (Rep) technique has no apparent advantages in regions with smooth textures, and (2) the accuracy drop after quantization compared with SR methods based on No-Rep because the structure information of the image will be weakened caused by the multi-branch fusion in Rep technique. Inspired by the above phenomenon, we innovatively combine texture classification and Rep techniques to propose a generic TARepSR framework (consisting of TA-Module and VarRepSR-Module) to adjust the branching of Rep blocks texture-awarely. Specifically, the TA-Module is a lightweight classification network to classify textures in different regions. An existing SR network using texture-aware Rep techniques can be used as the VarRepSR-Module to super-resolute images with higher imaging quality without additional computational costs. Moreover, we propose a TC loss to avoid over-fitting caused by an unbalanced degree of the tendency of classification results to classify different textures better. Experiments show that our TARepSR can not only improve the imaging quality of most existing methods (e.g., FSRCNN, CARN, EDSR, XLSR) on 4K/8K images with negligible increase in computational cost but also improve the accuracy after quantization compared with the state-of-the-art Rep methods. [ABSTRACT FROM AUTHOR]

Focusing on the issues of complex models, high computational cost, and low identification speed of existing coal gangue image identification object detection algorithms, an optimized YOLOv5s lightweight detection model for coal gangue is proposed. Using ShuffleNetV2 as the backbone network, a convolution pooling module is used at the input end instead of the original convolution module. Combining the re-parameterization idea of RepVGG and introducing depthwise separable convolution, a neck feature fusion network is constructed. And using the WIoU function as the loss function. The experimental findings indicate that the improved model maintains the same accuracy, the number of parameters is only 5.1% of the original, the computational effort is reduced to 6.3 % of the original, and the identification speed is improved by 30.9% on GPU and 4 times on CPU. This method significantly reduces model complexity and improves detection speed while maintaining detection accuracy. [ABSTRACT FROM AUTHOR]

Due to the considerable computational demands of physics-based numerical weather prediction, especially when modeling fine-grained spatio-temporal atmospheric phenomena, deep learning methods offer an advantageous approach by leveraging specialized computing devices to accelerate training and significantly reduce computational costs. Consequently, the application of deep learning methods has presented a novel solution in the field of weather forecasting. In this context, we introduce a groundbreaking deep learning-based weather prediction architecture known as Hierarchical U-Net (HU-Net) with re-parameterization techniques. The HU-Net comprises two essential components: a feature extraction module and a U-Net module with re-parameterization techniques. The feature extraction module consists of two branches. First, the global pattern extraction employs adaptive Fourier neural operators and self-attention, well-known for capturing long-term dependencies in the data. Second, the local pattern extraction utilizes convolution operations as fundamental building blocks, highly proficient in modeling local correlations. Moreover, a feature fusion block dynamically combines dual-scale information. The U-Net module adopts RepBlock with re-parameterization techniques as the fundamental building block, enabling efficient and rapid inference. In extensive experiments carried out on the large-scale weather benchmark dataset WeatherBench at a resolution of 1.40625 ∘ , the results demonstrate that our proposed HU-Net outperforms other baseline models in both prediction accuracy and inference time. [ABSTRACT FROM AUTHOR]

Stitched images can offer a broader field of view, but their boundaries can be irregular and unpleasant. To address this issue, current methods for rectangling images start by distorting local grids multiple times to obtain rectangular images with regular boundaries. However, these methods can result in content distortion and missing boundary information. We have developed an image rectangling solution using the reparameterized transformer structure, focusing on single distortion. Additionally, we have designed an assisted learning network to aid in the process of the image rectangling network. To improve the network's parallel efficiency, we have introduced a local thin-plate spline Transform strategy to achieve efficient local deformation. Ultimately, the proposed method achieves state-of-the-art performance in stitched image rectangling with a low number of parameters while maintaining high content fidelity. The code is available at https://github.com/MelodYanglc/TransRectangling. [ABSTRACT FROM AUTHOR]

In this paper, we aim to improve the imaging quality of super-resolution (SR) without increasing the inference time to address the difficulty of trading off between quality and inference time in many existing methods and design a deployment-friendly, lightweight model for mobile devices. Specifically, we propose a general RepDFSR framework to enhance the textures of SR images while avoiding additional inference time overhead, which can be applied to existing SR networks. It incorporates innovative convolutional block design, loss function design, and the re-parameterizable technique. In RepDFSR, we propose a re-parameterizable texture-enhanced convolution based on differential filters. It extracts texture information more advantageous and efficient in training than regular convolution. Secondly, we propose a DF loss function to compel the model to super-resolve the gradient mappings with high variance, thus reconstructing images with sharper textures. Moreover, we propose a TELNet network for mobile devices based on the RepDFSR framework to validate the effectiveness of RepDFSR and our thoughts on practical applications of SR on mobile devices. The experimental results demonstrate the successful integration of the RepDFSR framework with the existing SR methods. Additionally, the TELNet meets the requirements of mobile devices for hardware and quantization limitations, showcasing superior SR performance compared to classic and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The speaker encoder is an important front-end module that explores discriminative speaker features for many speech applications requiring speaker information. Current speaker encoders aggregate multi-scale features from utterances using multi-branch network architectures. However, naively adding many branches through a fully convolutional operation cannot efficiently improve its capability to capture multi-scale features due to the problem of rapid increase of model parameters and computational complexity. Therefore, in current network architectures, only a few branches corresponding to a limited number of temporal scales are designed for capturing speaker features. To address this problem, this paper proposes an effective temporal multi-scale (TMS) model where multi-scale branches could be efficiently designed in a speaker encoder while negligibly increasing computational costs. The TMS model is based on a time-delay neural network (TDNN), where the network architecture is separated into channel-modeling and temporal multi-branch modeling operators. In the TMS model, adding temporal multi-scale elements in the temporal multi-branch operator only slightly increases the model's parameters, thus saving more of the computational budget to add branches with large temporal scales. After model training, we further develop a systemic re-parameterization method to convert the multi-branch network topology into a single-path-based topology to increase the inference speed.We conducted automatic speaker verification (ASV) experiments under in-domain (VoxCeleb) and out-of-domain (CNCeleb) conditions to investigate the proposed TMS model's performance.Experimental results show that the TMS-method-based model outperformed state-of-the-art ASV models (e.g., ECAPA-TDNN) and improved robustness. Moreover, the proposed model achieved a 29%–46% increase in the inference speed compared to ECAPA-TDNN. [ABSTRACT FROM AUTHOR]

A joint-processing model for multi-GNSS (GPS, GLONASS, BDS and GALILEO) precise point positioning (PPP) is proposed, in which raw code and phase observations are used. In the proposed model, inter-system biases (ISBs) and GLONASS code inter-frequency biases (IFBs) are carefully considered, among which GLONASS code IFBs are modeled as a linear function of frequency numbers. To get the full rank function model, the unknowns are re-parameterized and the estimable slant ionospheric delays and ISBs/IFBs are derived and estimated simultaneously. One month of data in April, 2015 from 32 stations of the International GNSS Service (IGS) Multi-GNSS Experiment (MGEX) tracking network have been used to validate the proposed model. Preliminary results show that RMS values of the positioning errors (with respect to external double-difference solutions) for static/kinematic solutions (four systems) are 6.2 mm/2.1 cm (north), 6.0 mm/2.2 cm (east) and 9.3 mm/4.9 cm (up). One-day stabilities of the estimated ISBs described by STD values are 0.36 and 0.38 ns, for GLONASS and BDS, respectively. Significant ISB jumps are identified between adjacent days for all stations, which are caused by the different satellite clock datums in different days and for different systems. Unlike ISBs, the estimated GLONASS code IFBs are quite stable for all stations, with an average STD of 0.04 ns over a month. Single-difference experiment of short baseline shows that PPP ionospheric delays are more precise than traditional leveling ionospheric delays. [ABSTRACT FROM AUTHOR]