Search

To mitigate the extrapolation error arising from offline reinforcement learning (RL) paradigm, existing methods typically make learned Q-functions over-conservative or enforce global policy constraints. In this article, we propose a dual behavior regularized offline deterministic Actor-Critic (DBRAC) by simultaneously performing behavior regularization on the coupling-iterative policy evaluation (PE) and policy improvement (PI) in the policy iteration process. In the PE phase, the difference between the Q-function and behavior value is first taken as the anti-exploration behavior value regularization term to drive the Q-function toward its true Q-value, which significantly reduces the conservatism of learned Q-function. In the PI phase, the estimated action variances of behavior policy in different states are then utilized for designing the weight and threshold of mild-local behavior cloning regularization term, which standardizes the local improvement potential of learned policy. Experiments on the well-known datasets for deep data-driven RL (D4RL) demonstrate that the DBRAC can quickly learn more competitive task-solving policies in various offline situations with different data qualities, significantly outperforming state-of-the-art offline RL baselines.

The pretrained vision-language models (VLMs) have achieved outstanding performance in various visual tasks, primarily due to the knowledge they have acquired from massive image-text pairs. This enables VLMs to generalize to a wide range of downstream tasks. This article presents the first attempt to adapt VLMs for the joint classification task of hyperspectral image (HSI) and LiDAR data, aiming to leverage the well-learned VLMs to extract more generalizable features from diverse remote sensing image sources. Initially, using a patch encoder (PE), low-dimensional patches of HSI and LiDAR data are transformed into high-dimensional latent feature representations, meeting the dimensional requirements of VLMs for visual input data. Unlike traditional classifiers that rely on discrete class labels, VLM-based classification methods depend on continuous vectors, which can be derived from textual templates with class names, i.e., prompts. The classification performance of VLM-based methods heavily relies on these prompts, but prompt engineering not only demands extensive expert knowledge but also is extremely time-consuming. To address this, prompt tuning (PT) methods are introduced to enhance the generalizability of VLMs by adding spectral-based prompts to the vision encoder and incorporating randomly initialized, learnable text prompts (TPs) into the text encoder. Finally, through a novel class-discriminative loss function, the distance between text features of different classes is increased, thereby enhancing the model’s discriminative ability. Experimental results on the Houston 2013, Trento, and MUUFL datasets demonstrate that the proposed method can achieve competitive classification accuracy with a limited number of labeled pixels.

In industrial scenarios, the source-domain (SD) data typically encompasses condition monitoring (CM) data from all machines within a workshop or factory setting, while the target-domain (TD) data may only include CM data from one or a small number of machines. The intelligent diagnostic method based on partial domain adaptation (PDA) represents a powerful tool for aligning features between SD and TD data within partial categories. However, existing PDA techniques can only align either the marginal or conditional distributions (CDs) between SD and TD data within the shared label space, but not both simultaneously. To overcome this limitation, our study introduces a dual structural consistent PDA network. This network leverages the vision transformer (ViT) as its foundation, ensuring effective extraction of distinguishable features from both SD and TD data. A weight balance mechanism is integrated into the partial adversarial training (PAT) process, facilitating marginal distribution alignment (MDA) between SD and TD data within the shared label space. Additionally, a knowledge distillation (KD)-based approach is employed for CD alignment (CDA) across the two structural consistent networks (SCNs), ensuring consistency in predictions for TD data. The effectiveness of our proposed method is demonstrated through its application on two sets of experimental faulty data, confirming its ability to provide a feature distribution that is not affected by domain changes but is discriminative for different classes when dealing with PDA tasks.

In this article, a novel coupled policy improvement mechanism is developed for improving policy iteration (PI) algorithms. In contrast to the common PI, the developed dual parallel policy iteration (DPPI) with coupled policy improvement mechanism consists of two parallel PIs. At each PI step, the performances of the two parallel policies are evaluated and the better one is defined as the dominant policy. Then, the dominant policy is used to guide the parallel policy improvement in a soft manner by constraining the Kullback–Liebler (KL) divergence between the dominant policy and the policy to be updated. It is proven that the convergence of DPPI can be guaranteed under the designed coupled policy improvement mechanism. Moreover, it is clearly shown that under certain conditions, the $Q$ -functions of the two new policies obtained in each parallel policy improvement are larger than those of all the previous dominant policies, which is conductive to accelerate the PI process and improve the policy learning efficiency to some extent. Furthermore, by combining DPPI with the twin delay deep deterministic (TD3) policy gradient, we propose a reinforcement learning (RL) algorithm: parallel TD3 (PTD3). Experimental results on continuous-action control tasks in the MuJoCo and OpenAI Gym platforms show that the proposed PTD3 outperforms the state-of-the-art RL algorithms.