Search

In recent years, significant progress has been made in multi-objective reinforcement learning (RL) research, which aims to balance multiple objectives by incorporating preferences for each objective. In most existing studies, specific preferences must be provided during deployment to indicate the desired policies explicitly. However, designing these preferences depends heavily on human prior knowledge, which is typically obtained through extensive observation of high-performing demonstrations with expected behaviors. In this work, we propose a simple yet effective offline adaptation framework for multi-objective RL problems without assuming handcrafted target preferences, but only given several demonstrations to implicitly indicate the preferences of expected policies. Additionally, we demonstrate that our framework can naturally be extended to meet constraints on safety-critical objectives by utilizing safe demonstrations, even when the safety thresholds are unknown. Empirical results on offline multi-objective and safe tasks demonstrate the capability of our framework to infer policies that align with real preferences while meeting the constraints implied by the provided demonstrations.

In this paper, we aim to utilize only offline trajectory data to train a policy for multi-objective RL. We extend the offline policy-regularized method, a widely-adopted approach for single-objective offline RL problems, into the multi-objective setting in order to achieve the above goal. However, such methods face a new challenge in offline MORL settings, namely the preference-inconsistent demonstration problem. We propose two solutions to this problem: 1) filtering out preference-inconsistent demonstrations via approximating behavior preferences, and 2) adopting regularization techniques with high policy expressiveness. Moreover, we integrate the preference-conditioned scalarized update method into policy-regularized offline RL, in order to simultaneously learn a set of policies using a single policy network, thus reducing the computational cost induced by the training of a large number of individual policies for various preferences. Finally, we introduce Regularization Weight Adaptation to dynamically determine appropriate regularization weights for arbitrary target preferences during deployment. Empirical results on various multi-objective datasets demonstrate the capability of our approach in solving offline MORL problems.

The thermophotovoltaic cells which convert the low temperature radiation into electricity are of significance due to their potential applications in many fields. In this work, Bi2Te3/Si thermophotovoltaic cells which work under the radiation from the blackbody with the temperature of 300 K-480 K are presented. The experimental results show that the cells can output electricity even under the radiation temperature of 300 K. The band structure of Bi2Te3/Si heterojunctions and the defects in Bi2Te3 thin films lower the conversion efficiency of the cells. It is also demonstrated that the resistivity of Si and the thickness of Bi2Te3 thin films have important effects on Bi2Te3/Si thermophotovoltaic cells. Although the cells' output power is small, this work provides a possible way to utilize the low temperature radiation., Comment: 6 pages, 7 figures

For TOF positron emission tomography (TOF PET) detectors, time-to-digital converters (TDCs) are essential to resolve the coincidence time of the photon pairs. Recently, an efficient TDC structure called ring-oscillator-based (RO-based) Vernier TDC using carry chains was reported by our team. The method is very promising due to its low linearity error and low resource cost. However, the implementation complexity is rather high especially when moving to multi-channels TDC designs, since this method calls for a manual intervention to the initial fitting results of the compilation software. In this paper, we elaborate the key points toward implementing high performance multi-channels TDCs of this kind while keeping the least implementation complexity. Furthermore, we propose an efficient fine time interpolator construction method called the period difference recording which only needs at most 31 adjustment trials to obtain a targeted TDC resolution. To validate the techniques proposed in this paper, we built a 32-channels TDC on a Stratix III FPGA chip and fully evaluated its performance. Code density tests show that the obtained resolution results lie in the range of (23 ps ~ 37 ps), the differential nonlinearity (DNL) results lie in the range of (-0.4 LSB ~ 0.4 LSB) and the integral nonlinearity (INL) results lie in the range of (-0.7 LSB ~ 0.7 LSB) for each of the 32 TDC channels. This paper greatly eases the designing difficulty of the carry chain RO-based TDCs and can significantly propel their development in practical use., Comment: 9 pages,9 figures, 4 tables; published on IEEE Transactions on Radiation and Plasma Medical Sciences