Search

Recent advancements in reinforcement learning (RL) have been fueled by large-scale data and deep neural networks, particularly for high-dimensional and complex tasks. Online RL methods like Proximal Policy Optimization (PPO) are effective in dynamic scenarios but require substantial real-time data, posing challenges in resource-constrained or slow simulation environments. Offline RL addresses this by pre-learning policies from large datasets, though its success depends on the quality and diversity of the data. This work proposes a framework that enhances PPO algorithms by incorporating a diffusion model to generate high-quality virtual trajectories for offline datasets. This approach improves exploration and sample efficiency, leading to significant gains in cumulative rewards, convergence speed, and strategy stability in complex tasks. Our contributions are threefold: we explore the potential of diffusion models in RL, particularly for offline datasets, extend the application of online RL to offline environments, and experimentally validate the performance improvements of PPO with diffusion models. These findings provide new insights and methods for applying RL to high-dimensional, complex tasks. Finally, we open-source our code at https://github.com/TianciGao/DiffPPO

We study the frustrated spin-$\frac{1}{2}$ model consisting of a linear chain of triangles with ferro (F)- and antiferromagnetic (AF) interactions connected by ferromagnetic interactions (triangles chain). The ground state phase diagram as a function of the frustration parameter (a ratio of F and AF interactions) and the interaction between triangles consists of the ferromagnetic, two ferrimagnetic and the singlet phases. We study the magnetic properties in these phases and analyze the magnetization curves. We show that there are the magnetization plateau in the singlet phase and the magnetization jumps in some region of this phase. We study the low-thermodynamics and its relation to the specific structure of the excitation spectrum of the triangle chain., Comment: 20 pages, 17 figures

We study the delta-chain with spin-$1$ on basal sites and spin-$\frac{1}{2}$ on apical sites. The Heisenberg interaction between neighbor basal spins is antiferromagnetic (AF) and the interaction between basal and apical spins is ferromagnetic (F). We show that the magnetization curve of this model is the same as that of the spin-$\frac{1}{2}$ kagome-like chain with competing Heisenberg interactions. The ground state phase diagram of the latter as a function of the ratio between the AF and F interaction, $\alpha $, consists of the ferromagnetic, ferrimagnetic and singlet phases. We study the magnetic properties in each ground state phase and analyze the magnetization curves. We show that there are magnetization plateaus and jumps in definite regions of value $\alpha$. We compare the magnetic properties of considered models with those of the spin-$\frac{1}{2}$ delta chain., Comment: 20 pages, 9 figures

It has been shown that during the outburst of accretion activity observed in UX Ori type star RZ Psc in 2013, the accretion rate increased approximately by an order of magnitude. This means that the accretion process at the late stages of the Pre-Main Sequence evolution is very unstable. Using the spectra obtained during this episode we have studied the magnetospheric emission in the H$\alpha$ line. Models of magnetospheric accretion are calculated to obtain the parameters of the magnetosphere from this observation. In present work we have taken into account the influence of the recombination delay effect during gas motion in the stellar magnetosphere. The accounting for this effect and the presence of the magnetospheric absorption in the IR CaII triplet lines and its absence in D Na I resonance lines allowed us to place a lower limit on the temperature in the magnetosphere at $\approx$ 10000 K, which significantly improved precision of our estimate of accretion rate. According to the best fit model the logarithm of accretion rate is $\log\dot{M} = -10.1\pm0.3$ ($\dot{M} \approx 7\times10^{-11}$ M$_\odot$yr$^{-1}$) and the inclination angle of RZ Psc is $43\pm 3^\circ$. It is less than the inclination, typical for the UX Ori stars (about 70$^\circ$), that explains the weak photometric variability of this star. Using the obtained accretion rate and magnetosphere radius we estimate the strength of the dipole component of the magnetic field of RZ Psc $\approx$ 0.1 kGs., Comment: This article has been accepted for publication in Monthly Notices of the Royal Astronomical Society Published by Oxford University Press on behalf of the Royal Astronomical Society

We investigate spin-$\frac{1}{2}$ anisotropic model of a linear chain of triangles with competing ferro- and antiferromagnetic interactions and ferromagneic Heisenberg interactions between triangles. For a certain ratio between interactions the one-magnon excitation band is dispersionless leading to an existence of localized-magnon states which form macroscopically degenerated ground states. The spectrum of excitations has a specific structure depending on the value of the triangle-triangle interaction. Such specific structure determines the low-temperature thermodynamics and, in particular, the temperature dependence of a specific heat. In the limit of strong anisotropy of interactions the spectrum has a multi-scale structure which consists of subsets rank-ordered on small parameter. Each subset is responsible for the appearance of the peak in the temperature dependence of the specific heat., Comment: 23 pages, 13 figures

We consider a class of anisotropic spin-$\frac{1}{2}$ models with competing ferro- and antiferromagnetic interactions on two-dimensional Tasaki and kagome lattices consisting of corner sharing triangles. For certain values of the interactions the ground state is macroscopically degenerated in zero magnetic field. In this case the ground state manifold consists of isolated magnons as well as the bound magnon complexes. The ground state degeneracy is estimated using a special form of exact wave function which admits arrow configuration representation on two-dimensional lattice. The comparison of this estimate with the result for some special exactly solved models shows that the used approach determines the number of the ground states with exponential accuracy. It is shown that the main contribution to the ground state degeneracy and the residual entropy is given by the bound magnon complexes., Comment: 20 pages, 8 figures

From superconductors to atomic nuclei, strongly-interacting many-body systems are ubiquitous in nature. Measuring the microscopic structure of such systems is a formidable challenge, often met by particle knockout scattering experiments. While such measurements are fundamental for mapping the structure of atomic nuclei, their interpretation is often challenged by quantum mechanical initial- and final-state interactions (ISI/FSI) of the incoming and scattered particles. Here we overcome this fundamental limitation by measuring the quasi-free scattering of 48 GeV/c 12C ions from hydrogen. The distribution of single protons is studied by detecting two protons at large angles in coincidence with an intact 11B nucleus. The 11B detection is shown to select the transparent part of the reaction and exclude the otherwise large ISI/FSI that would break the 11B apart. By further detecting residual 10B and 10Be nuclei, we also identified short-range correlated (SRC) nucleon-nucleon pairs, and provide direct experimental evidence for the separation of the pair wave-function from that of the residual many-body nuclear system. All measured reactions are well described by theoretical calculations that do not contain ISI/FSI distortions. Our results thus showcase a new ability to study the short-distance structure of short-lived radioactive atomic nuclei at the forthcoming FAIR and FRIB facilities. These studies will be pivotal for developing a ground-breaking microscopic understanding of the structure and properties of nuclei far from stability and the formation of visible matter in the universe., Comment: Accepted for publication in Nature Physics. 28 pages, 19 figures, and 1 table including main text, Methods, and Supplementary materials

In contrast to strongly frustrated classical systems, their quantum counterparts typically have a non-degenerate ground state. A counterexample is the celebrated Heisenberg sawtooth spin chain with ferromagnetic zigzag bonds $J_1$ and competing antiferromagnetic basal bonds $J_2$. At a quantum phase transition point $|J_2/J_1|=1/2$, this model exhibits a flat one-magnon excitation band leading to a massively degenerate ground-state manifold which results in a large residual entropy. Thus, for the spin-half model, the residual entropy amounts to exactly one half of its maximum value $\lim_{T\to\infty} S(T)/N = \ln2$. In the present paper we study in detail the role of the spin quantum number $s$ and the magnetic field $H$ in the parameter region around the transition (flat-band) point. For that we use full exact diagonalization up to $N=20$ lattice sites and the finite-temperature Lanczos method up to $N=36$ sites to calculate the density of states as well as the temperature dependence of the specific heat, the entropy and the susceptibility. The study of chain lengths up to $N=36$ allows a careful finite-size analysis. At the flat-band point we find extremely small finite-size effects for spin $s=1/2$, i.e., the numerical data virtually correspond to the thermodynamic limit. In all other cases the finite-size effects are still small and become visible at very low temperatures. In a sizeable parameter region around the flat-band point the former massively degenerate ground-state manifold acts as a large manifold of low-lying excitations leading to extraordinary thermodynamic properties at the transition point as well as in its vicinity such as an additional low-temperature maximum in the specific heat. Moreover, there is a very strong influence of the magnetic field on the low-temperature thermodynamics including an enhanced magnetocaloric effect., Comment: 13 pages, 10 figures, 1 table

We consider the strongly anisotropic spin-1/2 $XXZ$ model on the sawtooth-chain lattice with ferromagnetic longitudinal interaction $J^{zz}=\Delta J$ and aniferromagnetic transversal interaction $J^{xx}=J^{yy}=J>0$. At $\Delta=-1/2$ the lowest one-magnon excitation band is dispersionless (flat) leading to a massively degenerate set of ground states. Interestingly, this model admits a three-coloring representation of the ground-state manifold [H.~J.~Changlani et at., Phys. Rev. Lett. {\bf 120}, 117202 (2018)]. We characterize this ground-state manifold and elaborate the low-temperature thermodynamics of the system. We illustrate the manifestation of the flat-band physics of the anisotropic model by comparison with two isotropic flat-band Heisenberg sawtooth chains. Our analytical consideration is complemented by exact diagonalization and finite-temperature Lanczos method calculations., Comment: 12 pages,8 figures

We study the thermodynamics of the delta-chain with competing ferro- and antiferromagnetic interactions in an external magnetic field which generalizes the field-free case studied previously. This model plays an important role for the recently synthesized compound Fe$_{10}$Gd$_{10}$ which is nearly quantum critical. The classical version of the model is solved exactly and explicit analytical results for the low-temperature thermodynamics are obtained. The spin-$s$ quantum model is studied using exact diagonalization and finite-temperature Lanzos techniques. Particular attention is focused on the magnetization and the susceptibility. The magnetization of the classical model in the ferromagnetic part of the phase diagram defines the universal scaling function which is valid for the quantum model. The dependence of the susceptibility on the spin quantum number $s$ at the critical point between the ferro- and ferrimagnetic phases is studied and the relation to Fe$_{10}$Gd$_{10}$ is discussed., Comment: 30 pages, 14 figures

Motivated by a novel cyclic compound $Fe_{10}Gd_{10}$ with record ground state spin in which the arrangement of magnetic ions with $s=\frac{5}{2}$ and $s=\frac{7}{2}$ corresponds to a saw-tooth chain we investigate the thermodynamics of the delta-chain with competing ferro- and antiferromagnetic interactions. We study both classical and quantum versions of the model. The classical model is exactly solved and quantum effects are studied using full diagonalization and a finite temperature Lanczos technique for finite delta-chains as well as modified spin wave theory. It is shown that the main features of the magnetic susceptibility of the quantum spin delta chain are correctly described by the classical spin model, while quantum effects significantly change the low-temperature behavior of the specific heat. The relation of the obtained results to the $Fe_{10}Gd_{10}$ system is discussed., Comment: 28 pages, 11 figures

The spin-$\frac{1}{2}$ delta-chain (sawtooth chain) with antiferromagnetic Heisenberg basal chain and Ising apical-basal interactions is studied. The basal-apical interactions involve the bond alternation. The limiting cases of the model include the symmetrical delta-chain and the antiferromagnetic chain in the staggered magnetic field. We study ground state properties of the model by the exact diagonalization and density matrix renormalization group methods. The ground state phase diagram as a function of the bond alternation consists of magnetic and various non-magnetic phases. All phases excluding the ferrimagnetic phase are gapped and an origin of the gaps is cleared., Comment: 14 pages, 6 figures

We consider a spin-$\frac{1}{2}$ kagome-like chain with competing ferro- and antiferromagnetic anisotropic exchange interactions. The ground state phase diagram of this model consists of the ferromagnetic and ferrimagnetic phases. We study the ground state and the low-temperature properties on the phase boundary between these phases. The ground state on this phase boundary is macroscopically degenerate and consists of localized magnon states. We calculate the ground state degeneracy and corresponding residual entropy. The spontaneous magnetization has a jump on the phase boundary confirming the first-order type of the phase transition. In the limit of a strong anisotropy the spectrum of the low-energy excitations has multi-scale structure governing the peculiar features of the specific heat behavior., Comment: 19 pages, 7 figures

We consider analytically and numerically an anisotropic spin-$\frac{1}{2}$ delta-chain (sawtooth chain) in which exchange interactions between apical and basal spins are ferromagnetic and those between basal spins are antiferromagnetic. In the limit of strong anisotropy of exchange interactions this model can be considered as the Ising delta chain with macroscopic degenerate ground state perturbed by transverse quantum fluctuations. These perturbations lift the ground state degeneracy and the model reduces to the basal XXZ spin chain in the magnetic field induced by static apical spins. We show that the ground state of such model is ferrimagnetic. The excitations of the model are formed by ferrimagnetic domains separated by domain walls with a finite energy. At low temperatures the system is effectively divided into two independent subsystems, the apical subsystem described by the Ising spin-$\frac{1}{2}$ chain and the basal subsystem described by the XXZ chain with infinite $zz$ interactions., Comment: 18 pages, 9 figures

We consider analytically and numerically an anisotropic spin-$\frac{1}{2}$ delta-chain (sawtooth chain) with nearest-neighbor ferromagnetic and next-nearest-neighbor antiferromagnetic interactions. For certain values of the interactions a lowest one-particle band becomes flat and there is a class of localized-magnon eigenstates which form a ground state with a macroscopic degeneracy. In this case the model depends on a single parameter which can be chosen as the anisotropy of the exchange interactions. When this parameter changes from zero to infinity the model interpolates between the one-dimensional isotropic ferromagnet and the frustrated Ising model on the delta-chain. It is shown that the low-temperature thermodynamic properties in these limiting cases are governed by the specific structure of the excitation spectrum. In particular, the specific heat has one or infinite number of low-temperature maxima for the small or the large anisotropy parameter, correspondingly. Numerical calculations of finite chains demonstrate that this behavior is generic for definite values of the anisotropy parameter.

We investigate the spin-1/2 Heisenberg model on the delta chain (sawtooth chain) with ferromagnetic nearest-neighbor and antiferromagnetic next-neighbor interactions. For a special ratio between these interactions there is a class of exact ground states formed by localized magnons and the ground state is macroscopically degenerate with a large residual entropy per spin $s_0=\frac{1}{2}\ln 2$. An important feature of this model is a sharp decrease of the gaps for excited states with an increase of the number of magnons. These excitations give an essential contribution to the low-temperature thermodynamics. The behavior of the considered model is compared with that of the delta chain with both antiferromagnetic interactions., Comment: 24 pages, 10 figures

The mobility of polymer chains in perfect polyethylene (PE) crystal was calculated as a function of temperature and chain length through Molecular dynamics (MD) in united atom approximation. The results demonstrate that the chain mobility drastically increases in the vicinity of the phase transition from the orthorhombic to quasi-hexagonal phase. In the quasi-hexagonal phase, the chain mobility is almost independent on temperature and inversely proportional to the chain length., Comment: 5 pages, 6 figures

Under typical PEM fuel cell operating conditions, part of membrane electrode assembly is subjected to humidity cycling due to variation of inlet gas RH and/or flow rate. Cyclic membrane hydration/dehydration would cause cyclic swelling/shrinking of the unconstrained membrane. In a constrained membrane, it causes cyclic stress resulting in mechanical failure in the area adjacent to the gas inlet. A mathematical modeling framework for prediction of the lifetime of a PEM FC membrane subjected to hydration cycling is developed in this paper. The model predicts membrane lifetime as a function of RH cycling amplitude and membrane mechanical properties. The modeling framework consists of three model components: a fuel cell RH distribution model, a hydration/dehydration induced stress model that predicts stress distribution in the membrane, and a damage accrual model that predicts membrane life-time. Short descriptions of the model components along with overall framework are presented in the paper. The model was used for lifetime prediction of a GORE-SELECT membrane.