Search

The Circular Electron-Positron Collider (CEPC) can also work as a powerful and excellent synchrotron light source, which can generate high-quality synchrotron radiation. This synchrotron radiation has potential advantages in the medical field, with a broad spectrum, with energies ranging from visible light to x-rays used in conventional radiotherapy, up to several MeV. FLASH radiotherapy is one of the most advanced radiotherapy modalities. It is a radiotherapy method that uses ultra-high dose rate irradiation to achieve the treatment dose in an instant; the ultra-high dose rate used is generally greater than 40 Gy/s, and this type of radiotherapy can protect normal tissues well. In this paper, the treatment effect of CEPC synchrotron radiation for FLASH radiotherapy was evaluated by simulation. First, Geant4 simulation was used to build a synchrotron radiation radiotherapy beamline station, and then the dose rate that CEPC can produce was calculated. Then, a physicochemical model of radiotherapy response kinetics was established, and a large number of radiotherapy experimental data were comprehensively used to fit and determine the functional relationship between the treatment effect, dose rate and dose. Finally, the macroscopic treatment effect of FLASH radiotherapy was predicted using CEPC synchrotron radiation light through the dose rate and the above-mentioned functional relationship. The results show that CEPC synchrotron radiation beam is one of the best beams for FLASH radiotherapy., Comment: 18 pages, 7 figures

Large-scale generative models, such as text-to-image diffusion models, have garnered widespread attention across diverse domains due to their creative and high-fidelity image generation. Nonetheless, existing large-scale diffusion models are confined to generating images of up to 1K resolution, which is far from meeting the demands of contemporary commercial applications. Directly sampling higher-resolution images often yields results marred by artifacts such as object repetition and distorted shapes. Addressing the aforementioned issues typically necessitates training or fine-tuning models on higher-resolution datasets. However, this poses a formidable challenge due to the difficulty in collecting large-scale high-resolution images and substantial computational resources. While several preceding works have proposed alternatives to bypass the cumbersome training process, they often fail to produce convincing results. In this work, we probe the generative ability of diffusion models at higher resolution beyond their original capability and propose a novel progressive approach that fully utilizes generated low-resolution images to guide the generation of higher-resolution images. Our method obviates the need for additional training or fine-tuning which significantly lowers the burden of computational costs. Extensive experiments and results validate the efficiency and efficacy of our method. Project page: https://yhyun225.github.io/DiffuseHigh/, Comment: Project page: https://yhyun225.github.io/DiffuseHigh/

Temporal difference (TD) learning algorithms with neural network function parameterization have well-established empirical success in many practical large-scale reinforcement learning tasks. However, theoretical understanding of these algorithms remains challenging due to the nonlinearity of the action-value approximation. In this paper, we develop an improved non-asymptotic analysis of the neural TD method with a general $L$-layer neural network. New proof techniques are developed and an improved new $\tilde{\mathcal{O}}(\epsilon^{-1})$ sample complexity is derived. To our best knowledge, this is the first finite-time analysis of neural TD that achieves an $\tilde{\mathcal{O}}(\epsilon^{-1})$ complexity under the Markovian sampling, as opposed to the best known $\tilde{\mathcal{O}}(\epsilon^{-2})$ complexity in the existing literature.

This paper considers smooth strongly convex and strongly concave (SC-SC) stochastic saddle point (SSP) problems. Suppose there is an arbitrary oracle that in expectation returns an $\epsilon$-solution in the sense of certain gaps, which can be the duality gap or its weaker variants. We propose a general PB-SSP framework to guarantee an $\epsilon$ small duality gap solution with high probability via only $\mathcal{O}\big(\log \frac{1}{p}\cdot\text{poly}(\log \kappa)\big)$ calls of this oracle, where $p\in(0,1)$ is the confidence level and $\kappa$ is the condition number. When applied to the sample average approximation (SAA) oracle, in addition to equipping the solution with high probability, our approach even improves the sample complexity by a factor of $\text{poly}(\kappa)$, since the high-probability argument enables us to circumvent some key difficulties of the uniform stability analysis of SAA.

This paper considers the discounted criterion of nonzero-sum decentralized stochastic games with prospect players. The state and action spaces are finite. The state transition probability is nonstationary. Each player independently controls their own Markov chain. The subjective behavior of players is described by the prospect theory (PT). Compared to the average criterion of stochastic games under PT studied firstly in 2018, we are concerned with the time value of utility, i.e., the utility should be discounted in the future. Since PT distorts the probability, the optimality equation that plays a significant role in proving the existence of equilibrium does not exist. On the other hand, the games change into Markov decision processes (MDPs) with nonstationary payoff function when fixing others' stationary Markov strategies, then the occupation measure and the linear programming of stationary MDPs are no longer suitable. Therefore, we explore a new technique by constructing the marginal distribution on the state-action pairs at any time, and establish the existence of Nash equilibrium. When the initial state is given, there exists a Markov Nash equilibrium. Furthermore, this novel technique can be extended to the finite horizon criterion. Then, we present an algorithm to find a Markov $\varepsilon$-equilibrium. Finally, the model is applied to a noncooperative stochastic game among prosumers who can produce and consume energy in the smart grid, and we give some simulation results.

We investigate Bregman proximal gradient (BPG) methods for solving nonconvex composite stochastic optimization problems. Instead of the standard gradient Lipschitz continuity (GLC) assumption, the objective function only satisfies a smooth-adaptability assumption w.r.t. some kernel function. An in-depth analysis of the stationarity measure is made in this paper, where we reveal an interesting fact that the widely adopted Bregman proximal gradient mapping in the existing works may not correctly depict the near stationarity of the solutions. To resolve this issue, a new Bregman proximal gradient mapping is proposed and analyzed in this paper. Second, a thorough analysis is made on the sample complexities of the stochastic Bregman proximal gradient methods under both the old and the newly proposed gradient mappings are analyzed. Note that the absence of GLC disables the standard analysis of the stochastic variance reduction techniques, existing stochastic BPG methods only obtain an $O(\epsilon^{-2})$ sample complexity under the old gradient mapping, we show that such a limitation in the existing analyses mainly comes from the insufficient exploitation of the kernel's properties. By proposing a new kernel-conditioning regularity assumption on the kernel, we show that a simple epoch bound mechanism is enough to enable all the existing variance reduction techniques for stochastic BPG methods. Combined with a novel probabilistic argument, we show that there is a high probability event conditioning on which $O(\sqrt{n}\epsilon^{-1})$ sample complexity for the finite-sum stochastic setting can be derived for the old gradient mapping. Moreover, with a novel adaptive step size control mechanism, we also show an $\tilde{O}(\sqrt{n}L_h(\mathcal{X}_\epsilon)\epsilon^{-1})$ complexity under the new gradient mapping.

In a cost sharing problem on a weighted undirected graph, all other nodes want to connect to the source node for some service. Each edge has a cost denoted by a weight and all the connected nodes should share the total cost for the connectivity. The goal of the existing solutions (e.g. folk solution and cycle-complete solution) is to design cost sharing rules with nice properties, e.g. budget balance and cost monotonicity. However, they did not consider the cases that each non-source node has a budget which is the maximum it can pay for its cost share and may cut its adjacent edges to reduce its cost share. In this paper, we design two cost sharing mechanisms taking into account the nodes' budgets and incentivizing all nodes to report all their adjacent edges so that we can minimize the total cost for the connectivity., Comment: arXiv admin note: substantial text overlap with arXiv:2201.05976

We consider a cost sharing problem on a weighted directed acyclic graph (DAG) with a source node to which all the other nodes want to connect. The cost (weight) of each edge is private information reported by multiple contractors, and among them, only one contractor is selected as the builder. All the nodes except for the source need to share the total cost of the used edges. However, they may block others' connections to the source by strategically cutting their outgoing edges to reduce their cost share, which may increase the total cost of connectivity. To minimize the total cost of connectivity, we design a cost sharing mechanism to incentivize each node to offer all its outgoing edges and each contractor to report all the edges' weights truthfully, and show the properties of the proposed mechanism. In addition, our mechanism outperforms the two benchmark mechanisms.

Multi-cellular robot design aims to create robots comprised of numerous cells that can be efficiently controlled to perform diverse tasks. Previous research has demonstrated the ability to generate robots for various tasks, but these approaches often optimize robots directly in the vast design space, resulting in robots with complicated morphologies that are hard to control. In response, this paper presents a novel coarse-to-fine method for designing multi-cellular robots. Initially, this strategy seeks optimal coarse-grained robots and progressively refines them. To mitigate the challenge of determining the precise refinement juncture during the coarse-to-fine transition, we introduce the Hyperbolic Embeddings for Robot Design (HERD) framework. HERD unifies robots of various granularity within a shared hyperbolic space and leverages a refined Cross-Entropy Method for optimization. This framework enables our method to autonomously identify areas of exploration in hyperbolic space and concentrate on regions demonstrating promise. Finally, the extensive empirical studies on various challenging tasks sourced from EvoGym show our approach's superior efficiency and generalization capability.

Synthesizing physically plausible human motions in 3D scenes is a challenging problem. Kinematics-based methods cannot avoid inherent artifacts (e.g., penetration and foot skating) due to the lack of physical constraints. Meanwhile, existing physics-based methods cannot generalize to multi-object scenarios since the policy trained with reinforcement learning has limited modeling capacity. In this work, we present a framework that enables physically simulated characters to perform long-term interaction tasks in diverse, cluttered, and unseen scenes. The key idea is to decompose human-scene interactions into two fundamental processes, Interacting and Navigating, which motivates us to construct two reusable Controller, i.e., InterCon and NavCon. Specifically, InterCon contains two complementary policies that enable characters to enter and leave the interacting state (e.g., sitting on a chair and getting up). To generate interaction with objects at different places, we further design NavCon, a trajectory following policy, to keep characters' locomotion in the free space of 3D scenes. Benefiting from the divide and conquer strategy, we can train the policies in simple environments and generalize to complex multi-object scenes. Experimental results demonstrate that our framework can synthesize physically plausible long-term human motions in complex 3D scenes. Code will be publicly released at https://github.com/liangpan99/InterScene.

We study the optimal lower and upper complexity bounds for finding approximate solutions to the composite problem $\min_x\ f(x)+h(Ax-b)$, where $f$ is smooth and $h$ is convex. Given access to the proximal operator of $h$, for strongly convex, convex, and nonconvex $f$, we design efficient first order algorithms with complexities $\tilde{O}\left(\kappa_A\sqrt{\kappa_f}\log\left(1/{\epsilon}\right)\right)$, $\tilde{O}\left(\kappa_A\sqrt{L_f}D/\sqrt{\epsilon}\right)$, and $\tilde{O}\left(\kappa_A L_f\Delta/\epsilon^2\right)$, respectively. Here, $\kappa_A$ is the condition number of the matrix $A$ in the composition, $L_f$ is the smoothness constant of $f$, and $\kappa_f$ is the condition number of $f$ in the strongly convex case. $D$ is the initial point distance and $\Delta$ is the initial function value gap. Tight lower complexity bounds for the three cases are also derived and they match the upper bounds up to logarithmic factors, thereby demonstrating the optimality of both the upper and lower bounds proposed in this paper.

Robot design aims at learning to create robots that can be easily controlled and perform tasks efficiently. Previous works on robot design have proven its ability to generate robots for various tasks. However, these works searched the robots directly from the vast design space and ignored common structures, resulting in abnormal robots and poor performance. To tackle this problem, we propose a Symmetry-Aware Robot Design (SARD) framework that exploits the structure of the design space by incorporating symmetry searching into the robot design process. Specifically, we represent symmetries with the subgroups of the dihedral group and search for the optimal symmetry in structured subgroups. Then robots are designed under the searched symmetry. In this way, SARD can design efficient symmetric robots while covering the original design space, which is theoretically analyzed. We further empirically evaluate SARD on various tasks, and the results show its superior efficiency and generalizability., Comment: The Fortieth International Conference on Machine Learning (ICML 2023)

Recent offline meta-reinforcement learning (meta-RL) methods typically utilize task-dependent behavior policies (e.g., training RL agents on each individual task) to collect a multi-task dataset. However, these methods always require extra information for fast adaptation, such as offline context for testing tasks. To address this problem, we first formally characterize a unique challenge in offline meta-RL: transition-reward distribution shift between offline datasets and online adaptation. Our theory finds that out-of-distribution adaptation episodes may lead to unreliable policy evaluation and that online adaptation with in-distribution episodes can ensure adaptation performance guarantee. Based on these theoretical insights, we propose a novel adaptation framework, called In-Distribution online Adaptation with uncertainty Quantification (IDAQ), which generates in-distribution context using a given uncertainty quantification and performs effective task belief inference to address new tasks. We find a return-based uncertainty quantification for IDAQ that performs effectively. Experiments show that IDAQ achieves state-of-the-art performance on the Meta-World ML1 benchmark compared to baselines with/without offline adaptation.

This paper studies the complexity of finding approximate stationary points for nonconvex-strongly-concave (NC-SC) smooth saddle point problem: $\min_x\max_yf(x,y)$. Under first-order smoothness conditions, and denote $\kappa_y$ the condition number for the inner strongly-concave problem, existing literature shows that the optimal complexity for first-order methods to obtain an $\epsilon$-stationary point is $\tilde{\Theta}(\sqrt{\kappa_y}\epsilon^{-2})$. However, when the objective function further satisfies certain high-order smoothness conditions, both the iteration complexity upper and lower bounds can be improved for first-order methods. In detail, given the additional assumption that the function $\Phi(x):=\max_y f(x,y)$ has Lipschitz continuous Hessian, we derive a first-order method with an improved $\tilde{O}(\sqrt{\kappa_y}\epsilon^{-7/4})$ complexity, by designing an inexact proximal point variant of the "Convex Until Proven Guilty" method that is compatible with the APPA acceleration technique for saddle point problems. In addition to an upper bound, we also construct an $\Omega(\sqrt{\kappa_y}\epsilon^{-12/7})$ lower bound for NC-SC problems with second-order smoothness. As a result, the complexity of our proposed algorithm is near-optimal and it almost matches the lower bound barring a negligible factor of $\tilde{O}(\epsilon^{-1/28})$.

We consider a cost sharing problem on a weighted undirected graph, where all the nodes want to connect to a special node called source, and they need to share the total cost (weights) of the used edges. Each node except for the source has a private valuation of the connection, and it may block others' connections by strategically cutting its adjacent edges to reduce its cost share, which may increase the total cost. We aim to design mechanisms to prevent the nodes from misreporting their valuations and cutting their adjacent edges. We first show that it is impossible for such a mechanism to further satisfy budget balance (cover the total cost) and efficiency (maximize social welfare). Then, we design two feasible cost sharing mechanisms that incentivize each node to offer all its adjacent edges and truthfully report its valuation, and also satisfy either budget balance or efficiency.

In this paper, a Gauss-Newton Temporal Difference (GNTD) learning method is proposed to solve the Q-learning problem with nonlinear function approximation. In each iteration, our method takes one Gauss-Newton (GN) step to optimize a variant of Mean-Squared Bellman Error (MSBE), where target networks are adopted to avoid double sampling. Inexact GN steps are analyzed so that one can safely and efficiently compute the GN updates by cheap matrix iterations. Under mild conditions, non-asymptotic finite-sample convergence to the globally optimal Q function is derived for various nonlinear function approximations. In particular, for neural network parameterization with relu activation, GNTD achieves an improved sample complexity of $\tilde{\mathcal{O}}(\varepsilon^{-1})$, as opposed to the $\mathcal{\mathcal{O}}(\varepsilon^{-2})$ sample complexity of the existing neural TD methods. An $\tilde{\mathcal{O}}(\varepsilon^{-1.5})$ sample complexity of GNTD is also established for general smooth function approximations. We validate our method via extensive experiments in several RL benchmarks, where GNTD exhibits both higher rewards and faster convergence than TD-type methods.

Rejuvenating galaxies are unusual galaxies that fully quench and then subsequently experience a "rejuvenation" event to become star-forming once more. Rejuvenation rates vary substantially in models of galaxy formation: 10%-70% of massive galaxies are expected to experience rejuvenation by z = 0. Measuring the rate of rejuvenation is therefore important for calibrating the strength of star formation feedback mechanisms. However, these observations are challenging because rejuvenating systems blend in with normal star-forming galaxies in broadband photometry. In this paper, we use the galaxy spectral energy distribution (SED)-fitting code Prospector to search for observational markers that distinguish normal star-forming galaxies from rejuvenating galaxies. We find that rejuvenating galaxies have smaller Balmer absorption line equivalent widths (EWs) than star-forming galaxies. This is analogous to the well-known "K + A" or post-starburst galaxies, which have strong Balmer absorption due to A-stars dominating the light: in this case, rejuvenating systems have a lack of A-stars, instead resembling "O - A" systems. We find star-forming galaxies that have H$\beta$, H$\gamma$, and/or H$\delta$ absorption EWs $\lesssim 3${\AA} corresponds to a highly pure selection of rejuvenating systems. Interestingly, while this technique is highly effective at identifying mild rejuvenation, "strongly" rejuvenating systems remain nearly indistinguishable from star-forming galaxies due to the well-known stellar outshining effect. We conclude that measuring Balmer absorption line EWs in star-forming galaxy populations is an efficient method to identify rejuvenating populations, and discuss several techniques to either remove or resolve the nebular emission which typically lies on top of these absorption lines., Comment: Published in ApJ, 16 pages, 9 figures

In this paper, we propose a unified primal-dual algorithm framework based on the augmented Lagrangian function for composite convex problems with conic inequality constraints. The new framework is highly versatile. First, it not only covers many existing algorithms such as PDHG, Chambolle-Pock (CP), GDA, OGDA and linearized ALM, but also guides us to design a new efficient algorithm called Simi-OGDA (SOGDA). Second, it enables us to study the role of the augmented penalty term in the convergence analysis. Interestingly, a properly selected penalty not only improves the numerical performance of the above methods, but also theoretically enables the convergence of algorithms like PDHG and SOGDA. Under properly designed step sizes and penalty term, our unified framework preserves the $\mathcal{O}(1/N)$ ergodic convergence while not requiring any prior knowledge about the magnitude of the optimal Lagrangian multiplier. Linear convergence rate for affine equality constrained problem is also obtained given appropriate conditions. Finally, numerical experiments on linear programming, $\ell_1$ minimization problem, and multi-block basis pursuit problem demonstrate the efficiency of our methods.

As an important framework for safe Reinforcement Learning, the Constrained Markov Decision Process (CMDP) has been extensively studied in the recent literature. However, despite the rich results under various on-policy learning settings, there still lacks some essential understanding of the offline CMDP problems, in terms of both the algorithm design and the information theoretic sample complexity lower bound. In this paper, we focus on solving the CMDP problems where only offline data are available. By adopting the concept of the single-policy concentrability coefficient $C^*$, we establish an $\Omega\left(\frac{\min\left\{|\mathcal{S}||\mathcal{A}|,|\mathcal{S}|+I\right\} C^*}{(1-\gamma)^3\epsilon^2}\right)$ sample complexity lower bound for the offline CMDP problem, where $I$ stands for the number of constraints. By introducing a simple but novel deviation control mechanism, we propose a near-optimal primal-dual learning algorithm called DPDL. This algorithm provably guarantees zero constraint violation and its sample complexity matches the above lower bound except for an $\tilde{\mathcal{O}}((1-\gamma)^{-1})$ factor. Comprehensive discussion on how to deal with the unknown constant $C^*$ and the potential asynchronous structure on the offline dataset are also included.

We consider a cost sharing problem to connect all nodes in a weighted undirected graph, where the weight of each edge represents the cost to use the edge for the connectivity and the cost has to be shared among all connected nodes. There is one node called the source to which all the other nodes want to connect and it does not share the costs of the connectivity. As a node may need to go through other nodes to reach the source, the intermediate nodes may behave strategically to block the connection by cutting the edges adjacent to them. To prevent such strategical behavior, we design cost sharing mechanisms to incentivize all nodes not to cut any edge so that we can minimize the total cost for connecting all the nodes., Comment: The content are totally reorganized in a new paper

We consider the convex-concave saddle point problem $\min_{\mathbf{x}}\max_{\mathbf{y}}\Phi(\mathbf{x},\mathbf{y})$, where the decision variables $\mathbf{x}$ and/or $\mathbf{y}$ subject to a multi-block structure and affine coupling constraints, and $\Phi(\mathbf{x},\mathbf{y})$ possesses certain separable structure. Although the minimization counterpart of such problem has been widely studied under the topics of ADMM, this minimax problem is rarely investigated. In this paper, a convenient notion of $\epsilon$-saddle point is proposed, under which the convergence rate of several proposed algorithms are analyzed. When only one of $\mathbf{x}$ and $\mathbf{y}$ has multiple blocks and affine constraint, several natural extensions of ADMM are proposed to solve the problem. Depending on the number of blocks and the level of smoothness, $\mathcal{O}(1/T)$ or $\mathcal{O}(1/\sqrt{T})$ convergence rates are derived for our algorithms. When both $\mathbf{x}$ and $\mathbf{y}$ have multiple blocks and affine constraints, a new algorithm called ExtraGradient Method of Multipliers (EGMM) is proposed. Under desirable smoothness condition, an $\mathcal{O}(1/T)$ rate of convergence can be guaranteed regardless of the number of blocks in $\mathbf{x}$ and $\mathbf{y}$. In depth comparison between EGMM (fully primal-dual method) and ADMM (approximate dual method) is made over the multi-block optimization problems to illustrate the advantage of the EGMM., Comment: 25 pages