Search

Efficient collaboration in the centralized training with decentralized execution (CTDE) paradigm remains a challenge in cooperative multi-agent systems. We identify divergent action tendencies among agents as a significant obstacle to CTDE's training efficiency, requiring a large number of training samples to achieve a unified consensus on agents' policies. This divergence stems from the lack of adequate team consensus-related guidance signals during credit assignments in CTDE. To address this, we propose Intrinsic Action Tendency Consistency, a novel approach for cooperative multi-agent reinforcement learning. It integrates intrinsic rewards, obtained through an action model, into a reward-additive CTDE (RA-CTDE) framework. We formulate an action model that enables surrounding agents to predict the central agent's action tendency. Leveraging these predictions, we compute a cooperative intrinsic reward that encourages agents to match their actions with their neighbors' predictions. We establish the equivalence between RA-CTDE and CTDE through theoretical analyses, demonstrating that CTDE's training process can be achieved using agents' individual targets. Building on this insight, we introduce a novel method to combine intrinsic rewards and CTDE. Extensive experiments on challenging tasks in SMAC and GRF benchmarks showcase the improved performance of our method., Comment: The AAAI-2024 paper with the appendix

Mastering multiple tasks through exploration and learning in an environment poses a significant challenge in reinforcement learning (RL). Unsupervised RL has been introduced to address this challenge by training policies with intrinsic rewards rather than extrinsic rewards. However, current intrinsic reward designs and unsupervised RL algorithms often overlook the heterogeneous nature of collected samples, thereby diminishing their sample efficiency. To overcome this limitation, in this paper, we propose a reward-free RL algorithm called \alg. The key idea behind our algorithm is an uncertainty-aware intrinsic reward for exploring the environment and an uncertainty-weighted learning process to handle heterogeneous uncertainty in different samples. Theoretically, we show that in order to find an $\epsilon$-optimal policy, GFA-RFE needs to collect $\tilde{O} (H^2 \log N_{\mathcal F} (\epsilon) \mathrm{dim} (\mathcal F) / \epsilon^2 )$ number of episodes, where $\mathcal F$ is the value function class with covering number $N_{\mathcal F} (\epsilon)$ and generalized eluder dimension $\mathrm{dim} (\mathcal F)$. Such a result outperforms all existing reward-free RL algorithms. We further implement and evaluate GFA-RFE across various domains and tasks in the DeepMind Control Suite. Experiment results show that GFA-RFE outperforms or is comparable to the performance of state-of-the-art unsupervised RL algorithms., Comment: 32 pages, 5 figures, 4 tables, accepted by ICML 2024

This demo paper examines the susceptibility of Federated Learning (FL) systems to targeted data poisoning attacks, presenting a novel system for visualizing and mitigating such threats. We simulate targeted data poisoning attacks via label flipping and analyze the impact on model performance, employing a five-component system that includes Simulation and Data Generation, Data Collection and Upload, User-friendly Interface, Analysis and Insight, and Advisory System. Observations from three demo modules: label manipulation, attack timing, and malicious attack availability, and two analysis components: utility and analytical behavior of local model updates highlight the risks to system integrity and offer insight into the resilience of FL systems. The demo is available at https://github.com/CathyXueqingZhang/DataPoisoningVis.

Real-world multi-agent systems are often dynamic and continuous, where the agents co-evolve and undergo changes in their trajectories and interactions over time. For example, the COVID-19 transmission in the U.S. can be viewed as a multi-agent system, where states act as agents and daily population movements between them are interactions. Estimating the counterfactual outcomes in such systems enables accurate future predictions and effective decision-making, such as formulating COVID-19 policies. However, existing methods fail to model the continuous dynamic effects of treatments on the outcome, especially when multiple treatments (e.g., "stay-at-home" and "get-vaccine" policies) are applied simultaneously. To tackle this challenge, we propose Causal Graph Ordinary Differential Equations (CAG-ODE), a novel model that captures the continuous interaction among agents using a Graph Neural Network (GNN) as the ODE function. The key innovation of our model is to learn time-dependent representations of treatments and incorporate them into the ODE function, enabling precise predictions of potential outcomes. To mitigate confounding bias, we further propose two domain adversarial learning-based objectives, which enable our model to learn balanced continuous representations that are not affected by treatments or interference. Experiments on two datasets (i.e., COVID-19 and tumor growth) demonstrate the superior performance of our proposed model.

Discrete diffusion models have emerged as powerful tools for high-quality data generation. Despite their success in discrete spaces, such as text generation tasks, the acceleration of discrete diffusion models remains under explored. In this paper, we propose a discrete non-Markov diffusion model, which admits an accelerated reverse sampling for discrete data generation. Our method significantly reduces the number of function evaluations (i.e., calls to the neural network), making the sampling process much faster. Furthermore, we study the transition from finite to infinite step sampling, offering new insights into bridging the gap between discrete and continuous-time processes for discrete diffusion models. Extensive experiments on natural language generation and machine translation tasks demonstrate the superior performance of our method in terms of both generation speed and sample quality compared to existing methods for discrete diffusion models., Comment: 33 pages, 5 figures, 12 tables

In the current landscape of large models, the Transformer stands as a cornerstone, playing a pivotal role in shaping the trajectory of modern models. However, its application encounters challenges attributed to the substantial computational intricacies intrinsic to its attention mechanism. Moreover, its reliance on high-precision floating-point operations presents specific hurdles, particularly evident in computation-intensive scenarios such as edge computing environments. These environments, characterized by resource-constrained devices and a preference for lower precision, necessitate innovative solutions. To tackle the exacting data processing demands posed by edge devices, we introduce the Bitformer model, an inventive extension of the Transformer paradigm. Central to this innovation is a novel attention mechanism that adeptly replaces conventional floating-point matrix multiplication with bitwise operations. This strategic substitution yields dual advantages. Not only does it maintain the attention mechanism's prowess in capturing intricate long-range information dependencies, but it also orchestrates a profound reduction in the computational complexity inherent in the attention operation. The transition from an $O(n^2d)$ complexity, typical of floating-point operations, to an $O(n^2T)$ complexity characterizing bitwise operations, substantiates this advantage. Notably, in this context, the parameter $T$ remains markedly smaller than the conventional dimensionality parameter $d$. The Bitformer model in essence endeavors to reconcile the indomitable requirements of modern computing landscapes with the constraints posed by edge computing scenarios. By forging this innovative path, we bridge the gap between high-performing models and resource-scarce environments, thus unveiling a promising trajectory for further advancements in the field.

The challenge of overfitting, in which the model memorizes the training data and fails to generalize to test data, has become increasingly significant in the training of large neural networks. To tackle this challenge, Sharpness-Aware Minimization (SAM) has emerged as a promising training method, which can improve the generalization of neural networks even in the presence of label noise. However, a deep understanding of how SAM works, especially in the setting of nonlinear neural networks and classification tasks, remains largely missing. This paper fills this gap by demonstrating why SAM generalizes better than Stochastic Gradient Descent (SGD) for a certain data model and two-layer convolutional ReLU networks. The loss landscape of our studied problem is nonsmooth, thus current explanations for the success of SAM based on the Hessian information are insufficient. Our result explains the benefits of SAM, particularly its ability to prevent noise learning in the early stages, thereby facilitating more effective learning of features. Experiments on both synthetic and real data corroborate our theory., Comment: 52 pages, 4 figures, 2 tables. In NeurIPS 2023

We analyse the joint distribution of dust attenuation and projected axis ratios, together with galaxy size and surface brightness profile information, to infer lessons on the dust content and star/dust geometry within star-forming galaxies at 0 < z <2.5. To do so, we make use of large observational datasets from KiDS+VIKING+HSC-SSP and extend the analysis out to redshift z = 2.5 using the HST surveys CANDELS and 3D-DASH. We construct suites of SKIRT radiative transfer models for idealized galaxies observed under random viewing angles with the aim of reproducing the aforementioned distributions, including the level and inclination dependence of dust attenuation. We find that attenuation-based dust mass estimates are at odds with constraints from far-infrared observations, especially at higher redshifts, when assuming smooth star and dust geometries of equal extent. We demonstrate that UV-to-near-IR and far-infrared constraints can be reconciled by invoking clumpier dust geometries for galaxies at higher redshifts and/or very compact dust cores. We discuss implications for the significant wavelength- and redshift-dependent differences between half-light and half-mass radii that result from spatially varying dust columns within -- especially massive -- star-forming galaxies., Comment: Accepted for publication in MNRAS

This paper analyzes the performance of the 3rd Generation Partnership Project (3GPP)-inspired multi-cell wideband single-hop backhaul millimeter-wave-in-band-full-duplex (IBFD)-integrated access and backhaul (IAB) networks by using stochastic geometry. We model the wired-connected Next Generation NodeBs (gNBs) as the Mat\'ern hard-core point process (MHCPP) to meet the real-world deployment requirement and reduce the cost caused by wired connection in the network. We first derive association probabilities that reflect how likely the typical user-equipment is served by a gNB or an IAB-node based on the maximum long-term averaged biased-received-desired-signal power criteria. Further, by leveraging the composite Gamma-Lognormal distribution, we derive the closed-form signal to interference plus noise ratio coverage, capacity with outage, and ergodic capacity of the network. In order to avoid underestimating the noise, we consider the sidelobe gain on inter-cell interference links and the analog to digital converter quantization noise. Compared with the half-duplex transmission, numerical results show an enhanced capacity with outage and ergodic capacity provided by IBFD under successful self-interference cancellation. We also study how the power bias and density ratio of the IAB-node to gNB, and the hard-core distance can affect system performances.

We study reward-free reinforcement learning (RL) with linear function approximation, where the agent works in two phases: (1) in the exploration phase, the agent interacts with the environment but cannot access the reward; and (2) in the planning phase, the agent is given a reward function and is expected to find a near-optimal policy based on samples collected in the exploration phase. The sample complexities of existing reward-free algorithms have a polynomial dependence on the planning horizon, which makes them intractable for long planning horizon RL problems. In this paper, we propose a new reward-free algorithm for learning linear mixture Markov decision processes (MDPs), where the transition probability can be parameterized as a linear combination of known feature mappings. At the core of our algorithm is uncertainty-weighted value-targeted regression with exploration-driven pseudo-reward and a high-order moment estimator for the aleatoric and epistemic uncertainties. When the total reward is bounded by $1$, we show that our algorithm only needs to explore $\tilde O( d^2\varepsilon^{-2})$ episodes to find an $\varepsilon$-optimal policy, where $d$ is the dimension of the feature mapping. The sample complexity of our algorithm only has a polylogarithmic dependence on the planning horizon and therefore is "horizon-free". In addition, we provide an $\Omega(d^2\varepsilon^{-2})$ sample complexity lower bound, which matches the sample complexity of our algorithm up to logarithmic factors, suggesting that our algorithm is optimal., Comment: 37 pages, 1 figure, 2 tables. In ICML 2023

We study the intrinsic 3D shapes of quiescent galaxies over the last half of cosmic history based on their axial ratio distribution. To this end, we construct a sample of unprecedented size, exploiting multi-wavelength $u$-to-$K_s$ photometry from the deep wide area surveys KiDS+VIKING paired with high-quality $i$-band imaging from HSC-SSP. Dependencies of the shapes on mass, redshift, photometric bulge prominence and environment are considered. For comparison, the intrinsic shapes of quenched galaxies in the IllustrisTNG simulations are analyzed and contrasted to their formation history. We find that over the full $0 10^{11}\ M_{\odot}$, with the effect being most pronounced at lower redshifts. In TNG, the most massive galaxies feature the highest ex-situ stellar mass fractions, pointing to violent relaxation via mergers as the mechanism responsible for their 3D shape transformation. Larger differences between observed and simulated shapes are found at low to intermediate masses. At any mass, the most spheroidal quiescent galaxies in TNG feature the highest bulge mass fractions, and conversely observed quiescent galaxies with the highest bulge-to-total ratios are found to be intrinsically the roundest. Finally, we detect an environmental influence on galaxy shape, at least at the highest masses, such that at fixed mass and redshift quiescent galaxies tend to be rounder in denser environments., Comment: Accepted by MNRAS. 18 pages, 11 figures, 2 tables

This paper investigates the neutral gas phase of galactic winds via the Na I D$\lambda\lambda 5890,5895${\AA} feature within $z \sim 0.04$ MaNGA galaxies, and directly compares their incidence and strength to the ionized winds detected within the same parent sample. We find evidence for neutral outflows in 127 galaxies ($\sim 5$ per cent of the analysed line-emitting sample). Na I D winds are preferentially seen in galaxies with dustier central regions and both wind phases are more often found in systems with elevated SFR surface densities, especially when there has been a recent upturn in the star formation activity according to the SFR$_{5Myr}$/SFR$_{800Myr}$ parameter. We find the ionized outflow kinematics to be in line with what we measure in the neutral phase. This demonstrates that, despite their small contributions to the total outflow mass budget, there is value to collecting empirical measurements of the ionized wind phase to provide information on the bulk motion in the outflow. Depending on dust corrections applied to the ionized gas diagnostics, the neutral phase has $\sim 1.2 - 1.8$ dex higher mass outflow rates ($\dot{M}_{out}$), on average, compared to the ionized phase. We quantify scaling relations between $\dot{M}_{out}$ and the strengths of the physical wind drivers (SFR, $L_{AGN}$). Using a radial-azimuthal stacking method, and by considering inclination dependencies, we find results consistent with biconical outflows orthogonal to the disk plane. Our work complements other multi-phase outflow studies in the literature which consider smaller samples, more extreme objects, or proceed via stacking of larger samples., Comment: This is a pre-copyedited, author-produced PDF of an article accepted for publication in Monthly Notices of the Royal Astronomical Society (MNRAS) following peer review

In this work, we investigate the strength and impact of ionised gas outflows within $z \sim 0.04$ MaNGA galaxies. We find evidence for outflows in 322 galaxies ($12\%$ of the analysed line-emitting sample), 185 of which show evidence for AGN activity. Most outflows are centrally concentrated with a spatial extent that scales sublinearly with $R_{\rm e}$. The incidence of outflows is enhanced at higher masses, central surface densities and deeper gravitational potentials, as well as at higher SFR and AGN luminosity. We quantify strong correlations between mass outflow rates and the mechanical drivers of the outflow of the form $\dot{M}_{\rm out} \propto \rm SFR^{0.97}$ and $\dot{M}_{\rm out} \propto L_{\rm AGN}^{0.55}$. We derive a master scaling relation describing the mass outflow rate of ionised gas as a function of $M_{\star}$, SFR, $R_{\rm e}$ and $L_{\rm AGN}$. Most of the observed winds are anticipated to act as galactic fountains, with the fraction of galaxies with escaping winds increasing with decreasing potential well depth. We further investigate the physical properties of the outflowing gas finding evidence for enhanced attenuation in the outflow, possibly due to metal-enriched winds, and higher excitation compared to the gas in the galactic disk. Given that the majority of previous studies have focused on more extreme systems with higher SFRs and/or more luminous AGN, our study provides a unique view of the non-gravitational gaseous motions within `typical' galaxies in the low-redshift Universe, where low-luminosity AGN and star formation contribute jointly to the observed outflow phenomenology., Comment: Accepted for publication in MNRAS, 27 pages, Fig 7 & 8 for scaling wind strength with drivers, Fig 10 for master scaling

Simultaneous wireless information and power transfer (SWIPT) technique is a popular strategy to convey both information and RF energy for harvesting at receivers. In this regard, we consider a two-way relay system with multiple users and a multi-antenna relay employing SWIPT strategy, where splitting the received signal leads to a rate-energy trade-off. In literature, the works on transceiver design have been studied using computationally intensive and suboptimal convex relaxation based schemes. In this paper, we study the balanced precoder design using chordal distance (CD) decomposition, which incurs much lower complexity, and is flexible to dynamic energy requirements. It is analyzed that given a non-negative value of CD, the achieved harvested energy for the proposed balanced precoder is higher than that for the perfect interference alignment (IA) precoder. The corresponding loss in sum rates is also analyzed via an upper bound. Simulation results add that the IA schemes based on mean-squared error are better suited for the SWIPT maximization than the subspace alignment-based methods., Comment: arXiv admin note: text overlap with arXiv:2101.12161

This paper develops a 3GPP-inspired design for the in-band-full-duplex (IBFD) integrated access and backhaul (IAB) networks in the frequency range 2 (FR2) band, which can enhance the spectral efficiency (SE) and coverage while reducing the latency. However, the self-interference (SI), which is usually more than 100 dB higher than the signal-of-interest, becomes the major bottleneck in developing these IBFD networks. We design and analyze a subarray-based hybrid beamforming IBFD-IAB system with the RF beamformers obtained via RF codebooks given by a modified Linde-Buzo-Gray (LBG) algorithm. The SI is canceled in three stages, where the first stage of antenna isolation is assumed to be successfully deployed. The second stage consists of the optical domain (OD)-based RF cancellation, where cancelers are connected with the RF chain pairs. The third stage is comprised of the digital cancellation via successive interference cancellation followed by minimum mean-squared error baseband receiver. Multiuser interference in the access link is canceled by zero-forcing at the IAB-node transmitter. Simulations show that under 400 MHz bandwidth, our proposed OD-based RF cancellation can achieve around 25 dB of cancellation with 100 taps. Moreover, the higher the hardware impairment and channel estimation error, the worse digital cancellation ability we can obtain.

One of the key technologies for the future cellular networks is full duplex (FD)-enabled integrated access and backhaul (IAB) networks operating in the millimeter-wave (mmWave) frequencies. The main challenge in realizing FD-IAB networks is mitigating the impact of self-interference (SI) in the wideband mmWave frequencies. In this article, we first introduce the 3GPP IAB network architectures and wideband mmWave channel models. By utilizing the subarray-based hybrid precoding scheme at the FD-IAB node, multiuser interference is mitigated using zero-forcing at the transmitter, whereas the residual SI after successfully deploying antenna and analog cancellation is canceled by a minimum mean square error baseband combiner at the receiver. The spectral efficiency (SE) is evaluated for the RF insertion loss (RFIL) with different kinds of phase shifters and channel uncertainty. Simulation results show that, in the presence of the RFIL, the almost double SE, which is close to that obtained from fully connected hybrid precoding, can be achieved as compared to half duplex systems when the uncertainties are of low strength.