Search

Offline Reinforcement Learning (RL) is structured to derive policies from static trajectory data without requiring real-time environment interactions. Recent studies have shown the feasibility of framing offline RL as a sequence modeling task, where the sole aim is to predict actions based on prior context using the transformer architecture. However, the limitation of this single task learning approach is its potential to undermine the transformer model's attention mechanism, which should ideally allocate varying attention weights across different tokens in the input context for optimal prediction. To address this, we reformulate offline RL as a multi-objective optimization problem, where the prediction is extended to states and returns. We also highlight a potential flaw in the trajectory representation used for sequence modeling, which could generate inaccuracies when modeling the state and return distributions. This is due to the non-smoothness of the action distribution within the trajectory dictated by the behavioral policy. To mitigate this issue, we introduce action space regions to the trajectory representation. Our experiments on D4RL benchmark locomotion tasks reveal that our propositions allow for more effective utilization of the attention mechanism in the transformer model, resulting in performance that either matches or outperforms current state-of-the art methods.

We consider a cooperative X-channel with $\sf K$ transmitters (TXs) and $\sf K$ receivers (Rxs) where Txs and Rxs are gathered into groups of size $\sf r$ respectively. Txs belonging to the same group cooperate to jointly transmit a message to each of the $\sf K- \sf r$ Rxs in all other groups, and each Rx individually decodes all its intended messages. By introducing a new interference alignment (IA) scheme, we prove that when $\sf K/\sf r$ is an integer the sum Degrees of Freedom (SDoF) of this channel is lower bounded by $2\sf r$ if $\sf K/\sf r \in \{2,3\}$ and by $\frac{\sf K(\sf K-\sf r)-\sf r^2}{2\sf K-3\sf r}$ if $\sf K/\sf r \geq 4$. We also prove that the SDoF is upper bounded by $\frac{\sf K(\sf K-\sf r)}{2\sf K-3\sf r}$. The proposed IA scheme finds application in a wireless distributed MapReduce framework, where it improves the normalized data delivery time (NDT) compared to the state of the art.

We characterize the complexity of the lattice decoding problem from a neural network perspective. The notion of Voronoi-reduced basis is introduced to restrict the space of solutions to a binary set. On the one hand, this problem is shown to be equivalent to computing a continuous piecewise linear (CPWL) function restricted to the fundamental parallelotope. On the other hand, it is known that any function computed by a ReLU feed-forward neural network is CPWL. As a result, we count the number of affine pieces in the CPWL decoding function to characterize the complexity of the decoding problem. It is exponential in the space dimension $n$, which induces shallow neural networks of exponential size. For structured lattices we show that folding, a technique equivalent to using a deep neural network, enables to reduce this complexity from exponential in $n$ to polynomial in $n$. Regarding unstructured MIMO lattices, in contrary to dense lattices many pieces in the CPWL decoding function can be neglected for quasi-optimal decoding on the Gaussian channel. This makes the decoding problem easier and it explains why shallow neural networks of reasonable size are more efficient with this category of lattices (in low to moderate dimensions)., Comment: submitted to IEEE Transactions on Information Theory. arXiv admin note: text overlap with arXiv:1902.11294

This paper investigates the decoding of a remarkable set of lattices: We treat in a unified framework the Leech lattice in dimension 24, the Nebe lattice in dimension 72, and the Barnes-Wall lattices. A new interesting lattice is constructed as a simple application of single parity-check principle on the Leech lattice. The common aspect of these lattices is that they can be obtained via a single parity check or via the k-ing construction. We exploit these constructions to introduce a new efficient paradigm for decoding. This leads to efficient list decoders and quasi-optimal decoders on the Gaussian channel. Both theoretical and practical performance (point error probability and complexity) of the new decoders are provided., Comment: Submitted to IEEE Transactions on Information Theory

The problem of resource constrained scheduling in a dynamic and heterogeneous wireless setting is considered here. In our setup, the available limited bandwidth resources are allocated in order to serve randomly arriving service demands, which in turn belong to different classes in terms of payload data requirement, delay tolerance, and importance/priority. In addition to heterogeneous traffic, another major challenge stems from random service rates due to time-varying wireless communication channels. Various approaches for scheduling and resource allocation can be used, ranging from simple greedy heuristics and constrained optimization to combinatorics. Those methods are tailored to specific network or application configuration and are usually suboptimal. To this purpose, we resort to deep reinforcement learning (DRL) and propose a distributional Deep Deterministic Policy Gradient (DDPG) algorithm combined with Deep Sets to tackle the aforementioned problem. Furthermore, we present a novel way to use a Dueling Network, which leads to further performance improvement. Our proposed algorithm is tested on both synthetic and real data, showing consistent gains against state-of-the-art conventional methods from combinatorics, optimization, and scheduling metrics.

We consider updating strategies for a local cache which downloads time-sensitive files from a remote server through a bandwidth-constrained link. The files are requested randomly from the cache by local users according to a popularity distribution which varies over time according to a Markov chain structure. We measure the freshness of the requested time-sensitive files through their Age of Information (AoI). The goal is then to minimize the average AoI of all requested files by appropriately designing the local cache's downloading strategy. To achieve this goal, the original problem is relaxed and cast into a Constrained Markov Decision Problem (CMDP), which we solve using a Lagrangian approach and Linear Programming. Inspired by this solution for the relaxed problem, we propose a practical cache updating strategy that meets all the constraints of the original problem. Under certain assumptions, the practical updating strategy is shown to be optimal for the original problem in the asymptotic regime of a large number of files. For a finite number of files, we show the gain of our practical updating strategy over the traditional square-root-law strategy (which is optimal for fixed non time-varying file popularities) through numerical simulations., Comment: To appear ITW2020

We propose Graph Contrastive Learning (GraphCL), a general framework for learning node representations in a self supervised manner. GraphCL learns node embeddings by maximizing the similarity between the representations of two randomly perturbed versions of the intrinsic features and link structure of the same node's local subgraph. We use graph neural networks to produce two representations of the same node and leverage a contrastive learning loss to maximize agreement between them. In both transductive and inductive learning setups, we demonstrate that our approach significantly outperforms the state-of-the-art in unsupervised learning on a number of node classification benchmarks., Comment: Under review for Neurips 2020

We address the problem of resource allocation (RA) in a cognitive radio (CR) communication system with multiple secondary operators sharing spectrum with an incumbent primary operator. The key challenge of the RA problem is the inter-operator coordination arising in the optimization problem so that the aggregated interference at the primary users (PUs) does not exceed the target threshold. While this problem is easily solvable if a centralized unit could access information of all secondary operators, it becomes challenging in a realistic scenario. In this paper, considering a satellite setting, we alleviate this problem by proposing two approaches to reduce the information exchange level among the secondary operators. In the first approach, we formulate an RA scheme based on a partial information sharing method which enables distributed optimization across secondary operators. In the second approach, instead of exchanging secondary users (SUs) information, the operators only exchange their contributions of the interference-level and RA is performed locally across secondary operators. These two approaches, for the first time in this context, provide a trade-off between performance and level of inter-operator information exchange. Through the numerical simulations, we explain this trade-off and illustrate the penalty resulting from partial information exchange., Comment: arXiv admin note: text overlap with arXiv:2005.02746

Hybrid Automatic ReQuest (HARQ) protocol enables reliable communications in wireless systems. Usually, several parallel streams are sent in successive timeslots following a time-sharing approach. Recently, multi-layer HARQ has been proposed by superposing packets within a timeslot. In this paper, we evaluate the potential of this multi-layer HARQ by playing with some design parameters. We show that a gain in throughput is only obtained at mid-Signal-to-Noise Ratio (SNR).

We present new efficient recursive decoders for the Barnes-Wall lattices based on their squaring construction. The analysis of the new decoders reveals a quasi-quadratic complexity in the lattice dimension and a quasi-linear complexity in the list-size. The error rate is shown to be close to the universal lower bound in dimensions 64 and 128., Comment: Presented at ISIT 2020

We consider a scenario where a base station (BS) attempts to collect fresh information from power constrained sensors over time-varying band-limited wireless channels. We characterize the data freshness through the recently proposed metric--the Age of Information. We consider a time-varying channel model with power adaptation. Unlike previous work, packet loss may happen due to imperfect channel estimation or decoding error. We propose an asymptotic optimal scheduling algorithm minimizing AoI performance and satisfying both bandwidth and power constraint in such networks. Numerical simulations show that the proposed policy outperforms the greedy one, and we observe that sensors with poor channels are scheduled at higher AoI in order to limit the packet loss., Comment: to be submitted to ICC2020

We consider a system consisting of a library of time-varying files, a server that at all times observes the current version of all files, and a cache that at the beginning stores the current versions of all files but afterwards has to update %fresh versions of these files from the server. Unlike previous works, the update duration is not constant but depends on the file and its Age of Information (AoI), i.e., of the time elapsed since it was last updated. The goal of this work is to design an update policy that minimizes the average AoI of all files with respect to a given popularity distribution. Actually a relaxed problem, close to the original optimization problem, is solved and a practical update policy is derived. The update policy relies on the file popularity and on the functions that characterize the update durations of the files depending on their AoI. Numerical simulations show a significant improvement of this new update policy compared to the so-called square-root policy that is optimal under file-independent and constant update durations., Comment: To be submitted to ICC 2020

We present new families of continuous piecewise linear (CPWL) functions in Rn having a number of affine pieces growing exponentially in $n$. We show that these functions can be seen as the high-dimensional generalization of the triangle wave function used by Telgarsky in 2016. We prove that they can be computed by ReLU networks with quadratic depth and linear width in the space dimension. We also investigate the approximation error of one of these functions by shallower networks and prove a separation result. The main difference between our functions and other constructions is their practical interest: they arise in the scope of channel coding. Hence, computing such functions amounts to performing a decoding operation., Comment: arXiv admin note: text overlap with arXiv:1902.05146

Point lattices and their decoding via neural networks are considered in this paper. Lattice decoding in Rn, known as the closest vector problem (CVP), becomes a classification problem in the fundamental parallelotope with a piecewise linear function defining the boundary. Theoretical results are obtained by studying root lattices. We show how the number of pieces in the boundary function reduces dramatically with folding, from exponential to linear. This translates into a two-layer ReLU network requiring a number of neurons growing exponentially in n to solve the CVP, whereas this complexity becomes polynomial in n for a deep ReLU network., Comment: Submitted to 2019 IEEE International Symposium on Information Theory (ISIT)

A quasi-static flat multiple-antenna channel is considered. We show how real multilevel modulation symbols can be detected via deep neural networks. A multi-plateau sigmoid function is introduced. Then, after showing the DNN architecture for detection, we propose a twin-network neural structure. Batch size and training statistics for efficient learning are investigated. Near-Maximum-Likelihood performance with a relatively reasonable number of parameters is achieved., Comment: To Appear in the Proceedings of the 52nd Annual Asilomar Conference on Signals, Systems, and Computers

Lattice decoders constructed with neural networks are presented. Firstly, we show how the fundamental parallelotope is used as a compact set for the approximation by a neural lattice decoder. Secondly, we introduce the notion of Voronoi-reduced lattice basis. As a consequence, a first optimal neural lattice decoder is built from Boolean equations and the facets of the Voronoi cell. This decoder needs no learning. Finally, we present two neural decoders with learning. It is shown that L1 regularization and {\em a priori} information about the lattice structure lead to a simplification of the model., Comment: 6 pages, 5 figures, 2nd version of the 5-page paper initially submitted to the 2018 Sixth IEEE Global Conference on Signal and Information Processing

High-fidelity, real-time interactive applications are envisioned with the emergence of the Internet of Things (IoT) and tactile Internet by means of ultra-reliable low-latency communications (URLLC). Exploiting time diversity for fulfilling the URLLC requirements in an energy efficient manner is a challenging task due to the nontrivial interplay among packet size, retransmission rounds and delay, and transmit power. In this paper, we study the fundamental energy-latency tradeoff in URLLC systems employing incremental redundancy (IR) hybrid automatic repeat request (HARQ). We cast the average energy minimization problem with a finite blocklength (latency) constraint and feedback delay, which is non-convex. We propose a dynamic programming algorithm for energy efficient IR-HARQ optimization in terms of number of retransmissions, blocklength and power per round. Numerical results show that our IR-HARQ approach could provide around 25\% energy saving compared to one-shot transmission (no HARQ)., Comment: Journal submission

Optical communication systems represent the backbone of modern communication networks. Since their deployment, different fiber technologies have been used to deal with optical fiber impairments such as dispersion-shifted fibers and dispersion-compensation fibers. In recent years, thanks to the introduction of coherent detection based systems, fiber impairments can be mitigated using digital signal processing (DSP) algorithms. Coherent systems are used in the current 100 Gbps wavelength-division multiplexing (WDM) standard technology. They allow the increase of spectral efficiency by using multi-level modulation formats, and are combined with DSP techniques to combat the linear fiber distortions. In addition to linear impairments, the next generation 400 Gbps/1 Tbps WDM systems are also more affected by the fiber nonlinearity due to the Kerr effect. At high input power, the fiber nonlinear effects become more important and their compensation is required to improve the transmission performance. Several approaches have been proposed to deal with the fiber nonlinearity. In this paper, after a brief description of the Kerr-induced nonlinear effects, a survey on the fiber nonlinearity compensation (NLC) techniques is provided. We focus on the well-known NLC techniques and discuss their performance, as well as their implementation and complexity. An extension of the inter-subcarrier nonlinear interference canceler approach is also proposed. A performance evaluation of the well-known NLC techniques and the proposed approach is provided in the context of Nyquist and super-Nyquist superchannel systems., Comment: Accepted in the IEEE Communications Surveys and Tutorials

Consider a set of N agents seeking to solve distributively the minimization problem $\inf_{x} \sum_{n = 1}^N f_n(x)$ where the convex functions $f_n$ are local to the agents. The popular Alternating Direction Method of Multipliers has the potential to handle distributed optimization problems of this kind. We provide a general reformulation of the problem and obtain a class of distributed algorithms which encompass various network architectures. The rate of convergence of our method is considered. It is assumed that the infimum of the problem is reached at a point $x_\star$, the functions $f_n$ are twice differentiable at this point and $\sum \nabla^2 f_n(x_\star) > 0$ in the positive definite ordering of symmetric matrices. With these assumptions, it is shown that the convergence to the consensus $x_\star$ is linear and the exact rate is provided. Application examples where this rate can be optimized with respect to the ADMM free parameter $\rho$ are also given., Comment: 13 pages

Consider a set of networked agents endowed with private cost functions and seeking to find a consensus on the minimizer of the aggregate cost. A new class of random asynchronous distributed optimization methods is introduced. The methods generalize the standard Alternating Direction Method of Multipliers (ADMM) to an asynchronous setting where isolated components of the network are activated in an uncoordinated fashion. The algorithms rely on the introduction of randomized Gauss-Seidel iterations of a Douglas-Rachford operator for finding zeros of a sum of two monotone operators. Convergence to the sought minimizers is provided under mild connectivity conditions. Numerical results sustain our claims., Comment: 6 pages

Distributed estimation of the average value over a Wireless Sensor Network has recently received a lot of attention. Most papers consider single variable sensors and communications with feedback (e.g. peer-to-peer communications). However, in order to use efficiently the broadcast nature of the wireless channel, communications without feedback are advocated. To ensure the convergence in this feedback-free case, the recently-introduced Sum-Weight-like algorithms which rely on two variables at each sensor are a promising solution. In this paper, the convergence towards the consensus over the average of the initial values is analyzed in depth. Furthermore, it is shown that the squared error decreases exponentially with the time. In addition, a powerful algorithm relying on the Sum-Weight structure and taking into account the broadcast nature of the channel is proposed.

In this paper, we propose a resource allocation algorithm for the downlink of sectorized two-dimensional (2-D) OFDMA cellular networks assuming statistical Channel State Information (CSI) and fractional frequency reuse. The proposed algorithm can be implemented in a distributed fashion without the need to any central controlling units. Its performance is analyzed assuming fast fading Rayleigh channels and Gaussian distributed multicell interference. We show that the transmit power of this simple algorithm tends, as the number of users grows to infinity, to the same limit as the minimal power required to satisfy all users' rate requirements i.e., the proposed resource allocation algorithm is asymptotically optimal. As a byproduct of this asymptotic analysis, we characterize a relevant value of the reuse factor that only depends on an average state of the network., Comment: submitted to IEEE Transactions on Wireless Communications

In this work, a new static relaying protocol is introduced for half duplex single-relay networks, and its performance is studied in the context of communications over slow fading wireless channels. The proposed protocol is based on a Decode or Quantize and Forward (DoQF) approach. In slow fading scenarios, two performance metrics are relevant and complementary, namely the outage probability gain and the Diversity-Multiplexing Tradeoff (DMT). First, we analyze the behavior of the outage probability P_o associated with the proposed protocol as the SNR tends to infinity. In this case, we prove that SNR^2 P_o converges to a constant. We refer to this constant as the outage gain and we derive its closed-form expression for a general class of wireless channels that includes the Rayleigh and the Rice channels as particular cases. We furthermore prove that the DoQF protocol has the best achievable outage gain in the wide class of half-duplex static relaying protocols. A method for minimizing the outage gain with respect to the power distribution between the source and the relay, and with respect to the durations of the slots is also provided. Next, we focus on Rayleigh distributed fading channels to derive the DMT associated with the proposed DoQF protocol. Our results show that the DMT of DoQF achieves the 2 by 1 MISO upper-bound for multiplexing gains r < 0.25.

In this pair of papers (Part I and Part II in this issue), we investigate the issue of power control and subcarrier assignment in a sectorized two-cell downlink OFDMA system impaired by multicell interference. As recommended for WiMAX, we assume that the first part of the available bandwidth is likely to be reused by different base stations (and is thus subject to multicell interference) and that the second part of the bandwidth is shared in an orthogonal way between the different base stations (and is thus protected from multicell interference). Although the problem of multicell resource allocation is nonconvex in this scenario, we provide in Part I the general form of the global solution. In particular, the optimal resource allocation turns out to be "binary" in the sense that, except for at most one pivot-user in each cell, any user receives data either in the reused bandwidth or in the protected bandwidth, but not in both. The determination of the optimal resource allocation essentially reduces to the determination of the latter pivot-position.