Search

In this technical report, we present Falcon Mamba 7B, a new base large language model based on the novel Mamba architecture. Falcon Mamba 7B is trained on 5.8 trillion tokens with carefully selected data mixtures. As a pure Mamba-based model, Falcon Mamba 7B surpasses leading open-weight models based on Transformers, such as Mistral 7B, Llama3.1 8B, and Falcon2 11B. It is on par with Gemma 7B and outperforms models with different architecture designs, such as RecurrentGemma 9B and RWKV-v6 Finch 7B/14B. Currently, Falcon Mamba 7B is the best-performing Mamba model in the literature at this scale, surpassing both existing Mamba and hybrid Mamba-Transformer models, according to the Open LLM Leaderboard. Due to its architecture, Falcon Mamba 7B is significantly faster at inference and requires substantially less memory for long sequence generation. Despite recent studies suggesting that hybrid Mamba-Transformer models outperform pure architecture designs, we demonstrate that even the pure Mamba design can achieve similar, or even superior results compared to the Transformer and hybrid designs. We make the weights of our implementation of Falcon Mamba 7B publicly available on https://huggingface.co/tiiuae/falcon-mamba-7b, under a permissive license.

Edge Computing (EC) allows users to access computing resources at the network frontier, which paves the way for deploying delay-sensitive applications such as Mobile Augmented Reality (MAR). Under the EC paradigm, MAR users connect to the EC server, open sessions and send continuously frames to be processed. The EC server sends back virtual information to enhance the human perception of the world by merging it with the real environment. Resource allocation arises as a critical challenge when several MAR Service Providers (SPs) compete for limited resources at the edge of the network. In this paper, we consider EC in a multi-tenant environment where the resource owner, i.e., the Network Operator (NO), virtualizes the resources and lets SPs run their services using the allocated slice of resources. Indeed, for MAR applications, we focus on two specific resources: CPU and RAM, deployed in some edge node, e.g., a central office. We study the decision of the NO about how to partition these resources among several SPs. We model the arrival and service dynamics of users belonging to different SPs using Erlang queuing model and show that under perfect information, the interaction between the NO and SPs can be formulated as a sub-modular maximization problem under multiple Knapsack constraints. To solve the problem, we use an approximation algorithm, guaranteeing a bounded gap with respect to the optimal theoretical solution. Our numerical results show that the proposed algorithm outperforms baseline proportional allocation in terms of the number of sessions accommodated at the edge for each SP., Comment: 6 pages, 4 figures, IEEE ICC 2023

Reconfigurable Intelligent Surfaces (RIS) are considered a key enabler to achieve the vision of Smart Radio Environments, where the propagation environment can be programmed and controlled to enhance the efficiency of wireless systems. These surfaces correspond to planar sheets comprising a large number of small and low-cost reflecting elements whose parameters are adaptively selected with a programmable controller. Hence, by optimizing these coefficients, the information signals can be directed in a customized fashion. On the other hand, the initial access procedure used in 5G is beam sweeping, where the base station sequentially changes the active beam direction in order to scan all users in the cell. This conventional protocol results in an initial access latency. The aim of this paper is to minimize this delay by optimizing the activated beams in each timeslot, while leveraging the presence of the RIS in the network. The problem is formulated as a hard optimization problem. We propose an efficient solution based on jointly alternating optimization and Semi Definite Relaxation (SDR) techniques. Numerical results are provided to assess the superiority of our scheme as compared to conventional beam sweeping.

Reconfigurable Intelligent Surfaces (RIS) have recently emerged as a hot research topic, being widely advocated as a candidate technology for next generation wireless communications. These surfaces passively alter the behavior of propagation environments enhancing the performance of wireless communication systems. In this paper, we study the use of RIS in cell-free multiple-input multiple-output (MIMO) setting where distributed service antennas, called Access Points (APs), simultaneously serve the users in the network. While most existing works focus on the physical layer improvements RIS carry, less attention has been paid to the impact of dynamic arrivals and departures of the users. In such a case, ensuring the stability of the network is the main goal. For that, we propose an optimization framework of the phase shifts, for which we derived a low-complexity solution. We then provide a theoretical analysis of the network stability and show that our framework stabilizes the network whenever it is possible. We also prove that a low complexity solution of our framework stabilizes a guaranteed fraction (higher than 78.5%) of the stability region. We provide also numerical results that corroborate the theoretical claims.

We propose in this paper a coinvestment plan between several stakeholders of different types, namely a physical network owner, operating network nodes, e.g. a network operator or a tower company, and a set of service providers willing to use these resources to provide services as video streaming, augmented reality, autonomous driving assistance, etc. One such scenario is that of deployment of Edge Computing resources. Indeed, although the latter technology is ready, the high Capital Expenditure (CAPEX) cost of such resources is the barrier to its deployment. For this reason, a solid economical framework to guide the investment and the returns of the stakeholders is key to solve this issue. We formalize the coinvestment framework using coalitional game theory. We provide a solution to calculate how to divide the profits and costs among the stakeholders, taking into account their characteristics: traffic load, revenues, utility function. We prove that it is always possible to form the grand coalition composed of all the stakeholders, by showing that our game is convex. We derive the payoff of the stakeholders using the Shapley value concept, and elaborate on some properties of our game. We show our solution in simulation.

We consider in this work Edge Computing (EC) in a multi-tenant environment: the resource owner, i.e., the Network Operator (NO), virtualizes the resources and lets third party Service Providers (SPs - tenants) run their services, which can be diverse and with heterogeneous requirements. Due to confidentiality guarantees, the NO cannot observe the nature of the traffic of SPs, which is encrypted. This makes resource allocation decisions challenging, since they must be taken based solely on observed monitoring information. We focus on one specific resource, i.e., cache space, deployed in some edge node, e.g., a base station. We study the decision of the NO about how to partition cache among several SPs in order to minimize the upstream traffic. Our goal is to optimize cache allocation using purely data-driven, model-free Reinforcement Learning (RL). Differently from most applications of RL, in which the decision policy is learned offline on a simulator, we assume no previous knowledge is available to build such a simulator. We thus apply RL in an \emph{online} fashion, i.e., the policy is learned by directly perturbing the actual system and monitoring how its performance changes. Since perturbations generate spurious traffic, we also limit them. We show in simulation that our method rapidly converges toward the theoretical optimum, we study its fairness, its sensitivity to several scenario characteristics and compare it with a method from the state-of-the-art., Comment: 6 pages, 7 figures, accepted in IEEE ICC 2022 (Seoul, South Korea)

We study in this paper optimal control strategy for Advanced Sleep Modes (ASM) in 5G networks. ASM correspond to different levels of sleep modes ranging from deactivation of some components of the base station for several micro-seconds to switching off of almost all of them for one second or more. ASMs are made possible in 5G networks thanks to the definition of so-called lean carrier radio access which allows for configurable signaling periodicities. We model such a system using Markov Decision Processes (MDP) and find optimal sleep policy in terms of a trade-off between saved power consumption versus additional incurred delay for user traffic which has to wait for the network components to be woken-up and serve it. Eventually, for the system not to oscillate between sleep levels, we add a switching component in the cost function and show its impact on the energy reduction versus delay trade-off., Comment: The 18th IEEE International Symposium on Network Computing and Applications (NCA 2019) 26-28 September 2019 Cambridge, MA USA

We propose a setting for two-phase opinion dynamics in social networks, where a node's final opinion in the first phase acts as its initial biased opinion in the second phase. In this setting, we study the problem of two camps aiming to maximize adoption of their respective opinions, by strategically investing on nodes in the two phases. A node's initial opinion in the second phase naturally plays a key role in determining the final opinion of that node, and hence also of other nodes in the network due to its influence on them. More importantly, this bias also determines the effectiveness of a camp's investment on that node in the second phase. To formalize this two-phase investment setting, we propose an extension of Friedkin-Johnsen model, and hence formulate the utility functions of the camps. There is a tradeoff while splitting the budget between the two phases. A lower investment in the first phase results in worse initial biases for the second phase, while a higher investment spares a lower available budget for the second phase. We first analyze the non-competitive case where only one camp invests, for which we present a polynomial time algorithm for determining an optimal way to split the camp's budget between the two phases. We then analyze the case of competing camps, where we show the existence of Nash equilibrium and that it can be computed in polynomial time under reasonable assumptions. We conclude our study with simulations on real-world network datasets, in order to quantify the effects of the initial biases and the weightage attributed by nodes to their initial biases, as well as that of a camp deviating from its equilibrium strategy. Our main conclusion is that, if nodes attribute high weightage to their initial biases, it is advantageous to have a high investment in the first phase, so as to effectively influence the biases to be harnessed in the second phase.

We study a stochastic game framework with dynamic set of players, for modeling and analyzing their computational investment strategies in distributed computing. Players obtain a certain reward for solving the problem or for providing their computational resources, while incur a certain cost based on the invested time and computational power. We first study a scenario where the reward is offered for solving the problem, such as in blockchain mining. We show that, in Markov perfect equilibrium, players with cost parameters exceeding a certain threshold, do not invest; while those with cost parameters less than this threshold, invest maximal power. Here, players need not know the system state. We then consider a scenario where the reward is offered for contributing to the computational power of a common central entity, such as in volunteer computing. Here, in Markov perfect equilibrium, only players with cost parameters in a relatively low range in a given state, invest. For the case where players are homogeneous, they invest proportionally to the 'reward to cost' ratio. For both the scenarios, we study the effects of players' arrival and departure rates on their utilities using simulations and provide additional insights.

We study the problem of optimally investing in nodes of a social network in a competitive setting, where two camps aim to maximize adoption of their opinions by the population. In particular, we consider the possibility of campaigning in multiple phases, where the final opinion of a node in a phase acts as its initial biased opinion for the following phase. Using an extension of the popular DeGroot-Friedkin model, we formulate the utility functions of the camps, and show that they involve what can be interpreted as multiphase Katz centrality. Focusing on two phases, we analytically derive Nash equilibrium investment strategies, and the extent of loss that a camp would incur if it acted myopically. Our simulation study affirms that nodes attributing higher weightage to initial biases necessitate higher investment in the first phase, so as to influence these biases for the terminal phase. We then study the setting in which a camp's influence on a node depends on its initial bias. For single camp, we present a polynomial time algorithm for determining an optimal way to split the budget between the two phases. For competing camps, we show the existence of Nash equilibria under reasonable assumptions, and that they can be computed in polynomial time.