Search

In most urban areas, traffic congestion is a vexing, complex and growing issue day by day. Reinforcement learning (RL) enables a single decision maker (or an agent) to learn and make optimal actions in an independent manner, while multi-agent reinforcement learning (MARL) enables multiple agents to exchange knowledge, learn, and make optimal joint actions in a collaborative manner. The integration of the newly emerging deep learning and the traditional RL approach has created an advanced technique called deep Q -network (DQN) that has shown promising results in solving high-dimensional and complex problems, including traffic congestion. In this paper, DQN is embedded in traffic signal control to solve traffic congestion issue, which has been plagued with the curse of dimensionality whereby the representation of the operating environment can be highly dimensional and complex when the traditional RL approach is used. Most importantly, this paper proposes multi-agent DQN (MADQN) and investigates its use to further address the curse of dimensionality under traffic network scenarios with high traffic volume and disturbances. To investigate the effectiveness of our proposed scheme, a case study based on an urban area, namely Sunway city in Malaysia, is conducted. We evaluate our scheme via simulation using a traffic network simulator called simulation of urban mobility (SUMO) and a simulation tool called MATLAB. Simulation results show that our proposed scheme reduces the total travel time of the vehicles.

Cognitive radio (CR) is the next-generation wireless communication system that allows unlicensed users (or secondary users, SUs) to explore and exploit the underutilized licensed spectrum (or white spaces) owned by licensed users (or primary users, PUs) in an opportunistic manner. This paper proposes a route selection scheme over a clustered cognitive radio network (CRN) that enables SUs to form clusters, and a SU source node to search for a route to its destination node. An intrinsic characteristic of CRN is the dynamicity of operating environment in which network conditions (i.e., PUs’ activities) change as time goes by. Based on the network conditions, SUs form clusters whose cluster sizes are based on the number of available common channels in a cluster, select a common operating channel for each cluster, and search for a route over a clustered CRN using an artificial intelligence approach called reinforcement learning. Majority of the research related to CRNs has been limited to theoretical and simulation studies, and testbed investigation focusing on physical and data link layers. This investigation is a proof of concept focusing on the network layer of a route selection scheme over a clustered CRN in a universal software radio peripheral (USRP)/ GNU radio platform. Experimental results show that the proposed route selection scheme improves cluster stability by reducing the number of route breakages caused by route switches, and network scalability by reducing the number of clusters in the network without significant deterioration of quality of service, including throughput, packet delivery rate, and end-to-end delay.

Cognitive radio leverages on reinforcement learning (RL) to enhance network security.There is lack of reviews on the application of RL to based security schemes.We cover the challenges, characteristics, performance enhancements, and others. Cognitive radio network (CRN) enables unlicensed users (or secondary users, SUs) to sense for and opportunistically operate in underutilized licensed channels, which are owned by the licensed users (or primary users, PUs). Cognitive radio network (CRN) has been regarded as the next-generation wireless network centered on the application of artificial intelligence, which helps the SUs to learn about, as well as to adaptively and dynamically reconfigure its operating parameters, including the sensing and transmission channels, for network performance enhancement. This motivates the use of artificial intelligence to enhance security schemes for CRNs. Provisioning security in CRNs is challenging since existing techniques, such as entity authentication, are not feasible in the dynamic environment that CRN presents since they require pre-registration. In addition these techniques cannot prevent an authenticated node from acting maliciously. In this article, we advocate the use of reinforcement learning (RL) to achieve optimal or near-optimal solutions for security enhancement through the detection of various malicious nodes and their attacks in CRNs. RL, which is an artificial intelligence technique, has the ability to learn new attacks and to detect previously learned ones. RL has been perceived as a promising approach to enhance the overall security aspect of CRNs. RL, which has been applied to address the dynamic aspect of security schemes in other wireless networks, such as wireless sensor networks and wireless mesh networks can be leveraged to design security schemes in CRNs. We believe that these RL solutions will complement and enhance existing security solutions applied to CRN To the best of our knowledge, this is the first survey article that focuses on the use of RL-based techniques for security enhancement in CRNs.

The large-scale deployment of fifth generation (5G) is expected to produce a massive amount of data with high variability due to ultra-densification and the rapid increase in a heterogeneous range of applications and services (e.g., virtual reality, augmented reality, and driver-less vehicles), and network devices (e.g., smart gadgets and sensors). Clustering organizes network topology by segregating nodes with similar interests or behaviors in a network into logical groups in order to achieve network-level and cluster-level enhancements, particularly cluster stability, load balancing, social awareness, fairness, and quality of service. Clustering has been investigated to support mobile user equipment (UE) in access networks, whereby UEs form clusters themselves and may connect to BSs. In this paper, we present a comprehensive survey of the research work of clustering schemes proposed for various scenarios in 5G networks and highlight various aspects of clustering schemes, including objectives, challenges, metrics, characteristics, performance measures. Furthermore, we present open issues of clustering in 5G.

Cognitive Radio (CR) networks enable unlicensed or Secondary Users (SUs) to sense for and operate in the underutilized spectrum (or white spaces) owned by licensed or Primary Users (PUs) without causing unacceptable interference to the PUs' activities. Clustering, which is a topology management mechanism, organizes nodes into logical groups in order to provide network-wide performance enhancement. Clustering aims to achieve network scalability and stability, as well as to support cooperative tasks, such as channel sensing and channel access, which are essential to CR operations. While clustering has been well investigated in traditional networks such as mobile ad hoc networks, similar investigations in CR networks remain in the infancy stage. New clustering algorithms must be designed to address new challenges associated with the intrinsic characteristics of CR, namely the dynamicity of channel availability that changes with time and location. This article reviews clustering algorithms, and they are characterized by clustering objectives, metrics and the number of hops in each cluster. We also present complexity analysis, performance enhancements achieved by the clustering algorithms, as well as open issues, in order to establish a foundation for further research and to spark new research interests in this area. Clustering organizes nodes into logical groups to provide scalability, stability and cooperative tasks support, and the dynamicity of channel availability in Cognitive Radio networks has brought about challenges to clustering. This article presents an extensive review on various aspects of clustering algorithms in Cognitive Radio networks, including clustering objectives, characteristics, performance enhancements, complexity analysis, and open issues. Of particular focus is clustering metrics and how these metrics have been applied to form clusters in Cognitive Radio networks.Display Omitted We review clustering algorithms for Cognitive Radio networks.We present taxonomy of the attributes of clustering algorithms.We present advantages, challenges, objectives and characteristics of clustering.We compare algorithms in terms of complexity and performance enhancement.We propose open issues for further research.

In wireless networks, context awareness and intelligence are capabilities that enable each host to observe, learn, and respond to its complex and dynamic operating environment in an efficient manner. These capabilities contrast with traditional approaches where each host adheres to a predefined set of rules, and responds accordingly. In recent years, context awareness and intelligence have gained tremendous popularity due to the substantial network-wide performance enhancement they have to offer. In this article, we advocate the use of reinforcement learning (RL) to achieve context awareness and intelligence. The RL approach has been applied in a variety of schemes such as routing, resource management and dynamic channel selection in wireless networks. Examples of wireless networks are mobile ad hoc networks, wireless sensor networks, cellular networks and cognitive radio networks. This article presents an overview of classical RL and three extensions, including events, rules and agent interaction and coordination, to wireless networks. We discuss how several wireless network schemes have been approached using RL to provide network performance enhancement, and also open issues associated with this approach. Throughout the paper, discussions are presented in a tutorial manner, and are related to existing work in order to establish a foundation for further research in this field, specifically, for the improvement of the RL approach in the context of wireless networking, for the improvement of the RL approach through the use of the extensions in existing schemes, as well as for the design and implementation of RL in new schemes.