Search

Integrating sensing and communication is a defining theme for future wireless systems. This is motivated by the promising performance gains, especially as they assist each other, and by the better utilization of the wireless and hardware resources. Realizing these gains in practice, however, is subject to several challenges where leveraging machine learning can provide a potential solution. This article focuses on ten key machine learning roles for joint sensing and communication, sensing-aided communication, and communication-aided sensing systems, explains why and how machine learning can be utilized, and highlights important directions for future research. The article also presents real-world results for some of these machine learning roles based on the large-scale real-world dataset DeepSense 6G, which could be adopted in investigating a wide range of integrated sensing and communication problems., Accepted in IEEE Communications Magazine

This paper considers a cell-free integrated sensing and communication (ISAC) MIMO system, where distributed MIMO access points are jointly serving the communication users and sensing the targets. For this setup, we first develop two baseline approaches that separately design the sensing and communication beamforming vectors, namely communication-prioritized sensing beamforming and sensing-prioritized communication beamforming. Then, we consider the joint sensing and communication (JSC) beamforming design and derive the optimal structure of these JSC beamforming vectors based on a max-min fairness formulation. The results show that the developed JSC beamforming is capable of achieving nearly the same communication signal-to-interference-plus-noise ratio (SINR) that of the communication-prioritized sensing beamforming solutions with almost the same sensing SNR of the sensing-prioritized communication beamforming approaches, yielding a promising strategy for cell-free ISAC MIMO systems.

Realizing the potential gains of large-scale MIMO systems requires the accurate estimation of their channels or the fine adjustment of their narrow beams. This, however, is typically associated with high channel acquisition/beam sweeping overhead that scales with the number of antennas. Machine and deep learning represent promising approaches to overcome these challenges thanks to their powerful ability to learn from prior observations and side information. Training machine and deep learning models, however, requires large-scale datasets that are expensive to collect in deployed systems. To address this challenge, we propose a novel direction that utilizes digital replicas of the physical world to reduce or even eliminate the MIMO channel acquisition overhead. In the proposed digital twin aided communication, 3D models that approximate the real-world communication environment are constructed and accurate ray-tracing is utilized to simulate the site-specific channels. These channels can then be used to aid various communication tasks. Further, we propose to use machine learning to approximate the digital replicas and reduce the ray tracing computational cost. To evaluate the proposed digital twin based approach, we conduct a case study focusing on the position-aided beam prediction task. The results show that a learning model trained solely with the data generated by the digital replica can achieve relatively good performance on the real-world data. Moreover, a small number of real-world data points can quickly achieve near-optimal performance, overcoming the modeling mismatches between the physical and digital worlds and significantly reducing the data acquisition overhead., Comment: The dataset is available on the DeepSense 6G website

This article presents a vision where \textit{real-time} digital twins of the physical wireless environments are continuously updated using multi-modal sensing data from the distributed infrastructure and user devices, and are used to make communication and sensing decisions. This vision is mainly enabled by the advances in precise 3D maps, multi-modal sensing, ray-tracing computations, and machine/deep learning. This article details this vision, explains the different approaches for constructing and utilizing these real-time digital twins, discusses the applications and open problems, and presents a research platform that can be used to investigate various digital twin research directions., The 6G digital twin research platform will be available soon on https://deepverse6g.net/

Millimeter-wave (mmWave) and terahertz (THz) communication systems adopt large antenna arrays to ensure adequate receive signal power. However, adjusting the narrow beams of these antenna arrays typically incurs high beam training overhead that scales with the number of antennas. Recently proposed vision-aided beam prediction solutions, which utilize \textit{raw RGB images} captured at the basestation to predict the optimal beams, have shown initial promising results. However, they still have a considerable computational complexity, limiting their adoption in the real world. To address these challenges, this paper focuses on developing and comparing various approaches that extract lightweight semantic information from the visual data. The results show that the proposed solutions can significantly decrease the computational requirements while achieving similar beam prediction accuracy compared to the previously proposed vision-aided solutions., Comment: Based on the DeepSense dataset https://deepsense6g.net/. arXiv admin note: text overlap with arXiv:2205.12187

Current scene depth estimation approaches mainly rely on optical sensing, which carries privacy concerns and suffers from estimation ambiguity for distant, shiny, and transparent surfaces/objects. Reconfigurable intelligent surfaces (RISs) provide a path for employing a massive number of antennas using low-cost and energy-efficient architectures. This has the potential for realizing RIS-aided wireless sensing with high spatial resolution. In this paper, we propose to employ RIS-aided wireless sensing systems for scene depth estimation. We develop a comprehensive framework for building accurate depth maps using RIS-aided mmWave sensing systems. In this framework, we propose a new RIS interaction codebook capable of creating a sensing grid of reflected beams that meets the desirable characteristics of efficient scene depth map construction. Using the designed codebook, the received signals are processed to build high-resolution depth maps. Simulation results compare the proposed solution against RGB-based approaches and highlight the promise of adopting RIS-aided mmWave sensing in scene depth perception., Submitted to IEEE

Millimeter wave (mmWave) and terahertz (THz) drones have the potential to enable several futuristic applications such as coverage extension, enhanced security monitoring, and disaster management. However, these drones need to deploy large antenna arrays and use narrow directive beams to maintain a sufficient link budget. The large beam training overhead associated with these arrays makes adjusting these narrow beams challenging for highly-mobile drones. To address these challenges, this paper proposes a vision-aided machine learning-based approach that leverages visual data collected from cameras installed on the drones to enable fast and accurate beam prediction. Further, to facilitate the evaluation of the proposed solution, we build a synthetic drone communication dataset consisting of co-existing wireless and visual data. The proposed vision-aided solution achieves a top-$1$ beam prediction accuracy of $\approx 91\%$ and close to $100\%$ top-$3$ accuracy. These results highlight the efficacy of the proposed solution towards enabling highly mobile mmWave/THz drone communication., The mmWave drone dataset and code files will be available soon! arXiv admin note: text overlap with arXiv:2205.12187

Reconfigurable intelligent surfaces (RISs) have attracted increasing interest due to their ability to improve the coverage, reliability, and energy efficiency of millimeter wave (mmWave) communication systems. However, designing the RIS beamforming typically requires large channel estimation or beam training overhead, which degrades the efficiency of these systems. In this paper, we propose to equip the RIS surfaces with visual sensors (cameras) that obtain sensing information about the surroundings and user/basestation locations, guide the RIS beam selection, and reduce the beam training overhead. We develop a machine learning (ML) framework that leverages this visual sensing information to efficiently select the optimal RIS reflection beams that reflect the signals between the basestation and mobile users. To evaluate the developed approach, we build a high-fidelity synthetic dataset that comprises co-existing wireless and visual data. Based on this dataset, the results show that the proposed vision-aided machine learning solution can accurately predict the RIS beams and achieve near-optimal achievable rate while significantly reducing the beam training overhead., 6 pages, 6 figures. The RIS dataset and code files will be available soon!

Orthogonal time frequency space (OTFS) modulation has the potential to enable robust communications in highly-mobile scenarios. Estimating the channels for OTFS systems, however, is associated with high pilot signaling overhead that scales with the maximum delay and Doppler spreads. This becomes particularly challenging for massive MIMO systems where the overhead also scales with the number of antennas. An important observation however is that the delay, Doppler, and angle of departure/arrival information are directly related to the distance, velocity, and direction information of the mobile user and the various scatterers in the environment. With this motivation, we propose to leverage radar sensing to obtain this information about the mobile users and scatterers in the environment and leverage it to aid the OTFS channel estimation in massive MIMO systems. As one approach to realize our vision, this paper formulates the OTFS channel estimation problem in massive MIMO systems as a sparse recovery problem and utilizes the radar sensing information to determine the support (locations of the non-zero delay-Doppler taps). The proposed radar sensing aided sparse recovery algorithm is evaluated based on an accurate 3D ray-tracing framework with co-existing radar and communication data. The results show that the developed sensing-aided solution consistently outperforms the standard sparse recovery algorithms (that do not leverage radar sensing data) and leads to a significant reduction in the pilot overhead, which highlights a promising direction for OTFS based massive MIMO systems., submitted to IEEE

Employing large antenna arrays is a key characteristic of millimeter wave (mmWave) and terahertz communication systems. Due to the hardware constraints and the lack of channel knowledge, codebook based beamforming/combining is normally adopted to achieve the desired array gain. However, most of the existing codebooks focus only on improving the gain of their target user, without taking interference into account. This can incur critical performance degradation in dense networks. In this paper, we propose a sample-efficient online reinforcement learning based beam pattern design algorithm that learns how to shape the beam pattern to null the interfering directions. The proposed approach does not require any explicit channel knowledge or any coordination with the interferers. Simulation results show that the developed solution is capable of learning well-shaped beam patterns that significantly suppress the interference while sacrificing tolerable beamforming/combing gain from the desired user. Furthermore, a hardware proof-of-concept prototype based on mmWave phased arrays is built and used to implement and evaluate the developed online beam learning solutions in realistic scenarios. The learned beam patterns, measured in an anechoic chamber, show the performance gains of the developed framework and highlight a promising machine learning based beam/codebook optimization direction for mmWave and terahertz systems., Submitted to IEEE

Millimeter-wave (mmWave) communication systems rely on narrow beams for achieving sufficient receive signal power. Adjusting these beams is typically associated with large training overhead, which becomes particularly critical for highly-mobile applications. Intuitively, since optimal beam selection can benefit from the knowledge of the positions of communication terminals, there has been increasing interest in leveraging position data to reduce the overhead in mmWave beam prediction. Prior work, however, studied this problem using only synthetic data that generally does not accurately represent real-world measurements. In this paper, we investigate position-aided beam prediction using a real-world large-scale dataset to derive insights into precisely how much overhead can be saved in practice. Furthermore, we analyze which machine learning algorithms perform best, what factors degrade inference performance in real data, and which machine learning metrics are more meaningful in capturing the actual communication system performance., Comment: Submitted to IEEE. Datasets and code files are available on the DeepSense website: https://deepsense6g.net/

Massive multiple-input multiple-output (MIMO) system is promising in providing unprecedentedly high data rate. To achieve its full potential, the transceiver needs complete channel state information (CSI) to perform transmit/receive precoding/combining. This requirement, however, is challenging in the practical systems due to the unavoidable processing and feedback delays, which oftentimes degrades the performance to a great extent, especially in the high mobility scenarios. In this paper, we develop a deep learning based channel prediction framework that proactively predicts the downlink channel state information based on the past observed channel sequence. In its core, the model adopts a 3-D convolutional neural network (CNN) based architecture to efficiently learn the temporal, spatial and frequency correlations of downlink channel samples, based on which accurate channel prediction can be performed. Simulation results highlight the potential of the developed learning model in extracting information and predicting future downlink channels directly from the observed past channel sequence, which significantly improves the performance compared to the sample-and-hold approach, and mitigates the impact of the dynamic communication environment.

Can reconfigurable intelligent surfaces (RISs) operate in a standalone mode that is completely transparent to the 3GPP 5G initial access process? Realizing that may greatly simplify the deployment and operation of these surfaces and reduce the infrastructure control overhead. This paper investigates the feasibility of building standalone/transparent RIS systems and shows that one key challenge lies in determining the user equipment (UE)-side RIS beam reflection direction. To address this challenge, we propose to equip the RISs with multi-modal sensing capabilities (e.g., using wireless and visual sensors) that enable them to develop some perception of the surrounding environment and the mobile users. Based on that, we develop a machine learning framework that leverages the wireless and visual sensors at the RIS to select the optimal beams between the base station (BS) and users and enable 5G standalone/transparent RIS operation. Using a high-fidelity synthetic dataset with co-existing wireless and visual data, we extensively evaluate the performance of the proposed framework. Experimental results demonstrate that the proposed approach can accurately predict the BS and UE-side candidate beams, and that the standalone RIS beam selection solution is capable of realizing near-optimal achievable rates with significantly reduced beam training overhead., Comment: The RIS dataset and script files will be available soon. arXiv admin note: text overlap with arXiv:2211.07563

Millimeter-wave (mmWave) and terahertz (THz) communications require beamforming to acquire adequate receive signal-to-noise ratio (SNR). To find the optimal beam, current beam management solutions perform beam training over a large number of beams in pre-defined codebooks. The beam training overhead increases the access latency and can become infeasible for high-mobility applications. To reduce or even eliminate this beam training overhead, we propose to utilize the visual data, captured for example by cameras at the base stations, to guide the beam tracking/refining process. We propose a machine learning (ML) framework, based on an encoder-decoder architecture, that can predict the future beams using the previously obtained visual sensing information. Our proposed approach is evaluated on a large-scale real-world dataset, where it achieves an accuracy of $64.47\%$ (and a normalized receive power of $97.66\%$) in predicting the future beam. This is achieved while requiring less than $1\%$ of the beam training overhead of a corresponding baseline solution that uses a sequence of previous beams to predict the future one. This high performance and low overhead obtained on the real-world dataset demonstrate the potential of the proposed vision-aided beam tracking approach in real-world applications., Submitted to IEEE. The dataset and code files will be available on the DeepSense 6G website http://deepsense6g.net/

Reconfigurable intelligent surfaces (RIS) are expected to play an important role in future wireless communication systems. These surfaces typically rely on their reflection beamforming codebooks to reflect and focus the signal on the target receivers. Prior work has mainly considered pre-defined RIS beamsteering codebooks that do not adapt to the environment and hardware and lead to large beam training overhead. In this work, a novel deep reinforcement learning based framework is developed to efficiently construct the RIS reflection beam codebook. This framework adopts a multi-level design approach that transfers the learning between the multiple RIS subarrays, which speeds up the learning convergence and highly reduces the computational complexity for large RIS surfaces. The proposed approach is generic for co-located/distributed RIS surfaces with arbitrary array geometries and with stationary/non-stationary channels. Further, the developed solution does not require explicitly channel knowledge and adapts the codebook beams to the surrounding environment, user distribution, and hardware characteristics. Simulation results show that the proposed learning framework can learn optimized interaction codebooks within reasonable iterations. Besides, with only 6 beams, the learned codebook outperforms a 256-beam DFT codebook, which significantly reduces the beam training overhead., Submitted to IEEE Wireless Communications Letters, 5 pages, 3 figures

Enabling highly-mobile millimeter wave (mmWave) and terahertz (THz) wireless communication applications requires overcoming the critical challenges associated with the large antenna arrays deployed at these systems. In particular, adjusting the narrow beams of these antenna arrays typically incurs high beam training overhead that scales with the number of antennas. To address these challenges, this paper proposes a multi-modal machine learning based approach that leverages positional and visual (camera) data collected from the wireless communication environment for fast beam prediction. The developed framework has been tested on a real-world vehicular dataset comprising practical GPS, camera, and mmWave beam training data. The results show the proposed approach achieves more than $\approx$ 75\% top-1 beam prediction accuracy and close to 100\% top-3 beam prediction accuracy in realistic communication scenarios., Comment: Dataset and code files will be available on the DeepSense 6G website http://deepsense6g.net/

Vision-aided wireless communication is motivated by the recent advances in deep learning and computer vision as well as the increasing dependence on line-of-sight links in millimeter wave (mmWave) and terahertz systems. By leveraging vision, this new research direction enables an interesting set of new capabilities such as vision-aided mmWave beam and blockage prediction, proactive hand-off, and resource allocation among others. These capabilities have the potential of reliably supporting highly-mobile applications such as vehicular/drone communications and wireless virtual/augmented reality in mmWave and terahertz systems. Investigating these interesting applications, however, requires the development of special dataset and machine learning tasks. Based on the Vision-Wireless (ViWi) dataset generation framework [1], this paper develops an advanced and realistic scenario/dataset that features multiple base stations, mobile users, and rich dynamics. Enabled by this dataset, the paper defines the vision-wireless mmWave beam tracking task (ViWi-BT) and proposes a baseline solution that can provide an initial benchmark for the future ViWi-BT algorithms., The ViWi-BT Challenge at ICC 2020 - https://www.viwi-dataset.net/viwi-bt.html

Employing low-resolution analog-to-digital converters (ADCs) for millimeter wave receivers with large antenna arrays provides opportunity to efficiently reduce power consumption of the receiver. Reducing ADC resolution, however, results in performance degradation due to non-negligible quantization error. In addition, the large number of radio frequency (RF) chains is still not desirable. Accordingly, conventional low-resolution ADC systems require more efficient designs to minimize the cost and complexity while maximizing performance. In this article, we discuss advanced low-resolution ADC receiver architectures that further improve the spectral and energy efficiency tradeoff. To reduce both the numbers of RF chains and ADC bits, hybrid analog-and-digital beamforming is jointly considered with low-resolution ADCs. We explore the challenges in designing such receivers and present key insights on how the advanced architectures overcome such challenges. As an alternative low-resolution ADC receiver, we also introduce receivers with learning-based detection. The receiver does not require explicit channel estimation, thereby is suitable for one-bit ADC systems. Finally, future challenges and research issues are discussed., Comment: Accepted to IEEE Communications Magazine (7 pages, 5 figures)

This paper proposes a novel relay-aided intelligent reconfigurable surface (IRS) architecture for future wireless communication systems. The proposed architecture, which consists of two side-by-side intelligent surfaces connected via a full-duplex relay, has the potential of achieving the promising gains of intelligent surfaces while requiring much smaller numbers of reflecting elements. Consequently, the proposed IRS architecture needs significantly less channel estimation and beam training overhead and provides higher robustness compared to classical IRS approaches. Further, thanks to dividing the IRS reflection process over two surfaces, the position and orientation of these surfaces can be optimized to extend the wireless communication coverage and enhance the system performance. In this paper, the achievable rates and required numbers of elements using the proposed relay-aided IRS architecture are first analytically characterized and then evaluated using numerical simulations. The results show that the proposed architecture can achieve the data rate targets with much smaller numbers of elements compared to typical IRS solutions, which highlights a promising path towards the practical deployment of these intelligent surfaces., Comment: 8 pages; 7 figures

This paper investigates a novel research direction that leverages vision to help overcome the critical wireless communication challenges. In particular, this paper considers millimeter wave (mmWave) communication systems, which are principal components of 5G and beyond. These systems face two important challenges: (i) the large training overhead associated with selecting the optimal beam and (ii) the reliability challenge due to the high sensitivity to link blockages. Interestingly, most of the devices that employ mmWave arrays will likely also use cameras, such as 5G phones, self-driving vehicles, and virtual/augmented reality headsets. Therefore, we investigate the potential gains of employing cameras at the mmWave base stations and leveraging their visual data to help overcome the beam selection and blockage prediction challenges. To do that, this paper exploits computer vision and deep learning tools to predict mmWave beams and blockages directly from the camera RGB images and the sub-6GHz channels. The experimental results reveal interesting insights into the effectiveness of such solutions. For example, the deep learning model is capable of achieving over 90\% beam prediction accuracy, which only requires snapping a shot of the scene and zero overhead., submitted to IEEE Vehicular Technology Conference, Oct. 2019. The datasets/codes are available at https://www.viwi-dataset.net

Artificial intelligence (AI) based downlink channel state information (CSI) prediction for frequency division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems has attracted growing attention recently. However, existing works focus on the downlink CSI prediction for the users under a given environment and is hard to adapt to users in new environment especially when labeled data is limited. To address this issue, we formulate the downlink channel prediction as a deep transfer learning (DTL) problem, where each learning task aims to predict the downlink CSI from the uplink CSI for one single environment. Specifically, we develop the direct-transfer algorithm based on the fully-connected neural network architecture, where the network is trained on the data from all previous environments in the manner of classical deep learning and is then fine-tuned for new environments. To further improve the transfer efficiency, we propose the meta-learning algorithm that trains the network by alternating inner-task and across-task updates and then adapts to a new environment with a small number of labeled data. Simulation results show that the direct-transfer algorithm achieves better performance than the deep learning algorithm, which implies that the transfer learning benefits the downlink channel prediction in new environments. Moreover, the meta-learning algorithm significantly outperforms the direct-transfer algorithm in terms of both prediction accuracy and stability, which validates its effectiveness and superiority., Comment: Accepted by IEEE Transactions on Communications

Machine learning tools are finding interesting applications in millimeter wave (mmWave) and massive MIMO systems. This is mainly thanks to their powerful capabilities in learning unknown models and tackling hard optimization problems. To advance the machine learning research in mmWave/massive MIMO, however, there is a need for a common dataset. This dataset can be used to evaluate the developed algorithms, reproduce the results, set benchmarks, and compare the different solutions. In this work, we introduce the DeepMIMO dataset, which is a generic dataset for mmWave/massive MIMO channels. The DeepMIMO dataset generation framework has two important features. First, the DeepMIMO channels are constructed based on accurate ray-tracing data obtained from Remcom Wireless InSite. The DeepMIMO channels, therefore, capture the dependence on the environment geometry/materials and transmitter/receiver locations, which is essential for several machine learning applications. Second, the DeepMIMO dataset is generic/parameterized as the researcher can adjust a set of system and channel parameters to tailor the generated DeepMIMO dataset for the target machine learning application. The DeepMIMO dataset can then be completely defined by the (i) the adopted ray-tracing scenario and (ii) the set of parameters, which enables the accurate definition and reproduction of the dataset. In this paper, an example DeepMIMO dataset is described based on an outdoor ray-tracing scenario of 18 base stations and more than one million users. The paper also shows how this dataset can be used in an example deep learning application of mmWave beam prediction., Comment: to appear in Proc. of Information Theory and Applications Workshop (ITA), Feb., 2019

The sensitivity of millimeter wave (mmWave) signals to blockages is a fundamental challenge for mobile mmWave communication systems. The sudden blockage of the line-of-sight (LOS) link between the base station and the mobile user normally leads to disconnecting the communication session, which highly impacts the system reliability. Further, reconnecting the user to another LOS base station incurs high beam training overhead and critical latency problems. In this paper, we leverage machine learning tools and propose a novel solution for these reliability and latency challenges in mmWave MIMO systems. In the developed solution, the base stations learn how to predict that a certain link will experience blockage in the next few time frames using their past observations of adopted beamforming vectors. This allows the serving base station to proactively hand-over the user to another base station with a highly probable LOS link. Simulation results show that the developed deep learning based strategy successfully predicts blockage/hand-off in close to 95% of the times. This reduces the probability of communication session disconnection, which ensures high reliability and low latency in mobile mmWave systems., Comment: submitted to IEEE GlobalSIP 2018 (Invited paper)

Hybrid analog and digital precoding allows millimeter wave (mmWave) systems to achieve both array and multiplexing gain. The design of the hybrid precoders and combiners, though, is usually based on knowledge of the channel. Prior work on mmWave channel estimation with hybrid architectures focused on narrowband channels. Since mmWave systems will be wideband with frequency selectivity, it is vital to develop channel estimation solutions for hybrid architectures based wideband mmWave systems. In this paper, we develop a sparse formulation and compressed sensing based solutions for the wideband mmWave channel estimation problem for hybrid architectures. First, we leverage the sparse structure of the frequency selective mmWave channels and formulate the channel estimation problem as a sparse recovery in both time and frequency domains. Then, we propose explicit channel estimation techniques for purely time or frequency domains and for combined time/frequency domains. Our solutions are suitable for both SC-FDE and OFDM systems. Simulation results show that the proposed solutions achieve good channel estimation quality, while requiring small training overhead. Leveraging the hybrid architecture at the transceivers gives further improvement in estimation error performance and achievable rates., Comment: 31 pages, 13 figures