Search

Deep reinforcement learning (DRL) has been shown to be successful in many application domains. Combining recurrent neural networks (RNNs) and DRL further enables DRL to be applicable in non-Markovian environments by capturing temporal information. However, training of both DRL and RNNs is known to be challenging requiring a large amount of training data to achieve convergence. In many targeted applications, such as those used in the fifth generation (5G) cellular communication, the environment is highly dynamic while the available training data is very limited. Therefore, it is extremely important to develop DRL strategies that are capable of capturing the temporal correlation of the dynamic environment requiring limited training overhead. In this paper, we introduce the deep echo state Q-network (DEQN) that can adapt to the highly dynamic environment in a short period of time with limited training data. We evaluate the performance of the introduced DEQN method under the dynamic spectrum sharing (DSS) scenario, which is a promising technology in 5G and future 6G networks to increase the spectrum utilization. Compared to conventional spectrum management policy that grants a fixed spectrum band to a single system for exclusive access, DSS allows the secondary system to share the spectrum with the primary system. Our work sheds light on the application of an efficient DRL framework in highly dynamic environments with limited available training data., This work is accepted in IEEE Transactions on Neural Networks and Learning Systems

Current studies that Our records indicate that Hao-Hsuan Chang is a Graduate Student Member of the IEEE. Please verify. Our records indicate that Jonathan D. Ashdown is a Member of the IEEE. Please verify. apply reinforcement learning (RL) to dynamic spectrum access (DSA) problems in wireless communications systems mainly focus on model-free RL (MFRL). However, in practice, MFRL requires a large number of samples to achieve good performance making it impractical in real-time applications such as DSA. Combining model-free and model-based RL can potentially reduce the sample complexity while achieving a similar level of performance as MFRL as long as the learned model is accurate enough. However, in a complex environment, the learned model is never perfect. In this article, we combine model-free and model-based RL, and introduce an algorithm that can work with an imperfectly learned model to accelerate the MFRL. Results show our algorithm achieves higher sample efficiency than the standard MFRL algorithm and the Dyna algorithm (a standard algorithm integrating model-based RL and MFRL) with much lower computation complexity than the Dyna algorithm. For the extreme case where the learned model is highly inaccurate, the Dyna algorithm performs even worse than the MFRL algorithm while our algorithm can still outperform the MFRL algorithm.

In this letter we apply deep learning tools to conduct channel estimation for an orthogonal frequency division multiplexing (OFDM) system based on downlink pilots. To be specific, a residual learning based deep neural network specifically designed for channel estimation is introduced. Due to the compact network size as well as the underlying network architecture, the computation cost can be greatly reduced. Furthermore, this residual network architecture is compatible with any downlink pilot patterns making it compatible for modern wireless systems. The estimation error of the introduced residual learning approach is evaluated under 3rd Generation Partnership Project (3GPP) channel models. It outperforms other deep learning based estimation method with comparable to minimum mean square error (MMSE) estimation performance.

Device-to-device (D2D) communications provide efficient ways to increase spectrum utilization ratio with reduced power consumption for proximity wireless applications. In this paper, we investigate resource allocation strategies for D2D communications underlaying cellular networks. To be specific, we study the centralized resource allocation algorithm for controlling transmit powers of the underlying D2D pairs in order to maximize the weighted sum-rate while guaranteeing the quality of service (QoS) requirements for both D2D pairs and cellular users (CUs). A novel DC (difference of convex function) programming-based method, called alternative DC (ADC) programming, is introduced to effectively solve this complicated resource allocation problem. Through updating each D2D pair's power alternatively, the QoS requirement for each D2D pair can be solvable and incorporated systematically to the introduced ADC programming framework. The simulation results show that the introduced ADC programming achieves the highest weighted sum-rate compared to the state-of-the-art methods while ensuring that the QoS of each D2D pair and CU are satisfied. Air Force Research Laboratory (AFRL) through the National Spectrum Consortium [NSC-17-7030, 88ABW-2019-1851] Published version This material is based upon work funded by Air Force Research Laboratory (AFRL) through the National Spectrum Consortium under contract number NSC-17-7030. Approved for public release (reference number: 88ABW-2019-1851).

Dynamic spectrum access (DSA) is a promising technology to increase the spectrum efficiency of Internet of Things (IoT) networks, where the traffic demand grows up dramatically recently. In this paper, an intelligent DSA-assisted IoT network is introduced, where we investigate the spectrum sensing through neuromorphic computing (NC) and spectrum access through genetic algorithm (GA)-based power allocation. To be specific, we apply the NC's unconventional computing architectures that exploit and harness the intrinsic dynamics for computation, and thus provide increased functionality with better spectrum sensing performance requiring significantly lower size, weight, and power budgets. Furthermore, we design a GA algorithm to intelligently search the desirable transmission power for multiple IoT devices sharing the same channel to enhance the capacity of the highly dynamic DSA-assisted IoT network. Extensive simulation results have demonstrated the benefits of NC and GA compared to other baseline algorithms and methodologies.

Power control plays an important role in improving the system throughput in communication system since co-channel interference is a major limitation to the system throughput. The power control problem of maximizing the system throughput in the multiuser and multichannel communication system is a highly complicated nonconvex problem since user are interfered with one another if operating in the same wireless channel. We reformulate the nonconvex objective function of this problem as a difference of two convex functions, which is called DC (difference of convex function) programming. To reduce the computation complexity in the high dimensional space, we introduce an alternative power allocation scheme to search in the low dimensional space, where each user updates its power sequentially. A global optimal power allocation is found by utilizing the branch-and- bound algorithm for each user while taking other users' power allocation as constant value. Furthermore, we incorporate each user's maximum power and minimum data rate constraint into the optimization framework. We found that the minimum data rate constraint of each user can be turned into multiple linear inequalities and then be added to the DC programming optimization framework. The simulation results show that our introduced method achieves the highest sum data rate compared to the state-of-the-art methods, including iterative water filling and geometric programming.

Decoration of CuInS2 (CIS) quantum dots (QDs) on ZnO nanowires (NWs) with an interlayer of In2S3 as photoelectrode has been successfully fabricated on FTO via the simple solution routes for photoelectrochemical (PEC) application. Scanning electron microscopy, transmission electron microscopy, and X-ray diffraction are utilized to systematically analyze the morphology and structure of the CIS QD/In2S3/ZnO NWs heterostructure. The composition of this multilayer heterostructure and the removal of QD ligands by a thermal process are confirmed by X-ray photoelectron spectra. In comparison with CIS QDs/ZnO NWs, the CIS QD/In2S3/ZnO heterostructural photoelectrode displays an efficient charge separation and carrier transport path for photocurrent up to 2.4 mA·cm–2 that is competitive with other Cd- and Pb-free QD-based materials. In addition, Mott–Schottky analysis demonstrates the negative shift of the flat band in the CIS QD/In2S3/ZnO, which benefits the early onset potential. Significantly, this hierarchical ...

Since the conventional design strategies for interplant heat integration usually focused upon minimization of the overall utility cost, the optimal solutions may not be implementable due to the additional need to distribute the financial benefits “fairly.” To resolve this profit sharing issue, a Nash-equilibrium constrained optimization strategy has already been developed to sequentially synthesize heat exchanger networks (HENs) that facilitate direct heat transfers across plant boundaries. Although this available approach is thermodynamically viable, the resulting network may be highly coupled and therefore inoperable. To address the operability issues in any multi-plant HEN, the present study aims to modify the aforementioned strategy by considering only indirect interplant heat-exchange options. Two separate sets of mathematical programming models are developed in this work for generating the total-site heat integration schemes with the available utilities and an extra intermediate fluid, respectively. The negotiation powers of the participating plants are also considered for reasonably distributing the utility cost savings and also shouldering the capital cost hikes. Finally, extensive case studies are presented to demonstrate the effectiveness of the proposed procedures and to compare the pros and cons of these two indirect heat-exchange alternatives.

In this work, we propose to establish a mobile edge computing (MEC) network that considers computation, caching and communication jointly. Depending on the demanding categories, users in the network are partitioned into computation-driven and caching-driven users, both of which need memory resource to improve their quality of experiences (QoEs). Thus, a memory resource allocation problem is aroused to maximize the performance of the whole network. Due to the fact that the users' characterization plays an important role to the resource allocation scheme and with the help of machine learning techniques, we propose to study and predict the users' patterns by distributed learning methods which take the heterogeneity of base station type and users' mobility, etc into consideration. The proposed machine learning based distributed MEC system can maximize the efficiency of the network by optimizing the resource allocation scheme and perfectly predicting users' pattern.

Based on mathematics and engineering point of view, we aim to explore the establishment of model structure, and thus calculate the benefits and risks of a financial product. Specifically, we exploit a large amount of market data of futures options so as to address two issues in this work. The first issue is to discover an appropriate product choice and timing for profitable trading. Without loss of generality, we investigate the effectiveness of several trading constraints and technical indicators by scrutinizing and back testing with the long-term market data. The second issue is to use the spread strategies for risk control when being an options seller. Note that a spread position is constituted where one buys an option and sells another option against it. In general, we develop a scheme to simulate different trading strategies and thus identify some simple but profitable strategies. Experimental studies show that our strategies yield good profit in the TAIFEX market during 2009 to 2018.

Reservoir computing (RC) is a special recurrent neural network which consists of a fixed high dimensional feature mapping and trained readout weights. In this paper, we introduce a new RC structure for multiple-input, multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) symbol detection, namely windowed echo state network (WESN). The theoretical analysis shows that adding buffers in input layers can bring an enhanced short-term memory (STM) to the underlying neural network. Furthermore, a unified training framework is developed for the WESN MIMO-OFDM symbol detector using both comb and scattered pilot patterns that are compatible with the structure adopted in 3GPP LTE/LTE-Advanced systems. Complexity analysis suggests the advantages of WESN based symbol detector over state-of-the-art symbol detectors such as the linear minimum mean square error (LMMSE) detection and the sphere decoder, when the system is employed with a large number of OFDM sub-carriers. Numerical evaluations illustrate the advantage of the introduced WESN-based symbol detector and demonstrate that the improvement of STM can significantly improve symbol detection performance as well as effectively mitigate model mismatch effects compared to existing methods.

Dynamic spectrum access (DSA) is regarded as one of the key enabling technologies for future communication networks. In this paper, we introduce a power allocation strategy for distributed DSA networks using a powerful machine learning tool, namely deep reinforcement learning. The introduced power allocation strategy enables DSA users to conduct power allocation in a distributed fashion without relying on channel state information and cooperations among DSA users. Furthermore, to capture the temporal correlation of the underlying DSA network environments, the reservoir computing, a special class of recurrent neural network, is employed to realize the introduced deep reinforcement learning scheme. The combination of reservoir computing and deep reinforcement learning significantly improves the efficiency of the introduced resource allocation scheme. Simulation evaluations are conducted to demonstrate the effectiveness of the introduced power allocation strategy.

We successfully fabricated gate stacks (ZrO2/GeO x /Ge) with a subnanometer equivalent oxide thickness (EOT) and low-leakage current on n-/p-Ge through plasma-enhanced atomic layer deposition (ALD). A 0.78-nm-thick GeO x was formed through plasma oxidation (i.e., in situ plasma interfacial passivation, followed by 3.48-nm-thick ZrO2 growth in the same ALD reactor). A subnanometer EOT of $\sim 0.9$ nm was achieved with a relatively high dielectric constant (roughly 30) of tetragonal-phase ZrO2. The gate leakage was $\sim 1 \times 10^{-4}$ A/cm2 at $\text{V}_{\mathrm {\mathbf {FB}}} - 1$ V, and roughly $5 \times 10^{-5}$ A/cm2 at $\text{V}_{\mathrm {\mathbf {FB}}} + 1$ V on p- and n-type Ge, respectively. Our ZrO2 stabilized in the tetragonal phase, when the post-deposition annealing temperature, was higher than 500 °C. Therefore, the proposed scheme is simple and effective for use in pursuing an ultralow EOT gate dielectric on Ge.

Dynamic spectrum access (DSA) is regarded as an effective and efficient technology to share radio spectrum among different networks. As a secondary user (SU), a DSA device will face two critical problems: avoiding causing harmful interference to primary users (PUs), and conducting effective interference coordination with other secondary users. These two problems become even more challenging for a distributed DSA network where there is no centralized controllers for SUs. In this paper, we investigate communication strategies of a distributive DSA network under the presence of spectrum sensing errors. To be specific, we apply the powerful machine learning tool, deep reinforcement learning (DRL), for SUs to learn "appropriate" spectrum access strategies in a distributed fashion assuming NO knowledge of the underlying system statistics. Furthermore, a special type of recurrent neural network (RNN), called the reservoir computing (RC), is utilized to realize DRL by taking advantage of the underlying temporal correlation of the DSA network. Using the introduced machine learning-based strategy, SUs could make spectrum access decisions distributedly relying only on their own current and past spectrum sensing outcomes. Through extensive experiments, our results suggest that the RC-based spectrum access strategy can help the SU to significantly reduce the chances of collision with PUs and other SUs. We also show that our scheme outperforms the myopic method which assumes the knowledge of system statistics, and converges faster than the Q-learning method when the number of channels is large., Comment: This work is accepted in IEEE IoT Journal 2018

In this paper, an effective algorithm for compressing simple images, such as cartoons and man-drawn images, is proposed. Compared to existing methods, the proposed algorithm applies several new techniques. First, we classify the regions of an image into 4 classes (uniform, semi-uniform, multiple DCs, and non-uniform). For different classes, different coding algorithms are applied. Second, instead of calculating the average, we apply majority voting to determine DC terms. Moreover, a dividing and 2nd order polynomial approximation scheme is applied for boundary encoding. Simulations show that, when compressing simple images, the proposed algorithm much outperforms other state-of-the-art algorithms, especially in perception.

Surface plasmon resonance ( SPR) is one of the main mechanisms of Surface Raman Enhance S cattering (SERS) and it will depend on the morphology and free carrier density of s ubstrates, in many of discussions have been proved. Recently, the semiconductor copper(I) sulphide (Cu 2 S), the natural p -type semi conductor, exhibits remarkable SPR in the near -infrared region [1] and can be regards as best candidate for active SERS substrates. In this repor t, the successive ionic layer ad sorption and reaction (SILAR) process will be used to synthesis Cu 2 S nano structures [2] from ZnO nanorods as template deposited by electrochemical reaction. To further manipulate the different carrier densities of Cu 2 S nano structures s, the adjustment of Cu vacancy in Cu 2 S can be accomplished by thermal processes under noble gas . Taking 4 -aminothiophenol (4 -ATP) as probe molecule to measure the SERS performance by Cu 2 S nano structures made in this fabrication and also examines the effect on SERS by adjust ing Cu vacancy under an excit ed wavelength of 632.8 nm and light power of 15 mW. In fact, the modulation of Cu vacancy wi ll positively correlate to th e SPR frequencies and so could get the best enhancement factor under the limited condition of excit ed source. Therefore, our results could provide a new opportunity to use SERS to explore the molecule -semiconductor interaction, a fundamental but essential question for designing novel devices. Keywords: Cu

The stroke is a very important feature for a character and is helpful for word recognition and handwriting identification. Although thinning algorithms can be applied for stroke extraction, they always suffer from the problems of bifurcation and disconnection. Moreover, since the end points of strokes cannot be preserved by thinning, the stroke length cannot be accurately determined and the start and the end parts of a stroke, which are useful for identifying the writing habit of a person, are hard to be extracted explicitly. In this paper, we proposed a very accurate stroke extraction algorithm which can well preserve the ends of strokes. Simulations on some Chinese characters show that the proposed algorithm is reliable and can precisely extract the strokes of characters.