Search

Hierarchical temporal memory (HTM) is a promising unsupervised machine-learning algorithm that models key principles of neocortical computation. One of the main components of HTM is the spatial pooler (SP), which encodes binary input streams into sparse distributed representations (SDRs). In this paper, we propose an information-theoretic framework for the performance comparison of HTM-spatial pooler (SP) algorithms, specifically, for quantifying the similarities and differences between sparse distributed representations in SP algorithms. We evaluate SP's standalone performance, as well as HTM's overall performance. Our comparison of various SP algorithms using Renyi mutual information, Renyi divergence, and Henze–Penrose divergence measures reveals that the SP algorithm with learning and a logarithmic boosting function yields the most effective and useful data representation. Moreover, the most effective SP algorithm leads to superior HTM results. In addition, we utilize our proposed framework to compare HTM with other state-of-the-art sequential learning algorithms. We illustrate that HTM exhibits superior adaptability to pattern changes over time than long short term memory (LSTM), gated recurrent unit (GRU) and online sequential extreme learning machine (OS-ELM) algorithms. This superiority is evident from the lower Renyi divergence of HTM (0.23) compared to LSTM6000 (0.33), LSTM3000 (0.38), GRU (0.41), and OS-ELM (0.49). HTM also achieved the highest Renyi mutual information value of 0.79, outperforming LSTM6000 (0.73), LSTM3000 (0.71), GRU (0.68), and OS-ELM (0.62). These findings not only confirm the numerous advantages of HTM over other sequential learning algorithm, but also demonstrate the effectiveness of our proposed information-theoretic approach as a powerful framework for comparing and evaluating various learning algorithms. [ABSTRACT FROM AUTHOR]

In this paper, first, visible light communications system and corresponding wireless channel with related random variables are reviewed. Then, considering channel random variables, important communication performance, i.e., average channel capacity and channel outage probability are calculated. Finally, to evaluate the obtained results, numerical calculations are provided. [ABSTRACT FROM AUTHOR]

Information-theoretic analysis of the communication systems performances is of practical and theoretical importance. In this paper, (i) Specifically, we generalize the Cover–Chiang work on point to point channel with two-sided state information to two-receiver less noisy broadcast channel (2R-LN-BC) with different side information non-causally known at the encoder as well as the decoders, and obtain the capacity region including the previous works as its special cases. Then, (ii) the obtained result is expanded to the continuous alphabet Gaussian version. In this Gaussian version, the side information available at the receivers are supposed to be correlated on the channel noises and the impact of these receivers cognition of the noises on the capacity region is analyzed. [ABSTRACT FROM AUTHOR]

Analyzing wireless communication performances by using information-theoretic results is of practical importance. In this paper, first, an achievable secrecy rate region and an outer bound on the secrecy capacity region for the discrete alphabet and memoryless wiretap channel with side information non-causally known at the transmitter are obtained. Then, by extending the results to the continuous alphabet wireless wiretap channel and by deriving a closed-form expression on the secrecy coverage region (SCR), as a remarkable wireless performance factor, impact of side information on the SCR is analyzed and it is shown that side information increases the SCR as expected intuitively. Numerical evaluation of theoretical results is done finally. [ABSTRACT FROM AUTHOR]

Outsourcing data on cloud storage services has already attracted great attention due to the prospect of rapid data growth and storing efficiencies for customers. The coding-based cloud storage approach can offer a more reliable and faster solution with less storage space in comparison with replication-based cloud storage. LT codes are the famous member of the rateless code family that can improve performance of storage systems utilizing good degree distributions. Since degree distribution plays a key role in LT codes performance, recently introduced Poisson robust soliton distribution (PRSD) and combined Poisson robust soliton distribution (CPRSD) motivate us to investigate LT code-based cloud storage system. So, we exploit LT codes with new degree distributions to provide lower average degree and higher decoding efficiency, specifically when receiving fewer encoding symbols, compared with popular degree distribution, robust soliton distribution (RSD). In this paper, we show that proposed cloud storage outperforms traditional ones in terms of storage space and robustness encountering unavailability of encoding symbols, due to compatible properties of PRSD and CPRSD with cloud storage essence. Furthermore, a modified decoding process is presented based on required encoding symbols behavior to reduce data retrieval time. Numerical results confirm improvement of cloud storage performance. [ABSTRACT FROM AUTHOR]

In this paper, we focus on estimating Direction of Arrival (DOA) and removing heavy clutter embedded with measurement noise. A correlated Gaussian process is chosen to model destructive effects of clutter. Also, a white Gaussian process is selected to describe measurement noise caused by sensor array. After adding these distortions to the off-grid model, we utilize Sparse Bayesian Learning and principal component analysis (as a preprocessing stage) in order to remove these distortions as well as estimating of true DOAs. Finally, at the end we will show how ignorance of clutter from model or combine it with measurement noise degrade DOA estimation. This will be demonstrated by various numerical simulations. [ABSTRACT FROM AUTHOR]

A Grassmannian fusion frame is an optimal configuration of subspaces of a given vector space, that are useful in some applications related to representing data in signal processing. Grassmannian fusion frames are robust against noise and erasures when the signal is reconstructed. In this paper, we present an approach to construct optimal Grassmannian fusion frames based on a given Grassmannian frame. We also analyse an algorithm for sparse fusion frames which was introduced by Calderbank et al. and present necessary and sufficient conditions for the output of that algorithm to be an optimal Grassmannian fusion frame. [ABSTRACT FROM AUTHOR]

The shotgun cellular systems (SCSs) (modeling, generally, a dense cellular or wireless data network deployment and non-ideal regular hexagonal grid with site-acquisition difficulties, variable traffic load) are wireless communication systems with randomly placed base stations (BSs) over the entire plane according to a stochastic (e.g., Poisson) point process in n dimensions (n = 1, 2, and 3). Due to the good performance and the importance of the SCSs in today's communications, the physical layer security of these systems is practically of importance, specifically, in more realistic channel propagation models. In this paper, by assuming that the channel coefficients are known at the receiver, we analyze the performance of the physical layer security for the classic Wyner's three-node model in the SCSs over correlated/independent log-normal shadow fading channels. For this purpose, first, we calculate the signal-to-interference-plus-noise ratio (SINR) performance at a mobile station (MS) in an SCS. Then, the average secrecy capacity (ASC), the probability of non-zero secrecy capacity (PNSC), the secure outage probability (SOP), and the lower bound of SOP (LSOP) are obtained, and also, their approximated closed-form expressions are derived. Finally, analytical results are validated numerically. [ABSTRACT FROM AUTHOR]

In this paper, having reviewed necessary preliminaries, including sparsity, Tsallis entropy, diversity, preprocessing, fisher information, and Cramer–Rao bound, we analyze the impact of preprocessing a signal on the signal sparsity related to Cramer–Rao lower bound and its main feature, for example, its reconstruction error. The main idea of this paper is to increase the sparsity of a vector, or to decrease its nonzero elements, then to compute the estimation error bound before and after sparsifying the signal. Finally, the claims are validated numerically. We implement Savitzky–Golay filtering on some ECG signals (applying MIT-BIH database of cardiac signals) and then compress them, to illustrate that the sparsity (the reconstruction error) of non-filtered signal was less (more) than that of filtered one. The results can be useful in signal compression and transmission procedures to have fewer recovery errors. [ABSTRACT FROM AUTHOR]

Shotgun cellular systems (SCSs) are wireless communication systems with randomly placed base stations (BSs) over the entire plane according to a two-dimensional Poisson point process. Such a system can model a dense cellular or wireless data network deployment, where the BS locations end up being close to random due to constraints other than optimal coverage. SCSs have been studied by considering path-loss and independent shadowing paths between BS to mobile station (MS) pairs in the channel models. In this paper, we consider correlated shadowing paths between BS to MS pairs as a most important factor, and analyze the carrier to interference ratio (CIR), in a SCS over this correlation, and determine an expression for distribution of CIR, and obtain the tail probability of the CIR. [ABSTRACT FROM AUTHOR]

One of the most significant issues in machine learning is system identification with many applications, e.g., channel estimation (CE) in digital communications. Introducing a new correntropy-based method, this paper deals with the comparison between mean square error (MSE) and information theoretic measures in non-Gaussian noise channel estimation, by analyzing the MSE, minimum error entropy (MEE) and correntropy algorithms in several channel models utilizing neural networks. The first contribution of this paper is introducing a new correntropy-based conjugate gradient (CCG) method and applying it in the CE problem, which this new algorithm converges faster than standard maximum correntropy criterion algorithm. Aiming at this contribution, the better convergence rate is discussed analytically and it is proved that the CCG could converge to the optimal solution quadratically. Next, the performance of an extended MSE algorithm is compared with information theoretic criteria; in addition, a comparison between MEE and correntropy-based algorithm is presented. The Monte Carlo results illustrate that correntropy and MEE outperform MSE algorithm in low-SNR communications especially in the presence of impulsive noise. Then, we apply the trained neural networks in the receiver as an equalizer to obtain the intended performance for different SNR values. [ABSTRACT FROM AUTHOR]

Correntropy as an efficient information theoretic (ITL) criterion has been extensively applied in many non-Gaussian applications. In order to maximize correntropy, several optimization algorithms have been proposed. Fixed-point maximum correntropy (FP-MCC) and correntropy-based Levenberg-Marquardt (CLM) are the fastest of the proposed methods. As the convergence rate of these operational methods has not been studied before, in this paper we prove the ability of their quadratic convergence from a theoretic point of view and establish the influence of bandwidth on their convergence order. Then, theoretic results are validated through numerical experiments. [ABSTRACT FROM AUTHOR]