Search

With the rising emergence of decentralized and opportunistic approaches to machine learning, end devices are increasingly tasked with training deep learning models on-devices using crowd-sourced data that they collect themselves. These approaches are desirable from a resource consumption perspective and also from a privacy preservation perspective. When the devices benefit directly from the trained models, the incentives are implicit - contributing devices' resources are incentivized by the availability of the higher-accuracy model that results from collaboration. However, explicit incentive mechanisms must be provided when end-user devices are asked to contribute their resources (e.g., computation, communication, and data) to a task performed primarily for the benefit of others, e.g., training a model for a task that a neighbor device needs but the device owner is uninterested in. In this project, we propose a novel blockchain-based incentive mechanism for completely decentralized and opportunistic learning architectures. We leverage a smart contract not only for providing explicit incentives to end devices to participate in decentralized learning but also to create a fully decentralized mechanism to inspect and reflect on the behavior of the learning architecture.

In current blockchain systems, full nodes that perform all of the available functionalities need to store the entire blockchain. In addition to the blockchain, full nodes also store a blockchain-summary, called the \emph{state}, which is used to efficiently verify transactions. With the size of popular blockchains and their states growing rapidly, full nodes require massive storage resources in order to keep up with the scaling. This leads to a tug-of-war between scaling and decentralization since fewer entities can afford expensive resources. We present \emph{hybrid nodes} for proof-of-work (PoW) cryptocurrencies which can validate transactions, validate blocks, validate states, mine, select the main chain, bootstrap new hybrid nodes, and verify payment proofs. With the use of a protocol called \emph{trimming}, hybrid nodes only retain polylogarithmic number of blocks in the chain length in order to represent the proof-of-work of the blockchain. Hybrid nodes are also optimized for the storage of the state with the use of \emph{stateless blockchain} protocols. The lowered storage requirements should enable more entities to join as hybrid nodes and improve the decentralization of the system. We define novel theoretical security models for hybrid nodes and show that they are provably secure. We also show that the storage requirement of hybrid nodes is near-optimal with respect to our security definitions.

This paper develops a novel methodology for designing analog beamforming codebooks for full-duplex millimeter wave (mmWave) transceivers, the first such codebooks to the best of our knowledge. Our design reduces the self-interference coupled by transmit-receive beam pairs and simultaneously delivers high beamforming gain over desired coverage regions, allowing mmWave full-duplex systems to support beam alignment while minimizing self-interference. To do so, our methodology allows some variability in beamforming gain to strategically shape beams that reject self-interference while still having substantial gain. We present an algorithm for approximately solving our codebook design problem while accounting for the non-convexity posed by digitally-controlled phase shifters and attenuators. Numerical results suggest that our design can outperform or nearly match existing codebooks in sum spectral efficiency across a wide range of self-interference power levels. Results show that our design offers an extra 20-50 dB of robustness to self-interference, depending on hardware constraints.

In this paper, we use a model of large random network game where the agents plays selfishly and are affected by their neighbors, to explore the conditions under which the Nash equilibrium (NE) of the game is affected by a perturbation in the network. We use a phase transition phenomenon observed in finite rank deformations of large random matrices, to study how the NE changes on crossing critical threshold points. Our main contribution is as follows: when the perturbation strength is greater than a critical point, it impacts the NE of the game, whereas when this perturbation is below this critical point, the NE remains independent of the perturbation parameter. This demonstrates a phase transition in NE which alludes that perturbations can affect the behavior of the society only if their strength is above a critical threshold. We provide numerical examples for this result and present scenarios under which this phenomenon could potentially occur in real world applications.

In this paper, we present a sequential decomposition algorithm equivalent of Master equation to compute GMFE of GMFG and graphon optimal Markovian policies (GOMPs) of graphon mean field teams (GMFTs). We consider a large population of players sequentially making strategic decisions where the actions of each player affect their neighbors which is captured in a graph, generated by a known graphon. Each player observes a private state and also a common information as a graphon mean-field population state which represents the empirical networked distribution of other players' types. We consider non-stationary population state dynamics and present a novel backward recursive algorithm to compute both GMFE and GOMP that depend on both, a player's private type, and the current (dynamic) population state determined through the graphon. Each step in computing GMFE consists of solving a fixed-point equation, while computing GOMP involves solving for an optimization problem. We provide conditions on model parameters for which there exists such a GMFE. Using this algorithm, we obtain the GMFE and GOMP for a specific security setup in cyber physical systems for different graphons that capture the interactions between the nodes in the system., Comment: 26 pages, 6 figures. arXiv admin note: text overlap with arXiv:1905.04154

In this work, we introduce EQ-Net: the first holistic framework that solves both the tasks of log-likelihood ratio (LLR) estimation and quantization using a data-driven method. We motivate our approach with theoretical insights on two practical estimation algorithms at the ends of the complexity spectrum and reveal a connection between the complexity of an algorithm and the information bottleneck method: simpler algorithms admit smaller bottlenecks when representing their solution. This motivates us to propose a two-stage algorithm that uses LLR compression as a pretext task for estimation and is focused on low-latency, high-performance implementations via deep neural networks. We carry out extensive experimental evaluation and demonstrate that our single architecture achieves state-of-the-art results on both tasks when compared to previous methods, with gains in quantization efficiency as high as $20\%$ and reduced estimation latency by up to $60\%$ when measured on general purpose and graphical processing units (GPU). In particular, our approach reduces the GPU inference latency by more than two times in several multiple-input multiple-output (MIMO) configurations. Finally, we demonstrate that our scheme is robust to distributional shifts and retains a significant part of its performance when evaluated on 5G channel models, as well as channel estimation errors.

Equipping millimeter wave (mmWave) systems with full-duplex capability would accelerate and transform next-generation wireless applications and forge a path for new ones. Full-duplex mmWave transceivers could capitalize on the already attractive features of mmWave communication by supplying spectral efficiency gains and latency improvements while also affording future networks with deployment solutions in the form of interference management and wireless backhaul. Foreseeable challenges and obstacles in making mmWave full-duplex a reality are presented in this article along with noteworthy unknowns warranting further investigation. With these novelties of mmWave full-duplex in mind, we lay out potential solutions---beyond active self-interference cancellation---that harness the spatial degrees of freedom bestowed by dense antenna arrays to enable simultaneous transmission and reception in-band., Comment: Submitted to the IEEE Wireless Communications Magazine: Special Issue on Full Duplex Communications

The ability of deep learning (DL) to improve the practice of medicine and its clinical outcomes faces a looming obstacle: model interpretation. Without description of how outputs are generated, a collaborating physician can neither resolve when the model's conclusions are in conflict with his or her own, nor learn to anticipate model behavior. Current research aims to interpret networks that diagnose ECG recordings, which has great potential impact as recordings become more personalized and widely deployed. A generalizable impact beyond ECGs lies in the ability to provide a rich test-bed for the development of interpretive techniques in medicine. Interpretive techniques for Deep Neural Networks (DNNs), however, tend to be heuristic and observational in nature, lacking the mathematical rigor one might expect in the analysis of math equations. The motivation of this paper is to offer a third option, a scientific approach. We treat the model output itself as a phenomenon to be explained through component parts and equations governing their behavior. We argue that these component parts should also be "black boxes" --additional targets to interpret heuristically with clear functional connection to the original. We show how to rigorously factor a DNN into a hierarchical equation consisting of black box variables. This is not a subdivision into physical parts, like an organism into its cells; it is but one choice of an equation into a collection of abstract functions. Yet, for DNNs trained to identify normal ECG waveforms on PhysioNet 2017 Challenge data, we demonstrate this choice yields interpretable component models identified with visual composite sketches of ECG samples in corresponding input regions. Moreover, the recursion distills this interpretation: additional factorization of component black boxes corresponds to ECG partitions that are more morphologically pure.

Efficient modeling of relational data arising in physical, social, and information sciences is challenging due to complicated dependencies within the data. In this work, we build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in a low-dimensional graph latent representation. We incorporate hyperbolic geometry in the latent space through a Poincare embedding to efficiently represent graphs exhibiting hierarchical structure. To address the naive posterior latent distribution assumptions in classical variational inference, we use semi-implicit hierarchical variational Bayes to implicitly capture posteriors of given graph data, which may exhibit heavy tails, multiple modes, skewness, and highly correlated latent structures. We show that the existing semi-implicit variational inference objective provably reduces information in the observed graph. Based on this observation, we estimate and add an additional mutual information term to the semi-implicit variational inference learning objective to capture rich correlations arising between the input and latent spaces. We show that the inclusion of this regularization term in conjunction with the Poincare embedding boosts the quality of learned high-level representations and enables more flexible and faithful graphical modeling. We experimentally demonstrate that our approach outperforms existing graph variational auto-encoders both in Euclidean and in hyperbolic spaces for edge link prediction and node classification.

We consider the minimum cost intervention design problem: Given the essential graph of a causal graph and a cost to intervene on a variable, identify the set of interventions with minimum total cost that can learn any causal graph with the given essential graph. We first show that this problem is NP-hard. We then prove that we can achieve a constant factor approximation to this problem with a greedy algorithm. We then constrain the sparsity of each intervention. We develop an algorithm that returns an intervention design that is nearly optimal in terms of size for sparse graphs with sparse interventions and we discuss how to use it when there are costs on the vertices., Comment: In NIPS 2018

We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We extend this notion to Renyi common entropy by minimizing the Renyi entropy of the latent variable. To efficiently compute common entropy, we propose an iterative algorithm that can be used to discover the trade-off between the entropy of the latent variable and the conditional mutual information of the observed variables. We show two applications of common entropy in causal inference: First, under the assumption that there are no low-entropy mediators, it can be used to distinguish causation from spurious correlation among almost all joint distributions on simple causal graphs with two observed variables. Second, common entropy can be used to improve constraint-based methods such as PC or FCI algorithms in the small-sample regime, where these methods are known to struggle. We propose a modification to these constraint-based methods to assess if a separating set found by these algorithms is valid using common entropy. We finally evaluate our algorithms on synthetic and real data to establish their performance., Comment: In Proceedings of NeurIPS 2020

In this paper, an algebraic binning based coding scheme and its associated achievable rate for key generation using physically unclonable functions (PUFs) is determined. This achievable rate is shown to be optimal under the generated-secret (GS) model for PUFs. Furthermore, a polar code based polynomial-time encoding and decoding scheme that achieves this rate is also presented.

We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the unobserved exogenous variable: Inspired by Occam's razor, we assume that the exogenous variable is simple in the true causal direction. We quantify simplicity using R\'enyi entropy. Our main result is that, under natural assumptions, if the exogenous variable has low $H_0$ entropy (cardinality) in the true direction, it must have high $H_0$ entropy in the wrong direction. We establish several algorithmic hardness results about estimating the minimum entropy exogenous variable. We show that the problem of finding the exogenous variable with minimum entropy is equivalent to the problem of finding minimum joint entropy given $n$ marginal distributions, also known as minimum entropy coupling problem. We propose an efficient greedy algorithm for the minimum entropy coupling problem, that for $n=2$ provably finds a local optimum. This gives a greedy algorithm for finding the exogenous variable with minimum $H_1$ (Shannon Entropy). Our greedy entropy-based causal inference algorithm has similar performance to the state of the art additive noise models in real datasets. One advantage of our approach is that we make no use of the values of random variables but only their distributions. Our method can therefore be used for causal inference for both ordinal and also categorical data, unlike additive noise models., Comment: To appear in AAAI 2017

We propose an adversarial training procedure for learning a causal implicit generative model for a given causal graph. We show that adversarial training can be used to learn a generative model with true observational and interventional distributions if the generator architecture is consistent with the given causal graph. We consider the application of generating faces based on given binary labels where the dependency structure between the labels is preserved with a causal graph. This problem can be seen as learning a causal implicit generative model for the image and labels. We devise a two-stage procedure for this problem. First we train a causal implicit generative model over binary labels using a neural network consistent with a causal graph as the generator. We empirically show that WassersteinGAN can be used to output discrete labels. Later, we propose two new conditional GAN architectures, which we call CausalGAN and CausalBEGAN. We show that the optimal generator of the CausalGAN, given the labels, samples from the image distributions conditioned on these labels. The conditional GAN combined with a trained causal implicit generative model for the labels is then a causal implicit generative model over the labels and the generated image. We show that the proposed architectures can be used to sample from observational and interventional image distributions, even for interventions which do not naturally occur in the dataset.

We consider the problem of learning a causal graph over a set of variables with interventions. We study the cost-optimal causal graph learning problem: For a given skeleton (undirected version of the causal graph), design the set of interventions with minimum total cost, that can uniquely identify any causal graph with the given skeleton. We show that this problem is solvable in polynomial time. Later, we consider the case when the number of interventions is limited. For this case, we provide polynomial time algorithms when the skeleton is a tree or a clique tree. For a general chordal skeleton, we develop an efficient greedy algorithm, which can be improved when the causal graph skeleton is an interval graph.

This paper introduces a class of non-linear and continuous-time opinion dynamics model with additive noise and state dependent interaction rates between agents. The model features interaction rates which are proportional to a negative power of opinion distances. We establish a non-local partial differential equation for the distribution of opinion distances and use Mellin transforms to provide an explicit formula for the stationary solution of the latter, when it exists. Our approach leads to new qualitative and quantitative results on this type of dynamics. To the best of our knowledge these Mellin transform results are the first quantitative results on the equilibria of opinion dynamics with distance-dependent interaction rates. The closed form expressions for this class of dynamics are obtained for the two agent case. However the results can be used in mean-field models featuring several agents whose interaction rates depend on the empirical average of their opinions. The technique also applies to linear dynamics, namely with a constant interaction rate, on an interaction graph.

This paper presents a polar coding scheme to achieve secrecy in block fading binary symmetric wiretap channels without the knowledge of instantaneous channel state information (CSI) at the transmitter. For this model, a coding scheme that hierarchically utilizes polar codes is presented. In particular, on polarization of different binary symmetric channels over different fading blocks, each channel use is modeled as an appropriate binary erasure channel over fading blocks. Polar codes are constructed for both coding over channel uses for each fading block and coding over fading blocks for certain channel uses. In order to guarantee security, random bits are introduced at appropriate places to exhaust the observations of the eavesdropper. It is shown that this coding scheme, without instantaneous CSI at the transmitter, is secrecy capacity achieving for the simultaneous fading scenario. For the independent fading case, the capacity is achieved when the fading realizations for the eavesdropper channel is always degraded with respect to the receiver. For the remaining cases, the gap is analyzed by comparing lower and upper bounds. Remarkably, for the scenarios where the secrecy capacity is achieved, the results imply that instantaneous CSI does not increase the secrecy capacity.

Natural Disasters like hurricanes, floods or earthquakes can damage power grid devices and create cascading blackouts and islands. The nature of failure propagation and extent of damage is dependent on the structural features of the grid, which is different from that of random networks. This paper analyzes the structural vulnerability of real power grids to impending disasters and presents intuitive graphical metrics to quantify the extent of damage. Two improved graph eigen-value based bounds on the grid vulnerability are developed and demonstrated through simulations of failure propagation on IEEE test cases and real networks. Finally this paper studies adversarial attacks aimed at weakening the grid's structural resilience and presents two approximate schemes to determine the critical transmission lines that may be attacked to minimize grid resilience. The framework can be also be used to design protection schemes to secure the grid against such adversarial attacks. Simulations on power networks are used to compare the performance of the attack schemes in reducing grid resilience., Comment: 9 pages, 11 figures

We consider the problem of learning causal networks with interventions, when each intervention is limited in size under Pearl's Structural Equation Model with independent errors (SEM-IE). The objective is to minimize the number of experiments to discover the causal directions of all the edges in a causal graph. Previous work has focused on the use of separating systems for complete graphs for this task. We prove that any deterministic adaptive algorithm needs to be a separating system in order to learn complete graphs in the worst case. In addition, we present a novel separating system construction, whose size is close to optimal and is arguably simpler than previous work in combinatorics. We also develop a novel information theoretic lower bound on the number of interventions that applies in full generality, including for randomized adaptive learning algorithms. For general chordal graphs, we derive worst case lower bounds on the number of interventions. Building on observations about induced trees, we give a new deterministic adaptive algorithm to learn directions on any chordal skeleton completely. In the worst case, our achievable scheme is an $\alpha$-approximation algorithm where $\alpha$ is the independence number of the graph. We also show that there exist graph classes for which the sufficient number of experiments is close to the lower bound. In the other extreme, there are graph classes for which the required number of experiments is multiplicatively $\alpha$ away from our lower bound. In simulations, our algorithm almost always performs very close to the lower bound, while the approach based on separating systems for complete graphs is significantly worse for random chordal graphs., Comment: Accepted to NIPS 2015

Mobile cyberphysical systems have received considerable attention over the last decade, as communication, computing and control come together on a common platform. Understanding the complex interactions that govern the behavior of large complex cyberphysical systems is not an easy task. The goal of this paper is to address this challenge in the particular context of multimedia delivery over an autonomous aerial vehicle (AAV) network. Bandwidth requirements and stringent delay constraints of real-time video streaming, paired with limitations on computational complexity and power consumptions imposed by the underlying implementation platform, make cross-layer and cross-domain co-design approaches a necessity. In this paper, we propose a novel, low-complexity rate-distortion optimized (RDO) algorithms specifically targeted at video streaming over mobile embedded networks. We test the performance of our RDO algorithms using a network of AAVs both in simulation and implementation.

We consider the problem of network coding across three unicast sessions over a directed acyclic graph, where each sender and the receiver is connected to the network via a single edge of unit capacity. We consider a network model in which the middle of the network only performs random linear network coding, and restrict our approaches to precoding-based linear schemes, where the senders use precoding matrices to encode source symbols. We adapt a precoding-based interference alignment technique, originally developed for the wireless interference channel, to construct a precoding-based linear scheme, which we refer to as as a {\em precoding-based network alignment scheme (PBNA)}. A primary difference between this setting and the wireless interference channel is that the network topology can introduce dependencies between elements of the transfer matrix, which we refer to as coupling relations, and can potentially affect the achievable rate of PBNA. We identify all possible such coupling relations, and interpret these coupling relations in terms of network topology and present polynomial-time algorithms to check the presence of these coupling relations. Finally, we show that, depending on the coupling relations present in the network, the optimal symmetric rate achieved by precoding-based linear scheme can take only three possible values, all of which can be achieved by PBNA., Comment: arXiv admin note: text overlap with arXiv:1202.3405

The topological structure of the power grid plays a key role in the reliable delivery of electricity and price settlement in the electricity market. Incorporation of new energy sources and loads into the grid over time has led to its structural and geographical expansion and can affect its stable operation. This paper presents an intuitive analytical model for the temporal evolution of large grids and uses it to understand common structural features observed in grids across America. In particular, key graph parameters like degree distribution, graph diameter, betweenness centralities, eigen-spread and clustering coefficients, as well as graph processes like infection propagation are used to quantify the model's benefits through comparison with the Western US and ERCOT power grids. The most significant contribution of the developed model is its analytical tractability, that provides a closed form expression for the nodal degree distribution observed in large grids. The discussed model can be used to generate realistic test cases to analyze topological effects on grid functioning and new grid technologies.

This paper extends the literature on interference alignment to more general classes of deterministic channels which incorporate non-linear input-output relationships. It is found that the concept of alignment extends naturally to these deterministic interference channels, and in many cases, the achieved degrees of freedom (DoF) can be shown to be optimal., Comment: 5 pages, submitted to ISIT 2010

An analytically tractable model for Gaussian multiuser channels with fading is studied, and the capacity region of this model is found to be a good approximation of the capacity region of the original Gaussian network. This work extends the existing body of work on deterministic models for Gaussian multiuser channels to include the physical phenomenon of fading. In particular, it generalizes these results to a unicast, multiple node network setting with fading., Comment: 6 pages, 3 figures

We address the challenge of implementing reliable computation of Boolean functions in future nanocircuit fabrics. Such fabrics are projected to have very high defect rates. We overcome this limitation by using a combination of cheap but unreliable nanodevices and reliable but expensive CMOS devices. In our approach, defect tolerance is achieved through a novel coding of Boolean functions; specifically, we exploit the dont cares of Boolean functions encountered in multi-level Boolean logic networks for constructing better codes. We show that compared to direct application of existing coding techniques, the coding overhead in terms of extra bits can be reduced, on average by 23%, and savings can go up to 34%. We demonstrate that by incorporating efficient coding techniques more than a 40% average yield improvement is possible in case of 1% and 0.1% defect rates. With 0.1% defect density, the savings can be up to 90%., Comment: 6 pages, 5 Figures

This paper is motivated by a sensor network on a correlated field where nearby sensors share information, and can thus assist rather than interfere with one another. A special class of two-user Gaussian interference channels (IFCs) is considered where one of the two transmitters knows both the messages to be conveyed to the two receivers (called the IFC with degraded message sets). Both achievability and converse arguments are provided for this scenario for a class of discrete memoryless channels with weak interference. For the case of the Gaussian weak interference channel with degraded message sets, optimality of Gaussian inputs is also shown, resulting in the capacity region of this channel.