Search

The attention mechanism has been widely used in deep neural networks as a model component. By now, it has become a critical building block in many state-of-the-art natural language models. Despite its great success established empirically, the working mechanism of attention has not been investigated at a sufficient theoretical depth to date. In this paper, we set up a simple text classification task and study the dynamics of training a simple attention-based classification model using gradient descent. In this setting, we show that, for the discriminative words that the model should attend to, a persisting identity exists relating its embedding and the inner product of its key and the query. This allows us to prove that training must converge to attending to the discriminative words when the attention output is classified by a linear classifier. Experiments are performed, which validate our theoretical analysis and provide further insights.

Full quantum capability devices can provide secure communications, but they are challenging to make portable given the current technology. Besides, classical portable devices are unable to construct communication channels resistant to quantum computers. Hence, communication security on portable devices cannot be guaranteed. Semi-Quantum Communication (SQC) attempts to break the quandary by lowering the receiver's required quantum capability so that secure communications can be implemented on a portable device. However, all SQC protocols have low qubit efficiency and complex hardware implementations. The protocols involving quantum entanglement require linear Entanglement Preservation Time (EPT) and linear quregister size. In this paper, we propose two new keyless SQC protocols that address the aforementioned weaknesses. They are named Economic Keyless Semi-Quantum Point-to-point Communication (EKSQPC) and Rate Estimation EKSQPC (REKSQPC). They achieve theoretically constant minimal EPT and quregister size, regardless of message length. We show that the new protocols, with low overhead, can detect Measure and Replay Attacks (MRAs). REKSQDC is tolerant to transmission impairments and environmental perturbations. The protocols are based on a new quantum message transmission operation termed Tele-Fetch. Like QKD, their strength depends on physical principles rather than mathematical complexity.

Network structures, consisting of nodes and edges, have applications in almost all subjects. A set of nodes is called a community if the nodes have strong interrelations. Industries (including cell phone carriers and online social media companies) need community structures to allocate network resources and provide proper and accurate services. However, all the current detection algorithms are motivated by the practical problems, whose applicabilities in other fields are open to question. Thence, for a new community problem, researchers need to derive algorithms ad hoc, which is arduous and even unnecessary. In this paper, we represent a general procedure to find community structures in practice. We mainly focus on two typical types of networks: transmission networks and similarity networks. We reduce them to a unified graph model, based on which we propose a general method to define and detect communities. Readers can specialize our general algorithm to accommodate their problems. In the end, we also give a demonstration to show how the algorithm works.

Many TDMA- and CSMA-based protocols try to obtain fair channel access and to increase channel utilization. It is still challenging and crucial in Wireless Sensor Networks (WSNs), especially when the time synchronization cannot be well guaranteed and consumes much extra energy. This paper presents a localized and ondemand scheme ADC to adaptively adjust duty cycle based on quorum systems. ADC takes advantages of TDMA and CSMA and guarantees that (1) each node can fairly access channel based on its demand, (2) channel utilization can be increased by reducing competition for channel access among neighboring nodes, (3) every node has at least one rendezvous active time slot with each of its neighboring nodes even under asynchronization. The latency bound of data aggregation is analyzed under ADC to show that ADC can bound the latency under both synchronization and asynchronization. We conduct extensive experiments in TinyOS on a real test-bed with TelosB nodes to evaluate the performance of ADC. Comparing with B-MAC, ADC substantially reduces the contention for channel access and energy consumption, and improves network throughput., Comment: 11 pages, 13 figures, IEEE 8th International Conference on Mobile Ad hoc and Sensor Systems (MASS, 2011)

Sensor networks aim at monitoring their surroundings for event detection and object tracking. But due to failure or death of sensors, false signal can be transmitted. In this paper, we consider the problem of fault detection in wireless sensor network (WSN), in particular, addressing both the noise-related measurement error and sensor fault simultaneously in fault detection. We assume that the sensors are placed at the center of a square (or hexagonal) cell in region of interest (ROI) and, if the event occurs, it occurs at a particular cell of the ROI. We propose fault detection schemes that take into account error probabilities into the optimal event detection process. We develop the schemes under the consideration of Neyman-Pearson test and Bayes test.

We propose an integrated architecture for smart grids, that supports data aggregation and access control. Data can be aggregated by home area network, building area network and neighboring area network in such a way that the privacy of customers is protected. We use homomorphic encryption technique to achieve this. The consumer data that is collected is sent to the substations where it is monitored by remote terminal units (RTU). The proposed access control mechanism gives selective access to consumer data stored in data repositories and used by different smart grid users. Users can be maintenance units, utility centers, pricing estimator units or analyzing and prediction groups. We solve this problem of access control using cryptographic technique of attribute-based encryption. RTUs and users have attributes and cryptographic keys distributed by several key distribution centers (KDC). RTUs send data encrypted under a set of attributes. Users can decrypt information provided they have valid attributes. The access control scheme is distributed in nature and does not rely on a single KDC to distribute keys. Bobba \emph{et al.} \cite{BKAA09} proposed an access control scheme, which relies on a centralized KDC and is thus prone to single-point failure. The other requirement is that the KDC has to be online, during data transfer which is not required in our scheme. Our access control scheme is collusion resistant, meaning that users cannot collude and gain access to data, when they are not authorized to access. We theoretically analyze our schemes and show that the computation overheads are low enough to be carried out in smart grids. To the best of our knowledge, ours is the first work on smart grids, which integrates these two important security components (privacy preserving data aggregation and access control) and presents an overall security architecture in smart grids., Comment: 12 Pages, 3 figures

Detecting misbehavior (such as transmissions of false information) in vehicular ad hoc networks (VANETs) is very important problem with wide range of implications including safety related and congestion avoidance applications. We discuss several limitations of existing misbehavior detection schemes (MDS) designed for VANETs. Most MDS are concerned with detection of malicious nodes. In most situations, vehicles would send wrong information because of selfish reasons of their owners, e.g. for gaining access to a particular lane. Because of this (\emph{rational behavior}), it is more important to detect false information than to identify misbehaving nodes. We introduce the concept of data-centric misbehavior detection and propose algorithms which detect false alert messages and misbehaving nodes by observing their actions after sending out the alert messages. With the data-centric MDS, each node can independently decide whether an information received is correct or false. The decision is based on the consistency of recent messages and new alert with reported and estimated vehicle positions. No voting or majority decisions is needed, making our MDS resilient to Sybil attacks. Instead of revoking all the secret credentials of misbehaving nodes, as done in most schemes, we impose fines on misbehaving nodes (administered by the certification authority), discouraging them to act selfishly. This reduces the computation and communication costs involved in revoking all the secret credentials of misbehaving nodes., Comment: 12 pages