Search

With the advent of large models based on the Transformer architecture, researchers have observed an anomalous phenomenon in the Attention mechanism--there is a very high attention on the first element, which is prevalent across Transformer-based models. It is crucial to understand it for the development of techniques focusing on attention distribution, such as Key-Value (KV) Cache compression and infinite extrapolation; however, the latent cause leaves to be unknown. In this paper, we analyze such a phenomenon from the perspective of waiver phenomenon, which involves reducing the internal values of certain elements in the sequence, allowing them to absorb excess attention without affecting their contribution to information. In specific models, due to differences in positional encoding and attention patterns, we have found that the selection of waiver elements by the model can be categorized into two methods: positional-encoding-based and feature-distribution-within-elements-based.

Single-center two-electron integration is an important core technology in ab initio calculation of atomic and molecular structures. Therefore, this paper reviews and optimizes the method of Zhao et al., and draws a conclusion: Because this method is an accurate calculation without truncation error, it is superior to Slater-Condon integration method., Comment: Errors in the calculation process in the paper

Channel estimation and data transmission constitute the most fundamental functional modules of multiple-input multiple-output (MIMO) communication systems. The underlying key tasks corresponding to these modules are training sequence optimization and transceiver optimization. Hence, we jointly optimize the linear transmit precoder and the training sequence of MIMO systems using the metrics of their effective mutual information (MI), effective mean squared error (MSE), effective weighted MI, effective weighted MSE, as well as their effective generic Schur-convex and Schur-concave functions. Both statistical channel state information (CSI) and estimated CSI are considered at the transmitter in the joint optimization. A unified framework termed as joint matrix-monotonic optimization is proposed. Based on this, the optimal precoder matrix and training matrix structures can be derived for both CSI scenarios. Then, based on the optimal matrix structures, our linear transceivers and their training sequences can be jointly optimized. Compared to state-of-the-art benchmark algorithms, the proposed algorithms visualize the bold explicit relationships between the attainable system performance of our linear transceivers conceived and their training sequences, leading to implementation ready recipes. Finally, several numerical results are provided, which corroborate our theoretical results and demonstrate the compelling benefits of our proposed pilot-aided MIMO solutions., Comment: 39 pages, 9 figures, 1 table, manuscript accepted in IEEE TVT

This paper considers the joint channel estimation and device activity detection in the grant-free random access systems, where a large number of Internet-of-Things devices intend to communicate with a low-earth orbit satellite in a sporadic way. In addition, the massive multiple-input multiple-output (MIMO) with orthogonal time-frequency space (OTFS) modulation is adopted to combat the dynamics of the terrestrial-satellite link. We first analyze the input-output relationship of the single-input single-output OTFS when the large delay and Doppler shift both exist, and then extend it to the grant-free random access with massive MIMO-OTFS. Next, by exploring the sparsity of channel in the delay-Doppler-angle domain, a two-dimensional pattern coupled hierarchical prior with the sparse Bayesian learning and covariance-free method (TDSBL-CF) is developed for the channel estimation. Then, the active devices are detected by computing the energy of the estimated channel. Finally, the generalized approximate message passing algorithm combined with the sparse Bayesian learning and two-dimensional convolution (ConvSBL-GAMP) is proposed to decrease the computations of the TDSBL-CF algorithm. Simulation results demonstrate that the proposed algorithms outperform conventional methods., Comment: This paper has been accepted by IEEE JSAC Special Issue on Antenna Array Enabled Space/Air/Ground Communications and Networking

Terrestrial-satellite networks are envisioned to play a significant role in the sixth-generation (6G) wireless networks. In such networks, hot air balloons are useful as they can relay the signals between satellites and ground stations. Most existing works assume that the hot air balloons are deployed at the same height with the same minimum elevation angle to the satellites, which may not be practical due to possible route conflict with airplanes and other flight equipment. In this paper, we consider a TSN containing hot air balloons at different heights and with different minimum elevation angles, which creates the challenge of non-uniform available serving time for the communication between the hot air balloons and the satellites. Jointly considering the caching, computing, and communication (3C) resource management for both the ground-balloon-satellite links and inter-satellite laser links, our objective is to maximize the network energy efficiency. Firstly, by proposing a tapped water-filling algorithm, we schedule the traffic to relay among satellites according to the available serving time of satellites. Then, we generate a series of configuration matrices, based on which we formulate the relationship of relay time and the power consumption involved in the relay among satellites. Finally, the integrated system model of TSN is built and solved by geometric programming with Taylor series approximation. Simulation results demonstrate the effectiveness of our proposed scheme.

Mobile edge computing (MEC) is a promising technology to support mission-critical vehicular applications, such as intelligent path planning and safety applications. In this paper, a collaborative edge computing framework is developed to reduce the computing service latency and improve service reliability for vehicular networks. First, a task partition and scheduling algorithm (TPSA) is proposed to decide the workload allocation and schedule the execution order of the tasks offloaded to the edge servers given a computation offloading strategy. Second, an artificial intelligence (AI) based collaborative computing approach is developed to determine the task offloading, computing, and result delivery policy for vehicles. Specifically, the offloading and computing problem is formulated as a Markov decision process. A deep reinforcement learning technique, i.e., deep deterministic policy gradient, is adopted to find the optimal solution in a complex urban transportation network. By our approach, the service cost, which includes computing service latency and service failure penalty, can be minimized via the optimal workload assignment and server selection in collaborative computing. Simulation results show that the proposed AI-based collaborative computing approach can adapt to a highly dynamic environment with outstanding performance., Comment: 31 pages, single column, 12 figures. Accepted in IEEE Transactions on Cognitive Communications and Networking

In this paper, we study unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) with the objective to optimize computation offloading with minimum UAV energy consumption. In the considered scenario, a UAV plays the role of an aerial cloudlet to collect and process the computation tasks offloaded by ground users. Given the service requirements of users, we aim to maximize UAV energy efficiency by jointly optimizing the UAV trajectory, the user transmit power, and computation load allocation. The resulting optimization problem corresponds to nonconvex fractional programming, and the Dinkelbach algorithm and the successive convex approximation (SCA) technique are adopted to solve it. Furthermore, we decompose the problem into multiple subproblems for distributed and parallel problem solving. To cope with the case when the knowledge of user mobility is limited, we adopt a spatial distribution estimation technique to predict the location of ground users so that the proposed approach can still be applied. Simulation results demonstrate the effectiveness of the proposed approach for maximizing the energy efficiency of UAV., Comment: 33 pages, 7 figures, IEEE single-column journal

In this paper, the content service provision of information-centric vehicular networks (ICVNs) is investigated from the aspect of mobile edge caching, considering the dynamic driving-related context information. To provide up-to-date information with low latency, two schemes are designed for cache update and content delivery at the roadside units (RSUs). The roadside unit centric (RSUC) scheme decouples cache update and content delivery through bandwidth splitting, where the cached content items are updated regularly in a round-robin manner. The request adaptive (ReA) scheme updates the cached content items upon user requests with certain probabilities. The performance of both proposed schemes are analyzed, whereby the average age of information (AoI) and service latency are derived in closed forms. Surprisingly, the AoI-latency trade-off does not always exist, and frequent cache update can degrade both performances. Thus, the RSUC and ReA schemes are further optimized to balance the AoI and latency. Extensive simulations are conducted on SUMO and OMNeT++ simulators, and the results show that the proposed schemes can reduce service latency by up to 80\% while guaranteeing content freshness in heavily loaded ICVNs., Comment: IEEE Transactions on Mobile Computing (under revision)

The constructions of configuration interaction (CI) expansions from a matrix-product state (MPS) involves numerous matrix operations and the skillful sampling of important configurations when in a huge Hilbert space. In this work, we present an efficient procedure for constructing CI expansions from MPS using the Charm++ parallel programming framework, upon which automatic load balancing and object migration facilities can be employed. This procedure was employed in the MPS-to-CI utility (Moritz et al., J. Chem. Phys. 2007, 126, 224109), sampling-reconstructed complete active space algorithm (SR-CAS, Boguslawski et al., J. Chem. Phys. 2011, 134, 224101) and entanglement-driven genetic algorithm (EDGA, Luo et al., J. Chem. Theory Comput. 2017, 13, 4699-4710). It enhances productivity and allows the sampling programs evolve to their population-expansion versions (e.g., EDGA with population expansion [PE-EDGA]). Examples of 1,2-dioxetanone and firefly dioxetanone anion (FDO-) molecules demonstrated that 1) the procedure could be flexibly employed among various multi-core architectures; 2) the parallel efficiencies could be persistently improved simply by increasing the proportion of asynchronous executions; 3) PE-EDGA could construct a CAS-type CI wave function from a huge Hilbert space, with 0.9952 CI completeness and 96.7% correlation energy via ~1.66x10^6 configurations (only 0.0000028% of the total configurations) of a bi-radical state of FDO- molecule using the full valence active space within a few hours.

In this paper, we design dynamic probabilistic caching for the scenario when the instantaneous content popularity may vary with time while it is possible to predict the average content popularity over a time window. Based on the average content popularity, optimal content caching probabilities can be found, e.g., from solving optimization problems, and existing results in the literature can implement the optimal caching probabilities via static content placement. The objective of this work is to design dynamic probabilistic caching that: i) converge (in distribution) to the optimal content caching probabilities under time-invariant content popularity, and ii) adapt to the time-varying instantaneous content popularity under time-varying content popularity. Achieving the above objective requires a novel design of dynamic content replacement because static caching cannot adapt to varying content popularity while classic dynamic replacement policies, such as LRU, cannot converge to target caching probabilities (as they do not exploit any content popularity information). We model the design of dynamic probabilistic replacement policy as the problem of finding the state transition probability matrix of a Markov chain and propose a method to generate and refine the transition probability matrix. Extensive numerical results are provided to validate the effectiveness of the proposed design., Comment: 13 pages; 10 figures

In the second part of this two-part paper, we extend the study of dynamic caching via state transition field (STF) to the case of time-varying content popularity. The objective of this part is to investigate the impact of time-varying content popularity on the STF and how such impact accumulates to affect the performance of a replacement scheme. Unlike the case in the first part, the STF is no longer static over time, and we introduce instantaneous STF to model it. Moreover, we demonstrate that many metrics, such as instantaneous state caching probability and average cache hit probability over an arbitrary sequence of requests, can be found using the instantaneous STF. As a steady state may not exist under time-varying content popularity, we characterize the performance of replacement schemes based on how the instantaneous STF of a replacement scheme after a content request impacts on its cache hit probability at the next request. From this characterization, insights regarding the relations between the pattern of change in the content popularity, the knowledge of content popularity exploited by the replacement schemes, and the effectiveness of these schemes under time-varying popularity are revealed. In the simulations, different patterns of time-varying popularity, including the shot noise model, are experimented. The effectiveness of example replacement schemes under time-varying popularity is demonstrated, and the numerical results support the observations from the analytic results., Comment: 6 figures

This two-part paper investigates cache replacement schemes with the objective of developing a general model to unify the analysis of various replacement schemes and illustrate their features. To achieve this goal, we study the dynamic process of caching in the vector space and introduce the concept of state transition field (STF) to model and characterize replacement schemes. In the first part of this work, we consider the case of time-invariant content popularity based on the independent reference model (IRM). In such case, we demonstrate that the resulting STFs are static, and each replacement scheme leads to a unique STF. The STF determines the expected trace of the dynamic change in the cache state distribution, as a result of content requests and replacements, from any initial point. Moreover, given the replacement scheme, the STF is only determined by the content popularity. Using four example schemes including random replacement (RR) and least recently used (LRU), we show that the STF can be used to analyze replacement schemes such as finding their steady states, highlighting their differences, and revealing insights regarding the impact of knowledge of content popularity. Based on the above results, STF is shown to be useful for characterizing and illustrating replacement schemes. Extensive numeric results are presented to demonstrate analytical STFs and STFs from simulations for the considered example replacement schemes., Comment: 30 pages, 11 figures

In this paper, we propose a contention intensity based distributed coordination (CIDC) scheme for safety message broadcast. By exploiting the high-frequency and periodical features of the safety message broadcast, the application-layer design of the CIDC enables each vehicle to estimate the instantaneous channel contention intensity in a fully distributed manner. With the contention intensity information, the MAC layer design of CIDC allows vehicles to adopt a better channel access strategy compared to the 802.11p. This is because CIDC selects the initial back-off counter for each new packet deterministically, i.e., based on the contention intensity, instead of randomly. The proposed CIDC is modeled, and key performance indicators in terms of the packet collision probability and average contention delay, are derived. It is shown that the proposed change in the initial counter selection leads to a system model completely different from the classic Markov chain based model. Moreover, the proposed CIDC, fully distributed and compatible with the 802.11p, can achieve both a much lower collision probability and a smaller contention delay compared with 802.11p at the cost of a small communication and computation overhead. Extensive simulation results demonstrate the effectiveness of the CIDC in both of the accurate and the erroneous contention intensity estimation scenarios., Comment: 25 Pages, 8 Figures

Classic network utility maximization problems are usually solved assuming all information is available, implying that information not locally available is always truthfully reported. This may not be practical in all scenarios, especially in distributed/semi-distributed networks. In this paper, incentive for truthful reporting in network optimizations with local information is studied. A novel general model for extending network utility maximization (NUM) problems to incorporate local information is proposed, which allows each user to choose its own objective locally and/or privately. Two specific problems, i.e., a user-centric problem (UCP) and a network-centric problem (NCP), are studied. In the UCP, a network center aims to maximize the collective benefit of all users, and truthful reporting from the users regarding their local information is necessary for finding the solution. We show that the widely-adopted dual pricing cannot guarantee truthful information reporting from a user unless the resource is over-supplied or the price is too high for this user to afford. In the NCP, the network center has its own objective and preferred solution, and incentive needs to be provided for the users to adopt the solution it prefers. Truthful reporting from users is necessary for the center to determine the incentives and achieve its solution. For two-user and multiuser cases, we propose two mechanisms to motivate truthful reporting from users while guaranteeing nonnegative utility gains for both the users and the center. A case study on underlay D2D communication illustrates the application of the UCP and NCP. Simulations are conducted for the D2D application to validate the analytical results and demonstrate the proposed mechanisms., Comment: 29 Pages, 10 figures