Search

We introduce OmChat, a model designed to excel in handling long contexts and video understanding tasks. OmChat's new architecture standardizes how different visual inputs are processed, making it more efficient and adaptable. It uses a dynamic vision encoding process to effectively handle images of various resolutions, capturing fine details across a range of image qualities. OmChat utilizes an active progressive multimodal pretraining strategy, which gradually increases the model's capacity for long contexts and enhances its overall abilities. By selecting high-quality data during training, OmChat learns from the most relevant and informative data points. With support for a context length of up to 512K, OmChat demonstrates promising performance in tasks involving multiple images and videos, outperforming most open-source models in these benchmarks. Additionally, OmChat proposes a prompting strategy for unifying complex multimodal inputs including single image text, multi-image text and videos, and achieving competitive performance on single-image benchmarks. To further evaluate the model's capabilities, we proposed a benchmark dataset named Temporal Visual Needle in a Haystack. This dataset assesses OmChat's ability to comprehend temporal visual details within long videos. Our analysis highlights several key factors contributing to OmChat's success: support for any-aspect high image resolution, the active progressive pretraining strategy, and high-quality supervised fine-tuning datasets. This report provides a detailed overview of OmChat's capabilities and the strategies that enhance its performance in visual understanding., Comment: 14 pages

The wireless blockchain network (WBN) concept, born from the blockchain deployed in wireless networks, has appealed to many network scenarios. Blockchain consensus mechanisms (CMs) are key to enabling nodes in a wireless network to achieve consistency without any trusted entity. However, consensus reliability will be seriously affected by the instability of communication links in wireless networks. Meanwhile, it is difficult for nodes in wireless scenarios to obtain a timely energy supply. Energy-intensive blockchain functions can quickly drain the power of nodes, thus degrading consensus performance. Fortunately, a symbiotic radio (SR) system enabled by cognitive backscatter communications can solve the above problems. In SR, the secondary transmitter (STx) transmits messages over the radio frequency (RF) signal emitted from a primary transmitter (PTx) with extremely low energy consumption, and the STx can provide multipath gain to the PTx in return. Such an approach is useful for almost all vote-based CMs, such as the Practical Byzantine Fault-tolerant (PBFT)-like and the RAFT-like CMs. This paper proposes symbiotic blockchain consensus (SBC) by transforming 6 PBFT-like and 4 RAFT-like state-of-the-art (SOTA) CMs to demonstrate universality. These new CMs will benefit from mutualistic transmission relationships in SR, making full use of the limited spectrum resources in WBN. Simulation results show that SBC can increase the consensus success rate of PBFT-like and RAFT-like by 54.1% and 5.8%, respectively, and reduce energy consumption by 9.2% and 23.7%, respectively., Comment: arXiv admin note: text overlap with arXiv:2309.16692

In space-air-ground integrated networks (SAGINs), cognitive spectrum sharing has been regarded as a promising solution to improve spectrum efficiency by enabling a secondary network to access the spectrum of a primary network. However, different networks in SAGIN may have different quality of service (QoS) requirements, which can not be well satisfied with the traditional cognitive spectrum sharing architecture. For example, the aerial network typically has high QoS requirements, which however may not be met when it acts as a secondary network. To address this issue, in this paper, we propose a hierarchical cognitive spectrum sharing architecture (HCSSA) for SAGINs, where the secondary networks are divided into a preferential one and an ordinary one. Specifically, the aerial and terrestrial networks can access the spectrum of the satellite network under the condition that the caused interference to the satellite terminal is below a certain threshold. Besides, considering that the aerial network has a higher priority than the terrestrial network, we aim to use a rate constraint to ensure the performance of the aerial network. Subject to these two constraints, we consider a sum-rate maximization for the terrestrial network by jointly optimizing the transmit beamforming vectors of the aerial and terrestrial base stations. To solve this non-convex problem, we propose a penalty-based iterative beamforming (PIBF) scheme that uses the penalty method and the successive convex approximation technique. Moreover, we also develop three low-complexity schemes by optimizing the normalized beamforming vectors and power control. Finally, we provide extensive numerical simulations to compare the performance of the proposed PIBF scheme and the low-complexity schemes. The results also demonstrate the advantages of the proposed HCSSA compared with the traditional cognitive spectrum sharing architecture.

Symbiotic radio (SR) is a promising solution to achieve high spectrum- and energy-efficiency due to its spectrum sharing and low-power consumption properties, in which the secondary system achieves data transmissions by backscattering the signal originating from the primary system. In this paper, we are interested in the pilot design and signal detection when the primary transmission adopts orthogonal frequency division multiplexing (OFDM). In particular, to preserve the channel orthogonality among the OFDM sub-carriers, each secondary symbol is designed to span an entire OFDM symbol. The comb-type pilot structure is employed by the primary transmission, while the preamble pilot structure is used by the secondary transmission. With the designed pilot structures, the primary signal can be detected via the conventional methods by treating the secondary signal as a part of the composite channel, i.e., the effective channel of the primary transmission. Furthermore, the secondary signal can be extracted from the estimated composite channel with the help of the detected primary signal. The bit error rate (BER) performance with both perfect and estimated CSI, the diversity orders of the primary and secondary transmissions, and the sensitivity to symbol synchronization error are analyzed. Simulation results show that the performance of the primary transmission is enhanced thanks to the backscatter link established by the secondary transmission. More importantly, even without the direct link, the primary and secondary transmissions can be supported via only the backscatter link., Comment: This paper has been accepted for publication in IEEE Transactions on Wireless Communications

This paper studies the reconfigurable intelligent surface (RIS)-assisted symbiotic radio (SR) system, where an RIS acts as a secondary transmitter to transmit its information by leveraging the primary signal as its RF carrier and simultaneously assists the primary transmission. Conventionally, all reflecting elements of the RIS are used to transmit the secondary signal, which, however, would limit its capability for assisting the primary transmission. To address this issue, we propose a novel RIS partitioning scheme, where the RIS is partitioned into two sub-surfaces, one to assist the primary transmission and the other to transmit the secondary signal. Naturally, there exists a fundamental tradeoff between the assistance and transmission capabilities of RIS regarding the surface partitioning strategy. Considering the coupling effect between the primary and secondary transmissions, we focus on the detection of the composite signal formed by the primary and secondary ones, based on which we propose a novel two-step detector. Then, we formulate the assistance-transmission tradeoff problem to minimize the bit error rate (BER) of the composite signal by jointly optimizing the surface partitioning strategy and the phase shifts of the two sub-surfaces, such that the overall BER of RIS-assisted SR is minimized. By solving this problem, we show that the optimized surface partitioning strategy depends on the channel strength ratio of the direct link to the reflected link. Finally, extensive simulations show that our proposed RIS partitioning scheme outperforms the conventional schemes which use all reflecting elements for either assistance or transmission., Comment: 17 pages, 17 figures

In reconfigurable intelligent surface (RIS)-assisted symbiotic radio (SR), the RIS acts as a secondary transmitter by modulating its information bits over the incident primary signal and simultaneously assists the primary transmission, then a cooperative receiver is used to jointly decode the primary and secondary signals. Most existing works of SR focus on using RIS to enhance the reflecting link while ignoring the ambiguity problem for the joint detection caused by the multiplication relationship of the primary and secondary signals. Particularly, in case of a blocked direct link, joint detection will suffer from severe performance loss due to the ambiguity, when using the conventional on-off keying and binary phase shift keying modulation schemes for RIS. To address this issue, we propose a novel modulation scheme for RIS-assisted SR that divides the phase-shift matrix into two components: the symbol-invariant and symbol-varying components, which are used to assist the primary transmission and carry the secondary signal, respectively. To design these two components, we focus on the detection of the composite signal formed by the primary and secondary signals, through which a problem of minimizing the bit error rate (BER) of the composite signal is formulated to improve both the BER performance of the primary and secondary ones. By solving the problem, we derive the closed-form solution of the optimal symbol-invariant and symbol-varying components, which is related to the channel strength ratio of the direct link to the reflecting link. Moreover, theoretical BER performance is analyzed. Finally, simulation results show the superiority of the proposed modulation scheme over its conventional counterpart., Comment: 16 pages,16 figures

Quick UDP Internet Connection (QUIC) is an emerging end-to-end encrypted, transport-layer protocol, which has been increasingly adopted by popular web services to improve communication security and quality of experience (QoE) towards end-users. However, this tendency makes the traffic analysis more challenging, given the limited information in the QUIC packet header and full encryption on the payload. To address this challenge, a novel rule-based approach is proposed to estimate the application-level traffic attributes without decrypting QUIC packets. Based on the size, timing, and direction information, our proposed algorithm analyzes the associated network traffic to infer the identity of each HTTP request and response pair, as well as the multiplexing feature in each QUIC connection. The inferred HTTP attributes can be used to evaluate the QoE of application-layer services and identify the service categories for traffic classification in the encrypted QUIC connections.

In this paper, the finite horizon asymmetric information linear quadratic (LQ) control problem is investigated for a discrete-time mean field system. Different from previous works, multiple controllers with different information sets are involved in the mean field system dynamics. The coupling of different controllers makes it quite difficult in finding the optimal control strategy. Fortunately, by applying the Pontryagin's maximum principle, the corresponding decentralized control problem of the finite horizon is investigated. The contributions of this paper can be concluded as: For the first time, based on the solution of a group of mean-field forward and backward stochastic difference equations (MF-FBSDEs), the necessary and sufficient solvability conditions are derived for the asymmetric information LQ control for the mean field system with multiple controllers. Furthermore, by the use of an innovative orthogonal decomposition approach, the optimal decentralized control strategy is derived, which is based on the solution to a non-symmetric Riccati-type equation.

Object detection (OD) in computer vision has made significant progress in recent years, transitioning from closed-set labels to open-vocabulary detection (OVD) based on large-scale vision-language pre-training (VLP). However, current evaluation methods and datasets are limited to testing generalization over object types and referral expressions, which do not provide a systematic, fine-grained, and accurate benchmark of OVD models' abilities. In this paper, we propose a new benchmark named OVDEval, which includes 9 sub-tasks and introduces evaluations on commonsense knowledge, attribute understanding, position understanding, object relation comprehension, and more. The dataset is meticulously created to provide hard negatives that challenge models' true understanding of visual and linguistic input. Additionally, we identify a problem with the popular Average Precision (AP) metric when benchmarking models on these fine-grained label datasets and propose a new metric called Non-Maximum Suppression Average Precision (NMS-AP) to address this issue. Extensive experimental results show that existing top OVD models all fail on the new tasks except for simple object types, demonstrating the value of the proposed dataset in pinpointing the weakness of current OVD models and guiding future research. Furthermore, the proposed NMS-AP metric is verified by experiments to provide a much more truthful evaluation of OVD models, whereas traditional AP metrics yield deceptive results. Data is available at \url{https://github.com/om-ai-lab/OVDEval}, Comment: Long paper accepted at AAAI 2024

Wireless blockchain networks have played an important role in many network scenarios, among which wireless Practical Byzantine Fault Tolerance (PBFT) consensus is regarded as one of the most important consensus mechanisms. It enables nodes in wireless networks to reach consistency without any trusted entity. However, due to the instability of wireless communication links, the reliability of the PBFT consensus will be seriously affected. Meanwhile, it is difficult for nodes in wireless scenarios to obtain a timely energy supply. The high-energy-consumption blockchain functions will quickly consume the power of nodes, thus, affecting consensus performance. Fortunately, the symbiotic radio (SR) system enabled by cognitive backscatter communications can provide a solution to the above problems. In SR, the secondary transmitter (STx) transmits messages by modulating its information over the radio frequency (RF) signal of the primary transmitter (PTx) with extremely low energy consumption, and the STx can provide multipath gain to the PTx in return. In our paper, we propose the symbiotic PBFT (S-PBFT) consensus benefited from the mutualistic transmission in SR, which can increase the consensus security by 54.82%, and save energy consumption by about 10%., Comment: IEEE Global Communications Conference 2023

In this paper, we are interested in a reconfigurable intelligent surface (RIS)-assisted symbiotic radio (SR) system, where an RIS assists a primary transmission by passive beamforming and simultaneously acts as an information transmitter by periodically adjusting its reflection coefficients. The above RIS functions innately enable a new multiplicative multiple access channel (M-MAC), where the primary and secondary signals are superposed in a multiplicative manner. To pursue the fundamental performance limits of M-MAC, we focus on the characterization of the capacity region for such a system when the direct link is blocked. Due to the reflection nature of RIS, the signal transmitted from the RIS should satisfy a passive reflection constraint. We consider two types of passive reflection constraints, one for the case that only the phases of the RIS can be adjusted, while the other for the case that both the amplitude and the phase can be adjusted. Under the passive reflection constraints at the RIS as well as the average power constraint at the primary transmitter (PTx), we characterize the capacity region of RIS-assisted SR. It is observed that: 1) the number of sum-rate-optimal points on the boundary of the capacity region is infinite; 2) for the rate pairs with the maximum sum rate, the optimal distribution of the amplitude of the primary signal is a continuous Rayleigh distribution, while for the remaining rate pairs on the capacity region boundary, the optimal amplitude of the primary signal is discrete; 3) the adjustment of the amplitude for the RIS can enlarge the capacity region as compared to the phase-adjusted-only case.