Search

Rapid advancements have been made in extending Large Language Models (LLMs) to Large Multi-modal Models (LMMs). However, extending input modality of LLMs to video data remains a challenging endeavor, especially for long videos. Due to insufficient access to large-scale high-quality video data and the excessive compression of visual features, current methods exhibit limitations in effectively processing long videos. In this paper, we introduce Kangaroo, a powerful Video LMM aimed at addressing these challenges. Confronted with issue of inadequate training data, we develop a data curation system to build a large-scale dataset with high-quality annotations for vision-language pre-training and instruction tuning. In addition, we design a curriculum training pipeline with gradually increasing resolution and number of input frames to accommodate long videos. Evaluation results demonstrate that, with 8B parameters, Kangaroo achieves state-of-the-art performance across a variety of video understanding benchmarks while exhibiting competitive results on others. Particularly, on benchmarks specialized for long videos, Kangaroo excels some larger models with over 10B parameters and proprietary models.

High-quality 3D photoacoustic imaging (PAI) reconstruction under sparse view or limited view has long been challenging. Traditional 3D iterative-based reconstruction methods suffer from both slow speed and high memory consumption. Recently, in computer graphics, the differentiable rendering has made significant progress, particularly with the rise of 3D Gaussian Splatting. Inspired by these, we introduce differentiable radiation into PAI, developing a novel reconstruction algorithm: the Sliding Ball Adaptive Growth algorithm (SlingBAG) for 3D PAI, which shows ability in high-quality 3D PAI reconstruction both under extremely sparse view and limited view. We established the point cloud dataset in PAI, and used unique differentiable rapid radiator based on the spherical decomposition strategy and the randomly initialized point cloud adaptively optimized according to sparse sensor data. Each point undergoes updates in 3D coordinates, initial pressure, and resolution (denoted by the radius of ball). Points undergo adaptive growth during iterative process, including point destroying, splitting and duplicating along the gradient of their positions, manifesting the sliding ball effect. Finally, our point cloud to voxel grid shader renders the final reconstruction results. Simulation and in vivo experiments demonstrate that our SlingBAG reconstruction result's SNR can be more than 40 dB under extremely sparse view, while the SNR of traditional back-projection algorithm's result is less than 20 dB. Moreover, the result of SlingBAG's structural similarity to the ground truth is significantly higher, with an SSIM value of 95.6%. Notably, our differentiable rapid radiator can conduct forward PA simulation in homogeneous, non-viscous media substantially faster than current methods that numerically simulate the wave propagation, such as k-Wave. The dataset and all code will be open source.

With the explosive growth of vehicle applications, vehicular networks based on millimeter wave (mmWave) bands have attracted interests from both academia and industry. mmWave communications are able to utilize the huge available bandwidth to provide multiple Gbps transmission rates among vehicles. In this paper, we address the content distribution scheduling problem in mmWave vehicular networks. It has been challenging for all vehicles in the same network to complete content downloading due to the limited communication resources of roadside units (RSUs) and the high mobility of vehicles. We propose a joint vehicle-to-infrastructure (V2I) and vehicle-tovehicle (V2V) scheduling scheme to minimize the total number of content distribution time slots from a global optimization perspective. In the V2I phase, the RSU serially transmits integrity content to vehicles, which are selected according to the vehicular network topology and transmission scheduling scheme. In the V2V phase, full-duplex communications and concurrent transmissions are exploited to achieve content sharing between vehicles and improve transmission efficiency. Performance evaluations demonstrate that our proposed scheme reduces the number of time slots and significantly improves system throughput when compared with other schemes, especially under large-size file transfers and a large number of vehicles., Comment: 12 pages, 11 figures, IEEE Transactions on Vehicular Technology

With the explosive growth of mobile traffic demand, densely deployed small cells underlying macrocells have great potential for 5G and beyond wireless networks. In this paper, we consider the problem of supporting traffic flows with diverse QoS requirements by exploiting three high frequency bands, i.e., the 28GHz band, the E-band, and the Terahertz (THz) band. The cooperation of the three bands is helpful for maximizing the number of flows with their QoS requirements satisfied. To solve the formulated nonlinear integer programming problem, we propose a triple-band scheduling scheme which can select the optimum scheduling band for each flow among three different frequency bands. The proposed scheme also efficiently utilizes the resource to schedule flow transmissions in time slots. Extensive simulations demonstrate the superior performance of the proposed scheme over three baseline schemes with respect to the number of completed flows and the system throughput., Comment: 12 pages, 10 figures, Journal of Communications and Networks

To exploit richer spectrum resources for even better service quality, millimeter wave (mmWave) communication has been considered for high-speed railway (HSR) communication systems. In this paper, we focus on scheduling as many flows as possible while satisfying their QoS requirements. Due to interference, eavesdropping, or other problems, some flows may not be directly transmitted from the track-side BS. In this paper, we propose an UAV-assisted scheduling scheme which utilizes a UAV to serve as relay for such flows. The proposed scheme also utilize two mmWave bands, one for the BS links and the other for the UAV links. The proposed algorithm aims to maximize the number of flows with their QoS requirements satisfied. Simulations demonstrate that the proposed scheme achieves a superior performance on the number of completed flows and the system throughput over two baseline schemes., Comment: 11 pages, 7 figures, article

The millimeter wave (mmWave) communication has drawn intensive attention with abundant band resources. In this paper, we consider the popular content distribution (PCD) problem in the mmWave vehicular network. In order to offload the communication burden of base stations (BSs), vehicle-to-vehicle (V2V) communication is introduced into the PCD problem to transmit contents between on-board units (OBUs) and improve the transmission efficiency. We propose a full-duplex (FD) cooperative scheme based on coalition formation game, and the utility function is provided based on the maximization of the number of received contents. The contribution of each member in the coalition can be transferable to its individual profit. While maximizing the number of received contents in the fixed time, the cooperative scheme also ensures the individual profit of each OBU in the coalition. We evaluate the proposed scheme by extensive simulations in mmWave vehicular networks. Compared with other existing schemes, the proposed scheme has superior performances on the number of possessed contents and system fairness. Besides, the low complexity of the proposed algorithm is demonstrated by the switch operation number and CPU time., Comment: 12 pages, 8 figures, IEEE Transactions on Vehicular Technology