Search

We present MEGA-Bench, an evaluation suite that scales multimodal evaluation to over 500 real-world tasks, to address the highly heterogeneous daily use cases of end users. Our objective is to optimize for a set of high-quality data samples that cover a highly diverse and rich set of multimodal tasks, while enabling cost-effective and accurate model evaluation. In particular, we collected 505 realistic tasks encompassing over 8,000 samples from 16 expert annotators to extensively cover the multimodal task space. Instead of unifying these problems into standard multi-choice questions (like MMMU, MMBench, and MMT-Bench), we embrace a wide range of output formats like numbers, phrases, code, \LaTeX, coordinates, JSON, free-form, etc. To accommodate these formats, we developed over 40 metrics to evaluate these tasks. Unlike existing benchmarks, MEGA-Bench offers a fine-grained capability report across multiple dimensions (e.g., application, input type, output format, skill), allowing users to interact with and visualize model capabilities in depth. We evaluate a wide variety of frontier vision-language models on MEGA-Bench to understand their capabilities across these dimensions., Comment: Technical report. Project page: https://tiger-ai-lab.github.io/MEGA-Bench/

Traditional terrestrial communication infrastructures often fail to collect the timely information from Internet of Thing (IoT) devices in remote areas. To address this challenge, we investigate a Satellite-unmanned aerial vehicles (UAV) integrated Non-terrestrial network (NTN), where the UAV is controlled by remote control center via UAV-to-Satellite connections. To maximize the energy efficiency (EE) of the UAV, we optimize the UAV trajectory, power allocation, and state sensing strategies, while guaranteing the control stability and communication reliability. This challenging problem is addressed using an efficient algorithm, incorporating a Deep Q-Network (DQN)-based trajectory determination, a closed form of power allocation, and one-dimensional searching for sensing. Numerical simulations are conducted to validate the effectiveness of our approach. The results showcase the data size of collection has a greater impact than transmission power, and reveal the relationship among sensing interval, communication maximum power and control performance. This study provides promising solutions and valuable insights for efficient data collection in remote IoT.

In wireless sensors networks, integrating localization and communications techniques is crucial for efficient spectrum and hardware utilization. In this paper, we present a novel framework of unmanned aerial vehicle (UAV)-aided localization and communication for ground node (GN), where the average spectral efficiency (SE) is used to reveal the intricate relationship among frame structure, channel estimation error, and localization accuracy. In particular, we first derive the lower bounds for channel estimation error and the three dimensional location prediction error. Leveraging these comprehensive analysis, we formulate a problem to maximize the average SE in UAV-GN communication, where the frame structure, beamwidth and power allocation are jointly optimized. Subsequently, we propose an efficient iterative algorithm to address this non-convex problem with closed-form expressions for beamwidth and power allocation. Numerical results demonstrate that the performance of our proposed method can approach the upper bound with much lower complexity, and achieve over 70\% performance gain compared to non-localization benchmarks. Additionally, the analysis highlights the dominant impacts from the Doppler effect over noise on the average SE.

Since the traffic administration at road intersections determines the capacity bottleneck of modern transportation systems, intelligent cooperative coordination for connected autonomous vehicles (CAVs) has shown to be an effective solution. In this paper, we try to formulate a Bi-Level CAV intersection coordination framework, where coordinators from High and Low levels are tightly coupled. In the High-Level coordinator where vehicles from multiple roads are involved, we take various metrics including throughput, safety, fairness and comfort into consideration. Motivated by the time consuming space-time resource allocation framework in [1], we try to give a low complexity solution by transforming the complicated original problem into a sequential linear programming one. Based on the "feasible tunnels" (FT) generated from the High-Level coordinator, we then propose a rapid gradient-based trajectory optimization strategy in the Low-Level planner, to effectively avoid collisions beyond High-level considerations, such as the pedestrian or bicycles. Simulation results and laboratory experiments show that our proposed method outperforms existing strategies. Moreover, the most impressive advantage is that the proposed strategy can plan vehicle trajectory in milliseconds, which is promising in realworld deployments. A detailed description include the coordination framework and experiment demo could be found at the supplement materials, or online at https://youtu.be/MuhjhKfNIOg.