Search

Comprehensive evaluation of Multimodal Large Language Models (MLLMs) has recently garnered widespread attention in the research community. However, we observe that existing benchmarks present several common barriers that make it difficult to measure the significant challenges that models face in the real world, including: 1) small data scale leads to a large performance variance; 2) reliance on model-based annotations results in restricted data quality; 3) insufficient task difficulty, especially caused by the limited image resolution. To tackle these issues, we introduce MME-RealWorld. Specifically, we collect more than $300$K images from public datasets and the Internet, filtering $13,366$ high-quality images for annotation. This involves the efforts of professional $25$ annotators and $7$ experts in MLLMs, contributing to $29,429$ question-answer pairs that cover $43$ subtasks across $5$ real-world scenarios, extremely challenging even for humans. As far as we know, MME-RealWorld is the largest manually annotated benchmark to date, featuring the highest resolution and a targeted focus on real-world applications. We further conduct a thorough evaluation involving $28$ prominent MLLMs, such as GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet. Our results show that even the most advanced models struggle with our benchmarks, where none of them reach $60\%$ accuracy. The challenges of perceiving high-resolution images and understanding complex real-world scenarios remain urgent issues to be addressed. The data and evaluation code are released at https://mme-realworld.github.io/ ., Comment: Project Page: https://mme-realworld.github.io/

We present quasi-real-time dual-comb spectroscopy (DCS) using two Yb:fiber combs with ~750 MHz repetition rates. A computational coherent averaging technique is employed to correct timing and phase fluctuations of the measured dual-comb interferogram (IGM). Quasi-real-time phase correction of 1-ms long acquisitions occurs every 1.5 seconds and is assisted by coarse radio frequency (RF) phase-locking of an isolated RF comb mode. After resampling and global offset phase correction, the RF comb linewidth is reduced from 200 kHz to ~1 kHz, while the line-to-floor ratio increases 13 dB in power in 1 ms. Using simultaneous offset frequency correction in opposite phases, we correct the aliased RF spectrum spanning three Nyquist zones, which yields an optical coverage of ~180 GHz around 1.035 $\mu$m probed on a sub-microsecond timescale. The absorption profile of gaseous acetylene is observed to validate the presented technique., Comment: 11 pages, 7 figures

We demonstrate balanced-detection dual-comb spectroscopy (DCS) using two free-running mechanical sharing dual-comb fiber lasers assisted by an all-computational digital phase correction algorithm. The mutual coherence between the combs allows us perform mode-resolved spectroscopy of gaseous hydrogen cyanide by digitally compensating residual timing and offset frequency fluctuations of the dual-comb signal. Setting the repetition rate difference between the combs to 500 Hz (1.5 kHz) yields more than 2000 resolved radio frequency comb lines after phase correction in a 3-dB bandwidth centered at 1560 nm of wavelength. Through coadding the corrected interferograms (IGMs), we obtain a single time-domain trace with a SNR of 6378 (13960) and 12.64 (13.77) bits of dynamic range in 1 second of averaging. The spectral SNR of the coadded trace reaches 529 (585), corresponding to a figure of merit of SNR of 1.3$\times$10$^6$ (1.4$\times$10$^6$). The measured absorption spectrum of hydrogen cyanide agrees well with the HITRAN database., Comment: 12 pages, 5 figures

Optical frequency combs are laser sources which are capable of generating discrete, equal-spaced and highly coherent comb modes. Optical frequency comb technique provides a significant bridge to transfer the stability between optical frequency and radio frequency. The advances of this technology greatly promote the development of precision spectroscopy, optical time/frequency transfer, optical frequency division, long-distance transfer of time/frequency references and high-precision distance measurement. Benefiting from the wide spectral outputs, femtosecond lasers have become the best choice for the fulfillment of optical frequency combs. Within the precise control of the repetition frequency and carrier-envelope offset frequency of the pulse train from femtosecond lasers, a stable optical frequency comb both in the time domain and frequency domain can be obtained. This dissertation presents the precise control of repetition rate, carrier-envelope offset frequency and coherent pulse synthesis in optical frequency combs., Comment: in Chinese language. KEY WORDS: Timing jitter; carrier-envelope phase; optical frequency comb; coherent pulse synthesis

In this letter, we derive a fully-stabilized narrow-linewidth optical frequency comb (OFC) reference to a kilometer-long fiber delay line for the first time, to the best of our knowledge. The 1537-nm comb modes and 1566-nm comb modes in the OFC are phase-locked to the fiber delay line with 40-kHz locking bandwidth. From out-of-loop measurement, the 1542-nm comb mode has residual phase noise of 925 mrad (integrated from 10 MHz to 1 kHz), fractional frequency stability of 9.13*10(-13) at 12.8 ms average time and 580 Hz linewidth. The linewidth has been compressed by a factor of ~ 170 compared to the free-running condition. Short-term stability of presented OFC exceeds most commercial microwave oscillators. The entire phase-locking system is compact and highly-integrated benefiting from absence of optical amplifiers, f-2f interferometers and optical/radio references. The presented OFC shows significant potential of being reliable laser source in low-noise-OFC-based precise metrology, microwave generation and dual-comb spectroscopic applications outside the laboratory., Comment: 14 pages,9 figures

Investigation on the relative phase evolution between two bounded optical solitons is essential for its potential applications in development of larger telecommunication capacity of optical fiber transmission lines, resolution improvement in advancing ultrafast characterization approaches and development of all-optical information storage. Here we characterized relative comb-line phase jitter power spectral density (PSD) of a soliton molecule pair generated from a passively mode-locked Er:fiber laser. Through tracking the intensity difference between two selected wavelengths in one certain spectral interference fringe, the relative comb-line phase noise PSD is measured by balanced detection. The estimated measurement resolution is at 10^(-14) rad2/Hz level and the estimated integrated phase noise from 10 MHz to 100 Hz is 2.04 mrad. The estimated relative linewidth is far below 1 mHz. Comparison between phase noise PSD and intensity noise PSD indicates that AM-PM conversion plays an important role in relative phase jitter dynamics between the two solitons. Our spectral interference fringe tracking technique is attractive for its simplicity and shows potential in ultra-high resolution in phase noise measurement, providing an ultra-sensitive alternative approach for relative phase noise stabilization of optical soliton molecules, between two independent mode-locked lasers and optical length stabilization of interferometers., Comment: 9 pages, 5 figures, to be published in Optics Communications, CF-260