Search

The advent of Multimodal Large Language Models, leveraging the power of Large Language Models, has recently demonstrated superior multimodal understanding and reasoning abilities, heralding a new era for artificial general intelligence. However, achieving AGI necessitates more than just comprehension and reasoning. A crucial capability required is effective planning in diverse scenarios, which involves making reasonable decisions based on complex environments to solve real-world problems. Despite its importance, the planning abilities of current MLLMs in varied scenarios remain underexplored. In this paper, we introduce EgoPlan-Bench2, a rigorous and comprehensive benchmark designed to assess the planning capabilities of MLLMs across a wide range of real-world scenarios. EgoPlan-Bench2 encompasses everyday tasks spanning 4 major domains and 24 detailed scenarios, closely aligned with human daily life. EgoPlan-Bench2 is constructed through a semi-automatic process utilizing egocentric videos, complemented by manual verification. Grounded in a first-person perspective, it mirrors the way humans approach problem-solving in everyday life. We evaluate 21 competitive MLLMs and provide an in-depth analysis of their limitations, revealing that they face significant challenges in real-world planning. To further improve the planning proficiency of current MLLMs, we propose a training-free approach using multimodal Chain-of-Thought (CoT) prompting through investigating the effectiveness of various multimodal prompts in complex planning. Our approach enhances the performance of GPT-4V by 10.24 on EgoPlan-Bench2 without additional training. Our work not only sheds light on the current limitations of MLLMs in planning, but also provides insights for future enhancements in this critical area. We have made data and code available at https://qiulu66.github.io/egoplanbench2/., Comment: Code & data are available at: https://qiulu66.github.io/egoplanbench2/

With the exponential growth of data and evolving use cases, petabyte-scale OLAP data platforms are increasingly adopting a model that decouples compute from storage. This shift, evident in organizations like Uber and Meta, introduces operational challenges including massive, read-heavy I/O traffic with potential throttling, as well as skewed and fragmented data access patterns. Addressing these challenges, this paper introduces the Alluxio local (edge) cache, a highly effective architectural optimization tailored for such environments. This embeddable cache, optimized for petabyte-scale data analytics, leverages local SSD resources to alleviate network I/O and API call pressures, significantly improving data transfer efficiency. Integrated with OLAP systems like Presto and storage services like HDFS, the Alluxio local cache has demonstrated its effectiveness in handling large-scale, enterprise-grade workloads over three years of deployment at Uber and Meta. We share insights and operational experiences in implementing these optimizations, providing valuable perspectives on managing modern, massive-scale OLAP workloads., Comment: Accepted to the USENIX Annual Technical Conference (USENIX ATC) 2024

Van der Waals encapsulation of two-dimensional materials within hexagonal boron nitride (h-BN) stacks has proven to be a promising way to create ultrahigh-performance electronic devices. However, contemporary approaches for achieving van der Waals encapsulation, which involve artificial layer stacking using mechanical transfer techniques, are difficult to control, prone to contamination, and unscalable. Here, we report on the transfer-free direct growth of high-quality graphene nanoribbons (GNRs) within h-BN stacks. The as-grown embedded GNRs exhibit highly desirable features being ultralong (up to 0.25 mm), ultranarrow ( < 5 nm), and homochiral with zigzag edges. Our atomistic simulations reveal that the mechanism underlying the embedded growth involves ultralow GNR friction when sliding between AA'-stacked h-BN layers. Using the grown structures, we demonstrate the transfer-free fabrication of embedded GNR field-effect devices that exhibit excellent performance at room temperature with mobilities of up to 4,600 $cm^{2} V^{-1} s^{-1}$ and on-off ratios of up to $10^{6}$. This paves the way to the bottom-up fabrication of high-performance electronic devices based on embedded layered materials.

Digital twin is a modern technology for many advanced applications. To construct a digital twin of a thermal system, it is required to make online estimations of unknown time-varying boundary conditions from sensor measured data, which needs to solve inverse heat transfer problems (IHTPs). However, a fast and accurate solution is challenging since the measured data is normally contaminated with noise and the traditional method to solve IHTP involves significant amount of calculations. Therefore, in this work, a rapid yet robust inversion algorithm called ANN-based extended Kalman smoothing algorithm is developed to realize the online prediction of desired parameter based on the measurements. The fast prediction is realized by replacing the conventional CFD-based state transfer models in extended Kalman smoothing algorithm with pre-trained ANN. Then, a two-dimensional internal convective heat transfer problem was employed as the case study to test the algorithm. The results have proved that the proposed algorithm is a computational-light and robust approach for solving IHTPs. The proposed algorithm can achieve estimation of unknown boundary conditions with a dimensionless average error of 0.0580 under noisy temperature measurement with a standard deviation of 10 K with a drastic reduction of computational cost compared to the conventional approach. Moreover, the effects of training data, location of sensor, future time step selection on the performance of prediction are investigated.

Graphene nanoribbons (GNRs) with widths of a few nanometres are promising candidates for future nano-electronic applications due to their structurally tunable bandgaps, ultrahigh carrier mobilities, and exceptional stability. However, the direct growth of micrometre-long GNRs on insulating substrates, which is essential for the fabrication of nano-electronic devices, remains an immense challenge. Here, we report the epitaxial growth of GNRs on an insulating hexagonal boron nitride (h-BN) substrate through nanoparticle-catalysed chemical vapor deposition (CVD). Ultra-narrow GNRs with lengths of up to 10 {\mu}m are synthesized. Remarkably, the as-grown GNRs are crystallographically aligned with the h-BN substrate, forming one-dimensional (1D) moir\'e superlattices. Scanning tunnelling microscopy reveals an average width of 2 nm and a typical bandgap of ~1 eV for similar GNRs grown on conducting graphite substrates. Fully atomistic computational simulations support the experimental results and reveal a competition between the formation of GNRs and carbon nanotubes (CNTs) during the nucleation stage, and van der Waals sliding of the GNRs on the h-BN substrate throughout the growth stage. Our study provides a scalable, single-step method for growing micrometre-long narrow GNRs on insulating substrates, thus opening a route to explore the performance of high-quality GNR devices and the fundamental physics of 1D moir\'e superlattices.

Data-driven methods have increasingly been applied to the development of optical systems as inexpensive and effective inverse design approaches. Optical properties (e.g., band-gap properties) of photonic crystals (PCs) are closely associated with characteristics of their light reflection spectra. Finding optimal PC constructions (within a pre-specified parameter space) that generate reflection spectra closest to a targeted spectrum is thus an interesting and meaningful inverse design problem, although relevant studies are still limited. Here we report a generally effective machine learning-based inverse design approach for one-dimensional photonic crystals (1DPCs), focusing on visible light spectra which are of high practical relevance. For a given class of 1DPC system, a deep neural network (DNN) in a unified structure is first trained over data from sizeable forward calculations (from layer thicknesses to spectrum). An iterative optimization scheme is then developed based on a coherent integration of DNN backward predictions (from spectrum to layer thicknesses), forward calculations, and Monte Carlo moves. We employ this new approach to four representative 1DPC systems including periodic structures with two-, three-, and four-layer repeating units and a heterostructure. The approach successfully converges to solutions of optimal 1DPC constructions for various targeted spectra regardless of their exact achievability. As two demonstrating examples, inverse designs toward a specially constructed "rectangle-shaped" green-light or red-light reflection spectrum are presented and discussed in detail. Remarkably, the results show that the approach can efficiently find out optimal layer thicknesses even when they are far outside the range covered by the original training data of DNN.

One-dimensional photonic crystals (1D PC) represent a class of periodic optical material, composed of alternating media with different dielectric constants along one direction. The most important property of 1D PCs is their photonic band-gap. However, multiple reflective bands are rarely reported in this research area. In this paper we demonstrate the tunability of multiple reflective bands in conventional 1D PC structure and 1D PC heterostructure. For both two types of 1D PC construction, positions of multiple reflective bands can be regulated under certain principles. In addition, structural color is revealed by transforming reflection spectra into CIE coordinates. It is indicated that the CIE coordinate shifts caused by multiple reflective bands behave quite different compared to those caused by one major photonic band-gap. The two approaches reported in this work may provide insights for the application of 1D PC in areas such as displays, sensors, and decoration.

The relationships between material constructions and reflective spectrum patterns are important properties of photonic crystals. One particular interesting reflectance profile is a high-intensity and uniform three-peak pattern with peak positions right located at the red, green, and blue (RGB, three original colors) region. For ease of construction, a seek for using one-dimensional photonic crystals to achieve RGB triple reflective bands is a meaningful endeavor. Only very limited previous studies exist, all relying on traditional periodic photonic crystals (PPCs) and of large thickness. The underlying physical principles remain elusive, leaving the question of thickness limit to achieve RGB bands unaddressed. Here, we present the first detailed work to explore the thickness limit issue based on both theoretical and experimental investigation. A set of heuristically derived design principles are used to uncover that the break of translational symmetry, thus introducing heterostructure photonic crystals (HPCs), is essential to reduce the total optical path difference (OPD) to ~ 3200nm (the theoretical limit) while still exhibiting high-quality RGB bands. A systematic experiment based on a 12-layer heterostructure construction was performed and well confirmed the theoretical predictions. The associated three-peak properties are successfully used to realize quantum dot fluorescent enhancement phenomena. Furthermore, the HPC exhibits unusually stability against solvent stimulus, in strong contrast to typical behaviors reported in PPCs. Our work for the first time proposes and verifies important rational rules for designing ultrathin HPCs toward RGB reflective bands, and provides insights for a wider range of explorations of light manipulation in photonic crystals.