Search

Your search keyword '"Song, Xinshuai"' showing total 12 results
Start Over Author "Song, Xinshuai"
12 results on '"Song, Xinshuai"'

1. Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method

Author

Song, Xinshuai, Chen, Weixing, Liu, Yang, Chen, Weikai, Li, Guanbin, and Lin, Liang

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

Existing Vision-Language Navigation (VLN) methods primarily focus on single-stage navigation, limiting their effectiveness in multi-stage and long-horizon tasks within complex and dynamic environments. To address these limitations, we propose a novel VLN task, named Long-Horizon Vision-Language Navigation (LH-VLN), which emphasizes long-term planning and decision consistency across consecutive subtasks. Furthermore, to support LH-VLN, we develop an automated data generation platform NavGen, which constructs datasets with complex task structures and improves data utility through a bidirectional, multi-granularity generation approach. To accurately evaluate complex tasks, we construct the Long-Horizon Planning and Reasoning in VLN (LHPR-VLN) benchmark consisting of 3,260 tasks with an average of 150 task steps, serving as the first dataset specifically designed for the long-horizon vision-language navigation task. Furthermore, we propose Independent Success Rate (ISR), Conditional Success Rate (CSR), and CSR weight by Ground Truth (CGT) metrics, to provide fine-grained assessments of task completion. To improve model adaptability in complex tasks, we propose a novel Multi-Granularity Dynamic Memory (MGDM) module that integrates short-term memory blurring with long-term memory retrieval to enable flexible navigation in dynamic environments. Our platform, benchmark and method supply LH-VLN with a robust data generation pipeline, comprehensive model evaluation dataset, reasonable metrics, and a novel VLN model, establishing a foundational framework for advancing LH-VLN., Comment: A novel Vision-Language Navigation task: Long-Horizon Vision-Language Navigation

Published
2024

2. InfiniteWorld: A Unified Scalable Simulation Framework for General Visual-Language Robot Interaction

Author

Ren, Pengzhen, Li, Min, Luo, Zhen, Song, Xinshuai, Chen, Ziwei, Liufu, Weijia, Yang, Yixuan, Zheng, Hao, Xu, Rongtao, Huang, Zitong, Ding, Tongsheng, Xie, Luyang, Zhang, Kaidong, Fu, Changfei, Liu, Yang, Lin, Liang, Zheng, Feng, and Liang, Xiaodan

Subjects
Computer Science - Robotics
Abstract

Realizing scaling laws in embodied AI has become a focus. However, previous work has been scattered across diverse simulation platforms, with assets and models lacking unified interfaces, which has led to inefficiencies in research. To address this, we introduce InfiniteWorld, a unified and scalable simulator for general vision-language robot interaction built on Nvidia Isaac Sim. InfiniteWorld encompasses a comprehensive set of physics asset construction methods and generalized free robot interaction benchmarks. Specifically, we first built a unified and scalable simulation framework for embodied learning that integrates a series of improvements in generation-driven 3D asset construction, Real2Sim, automated annotation framework, and unified 3D asset processing. This framework provides a unified and scalable platform for robot interaction and learning. In addition, to simulate realistic robot interaction, we build four new general benchmarks, including scene graph collaborative exploration and open-world social mobile manipulation. The former is often overlooked as an important task for robots to explore the environment and build scene knowledge, while the latter simulates robot interaction tasks with different levels of knowledge agents based on the former. They can more comprehensively evaluate the embodied agent's capabilities in environmental understanding, task planning and execution, and intelligent interaction. We hope that this work can provide the community with a systematic asset interface, alleviate the dilemma of the lack of high-quality assets, and provide a more comprehensive evaluation of robot interactions., Comment: 8 pages, 5 figures

Published
2024

3. MEIA: Multimodal Embodied Perception and Interaction in Unknown Environments

Author

Liu, Yang, Song, Xinshuai, Jiang, Kaixuan, Chen, Weixing, Luo, Jingzhou, Li, Guanbin, and Lin, Liang

Subjects
Computer Science - Computer Vision and Pattern Recognition
Abstract

With the surge in the development of large language models, embodied intelligence has attracted increasing attention. Nevertheless, prior works on embodied intelligence typically encode scene or historical memory in an unimodal manner, either visual or linguistic, which complicates the alignment of the model's action planning with embodied control. To overcome this limitation, we introduce the Multimodal Embodied Interactive Agent (MEIA), capable of translating high-level tasks expressed in natural language into a sequence of executable actions. Specifically, we propose a novel Multimodal Environment Memory (MEM) module, facilitating the integration of embodied control with large models through the visual-language memory of scenes. This capability enables MEIA to generate executable action plans based on diverse requirements and the robot's capabilities. Furthermore, we construct an embodied question answering dataset based on a dynamic virtual cafe environment with the help of the large language model. In this virtual environment, we conduct several experiments, utilizing multiple large models through zero-shot learning, and carefully design scenarios for various situations. The experimental results showcase the promising performance of our MEIA in various embodied interactive tasks., Comment: Codes will be available at https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List

Published
2024

8. The variations in volatile organic compounds based on the policy change for Omicron in the traffic hub of Zhengzhou.

Author

Zhang, Bowen, Zhang, Dong, Dong, Zhe, Song, Xinshuai, Zhang, Ruiqin, and Li, Xiao

Subjects
EMISSIONS (Air pollution), AIR quality standards, VOLATILE organic compounds, SARS-CoV-2 Omicron variant, ORGANIC bases
Abstract

Online volatile organic compounds (VOCs) were monitored before and after the Omicron policy change at an urban site in polluted Zhengzhou from 1 December 2022 to 31 January 2023. The characteristics and sources of VOCs were investigated. The daily mean concentrations of PM2.5 and total VOCs (TVOCs) ranged from 53.5 to 239.4 µ g m−3 and 15.6 to 57.1 ppbv, respectively, with mean values of 111.5 ± 45.1 µ g m−3 and 36.1 ± 21.0 ppbv, respectively, throughout the period. Two severe pollution events (designated as case 1 and case 2) were identified in accordance with the National Ambient Air Quality Standards (NAAQS) (China's National Ambient Air Quality Standards (NAAQS) from 2012). Case 1 (5 to 10 December PM2.5 daily mean = 142.5 µ g m−3) and case 2 (1 to 8 January PM2.5 daily mean = 181.5 µ g m−3) occurred during the infection period (when the policy of "full nucleic acid screening measures" was in effect) and the recovery period (after the policy was canceled), respectively. The PM2.5 and TVOC values for case 2 are, respectively, 1.3 and 1.8 times higher than those for case 1. The precise influence of disparate meteorological circumstances on the two pollution incidents is not addressed in this study. The results of the positive matrix factor modeling demonstrated that the primary source of VOCs during the observation period was industrial emissions, which constituted 32 % of the total VOCs, followed by vehicle emissions (27 %) and combustion (21 %). In case 1, industrial emissions constituted the primary source of VOCs, accounting for 32 % of the total VOCs. In contrast, in case 2, the contribution of vehicular emission sources increased to 33 % and became the primary source of VOCs. The secondary organic aerosol formation potential for case 1 and case 2 were found to be 37.6 and 65.6 µ g m−3, respectively. In case 1, the largest contribution of SOA formation potential (SOAP) from industrial sources accounted for the majority (63 %; 23.8 µ g m−3), followed by vehicular sources (18 %). After the end of the epidemic and the resumption of productive activities in the society, the difference in the proportion of secondary organic aerosol (SOA) generated from various sources decreased. Most of the SOAP came from solvent use and fuel evaporation sources, accounting for 32 % (20.9 µ g m−3) and 26 % (16.8 µ g m−3), respectively. On days with minimal pollution, industrial sources and solvent use remain the main contributors to SOA formation. Therefore, the regulation of emissions from industry, solvent-using industries, and motor vehicles needs to be prioritized to control the PM2.5 pollution problem. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

9. MEIA: Towards Realistic Multimodal Interaction and Manipulation for Embodied Robots

Author

Liu, Yang, Song, Xinshuai, Jiang, Kaixuan, Chen, Weixing, Luo, Jingzhou, Li, Guanbin, Lin, Liang, Liu, Yang, Song, Xinshuai, Jiang, Kaixuan, Chen, Weixing, Luo, Jingzhou, Li, Guanbin, and Lin, Liang

Abstract

With the surge in the development of large language models, embodied intelligence has attracted increasing attention. Nevertheless, prior works on embodied intelligence typically encode scene or historical memory in an unimodal manner, either visual or linguistic, which complicates the alignment of the model's action planning with embodied control. To overcome this limitation, we introduce the Multimodal Embodied Interactive Agent (MEIA), capable of translating high-level tasks expressed in natural language into a sequence of executable actions. Specifically, we propose a novel Multimodal Environment Memory (MEM) module, facilitating the integration of embodied control with large models through the visual-language memory of scenes. This capability enables MEIA to generate executable action plans based on diverse requirements and the robot's capabilities. Furthermore, we construct an embodied question answering dataset based on a dynamic virtual cafe environment with the help of the large language model. In this virtual environment, we conduct several experiments, utilizing multiple large models through zero-shot learning, and carefully design scenarios for various situations. The experimental results showcase the promising performance of our MEIA in various embodied interactive tasks., Comment: Codes will be available at https://github.com/HCPLab-SYSU/CausalVLR

Published
2024

11. The variations of VOCs based on the policy change of Omicron in polluted winter in traffic-hub city, China.

Author

Zhang, Bowen, Zhang, Dong, Dong, Zhe, Song, Xinshuai, Zhang, Ruiqin, and Li, Xiao

Subjects
SARS-CoV-2 Omicron variant, EMISSIONS (Air pollution), HALOCARBONS, WINTER, VOLATILE organic compounds
Abstract

Online volatile organic compounds (VOCs) were continuous monitored before and after the Omicron policy change at an urban site in polluted Zhengzhou from December 1, 2022, to January 31, 2023. The characteristics and sources of VOCs were explored. The daily average concentration of PM2.5 and total VOCs (TVOCs) ranged from 54 to 239 µg/m3andfrom 15.6 to 57.1 ppbv with an average value of 112 ± 45 µg/m3 and 36.1 ± 21.0 ppbv, respectively during the entire period. The values of PM2.5 and TVOCs in Case 3 (pollution episode after the abolishment of "Nucleic Acid Screening Measures for all staff" policy) were 1.3 and 1.8 times of the values in the Case 1 (pollution episode during "Nucleic Acid Screening Measures for all staff" policy). The concentration of TVOCs in Case 1 and Case 3 were 48.4 ± 20.4 and 67.6 ± 19.6 ppbv, respectively, increased by 63 % and 188 % compared with values during clean days. Alkanes were found to be the most abundant compounds during the entire period. Equivalent volume contribution of halogenated hydrocarbon and oxygenated VOCs (15 %) were found the most in Case 3, followed by alkenes (10 %). Though the volume contributions of aromatics were the lowest (6 % in Case 1 and 7 % in Case 3), the highest increasing ratio was found from clean days to polluted episodes. Positive Matrix Factor model results showed that the main source of VOCs during the observation period was industrial emissions, which accounted for 30 % of the TVOCs, followed by vehicular emission (24 %) and combustion (23 %). The vehicular emission became the largest source during Case 1 (40 %) and Case 3 (29 %), consisting of large numbers of people going out after the blockade. Secondary organic aerosol formation potential (SOAFP) values were 37 and 109 µg/m3, respectively with the highest SOAFP contribution (17–19 μg/m3and 31–51 %) from vehicular emission both in Case 1 and Case 3. Solvent usage sources had the second highest SOAFP value (9 and 16 μg/m3) with the contributions of 23 and 31 % in Case 1 and Case 3 respectively. The control of vehicular emission, and solvent usage should be focused in Zhengzhou, and combustion was also important for the control of PM2.5 pollution in winter. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results