Back to Search Start Over

MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task

Authors :
Li, Yanjie
Liu, Jingyi
Li, Weijun
Yu, Lina
Wu, Min
Li, Wenqiang
Hao, Meilan
Wei, Su
Deng, Yusong
Publication Year :
2024

Abstract

Mathematical formulas are the crystallization of human wisdom in exploring the laws of nature for thousands of years. Describing the complex laws of nature with a concise mathematical formula is a constant pursuit of scientists and a great challenge for artificial intelligence. This field is called symbolic regression (SR). Symbolic regression was originally formulated as a combinatorial optimization problem, and Genetic Programming (GP) and Reinforcement Learning algorithms were used to solve it. However, GP is sensitive to hyperparameters, and these two types of algorithms are inefficient. To solve this problem, researchers treat the mapping from data to expressions as a translation problem. And the corresponding large-scale pre-trained model is introduced. However, the data and expression skeletons do not have very clear word correspondences as the two languages do. Instead, they are more like two modalities (e.g., image and text). Therefore, in this paper, we proposed MMSR. The SR problem is solved as a pure multi-modal problem, and contrastive learning is also introduced in the training process for modal alignment to facilitate later modal feature fusion. It is worth noting that to better promote the modal feature fusion, we adopt the strategy of training contrastive learning loss and other losses at the same time, which only needs one-step training, instead of training contrastive learning loss first and then training other losses. Because our experiments prove training together can make the feature extraction module and feature fusion module wearing-in better. Experimental results show that compared with multiple large-scale pre-training baselines, MMSR achieves the most advanced results on multiple mainstream datasets including SRBench. Our code is open source at https://github.com/1716757342/MMSR<br />Comment: The Information Fusion has accepted this paper

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2402.18603
Document Type :
Working Paper
Full Text :
https://doi.org/10.1016/j.inffus.2024.102681