401. TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
- Author
-
Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, Yun Wang, Linjun Shou, Ming Gong, and Nan Duan
- Subjects
Electronic computers. Computer science ,QA75.5-76.95 - Abstract
In recent years, artificial intelligence (AI) has made incredible progress. Advanced foundation models such as ChatGPT can offer powerful conversation, in-context learning, and code generation abilities for a broad range of open-domain tasks. They can also generate high-level solution outlines for domain-specific tasks based on their acquired common-sense knowledge. Nonetheless, they still face difficulties in specialized tasks because they lack sufficient domain-specific data during pretraining and can make errors in neural network computations requiring accurate execution. However, many existing models and systems can perform domain-specific tasks very well, although they are not easily accessible or compatible with foundation models because of the different implementations or working mechanisms. Therefore, there is a clear and pressing need for a mechanism that can leverage foundation models to propose task solution outlines and then automatically match the subtasks in the outlines to off-the-shelf models and systems with special functionalities to complete these subtasks. Inspired by this, we introduce TaskMatrix.AI as a new AI ecosystem that connects foundation models to millions of application programming interfaces (APIs) for task completion. Unlike most previous studies, which aimed to improve a single AI model, TaskMatrix.AI focuses on using an existing foundation model (as a brain-like central system) and APIs of other AI models and systems (as subtask solvers) to realize diversified tasks in both the digital and physical domains.
- Published
- 2024
- Full Text
- View/download PDF