1. Quantum Large Language Models via Tensor Network Disentanglers
- Author
-
Aizpurua, Borja, Jahromi, Saeed S., Singh, Sukhbinder, and Orus, Roman
- Subjects
Quantum Physics ,Computer Science - Artificial Intelligence ,Computer Science - Machine Learning - Abstract
We propose a method to enhance the performance of Large Language Models (LLMs) by integrating quantum computing and quantum-inspired techniques. Specifically, our approach involves replacing the weight matrices in the Self-Attention and Multi-layer Perceptron layers with a combination of two variational quantum circuits and a quantum-inspired tensor network, such as a Matrix Product Operator (MPO). This substitution enables the reproduction of classical LLM functionality by decomposing weight matrices through the application of tensor network disentanglers and MPOs, leveraging well-established tensor network techniques. By incorporating more complex and deeper quantum circuits, along with increasing the bond dimensions of the MPOs, our method captures additional correlations within the quantum-enhanced LLM, leading to improved accuracy beyond classical models while maintaining low memory overhead., Comment: 4 pages, 2 figures
- Published
- 2024