Back to Search Start Over

From Cognition to Computation: A Comparative Review of Human Attention and Transformer Architectures

Authors :
Zhao, Minglu
Xu, Dehong
Gao, Tao
Publication Year :
2024

Abstract

Attention is a cornerstone of human cognition that facilitates the efficient extraction of information in everyday life. Recent developments in artificial intelligence like the Transformer architecture also incorporate the idea of attention in model designs. However, despite the shared fundamental principle of selectively attending to information, human attention and the Transformer model display notable differences, particularly in their capacity constraints, attention pathways, and intentional mechanisms. Our review aims to provide a comparative analysis of these mechanisms from a cognitive-functional perspective, thereby shedding light on several open research questions. The exploration encourages interdisciplinary efforts to derive insights from human attention mechanisms in the pursuit of developing more generalized artificial intelligence.

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2407.01548
Document Type :
Working Paper