Back to Search
Start Over
Hierarchical Bayesian LSTM for Head Trajectory Prediction on Omnidirectional Images
- Source :
- IEEE Transactions on Pattern Analysis and Machine Intelligence. 44:7563-7580
- Publication Year :
- 2022
- Publisher :
- Institute of Electrical and Electronics Engineers (IEEE), 2022.
-
Abstract
- When viewing omnidirectional images (ODIs), viewers can access different viewports via head movement (HM), which sequentially forms head trajectories in spatial-temporal domain. Thus, head trajectories play a key role in modeling human attention on ODIs. In this paper, we establish a large-scale dataset collecting 21,600 head trajectories on 1,080 ODIs. By mining our dataset, we find two important factors influencing head trajectories, i.e., temporal dependency and subject-specific variance. Accordingly, we propose a novel approach integrating hierarchical Bayesian inference into long short-term memory (LSTM) network for head trajectory prediction on ODIs, which is called HiBayes-LSTM. In HiBayes-LSTM, we develop a mechanism of Future Intention Estimation (FIE), which captures the temporal correlations from previous, current and estimated future information, for predicting viewport transition. Additionally, a training scheme called Hierarchical Bayesian inference (HBI) is developed for modeling inter-subject uncertainty in HiBayes-LSTM. For HBI, we introduce a joint Gaussian distribution in a hierarchy, to approximate the posterior distribution over network weights. By sampling subject-specific weights from the approximated posterior distribution, our HiBayes-LSTM approach can yield diverse viewport transition among different subjects and obtain multiple head trajectories. Extensive experiments validate that our HiBayes-LSTM approach significantly outperforms 9 state-of-the-art approaches for trajectory prediction on ODIs, and then it is successfully applied to predict saliency on ODIs.
- Subjects :
- Viewport
Dependency (UML)
business.industry
Computer science
Applied Mathematics
Gaussian
Posterior probability
Bayesian probability
Sampling (statistics)
Bayes Theorem
Pattern recognition
Bayesian inference
symbols.namesake
Computational Theory and Mathematics
Artificial Intelligence
symbols
Trajectory
Humans
Neural Networks, Computer
Computer Vision and Pattern Recognition
Artificial intelligence
business
Algorithms
Software
Subjects
Details
- ISSN :
- 19393539 and 01628828
- Volume :
- 44
- Database :
- OpenAIRE
- Journal :
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Accession number :
- edsair.doi.dedup.....a0630acb51e4c1122dd06254253cb6b9