Back to Search
Start Over
Efficient Deep Web Crawling Using Reinforcement Learning.
- Source :
- Advances in Knowledge Discovery & Data Mining: 14th Pacific-Asia Conference, Pakdd 2010, Hyderabad, India, June 21-24, 2010. Proceedings. Part I; 2010, p428-439, 12p
- Publication Year :
- 2010
-
Abstract
- Deep web refers to the hidden part of the Web that remains unavailable for standard Web crawlers. To obtain content of Deep Web is challenging and has been acknowledged as a significant gap in the coverage of search engines. To this end, the paper proposes a novel deep web crawling framework based on reinforcement learning, in which the crawler is regarded as an agent and deep web database as the environment. The agent perceives its current state and selects an action (query) to submit to the environment according to Q-value. The framework not only enables crawlers to learn a promising crawling strategy from its own experience, but also allows for utilizing diverse features of query keywords. Experimental results show that the method outperforms the state of art methods in terms of crawling capability and breaks through the assumption of full-text search implied by existing methods. [ABSTRACT FROM AUTHOR]
Details
- Language :
- English
- ISBNs :
- 9783642136566
- Database :
- Complementary Index
- Journal :
- Advances in Knowledge Discovery & Data Mining: 14th Pacific-Asia Conference, Pakdd 2010, Hyderabad, India, June 21-24, 2010. Proceedings. Part I
- Publication Type :
- Book
- Accession number :
- 76849361
- Full Text :
- https://doi.org/10.1007/978-3-642-13657-3_46