The FUN project proposed neural crawling to replace ranking via Page Rank logic – using language models to estimate the semantic quality of web pages and prioritise accordingly.
NGI-OpenWebSearchEU:FUN
FUN
Full description: The FUN project proposed neural crawling to replace ranking via Page Rank logic – using language models to estimate the semantic quality of web pages and prioritise accordingly. The team developed four strategies and tested them on 87 million pages from ClueWeb22-B. On natural language queries, their best approach (DomQ) consistently outperformed PageRank in both crawling effectiveness and downstream retrieval quality. On keyword queries, it remained competitive.
Giving endusers better quality results
Not available yet
n/a yet
Country: Italy
NGI Project: OpenWebSearch.EU
Status: Early research demo