Large scale web crawling with URL Frontier
The aim of the URL Frontier project is to provide an API and reference implementation for a crawl frontier, which can power various web crawlers, independently from their implementation language and scalability.
More
ProjectID
FED4FIRE:CC-URLFRONTIERCRAWL
Acronym
URL Frontier crawl
Additional Info
Not available yet
Enduser Relevance
Not available yet
Contact
https://www.fed4fire.eu/demo-stories/cc/url-frontier/
Endorsements
Not available yet
Disclaimer
Not available yet
Country:
NGI Project: Fed4fire
Status: -
Category: Measurement, monitoring, analysis and abuse handling
check other similar innovations
searx
A privacy-respecting, hackable metasearch engine
vdirsyncer
Synchronise calendars and contacts
p2panda – p2panda + Bamboo + OpenMLS.
p2panda is a user-friendly peer-to-peer communications protocol with browser support, local deletion, fork detection, efficient partial replication, large-scale group messaging...
SeedVault
Private backups of mobile applications
