Large scale web crawling with URL Frontier
The aim of the URL Frontier project is to provide an API and reference implementation for a crawl frontier, which can power various web crawlers, independently from their implementation language and scalability.
More
ProjectID
FED4FIRE:CC-URLFRONTIERCRAWL
Acronym
URL Frontier crawl
Additional Info
Not available yet
Enduser Relevance
Not available yet
Contact
https://www.fed4fire.eu/demo-stories/cc/url-frontier/
Endorsements
Not available yet
Disclaimer
Not available yet
Country:
NGI Project: Fed4fire
Status: -
Category: Measurement, monitoring, analysis and abuse handling
check other similar innovations
TrenchBoot for AMD platform in Linux kernel
Upstream TrenchBoot AMD support to the Linux kernel
SSI-as-a-Service
Simplifying SSI integration and adoption
MidScale
re-implement midPoint data store mechanisms to support large data sets and take full advantage of PostgreSQL open source database. We...
Owncast
ActivityPub powered Livecasting
