devrupt-io/squache

Caches HTTPS requests to reduce scraping bandwidth

View on index · View in 3D Map

Related repos

• Blocks bot scrapers by requiring proof-of-work on HTTP requests
• Scrapes websites and automates browsers
• Scrapes websites into markdown using remote browsers
• Adaptive web scraping for evolving websites
• tiny HTTP client for fetch with retries, timeouts, and simple API
• multi-engine web crawler for fast, large-scale data extraction
• Minimalist HTML fetch library
• Web app for cluster management and log analysis
• Web crawling and scraping tool
• tips for scraping the web without getting blocked