carlosplanchon/spidercreator

Automate web scraping spider creation with LLMs

turn web pages into LLM-ready data with WaterCrawl
watercrawl/WaterCrawl
extract structured data from any webpage using LLMs
mishushakov/llm-scraper
Python framework for web crawling and scraping with customizable spiders
scrapy/scrapy
lightweight Python tool for website scraping with LLMs
raznem/parsera
list of web crawler and spider projects in many languages
BruceDone/awesome-crawler
API that crawls a website and gives you clean data or markdown for LLMs
mendableai/firecrawl
Open-source tool for web crawling, scraping, and data extraction for LLMs and AI pipelines
unclecode/crawl4ai
Self-hosted web scraping solution, without writing code
jaypyles/Scraperr
Adaptive web scraping for evolving websites
D4Vinci/Scrapling
Build web scrapers with AI assistance
ScrapeGraphAI/scrapecraft