BruceDone/awesome-crawler

list of web crawler and spider projects in many languages

View on index · View in 3D Map

Related repos

• List of tools for building agents that browse the web
• Curated list of open source web and mobile applications
• Lists packages and resources for web scraping
• Automate web scraping spider creation with LLMs
• Python framework for web crawling and scraping with customizable spiders
• Scrapes websites and automates browsers
• Spiders websites to generate word lists
• Curated list of Scratch projects
• list of tools for searching the web using AI
• Collection of web development and programming resource links from the community