BuilderIO/gpt-crawler
crawls websites to collect data for AI and custom GPTs
View on index · View in 3D Map
Related repos
- AI agent for deep web and local research, generates reports with citations
- API for scraping and extracting website data for AI apps
- Build web scrapers with AI assistance
- Blocks AI crawlers from websites
- Open-source tool for web crawling, scraping, and data extraction for LLMs and AI pipelines
- AI-powered search, recommendations, and RAG API toolkit
- fetch a whole website and save as text for ai projects
- AI-powered Python tool for scraping websites and documents
- AI research assistant with real-time web scraping
- list of tools for searching the web using AI
