unclecode/crawl4ai

Open-source tool for web crawling, scraping, and data extraction for LLMs and AI pipelines

View on index · View in 3D Map

Related repos

• multi-engine web crawler for fast, large-scale data extraction
• Python library uses LLMs and graph logic for web scraping
• extract structured data from any webpage using LLMs
• Build web scrapers with AI assistance
• Scrapes websites with LLMs
• list of tools for searching the web using AI
• API that crawls a website and gives you clean data or markdown for LLMs
• AI-powered library for extracting and classifying data from docs with LLMs
• Extracts website data for agents