huggingface/datatrove
process, filter, and deduplicate large-scale text data with customizable pipelines
View on index · View in 3D Map
Related repos
- Python tools for loading and processing datasets for machine learning
- Control Blender with natural language AI agents
- no‑code AI spreadsheet to build and enrich datasets
- tool for finding and removing duplicate images in big datasets
- Data platform for analysis and visualization
- Integrates data sources for transformation and visualization
- text-to-speech library for generating voices with controllable style and speaker traits
- Toolkit for serving and running large language models through an inference server
- open source BI and data viz system originally built at Yandex
- Visual workflow automation platform with data lineage tracking
