huggingface/large-scale-image-deduplication
tool for finding and removing duplicate images in big datasets
View on index · View in 3D Map
Related repos
- Finds duplicates and outliers in image datasets
- cross-platform tool to find and manage duplicate files
- Lock-free deduplication cloud backup tool
- Desktop tool for image compression and processing
- process, filter, and deduplicate large-scale text data with customizable pipelines
- An online image compression tool with local processing
- Python tools for loading and processing datasets for machine learning
- visualize huge datasets as images with Python
- Finds and removes file duplicates
- Collection of PyTorch image models, training scripts, and utilities
