Repository Showcase
A curated collection of interesting GitHub repositories
huggingface/large-scale-image-deduplication
tool for finding and removing duplicate images in big datasets
Related repos
- cross-platform tool to find and manage duplicate files
- Desktop tool for image compression and processing
- process, filter, and deduplicate large-scale text data with customizable pipelines
- An online image compression tool with local processing
- Python tools for loading and processing datasets for machine learning
- visualize huge datasets as images with Python
- Collection of PyTorch image models, training scripts, and utilities
- finds similar code and patterns across files
- Open source toolset for image forensics
- Self-hosted open source photo management tool
