Repository Showcase
A curated collection of interesting GitHub repositories
MinishLab/semhash
by
Semantic text deduplication tool
tags:
Related repos
- tool for finding and removing duplicate images in big datasets
- cross-platform tool to find and manage duplicate files
- finds similar code and patterns across files
- process, filter, and deduplicate large-scale text data with customizable pipelines
- turns documents into LLM-ready data chunks
- CLI tools for parsing documents and running local semantic search
- FASTA and FASTQ file toolkit for sequence manipulation on the command line
- topic modeling and document similarity for large text datasets in Python
- Structured data extraction for LLMs
- alternative to RAG for LLMs, skips real-time retrieval by caching knowledge in context
