#Dedup

GitHub - MinishLab/semhash: Fast Multimodal Semantic Deduplication & Filtering Fast Multimodal Semantic Deduplication & Filtering - MinishLab/semhash

SemHash - a lightweight Semantic #Dedup & Filters
MIT lic
#AI #ML #OpenSource

github.com/MinishLab/se...

Is #dedup for NTFS on Server 2025 supported by
#Microsoft for #FSlogix or not?

Most AI says not but fails to give a definitive reference.

Was enabled & working on a system I inherited BUT there was also corruption on that volume.

Surely should be ok as a file system feature invisible to FSlogix?

📂 Data Prep Kit: #Unstructured #data prep for #LLM apps — #Python, #Ray, #Spark ⚙️ #Dedup, #PII redaction & doc #chunking for #RAG
🔗 Link in first 💬⤵️

Repost 🔁 #AI