Is your feature request related to a problem? Please describe.
Exact duplicate detection does not catch near-identical sequences (e.g. sequences with a single SNP or insertion). For epitope or primer datasets, high homology between sequences indicates redundancy that can bias downstream analyses such as MHCII binding prediction or PCR specificity.
Describe the solution you'd like
Add optional homology clustering to the DNA/RNA analysis pipeline maybe by using cd-hit-est (for nucleotide sequences) or a pure-Python fallback using pairwise alignment via Biopython:
- Cluster sequences at a configurable identity threshold (default: 90 %)
- Report number of clusters, mean cluster size, and the percentage of sequences in non-singleton clusters
- Store cluster membership for the top-N most frequent sequences
Activate via a new CLI flag --homology or automatically when the sequence column has fewer than 10,000 unique sequences (to keep runtime manageable). Display as a summary metric in the Statistics tab.
Describe alternatives you've considered
MinHash-based LSH (e.g. via datasketch) for approximate similarity without full alignment. Faster for large datasets but harder to interpret for a bioinformatics audience familiar with percent identity.
Additional context
CD-HIT is an optional external dependency; BioProfileKit should degrade gracefully to the Biopython fallback when it is not installed. Maybe it is possible to show the estimated runtime to the user before starting on large datasets?
Is your feature request related to a problem? Please describe.
Exact duplicate detection does not catch near-identical sequences (e.g. sequences with a single SNP or insertion). For epitope or primer datasets, high homology between sequences indicates redundancy that can bias downstream analyses such as MHCII binding prediction or PCR specificity.
Describe the solution you'd like
Add optional homology clustering to the DNA/RNA analysis pipeline maybe by using
cd-hit-est(for nucleotide sequences) or a pure-Python fallback using pairwise alignment viaBiopython:Activate via a new CLI flag
--homologyor automatically when the sequence column has fewer than 10,000 unique sequences (to keep runtime manageable). Display as a summary metric in the Statistics tab.Describe alternatives you've considered
MinHash-based LSH (e.g. via
datasketch) for approximate similarity without full alignment. Faster for large datasets but harder to interpret for a bioinformatics audience familiar with percent identity.Additional context
CD-HIT is an optional external dependency; BioProfileKit should degrade gracefully to the Biopython fallback when it is not installed. Maybe it is possible to show the estimated runtime to the user before starting on large datasets?