Discussion: Make LongSequence fixed-length?

This is obviously a breaking change, so this is for the far future if it will ever happen.

Currently, LongSequence is resizable: They support operations like:

julia> seq = dna"TAG";

julia> push!(seq, DNA_A)
4nt DNA Sequence:
TAGA

julia> append!(seq, rna"UAGA")
8nt DNA Sequence:
TAGATAGA

My proposal is to remove all methods on LongSequence that changes their size. This includes: resize!, pop!, popfirst!, push!, pushfirst!, filter!, deleteat!, and resizing during copy!.

Disadvantages of proposal

The disadvantages are obvious: Some users may want to do these operations. With my proposed suggestion, users would instead need to use immutable operations, just like if they were working with strings.
But: How often do people actually use these operations? I would guess that they are not used very often.

Advantage

It would allow us to change the storage of LongSequence from:

mutable struct OldSeq
    const data::Vector{UInt64}
    len::UInt
end

struct NewSeq
    data::Memory{UInt64}
    len::UInt
end

That is, it would allow making LongSequence a struct instead of a mutable struct, and it would also allow them to use Memory instead of Vector as backing storage. This saves two memory indirections.
More importantly, these indirections may inhibit memory optimisations such as stack-allocating some LongSequence, allocation hoisting, such. Such optimisations will be easier for Julia to do on an immutable struct containing Memory compared to a Memory hiding behind to mutable references.

At the moment, Julia doesn't really have any substantial memory optimisations so at the moment, there won't be much advantage to it.

@jakobnissen

At the moment, Julia doesn't really have any substantial memory optimisations so at the moment, there won't be much advantage to it.

If and when this stops being true, I'd be excited about having these changes and helping out where possible.

I personally never use the resizing and conceptually think of BioSequences like bio-flavored Strings with the appropriate validation checks and functions. For example, Instead of ASCII or UniCode alphabets with uppercasing and lowercasing functions the alphabets are nucleic acids or amino acids with (reverse)complement and/or (reverse)translation functions, etc.

If Strings in base Julia are already immutable as you suggest here:

With my proposed suggestion, users would instead need to use immutable operations, just like if they were working with strings.

then making this change would just make BioSequences more consistent with how I (personally, may not be universal) use the library

Related, my personal experience to:

But: How often do people actually use these operations? I would guess that they are not used very often.

is rarely if ever

Just my perspective but hopefully it is helpful. Thanks as always for your work on BioJulia @jakobnissen !

@jakobnissen

@jakobnissen Do we have a decent list of dependents? We could maybe use juliahub to query this, or use some of the work that Timothy has been doing. If we could add downstream tests, we could see how breaking this actually is.

I agree it unlikely that this is used frequently.

We have downstream tests already, so we coild just test it out (of course and then do more tests if it looks promising)

Hi
A biologist's point of view:
Many molecular biologists and population geneticists are concerned with sequence variations (SNPs, indels, CNVs...). SequenceVariation.jl is instrumental to manage efficiently short variant sequences. In case of deletion or insertion, the size of the sequence is obviously changed. I am preparing a package that will use hastables to map reference sequences and variants thereof to a limited number of loci on the human genome. If Biosequences are not resizable in future versions, I will have to rewrite everything. I will most likely useStrings as , unfortunately, NGS sequencers output text files (FASTA, FASTQ) and not 2/4-bit files and regex are quite efficient.
Thanks for the great job done for Biosequences.

I'm not sure I follow. Are the ability to resize biosequences important to represent variations? You could easily have a type that represents an indel without actually doing an operaton that changes the length of an existing biosequence. From what I can tell, SequenceVariation.jl does not actually do any operations that modify the length of a BioSequence.

Then you write you would prefer to use strings. But strings are also fixed-length (worse - they are immutable!). So I don't understand how switching to strings would help that issue.

Finally, (slightly offtopic, perhaps), check out the package FASTX.jl for working with FASTA + FASTQ files in Julia, and bio-regex for using regex with a BioSequence.

Sorry for not being clear enough.
When I get sequences from a FASTQ file (using FASTX), I get some that include deletions or insertions compared to the human genome reference sequence. I want to map those sequences to a limited number of loci on the genome using a hashtable (a la Novoalign), which, in my use case, is much faster than the classical kmer-based approach. This implies to precompute all observable variant sequences, including insertions and deletions. The variant sequences are actually stored as (reference sequence, variations) but I have to reconstruct the variant sequences when constructing the dictionary (sequence => locus).
For the Strings, you are right and, sorry, it is certainly offtopic.

@jakobnissen

It seems that the arbitrary length of LongSequence has its place given the existence of the Kmers.jl package.

P.S. Just adding this for context (@jakobnissen is clearly aware of this - since he contributed to the Kmers.jl package).

Hi! I’m still new to BioSequences, but I read through the discussion and wanted to add a small suggestion from a beginner perspective.
It seems like different users have different needs:
Some people mostly treat BioSequences like immutable strings (reverse-complement, translation, etc.).
Others have workflows (variant construction, alignment pre-processing, FASTQ → variant mapping) where resizing is genuinely needed.
Maybe a middle-ground approach could reduce the impact of this breaking change:
Possible middle solution
Instead of removing resizing entirely, BioSequences could provide:

A “builder” or “mutable buffer” type
– something lightweight that supports push!, pop!, resize!
– used only at construction time
– then converted to an immutable LongSequence for downstream tasks.

This is similar to how IOBuffer works for building strings before converting to a final String.

Clear documentation explaining:
when to use immutable sequences
when to use the mutable builder
how to convert between them

I think this could satisfy both groups without forcing every user to rewrite existing pipelines.

Thinking a bit more about it, I think the optimum would be:

mutable struct SeqBuf{A <: Alphabet}
    ref::RefVector{UInt64}
    offset_len::UInt
end

struct LongSeq{A <: Alphabet}
    ref::RefVector{UInt64}
    len::UInt
end

struct SeqView{A <: Alphabet}
    ref::RefVector{UInt64}
    offset_len::UInt
end

Where:

RefVector is from Add RefVector MemoryViews.jl#45
SeqBuf is resizable, front and back. offset_len uses 6 bits to encode the number of symbols offset in the first element of data, and the remaining bits for its length. Resizing means replacing the ref, like is done currently for Base.Vector
LongSeq is similar, except it's immutable and offset is always zero.
SeqView is similar to SeqBuf, except immutable

We could consider merging LongSeq and SeqView, but I think it might be useful to statically distinguish views from owned sequences in order to not make ownership too hard to reason about.

We could also give LongSeq an offset to allow zero-cost construction from the other types, but an offset does complicate some code and make it less efficient, since we always have to deal with a potential non-filled head data element. However this code would need to be written anyway for the other two types, so...

I'll have to mull it over, and also I think this should probably not be done until the new LTS which is approx. 2 years from now.

Thanks for the detailed follow-up! I think this design addresses the concern I raised earlier quite nicely.

The separation between SeqBuf, LongSeq, and SeqView seems particularly useful: SeqBuf can handle mutable construction, while LongSeq provides the immutable owned sequence and SeqView can avoid unnecessary copying.

One question from a user perspective: would it be possible to keep the migration from the current LongSequence API relatively straightforward by preserving the familiar mutation methods on SeqBuf (push!, pop!, resize!, append!, etc.)? That could make it easier for existing workflows to move to the new design without requiring major rewrites.

I also agree that clear documentation with a few examples showing when to use SeqBuf vs LongSeq vs SeqView would make the new model much easier to understand.

Navigation Menu

Repository navigation

Discussion: Make `LongSequence` fixed-length? #329

Disadvantages of proposal

Advantage

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Discussion: Make LongSequence fixed-length? #329

Description

Disadvantages of proposal

Advantage

Activity

cjprybol commented on Dec 22, 2024

kescobo commented on Dec 23, 2024

jakobnissen commented on Dec 23, 2024

ljournot commented on Jan 24, 2025

jakobnissen commented on Jan 24, 2025

ljournot commented on Jan 24, 2025

algunion commented on Jun 3, 2025

Bano733-code commented on Nov 30, 2025

jakobnissen commented on Aug 30, 2026

Bano733-code commented on Aug 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Discussion: Make `LongSequence` fixed-length? #329