torch_brain.nn

Embedding modules

InfiniteVocabEmbedding	Embedding layer with a vocabulary that can be extended.
RotaryTimeEmbedding	Rotary time/positional embedding layer.
SinusoidalTimeEmbedding	Sinusoidal time/position embedding layer.

Transformer related modules

RotaryCrossAttention	Cross-attention layer with rotary positional embeddings.
RotarySelfAttention	Self-attention layer with rotary positional embeddings.