SinusoidalTimeEmbedding

class torch_brain.nn.SinusoidalTimeEmbedding(dim, t_min, t_max)

Bases: torch.nn.modules.module.Module

Sinusoidal time/position embedding layer. These embeddings are generally added/concatenated to tokens to give them a sense of time/position. The timeperiods are logarithmically spaced between t_min and t_max (both inclusive).

Parameters:

• dim (int) – The dimension of the embedding needed (must be a multiple of 2)

• t_min (float) – Minimum period of the sinusoids. Set this to the smallest timescale you care about.

• t_max (float) – Maximum period of the sinusoids. Set this to the largest timescale you care about.

forward(timestamps)

Convert raw timestamps to sinusoidal embeddings

Parameters:

timestamps (torch.Tensor) – timestamps tensor

Return type:

Tensor