How models
pay attention.

Computes attention in on-chip tiles to avoid storing the full score matrix in GPU memory.

Shares key and value heads across query heads, reducing the KV cache.

Caches a compressed latent vector and a separate positional key for each token.

Restricts each token to a local window, reducing the number of attention pairs.

Uses a learned indexer to select token positions before main attention.

Updates a fixed-size state with learned decay gates and delta-rule corrections.

Applies channel-wise decay gates to a fixed-size recurrent state.

Allocates the KV cache in blocks, reducing unused reservations and supporting shared prefixes.