Local + global layers
all global
all sliding
5:1 · Gemma 3
1:1 · Gemma 2
▶ play
query
63
window W
5
keep first 4 (sink)
which tokens can still influence the query, layer by layer (toy: 64 tokens × 12 layers · tap the grid to move the query)
context for the cache bars
128K