Alternatives to attention sink frames
Options that appear in the same research papers as attention sink frames, by co-occurrence.
| Alternative | Papers (with attention sink frames) | Avg viability |
|---|---|---|
| RoPE | 1 | — |
| autoregressive models | 1 | — |
| multi-head attention | 1 | — |
| multi-head RoPE jitter | 1 | — |