Alternatives to Attention

Options that appear in the same research papers as Attention, by co-occurrence.

AlternativePapers (with Attention)Avg viability
PyTorch1
Deep Learning1
Mixture of Experts1
FilDeep1