Alternatives to DistilBERT

Options that appear in the same research papers as DistilBERT, by co-occurrence.

AlternativePapers (with DistilBERT)Avg viability
LoRA1—
RoBERTa1—
SMOTE1—
Gemma1—
DeBERTa1—
DistilGPT-21—
TinyT51—
gpt-oss1—