A comprehensive research framework for analyzing whether averaging adjacent tokens in a LLM can reduce the compute compared to standard model.
AI & ML interests
None defined yet.
Recent Activity
models 26
FAIRC/token-averaging-matched_mean-avg_50m_k8
Updated • 11
FAIRC/token-averaging-model1_50m
Updated • 23
FAIRC/token-averaging-avg_50m_k8_wexp
Updated • 11
FAIRC/token-averaging-avg_50m_k8_learnable_pos
Updated • 7
FAIRC/token-averaging-avg_50m_k8_learnable
Updated • 8
FAIRC/token-averaging-avg_50m_k2
Updated • 32
FAIRC/token-averaging-avg_50m_k4
Updated • 32
FAIRC/mowe-moe-768-19l
0.4B • Updated • 5
FAIRC/mowe-dense-768-19l
0.1B • Updated • 3
FAIRC/mowe-mowe-768-19l
0.3B • Updated • 3
datasets 0
None public yet