FAIRC/token-averaging-matched_mean-avg_50m_k8
Updated • 11
Token Averaging
A comprehensive research framework for analyzing whether averaging adjacent tokens in a LLM can reduce the compute compared to standard model.
-
-
FAIRC/token-averaging-model1_50m
Updated • 23 -
FAIRC/token-averaging-avg_50m_k8_wexp
Updated • 11 -
FAIRC/token-averaging-avg_50m_k8_learnable_pos
Updated • 7 -
FAIRC/token-averaging-avg_50m_k8_learnable
Updated • 8 -
FAIRC/token-averaging-avg_50m_k2
Updated • 32 -
FAIRC/token-averaging-avg_50m_k4
Updated • 32 -
FAIRC/token-averaging-update_control-model1_50m
Updated • 7 -
FAIRC/token-averaging-matched_mean-avg_50m_k4
Updated • 12 -
FAIRC/token-averaging-matched_mean-avg_50m_k2
Updated • 8 -
FAIRC/token-averaging-model2_50m_ctx2n
Updated • 8 -
FAIRC/token-averaging-avg_50m_k4_wexp
Updated • 11 -
FAIRC/token-averaging-avg_50m_k4_learnable_pos
Updated • 8 -
FAIRC/token-averaging-avg_50m_k4_learnable
Updated • 8 -
FAIRC/token-averaging-avg_50m_k2_word
Updated • 8 -
FAIRC/token-averaging-avg_50m_k2_wexp
Updated • 6 -
FAIRC/token-averaging-avg_50m_k2_ov4s2
Updated • 8 -
FAIRC/token-averaging-avg_50m_k2_learnable_pos
Updated • 10 -
FAIRC/token-averaging-avg_50m_k2_learnable
Updated • 11 -
FAIRC/token-averaging-model1_250m
Updated • 9 -
FAIRC/token-averaging-avg_250m_k2
Updated • 7 -
FAIRC/token-averaging-model1_500m
Updated • 12 -
FAIRC/token-averaging-avg_500m_k2
Updated • 17