s1ghhh/Llama-2-13b-Drop8Block
13B • Updated • 24 • 2
LLM-Drop
Model weights of paper "What Matters in Transformers? Not All Attention is Needed" (https://arxiv.org/abs/2406.15786)
-
-
s1ghhh/Llama-2-13b-Drop4Block
13B • Updated • 4 • 2 -
s1ghhh/Llama-2-13b-Drop4Attn
13B • Updated • 4 • 2 -
s1ghhh/Llama-2-13b-Drop8Attn
13B • Updated • 4 • 2 -
s1ghhh/Llama-2-13b-Drop4MLP
13B • Updated • 4 • 2 -
s1ghhh/Llama-2-13b-Drop8MLP
13B • Updated • 2 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4Block
7B • Updated • 4 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8Block
7B • Updated • 3 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4Attn
7B • Updated • 5 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8Attn
7B • Updated • 4 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4MLP
7B • Updated • 2 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8MLP
7B • Updated • 3 • 2 -
s1ghhh/Llama-2-70b-Drop
Text Generation • Updated • 3 • 2 -
s1ghhh/Llama-3-70b-Drop
Text Generation • 71B • Updated • 16 • 4 -
What Matters in Transformers? Not All Attention is Needed
Paper • 2406.15786 • Published • 33