MoritzMo123/kfac-svd_uniform_llama-3-8b_0.6
Text Generation • 5B • Updated
LEMS & KFAC-SVD Compression
This collection hosts the compressed models evaluated in our paper: "Layer-wise Error Modeling Search (LEMS) and KFAC-SVD".
Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.7
Text Generation • 6B • Updated • 23Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.8
Text Generation • 7B • UpdatedNote Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.9
Text Generation • 7B • Updated • 1Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.6
Text Generation • 5B • UpdatedNote Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.7
Text Generation • 6B • Updated • 1Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.8
Text Generation • 7B • Updated • 1Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.9
Text Generation • 7B • Updated • 1Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.6
Text Generation • 5B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.7
Text Generation • 6B • Updated • 1Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.8
Text Generation • 7B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.9
Text Generation • 7B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.6
Text Generation • 5B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.7
Text Generation • 6B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.8
Text Generation • 7B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.9
Text Generation • 7B • UpdatedNote Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.6
Text Generation • 4B • UpdatedNote Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.7
Text Generation • 5B • UpdatedNote Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.8
Text Generation • 6B • Updated • 1Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.9
Text Generation • 7B • Updated • 1Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.6
Text Generation • 4B • UpdatedNote Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.7
Text Generation • 5B • UpdatedNote Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.8
Text Generation • 6B • UpdatedNote Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.9
Text Generation • 7B • Updated • 1Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-2-7b_0.6
Text Generation • 4B • Updated • 1 • 1Note Llama-2-7B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-2-7b_0.8
Text Generation • 5B • Updated • 1Note Llama-2-7B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-2-7b_0.6
Text Generation • 4B • Updated • 1Note Llama-2-7B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-2-7b_0.8
Text Generation • 5B • UpdatedNote Llama-2-7B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.