LS3-Micro-Base

Dsg2 commited on 4 days ago

Commit

eeefab5

verified ·

1 Parent(s): 42681bf

Update README.md

Files changed (1) hide show

README.md CHANGED Viewed

@@ -29,7 +29,7 @@ This repo contains the Base (completed pretrain cudalm checkpoint) variant of LS
 LS3-Micro-Base has a 24.9% higher Intelligence Index than LS-Micro Base, while having 13% less total parameters. It also nearly matches LS2.5-108M-A17M with almost 11x less parameters.
  - Intelligence Index was calculated using the formula from [Open SLM Leaderboard](https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard). (*Results reproducable via Cudalm's built in bench tool, see bottom quickstart*)
- - Compute density derived using the following formula: Intelligence Index / sqrt(Active Params (millions) * Total Params (millions))
 ---

 LS3-Micro-Base has a 24.9% higher Intelligence Index than LS-Micro Base, while having 13% less total parameters. It also nearly matches LS2.5-108M-A17M with almost 11x less parameters.
  - Intelligence Index was calculated using the formula from [Open SLM Leaderboard](https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard). (*Results reproducable via Cudalm's built in bench tool, see bottom quickstart*)
+ - Intelligence density derived using the following formula: Intelligence Index / sqrt(Active Params (millions) * Total Params (millions))
 ---