Hugging Face's logo

fromziro
/
Width-Vs-Depth

Safetensors
English
width-vs-depth
slms
custom_tokenizer
llama
llama-architecture
tlms
ablation
Model card Files
xet
 Community
1
Harley-ml commited on
Commit
748910c
·
verified ·
1 Parent(s): 89cc828

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -22,7 +22,7 @@ We tested two different architecture configurations on 500M tokens of FineWeb to
22
 
23
  ### Config A: Deep-Narrow (code name: `depth_311`)
24
 
25
- - Base Architecture: `LlamaForCasualLM`
26
  - Tokenizer: [`Harley-ml/Dillionv2-1.3M`](https://huggingface.co/Harley-ml/Dillionv2-1.3M)
27
  - Transformers Version: `5.13.1`
28
  - Hidden Size: `128`
@@ -46,7 +46,7 @@ We tested two different architecture configurations on 500M tokens of FineWeb to
46
 
47
  ### Config B: Shallow-Wide (code name: `width_311`)
48
 
49
- - Base Architecture: `LlamaForCasualLM`
50
  - Tokenizer: [`Harley-ml/Dillionv2-1.3M`](https://huggingface.co/Harley-ml/Dillionv2-1.3M)
51
  - Transformers Version: `5.13.1`
52
  - Hidden Size: `192`
 
22
 
23
  ### Config A: Deep-Narrow (code name: `depth_311`)
24
 
25
+ - Base Architecture: `LlamaForCausalLM`
26
  - Tokenizer: [`Harley-ml/Dillionv2-1.3M`](https://huggingface.co/Harley-ml/Dillionv2-1.3M)
27
  - Transformers Version: `5.13.1`
28
  - Hidden Size: `128`
 
46
 
47
  ### Config B: Shallow-Wide (code name: `width_311`)
48
 
49
+ - Base Architecture: `LlamaForCausalLM`
50
  - Tokenizer: [`Harley-ml/Dillionv2-1.3M`](https://huggingface.co/Harley-ml/Dillionv2-1.3M)
51
  - Transformers Version: `5.13.1`
52
  - Hidden Size: `192`