Update README.md
Browse files
README.md
CHANGED
|
@@ -1 +1,8 @@
|
|
| 1 |
-
`31,870,464` non-embedding params, `38,598,913` embedding params, `70,469,377` total
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
`31,870,464` non-embedding params, `38,598,913` embedding params, `70,469,377` total.
|
| 2 |
+
```
|
| 3 |
+
"n_embd": 768
|
| 4 |
+
"n_head": 6
|
| 5 |
+
"n_inner": 1920
|
| 6 |
+
"n_layer": 6
|
| 7 |
+
"n_positions": 4096
|
| 8 |
+
```
|