`31,870,464` non-embedding params, `38,598,913` embedding params, `70,469,377` total. 
```
"n_embd": 768
"n_head": 6
"n_inner": 1920
"n_layer": 6
"n_positions": 4096
```