Hugging Face's logo

cbspace
/
gpt

Safetensors
model_hub_mixin
pytorch_model_hub_mixin
Model card Files
xet
 Community
cbspace commited on
Commit
23deb73
·
verified ·
1 Parent(s): 3908c32

Upload 2 files

Browse files

New model with max_seq_len=680 and loss of 3.548

Files changed (2) hide show
  1. config.json +2 -2
  2. model.safetensors +2 -2
config.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "dropout": 0.1,
3
  "embed_dim": 768,
4
  "ffn_dim": 3072,
5
- "max_seq_len": 576,
6
  "n_heads": 12,
7
  "n_layers": 12,
8
  "n_vocab": 50257
 
1
  {
2
+ "dropout": 0.0,
3
  "embed_dim": 768,
4
  "ffn_dim": 3072,
5
+ "max_seq_len": 680,
6
  "n_heads": 12,
7
  "n_layers": 12,
8
  "n_vocab": 50257
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2e8d7eb9ffdce0161253f45eaf202dc0fbae5f81ffa76a36d7440e0b7f91fa7a
3
- size 496398496
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:314d85386926ca222796a9ff57e16114519959df7eb96ed2092bde69b432c5b4
3
+ size 496717984