Hugging Face's logo

Devops-hestabit
/
airoboros-2.2.1-y34b-tensorRT

Model card Files
xet
 Community
Devops-hestabit commited on
Commit
7854480
·
1 Parent(s): 131445c

Upload folder using huggingface_hub

Browse files
Files changed (7) hide show
  1. yi-34b-tp1-awq/config.json +2 -2
  2. yi-34b-tp1-awq/llama_float16_tp1_rank0.engine +2 -2
  3. yi-34b-tp1-awq/model.cache +0 -0
  4. yi-34b-tp2-awq/config.json +2 -2
  5. yi-34b-tp2-awq/llama_float16_tp2_rank0.engine +2 -2
  6. yi-34b-tp2-awq/llama_float16_tp2_rank1.engine +2 -2
  7. yi-34b-tp2-awq/model.cache +0 -0
yi-34b-tp1-awq/config.json CHANGED
@@ -5,9 +5,9 @@
5
  "hidden_size": 7168,
6
  "int8": false,
7
  "max_batch_size": 8,
8
- "max_input_len": 2048,
9
  "max_num_tokens": null,
10
- "max_output_len": 512,
11
  "max_position_embeddings": 4096,
12
  "name": "llama",
13
  "num_heads": 56,
 
5
  "hidden_size": 7168,
6
  "int8": false,
7
  "max_batch_size": 8,
8
+ "max_input_len": 3800,
9
  "max_num_tokens": null,
10
+ "max_output_len": 250,
11
  "max_position_embeddings": 4096,
12
  "name": "llama",
13
  "num_heads": 56,
yi-34b-tp1-awq/llama_float16_tp1_rank0.engine CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cd8a9aa45eed94a430beb3c30253b107184b2f16abcf56b0d0307e2882b8ff8b
3
- size 18429951580
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:02e9fc4ad0b9c0251ad9c2a4ec72491c17042400b085aa0aa491f02090921507
3
+ size 18429951452
yi-34b-tp1-awq/model.cache CHANGED
Binary files a/yi-34b-tp1-awq/model.cache and b/yi-34b-tp1-awq/model.cache differ
 
yi-34b-tp2-awq/config.json CHANGED
@@ -5,9 +5,9 @@
5
  "hidden_size": 7168,
6
  "int8": false,
7
  "max_batch_size": 8,
8
- "max_input_len": 2048,
9
  "max_num_tokens": null,
10
- "max_output_len": 512,
11
  "max_position_embeddings": 4096,
12
  "name": "llama",
13
  "num_heads": 56,
 
5
  "hidden_size": 7168,
6
  "int8": false,
7
  "max_batch_size": 8,
8
+ "max_input_len": 3800,
9
  "max_num_tokens": null,
10
+ "max_output_len": 250,
11
  "max_position_embeddings": 4096,
12
  "name": "llama",
13
  "num_heads": 56,
yi-34b-tp2-awq/llama_float16_tp2_rank0.engine CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:284da8d17fac42bd90c773e980f1f573aeb646f7a38090e4afa77405757f54e2
3
- size 9681069212
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:304dafbc50e412895f62b10a19a1b2228ac83a241ba06733145056d602487184
3
+ size 9681068692
yi-34b-tp2-awq/llama_float16_tp2_rank1.engine CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5bb838ddd5040626e1eb6cb07f69cce8d6a0623843244ce4056f6bd34f625759
3
- size 9681069212
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92752876dcb30b6e657ee1077448e4ebb074550ec83293fc6631c7bf82b8443e
3
+ size 9681068692
yi-34b-tp2-awq/model.cache CHANGED
Binary files a/yi-34b-tp2-awq/model.cache and b/yi-34b-tp2-awq/model.cache differ