QipengGuo commited on
Commit
0ac9198
·
verified ·
1 Parent(s): 32b3b66

Enlarging max_position_embeddings to support long-context generation for the specific inference engine. (#8)

Browse files

- Enlarging max_position_embeddings to support long-context generation for the specific inference engine. (d806df7d7c1007e08dbd7e161f3c3243e4f46b80)

Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -26,7 +26,7 @@
26
  "hidden_size": 4096,
27
  "initializer_range": 0.02,
28
  "intermediate_size": 8192,
29
- "max_position_embeddings": 40960,
30
  "max_window_layers": 94,
31
  "mlp_only_layers": [],
32
  "model_type": "qwen3_moe",
 
26
  "hidden_size": 4096,
27
  "initializer_range": 0.02,
28
  "intermediate_size": 8192,
29
+ "max_position_embeddings": 65536,
30
  "max_window_layers": 94,
31
  "mlp_only_layers": [],
32
  "model_type": "qwen3_moe",