SLAI-AITP commited on
Commit
6cabedf
·
verified ·
1 Parent(s): 50459f9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +101 -0
README.md CHANGED
@@ -1,3 +1,104 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+ # SLAI T-Rex-Flash (DeepSeek-V4-Flash-OR)
5
+
6
+ SLAI T-Rex-Flash is an Operations Research (OR)–specialized model built from DeepSeek-V4-Flash through full-parameter continued pre-training (CPT) followed by supervised fine-tuning (SFT). It is designed for solver-oriented mathematical modeling, including formulation, Gurobi program generation, feasibility-oriented tasks, and structure-sensitive optimization modeling.
7
+
8
+ - Paper: [SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD](https://arxiv.org/abs/2607.20145)
9
+ - Paper PDF: [arXiv:2607.20145](https://arxiv.org/pdf/2607.20145)
10
+ - Code and training recipes: [SLAI-AITP/SLAI-T-Rex](https://github.com/SLAI-AITP/SLAI-T-Rex)
11
+ - ModelScope weights: [SLAIAITP/DeepSeek-V4-Flash-OR](https://www.modelscope.cn/models/SLAIAITP/DeepSeek-V4-Flash-OR)
12
+
13
+ ## Model Summary
14
+
15
+ | Item | Description |
16
+ | --- | --- |
17
+ | Model name | SLAI T-Rex-Flash |
18
+ | Base model | DeepSeek-V4-Flash |
19
+ | Domain | Operations Research and mathematical programming |
20
+ | Training | Full-parameter OR-oriented CPT followed by Clean-CoT SFT |
21
+ | Primary output | Mathematical formulation and/or solver-facing Python code |
22
+ | Training hardware | Ascend CloudMatrix384 SuperPOD with Ascend 910C NPUs |
23
+
24
+ The CPT stage supplies OR terminology, formulation patterns, solver APIs, and structural priors. The SFT stage aligns these capabilities with task instructions, output contracts, mathematical explanations, and executable Gurobi-style implementations. The reported SFT recipe uses an 8,192-token sequence length, AdamW, a global batch size of 128, and full-parameter optimization.
25
+
26
+ ## Download
27
+
28
+ Install ModelScope and download the checkpoint:
29
+
30
+ ```bash
31
+ python3 -m pip install -U modelscope
32
+ ```
33
+
34
+ ```python
35
+ from modelscope import snapshot_download
36
+
37
+ model_dir = snapshot_download("SLAIAITP/DeepSeek-V4-Flash-OR")
38
+ print(model_dir)
39
+ ```
40
+
41
+ The checkpoint should be served with a runtime that supports the DeepSeek-V4 architecture and the checkpoint format in this repository. Distributed serving flags depend on the target hardware, runtime version, and available memory. Check the repository configuration files before deployment.
42
+
43
+ ## Intended Use
44
+
45
+ SLAI T-Rex-Flash is intended for research and development involving:
46
+
47
+ - translating natural-language OR problems into mathematical programs;
48
+ - identifying variables, objectives, and constraints;
49
+ - generating Gurobi-style Python implementations;
50
+ - producing solver-compatible models under strict output contracts;
51
+ - studying domain-adaptive CPT and SFT for structured reasoning.
52
+
53
+ A typical prompt should state the optimization problem, all required data, the expected solver behavior, and the output contract. For tagged modeling tasks, the training data commonly use `<think>`, `<model>`, and `<python>` sections. Code-only and LP-writing tasks use their task-specific contracts instead.
54
+
55
+ ## Evaluation
56
+
57
+ The following results are reported in the technical report under zero-shot Pass@1 evaluation. `Overall` is the unweighted mean of the four OR benchmarks.
58
+
59
+ | Model | NL4OPT | OptiBench | B4O-Feasible | B4O-ORGEval | Overall |
60
+ | --- | ---: | ---: | ---: | ---: | ---: |
61
+ | DeepSeek-V4-Flash | 84.08 | 63.33 | 60.47 | 34.26 | 60.54 |
62
+ | SLAI T-Rex-Flash | **89.52** | **67.12** | **71.22** | **59.39** | **71.81** |
63
+
64
+ These results correspond to the complete CPT+SFT pipeline. The reported overall score improves by 11.27 percentage points over the original DeepSeek-V4-Flash checkpoint. The largest absolute gain is on B4O-ORGEval, which evaluates structural equivalence rather than execution success alone.
65
+
66
+ The report also evaluates general-capability retention:
67
+
68
+ | Benchmark | SLAI T-Rex-Flash | Base |
69
+ | --- | ---: | ---: |
70
+ | MMLU, 5-shot | 88.5 | 87.6 |
71
+ | MMLU-Pro, 5-shot | 69.2 | 71.0 |
72
+ | CMMLU, 5-shot | 92.4 | 92.1 |
73
+ | HumanEval, 0-shot | 68.8 | 69.4 |
74
+ | GSM8K, 8-shot | 90.3 | 89.8 |
75
+ | MATH, 4-shot | 56.7 | 58.4 |
76
+
77
+ Evaluation numbers should be compared only under the same prompt templates, decoding budgets, benchmark versions, solver environment, and scoring implementation described in the paper.
78
+
79
+ ## Limitations
80
+
81
+ - Generated optimization models may be executable but mathematically incorrect; solver execution is not proof of structural equivalence.
82
+ - The model can still make errors in variable domains, nonlinear expressions, ratios, quadratic terms, index alignment, and Gurobi-specific APIs.
83
+ - Performance is strongest on the OR task families represented by the training and evaluation pipeline. Results may not transfer to unrelated domains or unseen solver ecosystems.
84
+ - The reported results do not establish reliability for safety-critical, financial, medical, industrial, or legal decision making.
85
+ - Always validate generated formulations, data interfaces, solver status, constraints, and objective values before use.
86
+
87
+ ## License
88
+
89
+ Use of this checkpoint is governed by the license files and terms included in this repository. Users must also comply with any applicable terms of the DeepSeek-V4-Flash base checkpoint and third-party software such as Gurobi.
90
+
91
+ ## Citation
92
+
93
+ If you use this model, please cite the technical report:
94
+
95
+ ```bibtex
96
+ @misc{li2026slaitrexfullparameterposttraining,
97
+ title = {SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD},
98
+ author = {Dongfang Li and others},
99
+ year = {2026},
100
+ eprint = {2607.20145},
101
+ archivePrefix= {arXiv},
102
+ url = {https://arxiv.org/abs/2607.20145}
103
+ }
104
+ ```