Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,104 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
# SLAI T-Rex-Flash (DeepSeek-V4-Flash-OR)
|
| 5 |
+
|
| 6 |
+
SLAI T-Rex-Flash is an Operations Research (OR)–specialized model built from DeepSeek-V4-Flash through full-parameter continued pre-training (CPT) followed by supervised fine-tuning (SFT). It is designed for solver-oriented mathematical modeling, including formulation, Gurobi program generation, feasibility-oriented tasks, and structure-sensitive optimization modeling.
|
| 7 |
+
|
| 8 |
+
- Paper: [SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD](https://arxiv.org/abs/2607.20145)
|
| 9 |
+
- Paper PDF: [arXiv:2607.20145](https://arxiv.org/pdf/2607.20145)
|
| 10 |
+
- Code and training recipes: [SLAI-AITP/SLAI-T-Rex](https://github.com/SLAI-AITP/SLAI-T-Rex)
|
| 11 |
+
- ModelScope weights: [SLAIAITP/DeepSeek-V4-Flash-OR](https://www.modelscope.cn/models/SLAIAITP/DeepSeek-V4-Flash-OR)
|
| 12 |
+
|
| 13 |
+
## Model Summary
|
| 14 |
+
|
| 15 |
+
| Item | Description |
|
| 16 |
+
| --- | --- |
|
| 17 |
+
| Model name | SLAI T-Rex-Flash |
|
| 18 |
+
| Base model | DeepSeek-V4-Flash |
|
| 19 |
+
| Domain | Operations Research and mathematical programming |
|
| 20 |
+
| Training | Full-parameter OR-oriented CPT followed by Clean-CoT SFT |
|
| 21 |
+
| Primary output | Mathematical formulation and/or solver-facing Python code |
|
| 22 |
+
| Training hardware | Ascend CloudMatrix384 SuperPOD with Ascend 910C NPUs |
|
| 23 |
+
|
| 24 |
+
The CPT stage supplies OR terminology, formulation patterns, solver APIs, and structural priors. The SFT stage aligns these capabilities with task instructions, output contracts, mathematical explanations, and executable Gurobi-style implementations. The reported SFT recipe uses an 8,192-token sequence length, AdamW, a global batch size of 128, and full-parameter optimization.
|
| 25 |
+
|
| 26 |
+
## Download
|
| 27 |
+
|
| 28 |
+
Install ModelScope and download the checkpoint:
|
| 29 |
+
|
| 30 |
+
```bash
|
| 31 |
+
python3 -m pip install -U modelscope
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
```python
|
| 35 |
+
from modelscope import snapshot_download
|
| 36 |
+
|
| 37 |
+
model_dir = snapshot_download("SLAIAITP/DeepSeek-V4-Flash-OR")
|
| 38 |
+
print(model_dir)
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
The checkpoint should be served with a runtime that supports the DeepSeek-V4 architecture and the checkpoint format in this repository. Distributed serving flags depend on the target hardware, runtime version, and available memory. Check the repository configuration files before deployment.
|
| 42 |
+
|
| 43 |
+
## Intended Use
|
| 44 |
+
|
| 45 |
+
SLAI T-Rex-Flash is intended for research and development involving:
|
| 46 |
+
|
| 47 |
+
- translating natural-language OR problems into mathematical programs;
|
| 48 |
+
- identifying variables, objectives, and constraints;
|
| 49 |
+
- generating Gurobi-style Python implementations;
|
| 50 |
+
- producing solver-compatible models under strict output contracts;
|
| 51 |
+
- studying domain-adaptive CPT and SFT for structured reasoning.
|
| 52 |
+
|
| 53 |
+
A typical prompt should state the optimization problem, all required data, the expected solver behavior, and the output contract. For tagged modeling tasks, the training data commonly use `<think>`, `<model>`, and `<python>` sections. Code-only and LP-writing tasks use their task-specific contracts instead.
|
| 54 |
+
|
| 55 |
+
## Evaluation
|
| 56 |
+
|
| 57 |
+
The following results are reported in the technical report under zero-shot Pass@1 evaluation. `Overall` is the unweighted mean of the four OR benchmarks.
|
| 58 |
+
|
| 59 |
+
| Model | NL4OPT | OptiBench | B4O-Feasible | B4O-ORGEval | Overall |
|
| 60 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 61 |
+
| DeepSeek-V4-Flash | 84.08 | 63.33 | 60.47 | 34.26 | 60.54 |
|
| 62 |
+
| SLAI T-Rex-Flash | **89.52** | **67.12** | **71.22** | **59.39** | **71.81** |
|
| 63 |
+
|
| 64 |
+
These results correspond to the complete CPT+SFT pipeline. The reported overall score improves by 11.27 percentage points over the original DeepSeek-V4-Flash checkpoint. The largest absolute gain is on B4O-ORGEval, which evaluates structural equivalence rather than execution success alone.
|
| 65 |
+
|
| 66 |
+
The report also evaluates general-capability retention:
|
| 67 |
+
|
| 68 |
+
| Benchmark | SLAI T-Rex-Flash | Base |
|
| 69 |
+
| --- | ---: | ---: |
|
| 70 |
+
| MMLU, 5-shot | 88.5 | 87.6 |
|
| 71 |
+
| MMLU-Pro, 5-shot | 69.2 | 71.0 |
|
| 72 |
+
| CMMLU, 5-shot | 92.4 | 92.1 |
|
| 73 |
+
| HumanEval, 0-shot | 68.8 | 69.4 |
|
| 74 |
+
| GSM8K, 8-shot | 90.3 | 89.8 |
|
| 75 |
+
| MATH, 4-shot | 56.7 | 58.4 |
|
| 76 |
+
|
| 77 |
+
Evaluation numbers should be compared only under the same prompt templates, decoding budgets, benchmark versions, solver environment, and scoring implementation described in the paper.
|
| 78 |
+
|
| 79 |
+
## Limitations
|
| 80 |
+
|
| 81 |
+
- Generated optimization models may be executable but mathematically incorrect; solver execution is not proof of structural equivalence.
|
| 82 |
+
- The model can still make errors in variable domains, nonlinear expressions, ratios, quadratic terms, index alignment, and Gurobi-specific APIs.
|
| 83 |
+
- Performance is strongest on the OR task families represented by the training and evaluation pipeline. Results may not transfer to unrelated domains or unseen solver ecosystems.
|
| 84 |
+
- The reported results do not establish reliability for safety-critical, financial, medical, industrial, or legal decision making.
|
| 85 |
+
- Always validate generated formulations, data interfaces, solver status, constraints, and objective values before use.
|
| 86 |
+
|
| 87 |
+
## License
|
| 88 |
+
|
| 89 |
+
Use of this checkpoint is governed by the license files and terms included in this repository. Users must also comply with any applicable terms of the DeepSeek-V4-Flash base checkpoint and third-party software such as Gurobi.
|
| 90 |
+
|
| 91 |
+
## Citation
|
| 92 |
+
|
| 93 |
+
If you use this model, please cite the technical report:
|
| 94 |
+
|
| 95 |
+
```bibtex
|
| 96 |
+
@misc{li2026slaitrexfullparameterposttraining,
|
| 97 |
+
title = {SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD},
|
| 98 |
+
author = {Dongfang Li and others},
|
| 99 |
+
year = {2026},
|
| 100 |
+
eprint = {2607.20145},
|
| 101 |
+
archivePrefix= {arXiv},
|
| 102 |
+
url = {https://arxiv.org/abs/2607.20145}
|
| 103 |
+
}
|
| 104 |
+
```
|