vadery commited on
Commit
ff7fe1c
·
verified ·
1 Parent(s): 26cf85f

Add model card

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ - zh
6
+ base_model: Qwen/Qwen2.5-32B-Instruct
7
+ tags:
8
+ - aiops
9
+ - root-cause-analysis
10
+ - openrca
11
+ - tool-calling
12
+ ---
13
+
14
+ # Netis N1
15
+
16
+ Netis N1 is a 32B-parameter model specialized for **intelligent IT operations (AIOps)**, fine-tuned from Qwen2.5-32B-Instruct using Continual Pre-training, SFT, and RL.
17
+
18
+ ## Highlights
19
+
20
+ - **OpenRCA Scoring 26.0%** — on par with Claude Opus 4.5 (26.9%), surpassing GPT Codex (21.0%) and DeepSeek (13.6%)
21
+ - General capabilities preserved or improved (GPQA +9.6%, MMLU-Pro +3.3%, IFEval +1.0%)
22
+ - Two-stage diagnostic workflow: tool-call planning + fault localization
23
+
24
+ ## Evaluation Results
25
+
26
+ | Benchmark | Base (Qwen2.5-32B) | Netis N1 |
27
+ | ---------------------- | -----------------: | ----------------: |
28
+ | OpenRCA Bank (scoring) | — | **26.0%** |
29
+ | MMLU-Pro (300) | 63.7% | **67.0%** |
30
+ | GPQA Diamond (198) | 44.4% | **54.0%** |
31
+ | IFEval (100) | 86.0% / 90.5% | **87.0% / 91.2%** |
32
+ | GSM8K (1319) | 91.5% | 91.4% |
33
+ | AIME (90) | 16.7% | 11.1% |
34
+ | HLE (100) | 2.0% | 1.0% |
35
+
36
+ ## How to Evaluate
37
+
38
+ Serve with vLLM and run the eval suite:
39
+
40
+ ```bash
41
+ # Start vLLM server
42
+ python -m vllm.entrypoints.openai.api_server \
43
+ --model ./netis-n1 --port 6777 --max-model-len 20000
44
+
45
+ # OpenRCA eval (requires OpenRCA Bank dataset)
46
+ python eval_model.py --llm-backend vllm --vllm-base-url http://localhost:6777
47
+
48
+ # General benchmarks (MMLU-Pro, GPQA, AIME, IFEval)
49
+ python eval_multi_model.py --name netis-n1 --url http://localhost:6777 --model netis-n1
50
+ ```
51
+
52
+ ## License
53
+
54
+ Apache 2.0