File size: 4,451 Bytes
f214278 a421fc4 f214278 d8f7d9a f214278 a421fc4 f214278 a421fc4 f214278 a421fc4 f214278 d8f7d9a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | # KnowLine-9B-Gen1: serving and Decision Index reproduction
This file describes the serving path used for the KnowLine-9B-Gen1 evaluation. The model reads option-label probabilities directly; it does not generate free-form answers.
## Weights
- Base: Qwen3.5-9B.
- One epoch of LoRA SFT, rank 32, alpha 64, language-model linear layers, bf16.
- Training data: `mixU_clean_qwen35.jsonl`, 1,143,656 request rows after cleaning.
- Maximum training length: 12,288 tokens.
- Candidate: `a3-9b-u-ck4644-wise90-t118`.
- Readout temperature `T=1.18` is folded into `model.language_model.norm.weight`. The exact metadata is in `CALIBRATION.json`.
Do not apply `T=1.18` a second time. The front end reads at temperature 1 after the folded readout temperature.
## Software
The included front end consists of:
- `knowline_server.py`: `/v1/systemone` HTTP server;
- `knowline_engine.py`: backend and label-probability readout;
- `knowline_layout.py`: prompt layout and read-twice logic;
- `serve_knowline.sh`: SGLang launch helper.
The earlier internal candidate evaluation used FP8-at-load SGLang. The release launcher uses bf16 SGLang 0.5.21 with Qwen3.5's chat template, thinking disabled, 16 scoring workers, and read-twice enabled for short text-only requests. The full public DI run below completed on 2026-10-25 using this release package.
## Read twice
For a text-only request whose one-pass length is at most 3,000 tokens, the front end writes the user input a second time before the answer position. The second copy starts with:
```text
Read the same input again before answering:
```
Requests above the threshold and requests with images are read once. Disable this behavior with the front end's `--no-read-twice` option when comparing serving formats.
## Launch
```bash
bash serve_knowline.sh /path/to/KnowLine-9B-Gen1 0 8080
```
Equivalent shape:
```bash
CUDA_VISIBLE_DEVICES=0 python -m sglang.launch_server \
--model-path /path/to/KnowLine-9B-Gen1 \
--served-model-name m \
--tp 1 --mem-fraction-static 0.85 --enable-fp32-lm-head \
--mamba-radix-cache-strategy extra_buffer \
--port 9080
python knowline_server.py \
--model /path/to/KnowLine-9B-Gen1 \
--backend sglang \
--url http://127.0.0.1:9080 \
--served-model-name m \
--temperature 1 \
--workers 16 \
--port 8080
```
## Decision Index result
The official Decision Index 0.3 scorer reports **65.67 public index / 73.88 raw index**. All 140,620 requests were sent fresh; 140,178 are scoreable, and 442 are excluded by the official suite. Final scoreable counts: 140,178 ok, zero errors and zero unsupported; no truncation.
Two identical H20 servers processed 16 round-robin shards (eight per server). The initial pass had two HTTP504 transport errors, one SATA-Bench and one RouterBench request. Only those failed requests were retried, using the official runner and identical payload hashes, weights and serving. The original first-pass file and retry provenance were retained. The other 140,618 successful request responses were not changed or selected by correctness.
A separate 300-request serial pass compared 477 answers with this model's own final run: 476 agreed. The only disagreement had new/old margins 0.006/0.021, both below 0.06. All disagreements therefore qualify as near-ties; agreement after excluding near-ties is 100%. Single-request H20 latency: median 96.7 ms, mean 207.7 ms, p80 261.9 ms, p95 826.7 ms. Initial parallel-run request times include queueing and should not be substituted for these serial values.
The earlier 65.68 sample index and 59.55 predicted private Full were internal candidate-selection estimates, not this full public result. Public index and private leaderboard Full are different metrics and must not be ranked against each other.
## Limitations
- Images were not used in 9B Gen1 training.
- No block-bidirectional attention was used.
- Do not silently truncate long requests.
- The front end is for structured decisions, not open-ended chat.
`SHA256SUMS` covers every file in the repository.
## Serving correction (2026-10-25)
The launcher now uses Qwen's `locate`/`xyxy` box settings and no longer references the absent Gemma-specific `temperatures.json`. The folded T=1.18 remains the only readout scaling. Choice, yes/no and score requests passed an actual two-server bf16 smoke run before the full DI run was started. Image and box output quality is not established by that text-only smoke.
|