Instructions to use PelaAI/KnowLine-9B-Gen1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PelaAI/KnowLine-9B-Gen1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="PelaAI/KnowLine-9B-Gen1")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("PelaAI/KnowLine-9B-Gen1")
model = AutoModelForMultimodalLM.from_pretrained("PelaAI/KnowLine-9B-Gen1", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PelaAI/KnowLine-9B-Gen1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PelaAI/KnowLine-9B-Gen1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PelaAI/KnowLine-9B-Gen1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/PelaAI/KnowLine-9B-Gen1

SGLang

How to use PelaAI/KnowLine-9B-Gen1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PelaAI/KnowLine-9B-Gen1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PelaAI/KnowLine-9B-Gen1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PelaAI/KnowLine-9B-Gen1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PelaAI/KnowLine-9B-Gen1",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use PelaAI/KnowLine-9B-Gen1 with Docker Model Runner:
```
docker model run hf.co/PelaAI/KnowLine-9B-Gen1
```

KnowLine-9B-Gen1 / INFERENCE.md

PEScn

Report full DI 0.3 results and serial consistency validation

a421fc4 verified 10 days ago

preview code

Raw History Blame Contribute Delete

4.45 kB

KnowLine-9B-Gen1: serving and Decision Index reproduction

This file describes the serving path used for the KnowLine-9B-Gen1 evaluation. The model reads option-label probabilities directly; it does not generate free-form answers.

Weights

Base: Qwen3.5-9B.
One epoch of LoRA SFT, rank 32, alpha 64, language-model linear layers, bf16.
Training data: mixU_clean_qwen35.jsonl, 1,143,656 request rows after cleaning.
Maximum training length: 12,288 tokens.
Candidate: a3-9b-u-ck4644-wise90-t118.
Readout temperature T=1.18 is folded into model.language_model.norm.weight. The exact metadata is in CALIBRATION.json.

Do not apply T=1.18 a second time. The front end reads at temperature 1 after the folded readout temperature.

Software

The included front end consists of:

knowline_server.py: /v1/systemone HTTP server;
knowline_engine.py: backend and label-probability readout;
knowline_layout.py: prompt layout and read-twice logic;
serve_knowline.sh: SGLang launch helper.

The earlier internal candidate evaluation used FP8-at-load SGLang. The release launcher uses bf16 SGLang 0.5.21 with Qwen3.5's chat template, thinking disabled, 16 scoring workers, and read-twice enabled for short text-only requests. The full public DI run below completed on 2026-10-25 using this release package.

Read twice

For a text-only request whose one-pass length is at most 3,000 tokens, the front end writes the user input a second time before the answer position. The second copy starts with:

Read the same input again before answering:

Requests above the threshold and requests with images are read once. Disable this behavior with the front end's --no-read-twice option when comparing serving formats.

Launch

bash serve_knowline.sh /path/to/KnowLine-9B-Gen1 0 8080

Equivalent shape:

CUDA_VISIBLE_DEVICES=0 python -m sglang.launch_server \
  --model-path /path/to/KnowLine-9B-Gen1 \
  --served-model-name m \
  --tp 1 --mem-fraction-static 0.85 --enable-fp32-lm-head \
  --mamba-radix-cache-strategy extra_buffer \
  --port 9080

python knowline_server.py \
  --model /path/to/KnowLine-9B-Gen1 \
  --backend sglang \
  --url http://127.0.0.1:9080 \
  --served-model-name m \
  --temperature 1 \
  --workers 16 \
  --port 8080

Decision Index result

The official Decision Index 0.3 scorer reports 65.67 public index / 73.88 raw index. All 140,620 requests were sent fresh; 140,178 are scoreable, and 442 are excluded by the official suite. Final scoreable counts: 140,178 ok, zero errors and zero unsupported; no truncation.

Two identical H20 servers processed 16 round-robin shards (eight per server). The initial pass had two HTTP504 transport errors, one SATA-Bench and one RouterBench request. Only those failed requests were retried, using the official runner and identical payload hashes, weights and serving. The original first-pass file and retry provenance were retained. The other 140,618 successful request responses were not changed or selected by correctness.

A separate 300-request serial pass compared 477 answers with this model's own final run: 476 agreed. The only disagreement had new/old margins 0.006/0.021, both below 0.06. All disagreements therefore qualify as near-ties; agreement after excluding near-ties is 100%. Single-request H20 latency: median 96.7 ms, mean 207.7 ms, p80 261.9 ms, p95 826.7 ms. Initial parallel-run request times include queueing and should not be substituted for these serial values.

The earlier 65.68 sample index and 59.55 predicted private Full were internal candidate-selection estimates, not this full public result. Public index and private leaderboard Full are different metrics and must not be ranked against each other.

Limitations

Images were not used in 9B Gen1 training.
No block-bidirectional attention was used.
Do not silently truncate long requests.
The front end is for structured decisions, not open-ended chat.

SHA256SUMS covers every file in the repository.

Serving correction (2026-10-25)

The launcher now uses Qwen's locate/xyxy box settings and no longer references the absent Gemma-specific temperatures.json. The folded T=1.18 remains the only readout scaling. Choice, yes/no and score requests passed an actual two-server bf16 smoke run before the full DI run was started. Image and box output quality is not established by that text-only smoke.