Instructions to use Remek/basal-2.0-max-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Remek/basal-2.0-max-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Remek/basal-2.0-max-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Remek/basal-2.0-max-FP8")
model = AutoModelForCausalLM.from_pretrained("Remek/basal-2.0-max-FP8", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Remek/basal-2.0-max-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Remek/basal-2.0-max-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Remek/basal-2.0-max-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Remek/basal-2.0-max-FP8

SGLang

How to use Remek/basal-2.0-max-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Remek/basal-2.0-max-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Remek/basal-2.0-max-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Remek/basal-2.0-max-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Remek/basal-2.0-max-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Remek/basal-2.0-max-FP8 with Docker Model Runner:
```
docker model run hf.co/Remek/basal-2.0-max-FP8
```

basal-2.0-max-FP8

FP8 checkpoint for NVIDIA Hopper and Blackwell GPUs of basal-2.0-max (11B, base Bielik-PL-11B-v3.0-Instruct). Same model, same typed-decision API through basal-serve; the calibration files this folder ships are listed below.

Passes every bar of this tier against bf16 (agreement, agreement near ties, accuracy, ECE).

Measured against bf16

Agreement is the share of decisions where the port picks the same option as the bf16 release model on the development set dev v2 (MLX rows: every 8th dev v2 decision, 1,700 decisions). Bars for this tier: agreement 2-10 >= 0.9552, agreement at reference margin >= 0.2 >= 0.9893, accuracy within 1 point of bf16, ECE at most +0.01. All ports of this tier:

package	agreement with bf16 (options 2-10)	agreement at margin >= 0.2	accuracy diff	ECE diff	size	speed	gate
FP8	0.9704	0.9948	+0.0005	-0.0010	12 GB	19.9 dec/s c16, c1 p50 50 ms (H100)	PASS
GGUF Q8_0	0.9925	0.9997	+0.0002	+0.0018	12 GB	not measured in this release	PASS
GGUF Q5_K_M	0.9766	0.9970	-0.0001	+0.0032	7.5 GB	not measured in this release	PASS
MLX 8-bit	0.9933	0.9990	+0.0017	+0.0053	12 GB	622 ms c1 p50, 1506 ms for 5 questions (Mac)	PASS

The ports are decision-layer checks against bf16, not new benchmarks; the model's own results are on basal-2.0-max.

Use

The same install as the bf16 model (quick start); then:

basal-serve --model Remek/basal-2.0-max-FP8 --mode fp8 --port 8000

An NVIDIA ModelOpt FP8 checkpoint that the basal engine v2.0.0 loads natively; needs a GPU with FP8 (sm89+, Hopper or Blackwell). In FP8 the batch can change near-tie decisions; --deterministic runs every request alone with full FP8 accumulation.

Speed: 19.9 dec/s c16, c1 p50 50 ms (H100); one NVIDIA H100 80GB (cluster node), one option order.

Engine. FlashAttention 2; compact rows and template prefix off (FP8 keeps FlashAttention 2 unless --attn fa3, which is faster but was not gated and changes some decisions near ties). Engine v2.0.0 schedules long requests (over 2,048 tokens) behind short ones: in a mixed load at 32 clients short requests answer in 0.81 s instead of 1.38 s (p50), and long requests take about 7% longer (p50 1.96 s instead of 1.83 s); --scheduler off serves in arrival order. The same basal engine v2.0.0 also serves the experimental basal-omni-1.0 models (images and audio): README.

Calibration

CALIBRATION.json in this folder is a copy of the FP8 fit (CALIBRATION.fp8.json). basal-serve --mode fp8 loads CALIBRATION.<engine>.json for its engine when the folder has it, else CALIBRATION.json. A calibration fitted on bf16 readouts does not carry over exactly to a quantised port (an outside test of basal-1.5 on Ollama saw 2.3% real error at the 1% threshold with the bf16 file), so a port-specific fit, where it passed, uses the bf16 model's method on that port's own readouts of the same calibration splits, one option order:

basal-serve --mode fp8: Ships CALIBRATION.fp8.json, fitted on FP8 readouts of this port (same method as the bf16 fit, one option order); dev v2 ECE 0.0258 with the bf16 fit -> 0.0258 with this fit (in-sample), 0.0273 held-out (leave-one-family-out).

Where a port uses the bf16 calibration, treat its confidence thresholds as approximate and fit your own with basal-calibrate when they matter.

Results of the model

See the model card basal-2.0-max for Werdykt, JDI, LEK, PL-Holdout and PL-Real numbers against basal-1.5 and the evaluation notes. This port is not scored separately on those benchmarks.

Limitations

Validate on your own documents before relying on it: hidden and held-out sets do not cover every domain.
Polish world knowledge of a small model is limited: put the relevant facts in the state. Legal rules change.
Exact arithmetic and thresholds are a weak spot of one-pass decisions: compute them in code (or "facts": "auto").
The calibration is cautious on unseen tasks: no 1% threshold is certified in the shipped files, so "max_error": 0.01 on an act or tool question defers every decision; use 0.05, or certify thresholds on your own data with basal-calibrate.
act and multi over many labels are experimental. Up to 255 options per question (Ollama: 20); the context is 32,768 tokens. The 2.0 models ship no evidence head.
Decisions with serious consequences for people should be reviewed by a person.

Citation

@software{kinas2026basal20,
  title   = {basal-2.0: typed-decision models and inference engine for Polish and English},
  author  = {Kinas, Remigiusz},
  year    = {2026},
  version = {2.0.0},
  url     = {https://github.com/rkinas/basal}
}