Lucia III

Lucia III is the agent model from Praex, post-trained from GLM-5.3 for coding and terminal agents. It works in the shell: it reads and edits files, runs commands, installs what it needs, checks its own work and keeps going until the task is done. We test it in the open-source Praex agent. It is served with vLLM's OpenAI-compatible API, so other agents that use that API can run it too.

Results

Terminal-Bench, run by us in the Praex agent:

Benchmark and what was measured Lucia III GLM-5.3
TB 2.0, same 74 tasks, one try each, same agent and settings 68.9% (51 of 74) 58.1% (43 of 74)
TB 2.0, all 89 tasks, 5 tries each (445 trials), self-run with the leaderboard settings 61.3% (273 of 445) not run
TB 2.1, all 89 tasks, 5 tries each (445 trials), self-run with the leaderboard settings 68.8% (306 of 445) not run

Row 1 is a head-to-head comparison. Both models ran in the same agent (Praex CLI v0.1.32) with a 4,096-token thinking budget per turn and the benchmark's own task timeouts. The 74 tasks are the ones that were graded in every run we compared. The other 15 had a harness error in at least one run. With one try per task, expect a few points of noise either way.

Rows 2 and 3 are the whole benchmark. Terminal-Bench 2.1 is the current version of the benchmark. Both runs cover all 89 tasks with 5 tries each, using the leaderboard settings: timeout multiplier 1.0 and no timeout or resource overrides. Both use a 4,096-token thinking budget per turn.

  • TB 2.0: Praex CLI v0.1.32.
  • TB 2.1: Praex CLI v0.1.40, which keeps the agent alive when a build inside a task runs out of memory.
    • 95% confidence interval: plus or minus 8.0 points, clustered by task.
    • Solved at least once in 5 tries (pass@5): 82.0% of tasks. 43 tasks solved in all 5 tries, 16 never.
    • 3 trials are counted as failures (see the contamination checks below). Without that correction the score would be 69.4%.
    • Trial logs: https://hub.harborframework.com/jobs/30cac5da-1bb0-4750-8625-f28653ed44d2

These are self-run results, not official leaderboard entries. The official Terminal-Bench boards currently add only runs made by their maintainers.

About other published numbers. Z.ai reports GLM-5.3 results measured in Claude Code with timeouts of several hours. Our numbers use the Praex agent, a 4,096-token thinking budget per turn and the benchmark's own timeouts. The two sets of numbers are not comparable. Row 1 is the fair comparison with the base model.

Contamination checks.

  • Training tasks. We checked them against Terminal-Bench 2.0, 2.1 and 4.0 for 13-gram overlap in instructions, for task names and for the canary string. Nothing matched, so no task was removed.
  • What we scan for. In every benchmark trajectory, we look for visits to the benchmark's website, its repositories and copies of its files.
  • TB 2.0: none.
  • TB 2.1: in 4 of the 445 trials the agent searched the web for the task's source.
    • In 3 of them it found the benchmark's own solution and test files, once on a third-party copy, and used them. All 3 passed: two on mteb-retrieve, one on extract-elf. We count those 3 trials as failures.
    • The 4th search found nothing, and that trial failed anyway.
    • The trials are left unedited in the public logs.

How it was trained

Lucia III was trained in two stages on top of GLM-5.3, keeping its FP8 weights.

  1. Faber II is our earlier post-train of GLM-5.3, made for the Praex agent and its tool format.
  2. Lucia III was then trained on Faber II's own successful runs, an approach called rejection-sampled fine-tuning:
    • Faber II attempted 2,000 terminal tasks from NVIDIA's Nemotron-Terminal-Synthetic-Tasks in the Praex agent.
    • We kept the runs that passed the task's tests: 1,195 trajectories, 45.75M tokens.
    • We trained LoRA rank 64 on the attention layers for one epoch, at learning rate 5e-5 on a cosine schedule.
    • This release is the step 145 of 149 checkpoint, merged into the weights.

All training ran on one node with 8 × B200.

Use it

With vLLM v0.28.0 on one node with 8 × B200, which is what we run. The weights are about 716 GiB in FP8.

vllm serve praex/Lucia-III --served-model-name lucia-iii --tensor-parallel-size 8 \
  --trust-remote-code --max-model-len 262144 --kv-cache-dtype fp8 \
  --reasoning-parser glm45 --enable-auto-tool-choice --tool-call-parser glm45

The model has two chat templates:

  • chat_template.jinja, the default, has thinking on.
  • chat_template_nothink.jinja has thinking off.

The default sampling settings are in generation_config.json: temperature 1.0, top_p 0.95.

Limitations

  • We evaluated Lucia III mainly on terminal and coding tasks in English.
  • We have not run Terminal-Bench 4.0 in full, so this card makes no claim about it.
  • When it is stuck, Lucia III sometimes searches the web for the task's source, including benchmark solutions. Block benchmark repositories and their copies when you evaluate it.
  • Like any agent model, it runs real commands. Give it a sandbox or a container when the stakes are high.

License

Lucia III is a derivative of GLM-5.3 and is released under the GLM-5.3 License, an MIT-style license. See LICENSE. Its copyright notice, "Copyright (c) 2026 Z.AI", and its permission notice are kept unchanged, as the license requires.

Clause 2 of the license applies only to very large companies. If a company runs a Model as a Service business and its group revenue is above US$10 billion over 12 months, it must pass Z.AI's security review before commercial use.

Training tasks come from NVIDIA's Nemotron-Terminal-Synthetic-Tasks, licensed CC BY 4.0.

Credits

About Praex

Praex builds its own AI models for coding and terminal agents, and the open-source agent that runs them in the terminal, on desktop, on Android and on the web.

Downloads last month
180
Safetensors
Model size
753B params
Tensor type
BF16
·
F8_E4M3
·
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for praex/Lucia-III

Base model

zai-org/GLM-5.3
Finetuned
(21)
this model

Dataset used to train praex/Lucia-III