| --- |
| license: apache-2.0 |
| base_model: Ornith-1.0-35B |
| tags: |
| - agentic |
| - tool-use |
| - code |
| - reasoning |
| pipeline_tag: text-generation |
| library_name: transformers |
| --- |
| |
| # BTL-4 |
|
|
| A 35B agentic reasoning model from Bad Theory Labs, fine-tuned from |
| Ornith-1.0-35B on an execution-gated reasoning corpus. |
|
|
| Built for **tool use, software engineering and long-horizon agent work**. |
|
|
| --- |
|
|
| ## Benchmarks |
|
|
| | Benchmark | BTL-4 | Base Ornith-1.0-35B | Harness | |
| |---|---|---|---| |
| | **BFCL v4 (AST)** | **73.5%** | 69.2% | official `ast_checker`, all 1240 cases | |
| | **LiveCodeBench v6** | **66.1%** | — | official, 442 problems, 2024-08 → 2025-05 | |
| | **SWE-bench Verified** | **78.4%** | — | official harness | |
|
|
| **BFCL and LiveCodeBench were run in-house** with the official scorers, full |
| splits, no subsetting. The BFCL number is a paired comparison: identical |
| harness, identical decoding, only the weights differ. |
|
|
| ### LiveCodeBench by difficulty |
|
|
| | | pass@1 | |
| |---|---| |
| | easy | 99.1% | |
| | medium | 86.7% | |
| | hard | 60.5% | |
|
|
| The set is 45% hard problems, which is what pulls the aggregate down. |
|
|
| --- |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "badtheorylabs/BTL-4" |
| tok = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", |
| device_map="auto") |
| |
| messages = [{"role": "user", "content": "Refactor this function to be pure."}] |
| inputs = tok.apply_chat_template(messages, add_generation_prompt=True, |
| return_tensors="pt").to(model.device) |
| out = model.generate(inputs, max_new_tokens=2048) |
| print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ### Serving |
|
|
| ```bash |
| vllm serve badtheorylabs/BTL-4 \ |
| --max-model-len 131072 \ |
| --enable-auto-tool-choice --tool-call-parser qwen3_xml \ |
| --reasoning-parser qwen3 \ |
| --trust-remote-code |
| ``` |
|
|
| ### Generation settings |
|
|
| Ornith's published settings, used for every number above: |
|
|
| | | | |
| |---|---| |
| | temperature | 1.0 | |
| | top_p | 0.95 | |
| | context | 262144 native | |
| |
| **Give it room to think.** LiveCodeBench improved 60.9% → 66.1% purely by |
| raising the output budget from 16K to 32K. At 16K, 23.5% of problems were |
| truncated mid-solution and scored zero. Hard problems reason longer; cutting |
| them off costs real points. |
| |
| --- |
| |
| ## What it is good at |
| |
| - **Tool calling** — 73.5% BFCL v4 AST, +4.3 points over base |
| - **Competitive programming** — 99.1% easy / 86.7% medium on LiveCodeBench v6 |
| - **Long context** — 262K native, and it uses it |
| |
| ## What it is not |
| |
| - Not a chat model. It reasons before answering and is verbose by default. |
| - **Reasoning accumulates across agent turns.** The chat template strips prior |
| reasoning from older turns, but this only works if your harness separates it |
| into `reasoning_content`. With vLLM, that means `--reasoning-parser qwen3`. |
| Without it, thinking lands in `content`, accumulates every turn, and long |
| agent runs degrade. |
| - Token-hungry on hard problems. Budget accordingly. |
|
|
| --- |
|
|
| ## Training |
|
|
| Fine-tuned from Ornith-1.0-35B on an execution-gated reasoning corpus: |
| candidate trajectories were kept only where the resulting code actually ran and |
| passed its tests, so the reasoning that survived is reasoning that led |
| somewhere. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{btl4-2026, |
| title = {BTL-4: An Execution-Gated Agentic Reasoning Model}, |
| author = {Bad Theory Labs}, |
| year = {2026}, |
| url = {https://huggingface.co/badtheorylabs/BTL-4} |
| } |
| ``` |
|
|
