| --- |
| license: apache-2.0 |
| base_model: Tesslate/OmniCoder-9B |
| language: |
| - en |
| - zh |
| pipeline_tag: text-generation |
| tags: |
| - code |
| - reasoning |
| - compressed-reasoning |
| - chain-of-thought |
| - qwen3.5 |
| - lora |
| - vllm |
| model-index: |
| - name: Tessera-Preview-9B |
| results: |
| - task: |
| type: text-generation |
| name: Code generation |
| dataset: |
| name: LiveCodeBench release_v6 (full 1,055 problems, 16K budget) |
| type: livecodebench |
| metrics: |
| - type: pass@1 |
| name: "pass@1, greedy (base OmniCoder-9B: 39.5)" |
| value: 34.9 |
| verified: false |
| - type: pass@1 |
| name: "pass@1, temperature 0.6 (base: 45.9)" |
| value: 33.7 |
| verified: false |
| - task: |
| type: text-generation |
| name: Instruction following |
| dataset: |
| name: IFEval (full 541 prompts, greedy) |
| type: HuggingFaceH4/ifeval |
| metrics: |
| - type: accuracy |
| name: "prompt-strict (base: 59.9)" |
| value: 40.9 |
| verified: false |
| - type: accuracy |
| name: "instruction-strict, macro over 834 instructions (base: 60.7)" |
| value: 45.7 |
| verified: false |
| --- |
| |
|  |
|
|
| # Tessera-Preview-9B |
|
|
| A 9B coding model fine-tuned to reason internally in a compressed CJK register while emitting ordinary code and well-formed tool calls. On the full LiveCodeBench set it spends a median **17.7x fewer output tokens per problem** than its own base at a cost of **4.6 points** of pass@1 at matched greedy budget. This is a proof of concept: one LoRA run on a 10,205-example corpus. The costs are real and are reported next to the savings. |
|
|
| The reasoning channel is private by design. Users see normal code and normal tool calls; the compressed register never leaks into the output channel (0 of 46 completed problems in the measured run). |
|
|
| **Paper:** [Tessera-Preview-9B: Compressed Reasoning at 18x Fewer Tokens, and What It Costs](https://zellige.ai/research/compressed-cjk-reasoning) covers the method, the pre-registered protocol, and every number below with confidence intervals. |
|
|
| ## What the compression looks like |
|
|
| Private reasoning channel (real output, LiveCodeBench problem, solved): |
|
|
| ```text |
| 设 s 为 0-indexed 字符串。操作：选 i，c=s[i]，删除 i 左侧最近 c 和右侧最近 c（若存在）。 |
| 观察：每次操作删除两个相同字符。关键：操作不改变字符种类集合，仅减少计数。 |
| 故最终长度 = 不同字符数。算法：返回 len(set(s))。复杂度 O(n|1)。⊘：s 长度 1 → 返回 1。 |
| ``` |
|
|
| Shipped answer (verbatim, complete): |
|
|
| ```python |
| class Solution: |
| def minimizedStringLength(self, s: str) -> int: |
| return len(set(s)) |
| ``` |
|
|
| ## Results |
|
|
| All numbers are paired measurements against the model's own re-measured base (Tesslate/OmniCoder-9B) on one pinned serving stack: vLLM 0.21.0, CUDA graphs, A100-80G, evalscope 1.9.0, temperature pinned per condition, 16,384-token generation budget, ceiling hits scored as failures. |
|
|
| | Metric | Tessera | Base | Gap | |
| | --- | --- | --- | --- | |
| | LCB-1055 pass@1, greedy | **34.9%** [32.1, 37.8] | 39.5% [36.6, 42.5] | −4.6 (95% CI [−7.5, −1.8]) | |
| | LCB-1055 pass@1, temp 0.6 | 33.7% | 45.9% | −12.2 | |
| | IFEval-541 prompt-strict | 40.9% | 59.9% | −19.0 (95% CI [−24.5, −13.6]) | |
| | Median output tokens per LCB problem | **639** | 16,384 (at ceiling) | 17.7x (median paired ratio) | |
| | Budget deaths at greedy (LCB) | 21.6% | 58.8% | | |
|
|
| The two models fail differently. The base almost never writes wrong code (95.9% of its completions pass) but thinks into the 16K ceiling on 58.8% of problems. Tessera completes 78% of problems at a median 639 tokens end to end and errs by writing wrong code. Forcing an empty think collapses accuracy from 66% to 4% on the archived 50-problem protocol: the compressed channel is load-bearing, not decoration. |
|
|
| **Read the limitations before deploying.** The instruction-following gap is large: the training corpus is 100% code-agentic with zero IFEval-style prompts, and the model falls into 16K reasoning loops on 46.8% of such prompts. Sampling does not help this model (greedy is the intended operating point). Whether the gaps are a data-coverage artifact or intrinsic to the compression is the successor's question; the paper argues coverage is the likely major cause and says what would prove it. |
|
|
| ## Usage |
|
|
| The repo ships vLLM-ready weights (LoRA merged, keys repacked, Qwen3.5-9B text config). Serve: |
|
|
| ```bash |
| vllm serve ZelligeAI/tessera-preview-9b \ |
| --served-model-name tessera-preview-9b \ |
| --max-model-len 32768 --dtype bfloat16 \ |
| --language-model-only --gpu-memory-utilization 0.90 \ |
| --reasoning-parser qwen3 \ |
| --enable-auto-tool-choice --tool-call-parser qwen3_coder |
| ``` |
|
|
| Pinned versions matter. All published numbers come from vLLM **0.21.0**; vLLM 0.24.0 degraded this model in our validation and we do not recommend it. Greedy point estimates for this model family are stack-sensitive (the paper documents a 12-point spread across serving stacks), so treat scores measured on other stacks accordingly. |
|
|
| Two serving contracts to respect: |
|
|
| - **Pin temperature 0.0 for code.** The model has no sampling headroom; the shipped `generation_config.json` is greedy by default. |
| - **Do not disable thinking.** `enable_thinking: false` effectively disables the model (48 of 50 outputs empty in the ablation). There is no functional no-think mode. |
|
|
| The reasoning arrives on the standard reasoning channel (`reasoning_content` with the qwen3 parser) and the answer on `content`. You can discard the reasoning; it is not written for reading. |
|
|
| `adapter/` contains the original LoRA (r=16 on Q/K/V/O) plus tokenizer and chat template, for anyone who wants to re-merge against the base or continue training. The base ships behind a vision-language wrapper, so a bare adapter merge needs a key repack; the merge and repack recipe is in the paper's Appendix B. |
|
|
| ## Training |
|
|
| LoRA SFT on 10,205 examples (7.62M target tokens, 2 epochs, one A100-80G, 6h58m). The corpus is 100% code-agentic: single-turn compressed-reasoning items, tool-call wrapped items, 1,500 execution-verified agentic trajectories compressed through [tessera-compressor](https://huggingface.co/ZelligeAI/tessera-compressor), and 400 multi-turn recall items. Training targets were rendered inference-faithfully (history turns carry empty thinks exactly as the serving template produces them). The run was gated by ten pre-registered behavioral probes frozen before any data existed. |
|
|
| ## License |
|
|
| Apache-2.0, same as the base model. Trained on permissively licensed data (per-record licenses listed in the paper's Appendix B). |
|
|
