README.md

metadata

license: apache-2.0
base_model: Tesslate/OmniCoder-9B
language:
  - en
  - zh
pipeline_tag: text-generation
tags:
  - code
  - reasoning
  - compressed-reasoning
  - chain-of-thought
  - qwen3.5
  - lora
  - vllm
model-index:
  - name: Tessera-Preview-9B
    results:
      - task:
          type: text-generation
          name: Code generation
        dataset:
          name: LiveCodeBench release_v6 (full 1,055 problems, 16K budget)
          type: livecodebench
        metrics:
          - type: pass@1
            name: 'pass@1, greedy (base OmniCoder-9B: 39.5)'
            value: 34.9
            verified: false
          - type: pass@1
            name: 'pass@1, temperature 0.6 (base: 45.9)'
            value: 33.7
            verified: false
      - task:
          type: text-generation
          name: Instruction following
        dataset:
          name: IFEval (full 541 prompts, greedy)
          type: HuggingFaceH4/ifeval
        metrics:
          - type: accuracy
            name: 'prompt-strict (base: 59.9)'
            value: 40.9
            verified: false
          - type: accuracy
            name: 'instruction-strict, macro over 834 instructions (base: 60.7)'
            value: 45.7
            verified: false

Tessera-Preview-9B

A 9B coding model fine-tuned to reason internally in a compressed CJK register while emitting ordinary code and well-formed tool calls. On the full LiveCodeBench set it spends a median 17.7x fewer output tokens per problem than its own base at a cost of 4.6 points of pass@1 at matched greedy budget. This is a proof of concept: one LoRA run on a 10,205-example corpus. The costs are real and are reported next to the savings.

The reasoning channel is private by design. Users see normal code and normal tool calls; the compressed register never leaks into the output channel (0 of 46 completed problems in the measured run).

Paper: Tessera-Preview-9B: Compressed Reasoning at 18x Fewer Tokens, and What It Costs covers the method, the pre-registered protocol, and every number below with confidence intervals.

What the compression looks like

Private reasoning channel (real output, LiveCodeBench problem, solved):

设 s 为 0-indexed 字符串。操作：选 i，c=s[i]，删除 i 左侧最近 c 和右侧最近 c（若存在）。
观察：每次操作删除两个相同字符。关键：操作不改变字符种类集合，仅减少计数。
故最终长度 = 不同字符数。算法：返回 len(set(s))。复杂度 O(n|1)。⊘：s 长度 1 → 返回 1。

Shipped answer (verbatim, complete):

class Solution:
    def minimizedStringLength(self, s: str) -> int:
        return len(set(s))

Results

All numbers are paired measurements against the model's own re-measured base (Tesslate/OmniCoder-9B) on one pinned serving stack: vLLM 0.21.0, CUDA graphs, A100-80G, evalscope 1.9.0, temperature pinned per condition, 16,384-token generation budget, ceiling hits scored as failures.

Metric	Tessera	Base	Gap
LCB-1055 pass@1, greedy	34.9% [32.1, 37.8]	39.5% [36.6, 42.5]	−4.6 (95% CI [−7.5, −1.8])
LCB-1055 pass@1, temp 0.6	33.7%	45.9%	−12.2
IFEval-541 prompt-strict	40.9%	59.9%	−19.0 (95% CI [−24.5, −13.6])
Median output tokens per LCB problem	639	16,384 (at ceiling)	17.7x (median paired ratio)
Budget deaths at greedy (LCB)	21.6%	58.8%

The two models fail differently. The base almost never writes wrong code (95.9% of its completions pass) but thinks into the 16K ceiling on 58.8% of problems. Tessera completes 78% of problems at a median 639 tokens end to end and errs by writing wrong code. Forcing an empty think collapses accuracy from 66% to 4% on the archived 50-problem protocol: the compressed channel is load-bearing, not decoration.

Read the limitations before deploying. The instruction-following gap is large: the training corpus is 100% code-agentic with zero IFEval-style prompts, and the model falls into 16K reasoning loops on 46.8% of such prompts. Sampling does not help this model (greedy is the intended operating point). Whether the gaps are a data-coverage artifact or intrinsic to the compression is the successor's question; the paper argues coverage is the likely major cause and says what would prove it.

Usage

The repo ships vLLM-ready weights (LoRA merged, keys repacked, Qwen3.5-9B text config). Serve:

vllm serve ZelligeAI/tessera-preview-9b \
  --served-model-name tessera-preview-9b \
  --max-model-len 32768 --dtype bfloat16 \
  --language-model-only --gpu-memory-utilization 0.90 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

Pinned versions matter. All published numbers come from vLLM 0.21.0; vLLM 0.24.0 degraded this model in our validation and we do not recommend it. Greedy point estimates for this model family are stack-sensitive (the paper documents a 12-point spread across serving stacks), so treat scores measured on other stacks accordingly.

Two serving contracts to respect:

• Pin temperature 0.0 for code. The model has no sampling headroom; the shipped generation_config.json is greedy by default.
• Do not disable thinking. enable_thinking: false effectively disables the model (48 of 50 outputs empty in the ablation). There is no functional no-think mode.

The reasoning arrives on the standard reasoning channel (reasoning_content with the qwen3 parser) and the answer on content. You can discard the reasoning; it is not written for reading.

adapter/ contains the original LoRA (r=16 on Q/K/V/O) plus tokenizer and chat template, for anyone who wants to re-merge against the base or continue training. The base ships behind a vision-language wrapper, so a bare adapter merge needs a key repack; the merge and repack recipe is in the paper's Appendix B.

Training

LoRA SFT on 10,205 examples (7.62M target tokens, 2 epochs, one A100-80G, 6h58m). The corpus is 100% code-agentic: single-turn compressed-reasoning items, tool-call wrapped items, 1,500 execution-verified agentic trajectories compressed through tessera-compressor, and 400 multi-turn recall items. Training targets were rendered inference-faithfully (history turns carry empty thinks exactly as the serving template produces them). The run was gated by ten pre-registered behavioral probes frozen before any data existed.

License

Apache-2.0, same as the base model. Trained on permissively licensed data (per-record licenses listed in the paper's Appendix B).