talkie-1930-coder

13B model fine-tuned on agentic software-engineering trajectories from SWE-smith, starting from the talkie-1930 base. Tuned for the mini-swe-agent interaction format.

SWE-bench-Verified-Working-Harbor pass@1

metric	value
pass@1 (n=5 independent eval runs)	4.48% ± 0.69 pp
per-run resolved (out of 446)	23, 18, 20, 23, 16

Eval pipeline: vLLM (--model-impl transformers --max-model-len 32768 --dtype bfloat16) → mini-swe-agent (mini-extra swebench, temperature 0.7, max_tokens=4096), graded with the swebench harness against ricdomolm/SWE-bench_Verified-Working-Harbor.

Training recipe

Base model	talkie-1930-13b-base
Dataset	talkie-1930-swe-100k-64k (100k SWE-smith trajectories, packed at 64k)
Trainer	TRL SFTTrainer via accelerate (8× A100)
Optimizer	adamw_torch_fused, β=(0.9, 0.95), ε=1e-8
LR	2e-5, cosine_with_min_lr, warmup 3%
Precision	bf16
Weight decay	0.1
Max grad norm	30
Max length	65,536
Packing	bfd + padding-free
Loss	completion_only_loss=1 (loss only on assistant tokens)
Steps	2,016 (this is ckpt-2000)

Usage

This model uses custom modeling code (modeling_talkie.py, configuration_talkie.py). Load with trust_remote_code=True:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "ricdomolm/talkie-1930-coder",
    trust_remote_code=True,
    torch_dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained("ricdomolm/talkie-1930-coder")

For agentic eval, serve with vLLM and drive with mini-swe-agent:

vllm serve ricdomolm/talkie-1930-coder \
    --model-impl transformers --max-model-len 32768 --dtype bfloat16

Companion model

ricdomolm/talkie-web-coder — same recipe, same SFT data, but starting from a base model pre-trained on web-style data. Reaches 5.75% ± 1.04 pp on the same eval (n=3).