docs/agt11_scientist_model_selection.md

AGT 11 Scientist Model Selection

Decision

The primary Northflank and local training base for both role adapters is now Qwen/Qwen3.5-9B.

The reduced-scale fallback is Qwen/Qwen3.5-4B for lower-memory smoke runs, faster iteration, and notebook fallback paths.

The optional audit-only judge model candidate is Qwen/Qwen3.5-122B-A10B. It is not part of the deterministic reward loop.

Role Mapping

• Scientist: Qwen/Qwen3.5-9B + Unsloth GRPO LoRA
• Lab Manager / Lab Research Assistant: Qwen/Qwen3.5-9B + Unsloth SFT LoRA
• Fallback Scientist or Lab Manager: Qwen/Qwen3.5-4B
• Audit-only judge candidate: Qwen/Qwen3.5-122B-A10B

Why Qwen3.5-9B For The Two Trainable Roles

• It is a cleaner fit for the current Northflank H100 path than the older Qwen3-8B baseline and keeps both trainable roles on one family.
• It preserves enough planning headroom for strict JSON action output, paper-grounded reasoning, and negotiation under constraints.
• It still leaves a realistic fallback to the 4B variant when the team wants faster notebook iteration.

Why Keep The Judge Deterministic

• The reward source must stay reproducible across runs.
• A large model judge is useful for audits, narrative analysis, and post-run error review, but not for the scalar training reward.
• This keeps benchmark history and before/after graphs comparable across runs.

Current Training Priorities

1. Measure paper understanding explicitly on every evaluation run.
2. Expand Scientist prompt coverage around paper understanding, constraint grounding, and negotiation quality.
3. Keep cumulative benchmark graphs updating across runs instead of only saving one-off plots.
4. Treat the execution-style lab environment as the next architecture phase, not as an untracked reward change.