---
license: llama3.1
base_model: Raiff1982/codette-llama-3.1-8b-merged
library_name: peft
tags: [codette, star, lora, negative-result, research-artifact]
---

# newton-star — ⚠️ Research Artifact: Published Negative Result

**Do not use this adapter expecting improvement.** It is preserved as the
first arm of a controlled self-taught-reasoning (STaR) study, in which it
**regressed the base adapter from 34.0% to 25.0% (= chance) on GPQA-main.**

Trained on 500 of the model's own correct reasoning chains from *easy*
science QA (ARC/OpenBookQA/SciQ, 81% keep yield). The training fit was
clean (no overfit); the regression is a property of the method+data, not
the optimization: keep-correct STaR on data easier than the evaluation
teaches confident easy-mode reasoning that degrades hard-tier performance.

We publish failures with the same rigor as wins — a benchmark that can't
be trusted to report failure can't be trusted to report success.

## The STaR Study (GPQA-main, reason mode, n=100 per arm)

| Adapter | Training data | GPQA | Verdict |
|---|---|---|---|
| newton (untrained baseline) | — | **34.0%** | reproduced to the decimal, 4 days apart |
| newton-star-r | 350 keep-correct + 180 rationalized | *benchmark in progress* | complete STaR method |
| newton-star-hard | 350 MMLU-Pro STEM keep-correct | **28.0%** | attenuated the harm, below baseline |
| newton-star | 500 easy-science keep-correct | **25.0%** | regressed to chance |

**Finding:** keep-correct self-taught reasoning *consolidates existing ability
rather than extending it* — training data difficulty orders the outcome
perfectly, but no keep-correct arm matched the untrained baseline.
Full methodology, controls, and changelogs:
[Codette-Reasoning](https://github.com/Raiff1982/Codette-Reasoning)
(see `docs/CHANGELOG_2026-07-09.md`, `docs/CHANGELOG_2026-07-11.md`).

Created by Jonathan Harrison (Raiff1982) · Raiff's Bits LLC