Self-Forge paper (EN+PT) — judge-gated continual specialization; phases 1-4 measured

PAPER_EN.md

@@ -0,0 +1,92 @@
+# Self-Forge: Judge-Gated Continual Specialization of Small Local Language Models
+
+**Author:** MrWoRmMr · **Status:** DRAFT · rev. 2026-08-25
+
+> *Draft for a repository writeup. Names/infra of the author's personal agent ecosystem are generalized. Phases 1–4 are executed with measured results below: the first specialist has been forged and evaluated against its base.*
+
+## Abstract
+
+We describe **Self-Forge**, an architecture in which a small, locally-hosted language model is *quantized to fit commodity hardware*, then *continually specialized* toward a narrow domain through an autonomous, judge-gated fine-tuning loop. The central claim is practical rather than theoretical: for a bounded domain, a small model that is (a) run at a healable quantization level and (b) densely specialized can match a much larger general model on the domain, on hardware costing near-zero, entirely on-premises. We take this end-to-end on 2016-era commodity hardware: a 7B base runs on an 8GB GPU (fits, reasons, refuses fabricated data at ~19 tok/s), fine-tunes via LoRA sharded across three Pascal GPUs (loss 1.9→0.4), and the resulting specialist — evaluated against its base — internalizes the target house format, *eliminates* the base's fabricated figures, and **preserves groundedness**. The result is measured, fully on-premises, at near-zero marginal cost. We report each phase's evidence, including the failure modes (a judge-gate is essential and human verification still sits above it; the method-reasoning gain is bounded by dataset size).
+
+## 1. Motivation
+
+Retrieval-augmented generation (RAG) lets an agent *consult* external notes but does not make the model itself more capable: knowledge stays outside the weights, and every answer is a lookup. For a domain the agent uses constantly, we want the competence **internalized** — the model reasoning *from* the domain, not retrieving it. Two constraints shape our setting: (1) **sovereignty/privacy** — training and inference must stay on the owner's hardware, ruling out rented cloud GPUs; (2) **commodity hardware** — the deployment target is an 8GB consumer GPU. The question: *can a small local model be forged into a domain specialist that rivals a large general model, under these constraints?*
+
+## 2. The Self-Forge Loop
+
+The approach is fine-tuning wrapped in an autonomous, verified loop:
+
+1. **Ingest** the domain → generate candidate training examples (Q&A, rationales).
+2. **Judge-gate**: an independent verifier admits only correct/grounded examples.
+3. **Specialize**: low-rank fine-tuning (LoRA/QLoRA) on the admitted set.
+4. **Evaluate** on a held-out set; **promote** the adapter only if it improves the target *and* does not regress a frozen "must-not-break" set.
+5. Repeat on a cadence (weekly → daily) until held-out performance plateaus ("topic saturated").
+
+This mirrors human learning: test against reality, don't reinforce your own errors, consolidate periodically, unlearn bad habits.
+
+## 3. Method
+
+**3.1 Quantize-then-heal.** Quantization is treated as a lossy channel whose *specific* failures are measured (probing) and then *repaired* by the adapter, which compensates for quantization error on the targeted distribution. This works down to ~Q4 (a dented model the adapter can correct); below that (Q2/Q3) too much information is destroyed for a low-rank correction to recover.
+
+**3.2 Dense specialization.** A small model's limited capacity, concentrated on one domain, can exceed a large generalist on that domain. Fine-tuning targets the *stable* method (frameworks, reasoning, style); *volatile* facts (prices, current figures) remain in RAG and are never baked into weights.
+
+**3.3 Controls (anti-collapse, anti-forgetting).** Training a model on self-generated data risks *model collapse*; an external verifier (the judge) is the anchor that prevents it. Continual updates risk *catastrophic forgetting*; mitigations: a replay set, a **library of per-domain adapters** (hot-swapped rather than one growing blob), and periodic consolidation. Governance: versioned adapters with trivial rollback, shadow/canary before serving, and a drift detector that freezes learning when hallucination rises. Every step is small, verified, reversible, and human-approved.
+
+## 4. Results
+
+### 4.1 Inference feasibility (Phase 1)
+Measured on an 8GB Maxwell-class GPU sharing memory with a 0.5GB embedder. Base model: a general 7B (Q4, ~4.7GB), pre-fine-tune baseline.
+
+| Criterion | Result |
+|---|---|
+| Fits target GPU | ✅ ~5.2GB with embedder (< 8GB) |
+| Throughput | ~18.9 tok/s |
+| Domain reasoning | ✅ correct (ROE 22% > cost-of-equity 18% ⇒ price-to-book > 1) |
+| Groundedness | ✅ refused to fabricate price/earnings for a nonexistent ticker |
+
+Implication: the base is *already* a competent, grounded domain reasoner, so specialization is **narrow** (internalize method/style), not from-scratch teaching.
+
+A companion probe stress-tested quantization at the extreme: a **22B model crushed to ~2.2-bit (IQ2)** to fit the same 8GB. It did **not** collapse — it reasoned correctly and refused fabricated live data — but ran ~3× slower (6.4 vs 18.9 tok/s) and, on harder self-referential probes, confabulated about its own identity/config (≈7/10 grounded). Finding: aggressive quantization of a *large* base degrades **speed and self-knowledge before core correctness**; the practical winner on an 8GB target remains a small model at a healable (≥Q4) quant.
+
+### 4.2 Local training path (Phase 2)
+The critical bottleneck of earlier drafts — *does training run at all on this hardware?* — is resolved. A 7B base fine-tunes via LoRA (fp16) **sharded across three Pascal-class GPUs**: loss fell cleanly (3.64→2.48) over a smoke run, adapter saved, peak ~5–6GB/GPU. Two lessons: (a) the widespread belief "Pascal can't train modern PyTorch" holds only for 4-bit QLoRA (bitsandbytes); **plain fp16 LoRA runs fine**; (b) bleeding-edge trainer libraries broke in several ways (chunked-loss patches, collator format, a chat-template returning strings not ids), so the robust recipe is a **manual torch training loop** (forward→backward→step) rather than a high-level Trainer. Throughput is modest (~0.25 steps/s), making real fine-tunes minutes-to-hours, viable as batch jobs.
+
+### 4.3 Dataset construction (Phase 3)
+A 90-example domain dataset was built in two deliberately-separated slices:
+- **Persona/format (71 ex)** — generated *deterministically* by running the agent's own valuation function across the equity index; numbers are real, format is gold, **grounded by construction** (no LLM, no judge). Notably ~75% of these examples teach *honesty* — the model admitting when a DCF is inconclusive or inapplicable.
+- **Method/reasoning (19 ex)** — chunks from valuation textbooks → an LLM generates a Q&A → a judge filters. Here the judge-gate proved **essential and measurable**: naive generation was ~40% usable and produced *dangerous contradictions* (a 7B asserting DCF suits banks — the opposite of correct). Two guards fixed it: a retrieval-distance filter (killing off-topic chunks) and an LLM judge. A measured trade-off emerged — a strong slow cross-model judge (22B) yielded 100% clean but was impractically slow; a fast same-model judge was ~6× faster but more lenient (human inspection still caught ~24% meta/disclaimer junk it passed). **Human verification remained necessary above the automated gate.**
+
+### 4.4 First specialist + eval-gate (Phase 4)
+A 7B base was fine-tuned via LoRA (r=16, ~10M trainable params) on the 90-example dataset, 3 epochs, sharded across three Pascal GPUs (36 min, 270 steps). Training loss fell smoothly **1.885 → 0.553 → 0.408** — clean convergence, no divergence. The resulting adapter was evaluated against the raw base on the same probes, across three dimensions:
+
+| Dimension | Base (raw 7B) | Specialist (fine-tuned) |
+|---|---|---|
+| **Persona / format** | generic markdown; *fabricated numbers* (invented a share price and EPS) | reproduced the agent's exact house format (structured valuation card, deterministic tone) |
+| **Method reasoning** | correct (WACC, why-DCF-fails-for-banks) | correct — parity |
+| **Groundedness** | refused a fabricated ticker | refused — *preserved* |
+
+**Verdict: success where it was trained.** The fine-tune's clearest win is **format/persona** — the specialist internalized the house output style the base had no notion of, and in doing so *stopped fabricating figures* the base had invented. Crucially, **fine-tuning did not damage groundedness** (the primary risk — that training on generated data erodes the model's "I don't know"). Method reasoning was parity, expected given the method slice was small (19 vs 71 examples) and the base was already competent there.
+
+One nuance validates the architecture rather than undermining it: the specialist occasionally emits imprecise figures. This is *by design* — the persona slice teaches the *format*; the real numbers are injected at inference from a deterministic function (RAG/code), never baked into weights. Volatile facts stay out of the parameters; the model supplies structure and reasoning, the code supplies truth.
+
+## 5. Related Work
+
+QLoRA (fine-tuning a 4-bit-quantized base via low-rank adapters); self-improvement / self-taught reasoning (STaR) and RLAIF (learning from self-generated, verifier-filtered data); the model-collapse literature (why an external anchor is required); catastrophic forgetting and continual-learning mitigations (replay, adapter libraries); and the RAG-vs-fine-tuning trade-off (external knowledge vs internalized capability). Our contribution is not a new algorithm but a **composition**: quantize-then-heal + dense specialization + a judge-gated continual loop, under strict on-premises constraints.
+
+## 6. Limitations & Open Questions
+
+- **The win is format/persona, not (yet) reasoning.** The specialist clearly beats the base on house-format adherence and stops the base's number-fabrication, at parity on method reasoning and groundedness. Moving method reasoning from *parity* to *win* requires a larger method dataset — the current 19 method examples are a proof, not a corpus.
+- **A single eval run, few probes.** The verdict rests on 4 hand-inspected probes, not a held-out benchmark; a proper eval set (and comparison against the large incumbent, not just the base) is the next rigor step.
+- **Throughput, not feasibility, is the constraint.** Training and judging on 2016-era GPUs is slow (~0.25 train steps/s; a 22B judge at ~6 tok/s), so scaling the dataset to hundreds of examples is a batch/overnight job, not interactive.
+- **Dataset size.** 90 examples proves the pipeline; a production specialist wants substantially more method examples, bounded by textbook-chunk coverage.
+- **Hardware ceiling.** The 8GB target caps a healable (≥Q4) model at ~9B. Larger models require larger cards or a multi-node pool.
+- **Whether the specialist matches the incumbent** is the open empirical question the roadmap exists to answer.
+- **Quantization does not fix fabrication** — it addresses speed/size, not a model's intrinsic tendency to hallucinate; groundedness must be verified per model.
+
+## 7. Roadmap (phases)
+
+0. Lock plan ✅ · 1. Inference feasibility ✅ · 1b. base bake-off ✅ · 2. local training stack ✅ · 3. dataset (generate → judge-gate → split) ✅ · 4. first fine-tune + eval-gate ✅ · **5. close the loop** (collect → gate → train → eval → cadence) — next · 6. production controls (adapter library, canary, drift kill-switch) · 7. generalize (other domains; on-device/mobile). *Phases 0–4 done; the autonomous loop (5) and production controls (6) remain.*
+
+## 8. Availability
+
+Intended for release under a pseudonymous repository once Phase 2+ are validated: **method, evaluation harness, and the trained adapter** would be public; the domain dataset and the specific loop orchestration remain private (the practical moat). Sovereignty here means owning the *intelligence* (local model + data + loop), not reinventing the commodity substrate (quantization formats, adapter training) — which is used as-is, and improved only where it fails this hardware specifically.

PAPER_PT.md

@@ -0,0 +1,82 @@
+# Self-Forge: Especialização Contínua e Filtrada-por-Juiz de Modelos de Linguagem Pequenos e Locais
+
+**Autor:** MrWoRmMr · **Estado:** RASCUNHO · rev. 2026-08-25
+
+> *Rascunho para um writeup de repositório. Nomes/infra do ecossistema pessoal de agentes do autor estão generalizados. As Fases 1–4 estão executadas, com resultados medidos abaixo: o primeiro especialista foi forjado e avaliado contra seu base.*
+
+## Resumo
+Descrevemos o **Self-Forge**, uma arquitetura em que um modelo de linguagem pequeno, hospedado localmente, é *quantizado para caber em hardware comum* e então *continuamente especializado* rumo a um domínio estreito, por meio de um loop autônomo de fine-tuning filtrado por um juiz. A afirmação central é prática, não teórica: para um domínio delimitado, um modelo pequeno que seja (a) executado num nível de quantização *curável* e (b) densamente especializado pode **igualar um modelo geral muito maior** naquele domínio, em hardware de custo próximo de zero, inteiramente on-premises. Levamos isso de ponta a ponta em hardware comum de 2016: um base 7B roda numa GPU de 8GB (cabe, raciocina, recusa dado fabricado a ~19 tok/s), fine-tuna via LoRA sharded em três GPUs Pascal (loss 1,9→0,4), e o especialista resultante — avaliado contra seu base — internaliza o formato-casa alvo, *elimina* os números fabricados do base, e **preserva a groundedness**. O resultado é medido, inteiramente on-premises, a custo marginal próximo de zero. Reportamos a evidência de cada fase, incluindo os modos de falha (o juiz-gate é essencial e a verificação humana ainda fica acima dele; o ganho de raciocínio-de-método é limitado pelo tamanho do dataset).
+
+## 1. Motivação
+A geração aumentada por recuperação (RAG) permite que um agente *consulte* anotações externas, mas não torna o modelo em si mais capaz: o conhecimento fica fora dos pesos, e toda resposta é uma busca. Para um domínio que o agente usa o tempo todo, queremos a competência **internalizada** — o modelo raciocinando *a partir* do domínio, não o recuperando. Duas restrições moldam o cenário: (1) **soberania/privacidade** — treino e inferência devem permanecer no hardware do dono, o que descarta GPUs alugadas em nuvem; (2) **hardware comum** — o alvo de implantação é uma GPU de consumo de 8GB. A pergunta: *um modelo pequeno local pode ser forjado num especialista de domínio que rivalize com um modelo geral grande, sob essas restrições?*
+
+## 2. O Loop Self-Forge
+A abordagem é fine-tuning envolto num loop autônomo e verificado:
+1. **Ingerir** o domínio → gerar exemplos candidatos de treino (Q&A, raciocínios).
+2. **Filtro-do-juiz**: um verificador independente admite apenas exemplos corretos/fundamentados (grounded).
+3. **Especializar**: fine-tuning de baixo posto (LoRA/QLoRA) sobre o conjunto admitido.
+4. **Avaliar** num held-out; **promover** o adapter apenas se melhorar o alvo *e* não regredir um conjunto congelado "não-pode-piorar".
+5. Repetir numa cadência (semanal → diária) até o desempenho no held-out estabilizar ("tema saturado").
+
+Isso espelha o aprendizado humano: testar contra a realidade, não reforçar os próprios erros, consolidar periodicamente, desaprender maus hábitos.
+
+## 3. Método
+**3.1 Quantizar-e-curar.** A quantização é tratada como um canal com perda cujas falhas *específicas* são medidas (por probing) e então *reparadas* pelo adapter, que compensa o erro de quantização na distribuição alvo. Isso funciona até ~Q4 (um modelo "amassado" que o adapter corrige); abaixo disso (Q2/Q3) informação demais é destruída para uma correção de baixo posto recuperar.
+
+**3.2 Especialização densa.** A capacidade limitada de um modelo pequeno, concentrada num único domínio, pode exceder um generalista grande *naquele* domínio. O fine-tuning mira o *método estável* (frameworks, raciocínio, estilo); *fatos voláteis* (preços, números correntes) permanecem no RAG e nunca são assados nos pesos.
+
+**3.3 Controles (anti-colapso, anti-esquecimento).** Treinar um modelo com dados gerados por ele mesmo arrisca o *colapso do modelo* (model collapse); um verificador externo (o juiz) é a âncora que evita isso. Atualizações contínuas arriscam o *esquecimento catastrófico*; mitigações: um conjunto de replay, uma **biblioteca de adapters por domínio** (troca a quente, em vez de um único blob crescente) e consolidação periódica. Governança: adapters versionados com rollback trivial, shadow/canary antes de servir, e um detector de drift que congela o aprendizado quando a alucinação sobe. Cada passo é pequeno, verificado, reversível e aprovado por um humano.
+
+## 4. Resultados
+
+### 4.1 Viabilidade de inferência (Fase 1)
+Medida numa GPU de 8GB (classe Maxwell) dividindo memória com um embedder de 0,5GB. Modelo-base: um 7B geral (Q4, ~4,7GB), baseline pré-fine-tune.
+
+| Critério | Resultado |
+|---|---|
+| Cabe na GPU alvo | ✅ ~5,2GB com o embedder (< 8GB) |
+| Vazão | ~18,9 tok/s |
+| Raciocínio de domínio | �� correto (ROE 22% > custo de capital próprio 18% ⇒ preço/valor patrimonial > 1) |
+| Groundedness | ✅ recusou fabricar preço/lucro de um ticker inexistente |
+
+Implicação: o base *já* é um raciocinador competente e honesto, então a especialização é **estreita** (internalizar método/estilo), não ensino do zero.
+
+Um probe complementar testou a quantização no extremo: um **22B esmagado a ~2,2 bits (IQ2)** para caber nos mesmos 8GB. Ele **não** colapsou — raciocinou certo e recusou dado ao vivo fabricado — mas rodou ~3× mais lento (6,4 vs 18,9 tok/s) e, em probes mais difíceis de auto-referência, confabulou sobre a própria identidade/config (≈7/10 grounded). Achado: quantização agressiva de um base *grande* degrada **velocidade e auto-conhecimento antes da correção central**; o vencedor prático num alvo de 8GB continua sendo um modelo pequeno num quant curável (≥Q4).
+
+### 4.2 Caminho de treino local (Fase 2)
+O gargalo crítico dos rascunhos anteriores — *o treino roda neste hardware?* — está resolvido. Um base 7B fine-tuna via LoRA (fp16) **espalhado (sharded) em três GPUs Pascal**: a loss caiu limpo (3,64→2,48) num smoke run, adapter salvo, pico ~5–6GB/GPU. Duas lições: (a) a crença difundida "Pascal não treina PyTorch moderno" vale só pro QLoRA 4-bit (bitsandbytes); **LoRA fp16 puro roda liso**; (b) bibliotecas de treino bleeding-edge quebraram de várias formas (patches de chunked-loss, formato de collator, um chat-template retornando string em vez de ids), então a receita robusta é um **loop de treino manual em torch** (forward→backward→step), não um Trainer de alto nível. A vazão é modesta (~0,25 steps/s), tornando fine-tunes reais de minutos a horas, viáveis como jobs em lote.
+
+### 4.3 Construção do dataset (Fase 3)
+Um dataset de 90 exemplos foi construído em duas fatias deliberadamente separadas:
+- **Persona/formato (71 ex)** — gerados *deterministicamente* rodando a própria função de valuation do agente sobre o índice de ações; números reais, formato-ouro, **grounded por construção** (sem LLM, sem juiz). Notavelmente ~75% desses exemplos ensinam *honestidade* — o modelo admitindo quando o DCF é inconclusivo ou inaplicável.
+- **Método/raciocínio (19 ex)** — chunks de livros de valuation → um LLM gera Q&A → um juiz filtra. Aqui o juiz-gate provou-se **essencial e mensurável**: geração ingênua ficou ~40% aproveitável e produziu *contradições perigosas* (um 7B afirmando que DCF serve pra bancos — o oposto do correto). Dois guardas resolveram: um filtro de distância de retrieval (mata chunks fora do tópico) e um juiz LLM. Um trade-off medido emergiu — um juiz forte-lento cross-model (22B) deu 100% limpo mas foi impraticavelmente lento; um juiz rápido mesmo-modelo foi ~6× mais rápido, porém mais leniente (a inspeção humana ainda pegou ~24% de lixo meta/disclaimer que ele deixou passar). **A verificação humana continuou necessária acima do gate automático.**
+
+### 4.4 Primeiro especialista + eval-gate (Fase 4)
+Um base 7B foi fine-tunado via LoRA (r=16, ~10M params treináveis) sobre o dataset de 90 exemplos, 3 épocas, sharded em três GPUs Pascal (36 min, 270 steps). A loss de treino caiu suave **1,885 → 0,553 → 0,408** — convergência limpa, sem divergir. O adapter foi avaliado contra o base cru nos mesmos probes, em três dimensões:
+
+| Dimensão | Base (7B cru) | Especialista (fine-tunado) |
+|---|---|---|
+| **Persona / formato** | markdown genérico; *números fabricados* (inventou preço da ação e LPA) | reproduziu o formato-casa exato do agente (ficha de valuation estruturada, tom determinístico) |
+| **Raciocínio de método** | correto (WACC, por que DCF falha em bancos) | correto — paridade |
+| **Groundedness** | recusou um ticker fabricado | recusou — *preservada* |
+
+**Veredito: sucesso onde foi treinado.** A vitória mais clara é **formato/persona** — o especialista internalizou o estilo-casa que o base não tinha noção, e ao fazê-lo *parou de fabricar números* que o base inventava. Crucialmente, **o fine-tune não danificou a groundedness** (o risco nº1 — treinar em dado gerado erodir o "não sei"). O raciocínio de método ficou em paridade, esperado dado que a fatia de método era pequena (19 vs 71 exemplos) e o base já era competente ali.
+
+Uma nuance valida a arquitetura em vez de miná-la: o especialista às vezes emite números imprecisos. Isso é *por design* — a fatia de persona ensina o *formato*; os números reais são injetados na inferência por uma função determinística (RAG/código), nunca assados nos pesos. Fatos voláteis ficam fora dos parâmetros; o modelo fornece estrutura e raciocínio, o código fornece a verdade.
+
+## 5. Trabalho Relacionado
+QLoRA (fine-tuning de um base quantizado em 4-bit via adapters de baixo posto); auto-aprimoramento / raciocínio autodidata (STaR) e RLAIF (aprender de dados gerados por si mesmo e filtrados por verificador); a literatura de colapso de modelo (por que uma âncora externa é necessária); esquecimento catastrófico e mitigações de aprendizado contínuo (replay, bibliotecas de adapters); e o trade-off RAG-vs-fine-tuning (conhecimento externo vs capacidade internalizada). Nossa contribuição não é um algoritmo novo, e sim uma **composição**: quantizar-e-curar + especialização densa + um loop contínuo filtrado por juiz, sob restrições estritas de on-premises.
+
+## 6. Limitações & Questões em Aberto
+- **A vitória é formato/persona, não (ainda) raciocínio.** O especialista bate o base claramente em aderência ao formato-casa e para a fabricação de números do base, em paridade no raciocínio de método e na groundedness. Mover o método de *paridade* para *vitória* exige um dataset de método maior — os 19 exemplos atuais são prova, não corpus.
+- **Um único eval, poucos probes.** O veredito repousa em 4 probes inspecionados à mão, não num benchmark held-out; um conjunto de eval próprio (e comparação contra o incumbente grande, não só o base) é o próximo passo de rigor.
+- **Vazão, não viabilidade, é a restrição.** Treinar e julgar em GPUs de 2016 é lento (~0,25 steps/s de treino; juiz 22B a ~6 tok/s), então escalar o dataset pra centenas é job em lote/overnight, não interativo.
+- **Tamanho do dataset.** 90 exemplos provam o pipeline; um especialista de produção quer substancialmente mais exemplos de método, limitado pela cobertura de chunks dos livros.
+- **Teto de hardware.** O alvo de 8GB limita um modelo curável (≥Q4) a ~9B. Modelos maiores exigem placas maiores ou um pool multi-nó.
+- **Quantização não conserta fabricação** — trata velocidade/tamanho, não a tendência intrínseca a alucinar; groundedness deve ser verificada por modelo.
+
+## 7. Roadmap (fases)
+0. Travar plano ✅ · 1. Viabilidade de inferência ✅ · 1b. bake-off de base ✅ · 2. stack de treino local ✅ · 3. dataset (gerar → filtro-do-juiz → dividir) ✅ · 4. primeiro fine-tune + eval-gate ✅ · **5. fechar o loop** (coletar → filtrar → treinar → avaliar → cadência) — próximo · 6. controles de produção (biblioteca de adapters, canary, kill-switch) · 7. generalizar (outros domínios; on-device/celular). *Fases 0–4 feitas; faltam o loop autônomo (5) e os controles de produção (6).*
+
+## 8. Disponibilidade
+Pretendido para lançamento em repositório pseudônimo assim que a Fase 2+ for validada: **método, harness de avaliação e o adapter treinado** seriam públicos; o dataset do domínio e a orquestração específica do loop permanecem privados (o moat prático). Soberania aqui significa possuir a *inteligência* (modelo local + dado + loop), não reinventar o substrato commodity (formatos de quantização, treino de adapters) — que é usado como está, e melhorado apenas onde falha *especificamente* neste hardware.

README.md

@@ -0,0 +1,105 @@
+---
+license: mit
+language:
+- en
+- pt
+tags:
+- self-forge
+- fine-tuning
+- lora
+- local-llm
+- groundedness
+---
+
+# Self-Forge: Judge-Gated Continual Specialization of Small Local Language Models
+
+**Author:** MrWoRmMr · **Status:** DRAFT · rev. 2026-08-25
+
+> *Draft for a repository writeup. Names/infra of the author's personal agent ecosystem are generalized. Phases 1–4 are executed with measured results below: the first specialist has been forged and evaluated against its base.*
+
+## Abstract
+
+We describe **Self-Forge**, an architecture in which a small, locally-hosted language model is *quantized to fit commodity hardware*, then *continually specialized* toward a narrow domain through an autonomous, judge-gated fine-tuning loop. The central claim is practical rather than theoretical: for a bounded domain, a small model that is (a) run at a healable quantization level and (b) densely specialized can match a much larger general model on the domain, on hardware costing near-zero, entirely on-premises. We take this end-to-end on 2016-era commodity hardware: a 7B base runs on an 8GB GPU (fits, reasons, refuses fabricated data at ~19 tok/s), fine-tunes via LoRA sharded across three Pascal GPUs (loss 1.9→0.4), and the resulting specialist — evaluated against its base — internalizes the target house format, *eliminates* the base's fabricated figures, and **preserves groundedness**. The result is measured, fully on-premises, at near-zero marginal cost. We report each phase's evidence, including the failure modes (a judge-gate is essential and human verification still sits above it; the method-reasoning gain is bounded by dataset size).
+
+## 1. Motivation
+
+Retrieval-augmented generation (RAG) lets an agent *consult* external notes but does not make the model itself more capable: knowledge stays outside the weights, and every answer is a lookup. For a domain the agent uses constantly, we want the competence **internalized** — the model reasoning *from* the domain, not retrieving it. Two constraints shape our setting: (1) **sovereignty/privacy** — training and inference must stay on the owner's hardware, ruling out rented cloud GPUs; (2) **commodity hardware** — the deployment target is an 8GB consumer GPU. The question: *can a small local model be forged into a domain specialist that rivals a large general model, under these constraints?*
+
+## 2. The Self-Forge Loop
+
+The approach is fine-tuning wrapped in an autonomous, verified loop:
+
+1. **Ingest** the domain → generate candidate training examples (Q&A, rationales).
+2. **Judge-gate**: an independent verifier admits only correct/grounded examples.
+3. **Specialize**: low-rank fine-tuning (LoRA/QLoRA) on the admitted set.
+4. **Evaluate** on a held-out set; **promote** the adapter only if it improves the target *and* does not regress a frozen "must-not-break" set.
+5. Repeat on a cadence (weekly → daily) until held-out performance plateaus ("topic saturated").
+
+This mirrors human learning: test against reality, don't reinforce your own errors, consolidate periodically, unlearn bad habits.
+
+## 3. Method
+
+**3.1 Quantize-then-heal.** Quantization is treated as a lossy channel whose *specific* failures are measured (probing) and then *repaired* by the adapter, which compensates for quantization error on the targeted distribution. This works down to ~Q4 (a dented model the adapter can correct); below that (Q2/Q3) too much information is destroyed for a low-rank correction to recover.
+
+**3.2 Dense specialization.** A small model's limited capacity, concentrated on one domain, can exceed a large generalist on that domain. Fine-tuning targets the *stable* method (frameworks, reasoning, style); *volatile* facts (prices, current figures) remain in RAG and are never baked into weights.
+
+**3.3 Controls (anti-collapse, anti-forgetting).** Training a model on self-generated data risks *model collapse*; an external verifier (the judge) is the anchor that prevents it. Continual updates risk *catastrophic forgetting*; mitigations: a replay set, a **library of per-domain adapters** (hot-swapped rather than one growing blob), and periodic consolidation. Governance: versioned adapters with trivial rollback, shadow/canary before serving, and a drift detector that freezes learning when hallucination rises. Every step is small, verified, reversible, and human-approved.
+
+## 4. Results
+
+### 4.1 Inference feasibility (Phase 1)
+Measured on an 8GB Maxwell-class GPU sharing memory with a 0.5GB embedder. Base model: a general 7B (Q4, ~4.7GB), pre-fine-tune baseline.
+
+| Criterion | Result |
+|---|---|
+| Fits target GPU | ✅ ~5.2GB with embedder (< 8GB) |
+| Throughput | ~18.9 tok/s |
+| Domain reasoning | ✅ correct (ROE 22% > cost-of-equity 18% ⇒ price-to-book > 1) |
+| Groundedness | ✅ refused to fabricate price/earnings for a nonexistent ticker |
+
+Implication: the base is *already* a competent, grounded domain reasoner, so specialization is **narrow** (internalize method/style), not from-scratch teaching.
+
+A companion probe stress-tested quantization at the extreme: a **22B model crushed to ~2.2-bit (IQ2)** to fit the same 8GB. It did **not** collapse — it reasoned correctly and refused fabricated live data — but ran ~3× slower (6.4 vs 18.9 tok/s) and, on harder self-referential probes, confabulated about its own identity/config (≈7/10 grounded). Finding: aggressive quantization of a *large* base degrades **speed and self-knowledge before core correctness**; the practical winner on an 8GB target remains a small model at a healable (≥Q4) quant.
+
+### 4.2 Local training path (Phase 2)
+The critical bottleneck of earlier drafts — *does training run at all on this hardware?* — is resolved. A 7B base fine-tunes via LoRA (fp16) **sharded across three Pascal-class GPUs**: loss fell cleanly (3.64→2.48) over a smoke run, adapter saved, peak ~5–6GB/GPU. Two lessons: (a) the widespread belief "Pascal can't train modern PyTorch" holds only for 4-bit QLoRA (bitsandbytes); **plain fp16 LoRA runs fine**; (b) bleeding-edge trainer libraries broke in several ways (chunked-loss patches, collator format, a chat-template returning strings not ids), so the robust recipe is a **manual torch training loop** (forward→backward→step) rather than a high-level Trainer. Throughput is modest (~0.25 steps/s), making real fine-tunes minutes-to-hours, viable as batch jobs.
+
+### 4.3 Dataset construction (Phase 3)
+A 90-example domain dataset was built in two deliberately-separated slices:
+- **Persona/format (71 ex)** — generated *deterministically* by running the agent's own valuation function across the equity index; numbers are real, format is gold, **grounded by construction** (no LLM, no judge). Notably ~75% of these examples teach *honesty* — the model admitting when a DCF is inconclusive or inapplicable.
+- **Method/reasoning (19 ex)** — chunks from valuation textbooks → an LLM generates a Q&A → a judge filters. Here the judge-gate proved **essential and measurable**: naive generation was ~40% usable and produced *dangerous contradictions* (a 7B asserting DCF suits banks — the opposite of correct). Two guards fixed it: a retrieval-distance filter (killing off-topic chunks) and an LLM judge. A measured trade-off emerged — a strong slow cross-model judge (22B) yielded 100% clean but was impractically slow; a fast same-model judge was ~6× faster but more lenient (human inspection still caught ~24% meta/disclaimer junk it passed). **Human verification remained necessary above the automated gate.**
+
+### 4.4 First specialist + eval-gate (Phase 4)
+A 7B base was fine-tuned via LoRA (r=16, ~10M trainable params) on the 90-example dataset, 3 epochs, sharded across three Pascal GPUs (36 min, 270 steps). Training loss fell smoothly **1.885 → 0.553 → 0.408** — clean convergence, no divergence. The resulting adapter was evaluated against the raw base on the same probes, across three dimensions:
+
+| Dimension | Base (raw 7B) | Specialist (fine-tuned) |
+|---|---|---|
+| **Persona / format** | generic markdown; *fabricated numbers* (invented a share price and EPS) | reproduced the agent's exact house format (structured valuation card, deterministic tone) |
+| **Method reasoning** | correct (WACC, why-DCF-fails-for-banks) | correct — parity |
+| **Groundedness** | refused a fabricated ticker | refused — *preserved* |
+
+**Verdict: success where it was trained.** The fine-tune's clearest win is **format/persona** — the specialist internalized the house output style the base had no notion of, and in doing so *stopped fabricating figures* the base had invented. Crucially, **fine-tuning did not damage groundedness** (the primary risk — that training on generated data erodes the model's "I don't know"). Method reasoning was parity, expected given the method slice was small (19 vs 71 examples) and the base was already competent there.
+
+One nuance validates the architecture rather than undermining it: the specialist occasionally emits imprecise figures. This is *by design* — the persona slice teaches the *format*; the real numbers are injected at inference from a deterministic function (RAG/code), never baked into weights. Volatile facts stay out of the parameters; the model supplies structure and reasoning, the code supplies truth.
+
+## 5. Related Work
+
+QLoRA (fine-tuning a 4-bit-quantized base via low-rank adapters); self-improvement / self-taught reasoning (STaR) and RLAIF (learning from self-generated, verifier-filtered data); the model-collapse literature (why an external anchor is required); catastrophic forgetting and continual-learning mitigations (replay, adapter libraries); and the RAG-vs-fine-tuning trade-off (external knowledge vs internalized capability). Our contribution is not a new algorithm but a **composition**: quantize-then-heal + dense specialization + a judge-gated continual loop, under strict on-premises constraints.
+
+## 6. Limitations & Open Questions
+
+- **The win is format/persona, not (yet) reasoning.** The specialist clearly beats the base on house-format adherence and stops the base's number-fabrication, at parity on method reasoning and groundedness. Moving method reasoning from *parity* to *win* requires a larger method dataset — the current 19 method examples are a proof, not a corpus.
+- **A single eval run, few probes.** The verdict rests on 4 hand-inspected probes, not a held-out benchmark; a proper eval set (and comparison against the large incumbent, not just the base) is the next rigor step.
+- **Throughput, not feasibility, is the constraint.** Training and judging on 2016-era GPUs is slow (~0.25 train steps/s; a 22B judge at ~6 tok/s), so scaling the dataset to hundreds of examples is a batch/overnight job, not interactive.
+- **Dataset size.** 90 examples proves the pipeline; a production specialist wants substantially more method examples, bounded by textbook-chunk coverage.
+- **Hardware ceiling.** The 8GB target caps a healable (≥Q4) model at ~9B. Larger models require larger cards or a multi-node pool.
+- **Whether the specialist matches the incumbent** is the open empirical question the roadmap exists to answer.
+- **Quantization does not fix fabrication** — it addresses speed/size, not a model's intrinsic tendency to hallucinate; groundedness must be verified per model.
+
+## 7. Roadmap (phases)
+
+0. Lock plan ✅ · 1. Inference feasibility ✅ · 1b. base bake-off ✅ · 2. local training stack ✅ · 3. dataset (generate → judge-gate → split) ✅ · 4. first fine-tune + eval-gate ✅ · **5. close the loop** (collect → gate → train → eval → cadence) — next · 6. production controls (adapter library, canary, drift kill-switch) · 7. generalize (other domains; on-device/mobile). *Phases 0–4 done; the autonomous loop (5) and production controls (6) remain.*
+
+## 8. Availability
+
+Intended for release under a pseudonymous repository once Phase 2+ are validated: **method, evaluation harness, and the trained adapter** would be public; the domain dataset and the specific loop orchestration remain private (the practical moat). Sovereignty here means owning the *intelligence* (local model + data + loop), not reinventing the commodity substrate (quantization formats, adapter training) — which is used as-is, and improved only where it fails this hardware specifically.