README.md

---
license: gemma
library_name: peft
base_model: google/gemma-3-4b-it
base_model_relation: adapter
pipeline_tag: text-generation
language:
  - en
datasets:
  - manifesta/verified-agronomy-17k
tags:
  - lora
  - peft
  - adapter
  - agriculture
  - agronomy
  - reasoning
  - refusal
  - verified
  - gemma3
  - autoscientist
  - adaption
  - negative-result
inference: false
model-index:
  - name: adaption_agronomy_calc_problems
    results:
      - task:
          type: text-generation
          name: Agronomy calculation question answering
        dataset:
          name: Verified Agronomy 17k (held-out)
          type: manifesta/verified-agronomy-17k
          split: test
        metrics:
          - type: win-rate
            name: AutoScientist score vs base (adapted)
            value: 46.0
          - type: loss
            name: Held-out eval loss
            value: 1.0919
        source:
          name: Adaption AutoScientist held-out evaluation
          url: https://huggingface.co/manifesta/adaption_agronomy_calc_problems
---

[![Verify](https://github.com/A1VARA5/verified-agronomy-17k/actions/workflows/verify.yml/badge.svg)](https://github.com/A1VARA5/verified-agronomy-17k/actions/workflows/verify.yml)
[![Live interface](https://img.shields.io/badge/Live-MANIFESTA_Agriculture-000?logo=vercel&logoColor=white)](https://manifestaagriculture.adaptionlabs.app/)
[![Base model](https://img.shields.io/badge/Base-Gemma_3_4B_IT-1B72E8?logo=google&logoColor=white)](https://huggingface.co/google/gemma-3-4b-it)
[![Dataset](https://img.shields.io/badge/%F0%9F%A4%97_Dataset-verified--agronomy--17k-FFD21E)](https://huggingface.co/datasets/manifesta/verified-agronomy-17k)
[![Score](https://img.shields.io/badge/AutoScientist-46_vs_54_regression-c0392b)](#evaluation)
[![Data quality](https://img.shields.io/badge/Data_quality-%2B68.0%25-2ea44f)](#dataset)
[![Built with AutoScientist](https://img.shields.io/badge/Built_with-Adaption_AutoScientist-7C3AED)](https://adaptionlabs.ai)

# Agronomy Calculations (Gemma 3 4B LoRA)

A PEFT LoRA adapter trained on 17,199 source-cited agronomy calculations, where 10.4% of the
questions cannot be answered from what they give you and the correct response names the
missing input instead of estimating it.

**It lost to its base model. The AutoScientist head-to-head is 46 for the adapted model
against 54 for the base.** That is a regression, not a tie. This card gives the number
first and then the mechanism, read out of `trainer_state.json` in this repository rather
than guessed at.

I also edited the training recipe by hand instead of accepting the default: three epochs
instead of one, and a learning rate of 1e-4 instead of 1e-5. That was a hypothesis about
why two earlier runs had gone flat, and it did not work. A card that records a hypothesis
that failed is worth more than one that quietly reports the data-quality number instead.

## Contents

- [TL;DR](#tldr)
- [Quickstart](#quickstart)
- [What is actually in these weights](#what-is-actually-in-these-weights)
- [Evaluation](#evaluation)
- [Why it went backwards](#why-it-went-backwards)
- [Training](#training)
- [Intended use and limits](#intended-use-and-limits)
- [Dataset](#dataset)
- [Live interface](#live-interface)
- [Verify all of this yourself](#verify-all-of-this-yourself)
- [Related models](#related-models)
- [License](#license)
- [Citation](#citation)

## TL;DR

- **What it is:** a rank-16 LoRA adapter on the text tower of `google/gemma-3-4b-it`.
  29,802,496 trainable parameters, 119.3 MB, seven projections in all 34 decoder layers.
- **What it was for:** agronomy arithmetic with cited domain constants, and knowing when a
  question cannot be answered at all.
- **The number:** **46 for the adapted model, 54 for the base.** It got worse.
- **The recipe was hand-edited:** `n_epochs` 1 to 3, `learning_rate` 1e-5 to 1e-4, on a
  sub-10B base with domain and diversity expansion on. The change did not produce a better
  result.
- **The mechanism:** 66 optimizer steps over 3 epochs. Gradient norm ranged 0.0795 to
  2.6548 against a `max_grad_norm` of 2, and **5 of the 66 steps exceeded it**, all five in
  the first five steps while the learning rate was still warming up. Eval loss then fell
  monotonically across all five evaluations, 1.2917 down to 1.0919, and the model still lost
  the head-to-head.
- **Base model pointer, corrected:** `adapter_config.json` was exported declaring
  `togethercomputer/gemma-3-4b-it`, which returns HTTP 401. I changed it to
  `google/gemma-3-4b-it` on 2026-08-13 and kept the original string verbatim in the same
  file. See [Quickstart](#quickstart).
- **Built by:** MANIFESTA (Aivaras Navardauskas) for the Adaption AutoScientist Challenge,
  Agriculture.

## Quickstart

### The base model string was wrong. I fixed it, and the original is on the record

The export pipeline wrote this into `adapter_config.json`, and it shipped that way until
2026-08-13:

```json
"base_model_name_or_path": "togethercomputer/gemma-3-4b-it"
```

That repository does not exist. It returns HTTP 401 to an anonymous client, which is what
the Hub returns for a repo that is not there, and `togethercomputer` publishes no gemma-3
repository under any name. Nobody typed that string by hand. The pipeline invented it, and
I did not catch it before publishing.

The consequence was real: anything resolving the base from the config, including
`AutoPeftModelForCausalLM.from_pretrained` on the adapter alone, could not find a base to
load.

**Corrected 2026-08-13.** The file now reads:

```json
"base_model_name_or_path": "google/gemma-3-4b-it",
"original_base_model_name_or_path": "togethercomputer/gemma-3-4b-it"
```

No weight changed and nothing else in the config changed. The original exported value stays
in the same file so the artifact remains auditable: you can see what shipped first, what it
was changed to, and decide for yourself whether the change was right.

Check both in one line each:

```bash
curl -s -o /dev/null -w "%{http_code}\n" https://huggingface.co/api/models/google/gemma-3-4b-it          # 200
curl -s -o /dev/null -w "%{http_code}\n" https://huggingface.co/api/models/togethercomputer/gemma-3-4b-it # 401
```

### Load it against the text tower, not the multimodal wrapper

This is the part worth reading before you write any code. `google/gemma-3-4b-it` is a
vision language model, `Gemma3ForConditionalGeneration`, whose parameters are named
`model.language_model.layers.*`. **This adapter was trained on the text tower alone.** Every
one of its 476 tensor keys begins `base_model.model.model.layers.`, which is the naming of
a plain `Gemma3ForCausalLM`. The `config.json` shipped in this repo says the same thing:
`model_type: gemma3_text`, `architectures: ["Gemma3ForCausalLM"]`, 34 decoder layers,
hidden size 2560, 8 attention heads, 4 key-value heads, head dim 256.

So load the VLM, take its language tower, and attach the adapter to that:

```python
import torch
from transformers import AutoTokenizer, AutoConfig, AutoModelForImageTextToText, Gemma3ForCausalLM
from peft import PeftModel

BASE = "google/gemma-3-4b-it"                            # gated, accept the licence on the Hub first
ADAPTER = "manifesta/adaption_agronomy_calc_problems"

tok = AutoTokenizer.from_pretrained(ADAPTER)             # tokenizer ships in this repo
cfg = AutoConfig.from_pretrained(BASE)

vlm = AutoModelForImageTextToText.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto"
)

# the adapter targets the text tower, so rehost it as a causal LM
base = Gemma3ForCausalLM(cfg.text_config)
base.model = vlm.model.language_model
base.lm_head = vlm.lm_head

model = PeftModel.from_pretrained(base, ADAPTER).eval()

messages = [{"role": "user", "content":
    "A 24 ha maize field needs 150 kg N/ha. How much urea (46-0-0) do I need in total? "
    "If the question is missing something you need, say so and name it."}]

inputs = tok.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=384, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

Merging is optional and only touches the text tower:

```python
model = model.merge_and_unload()
```

### Confirm the adapter binds without downloading 8.6 GB of base weights

The failure this snippet exists to rule out is silent: attach a LoRA to the wrong module
tree and PEFT injects adapters that no saved tensor ever lands in, so you get a model that
loads cleanly and behaves exactly like the base. Build the architecture on the meta device
and count the keys instead. No weights are downloaded, and it finishes in seconds.

```python
import torch
from transformers import AutoConfig, AutoModelForImageTextToText, Gemma3ForCausalLM
from peft import PeftModel

cfg = AutoConfig.from_pretrained("google/gemma-3-4b-it")
with torch.device("meta"):
    vlm = AutoModelForImageTextToText.from_config(cfg)
    base = Gemma3ForCausalLM(cfg.text_config)
base.model, base.lm_head = vlm.model.language_model, vlm.lm_head

m = PeftModel.from_pretrained(base, "manifesta/adaption_agronomy_calc_problems",
                              low_cpu_mem_usage=True)
print(sum(1 for k, _ in m.named_parameters() if "lora_" in k))   # 476
```

476, with nothing missing and nothing extra, is the whole point: every tensor in
`adapter_model.safetensors` binds to a real module.

## What is actually in these weights

Read straight out of `adapter_model.safetensors` and `adapter_config.json`. Every row of
this table is reproducible from the repo.

| Property | Value |
| --- | --- |
| PEFT type | LoRA, `task_type: CAUSAL_LM` |
| Rank `r` | 16 |
| `lora_alpha` | 32 (so the scaling factor alpha/r is exactly 2.0) |
| `lora_dropout` | 0.0 |
| `bias` | none |
| `use_rslora` / `use_dora` | false / false |
| Target modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj` |
| Tensors in the file | 476 |
| Decoder layers touched | **34 of 34**, all seven projections in every layer, no gaps |
| Trainable parameters | 29,802,496 (float32, 119,273,568 bytes) |
| Share of the 3.88B text tower | roughly 0.77% |
| `exclude_modules` | empty, nothing was frozen out by name |

Worth naming the contrast with the two sibling runs in this series: this one put LoRA on
attention **and** the MLP, in every layer, at rank 16. The chart run used rank 8 on
`q_proj` and `v_proj` only. This adapter had more trainable surface than either sibling and
still went backwards, which is the reason the diagnosis below is about the optimisation and
the objective rather than about frozen modules.

## Evaluation

Judged by Adaption's AutoScientist evaluator on a held-out split the adapter never saw,
against the same base model it was trained from. Head-to-head, blind.

| Metric | Base | Adapted | Change |
| --- | --- | --- | --- |
| **AutoScientist score** | **54** | **46** | **-8** |
| Held-out eval loss (first eval, step 14) | | 1.2917 | |
| Held-out eval loss (final, step 66) | | 1.0919 | -0.1998 |
| Training loss | 1.6436 (step 1) | 0.9727 (step 66) | -0.6709 |

Eval loss across the five evaluations: **1.2917, 1.1639, 1.1239, 1.0999, 1.0919.** Monotonic,
textbook, and it bought a worse model.

**46 against 54 is a regression.** The adapter did not make Gemma 3 4B better at agronomy
calculations. It made it worse in the judge's view, on a split it had never seen. There is
no reading of that number that is a win, and separating it from the data-quality number
below is the honest way to present the entry.

## Why it went backwards

Three things, in order of how much I think they matter. All of them are checkable from the
files in this repository.

### 1. The eval loss curve was measuring the wrong thing

Eval loss fell at every one of the five evaluations and total training loss fell from 1.644
to 0.973. That is a model getting steadily better at reproducing the corpus, and this corpus
has a deliberately uniform shape: name the formula, substitute the numbers, carry the units,
close with `The final answer is $\boxed{ANSWER}$.` Three epochs at 1e-4 is enough to fit
that shape hard.

A judge comparing free-form answers does not reward the shape. It rewards the answer. So
the loss curve and the head-to-head are measuring different things, and only one of them
was ever the headline. If you take one thing from this card, take that: **a clean monotonic
eval curve is not evidence the model got better at the task.**

### 2. The recipe change was a hypothesis, and it was wrong

I raised `n_epochs` from 1 to 3 and `learning_rate` from 1e-5 to 1e-4 by hand, because the
two earlier runs in this series had looked starved of optimizer steps. Three epochs on a
sub-10B base did buy more steps: 66, against 34 for the chart run and 59 for the math run.

It did not buy a better model. Three passes over a templated corpus at ten times the default
learning rate is a good way to overfit a format, which is consistent with what section 1
describes. A single epoch at 1e-5 might have moved the model less and cost it less.

### 3. Five clipped steps, all of them at the start

| | |
| --- | ---: |
| Optimizer steps | 66 |
| `max_grad_norm` | 2 |
| Gradient norm range | 0.0795 to 2.6548 |
| **Steps that exceeded the threshold** | **5 of 66 (7.6%)** |
| Which steps | 1, 2, 3, 4, 5 |
| Their norms | 2.6548, 2.1871, 2.0625, 2.1209, 2.0619 |
| Learning rate at step 1 / step 8 | 0.0 / 1.0e-4 |

The learning rate warmed from 0 to its 1e-4 peak over eight steps. The gradient exceeded the
clipping threshold before that warmup finished and never did again after step 5: from step 6
onward the norm stayed under 2 the whole way down to 0.0795 by step 60.

I am listing this third because it is the smallest of the three effects. Five clipped steps
is a mild start, not a broken run. It is worth stating precisely rather than vaguely: this
is what "it clipped" actually amounted to, and it does not carry the explanation on its own.
Compare with the [math and code adapter](https://huggingface.co/manifesta/adaption_verified_math_code_instruct),
where 29 of 59 steps were clipped and the peak norm was 677, which is a genuinely unstable
run. This one was not unstable. It was stable and it fit the wrong objective.

**What would need to change for a real result:** one epoch at 1e-5, a held-out set scored
the way the judge scores rather than by loss, and a training target that varies its surface
form instead of the single boxed output contract this corpus enforces. The output contract
is right for machine grading and it may be exactly what a preference judge penalises.

## Training

Trained with [Adaption](https://adaptionlabs.ai) AutoScientist, SFT with LoRA, on Adaptive
Data output.

| Hyperparameter | Value |
| --- | --- |
| Base model | `google/gemma-3-4b-it`, declared correctly in `adapter_config.json` since 2026-08-13. The original export value `togethercomputer/gemma-3-4b-it` is preserved in the same file as `original_base_model_name_or_path` |
| Method | SFT, LoRA (PEFT), chat format |
| LoRA rank / alpha / dropout | 16 / 32 / 0.0 |
| Trainable modules | `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`, all 34 layers |
| **Learning rate** | **1e-4, raised by hand from the 1e-5 default** |
| **Epochs** | **3, raised by hand from the default 1** |
| Peak learning rate reached | step 8 |
| Max gradient norm | 2 |
| Optimizer steps | 66 |
| Per-device train batch size | 1, with accumulation, effective batch about 1,276 rows per step |
| Evaluations | 5, every 13 steps |
| Precision | bfloat16 base, float32 adapter weights |
| Total FLOPs | 1.3396e18 |
| Adaption dataset ID | `5b8890d0-96eb-433a-9f70-ae1d3dc1743f` |
| Training model ID | `adaption_gemma_3_4b_it_agronomy_calc_problems_587e58e3` |

Sixty-six optimizer steps over three epochs is 22 steps per epoch, so with roughly 28,069
rows in the expanded training set each step saw about 1,276 rows. Even at three epochs the
update budget stays small, because batch size scales with the corpus instead of the step
count doing so.

`trainer_state.json` ships in this repository, and the copy in the
[GitHub repo](https://github.com/A1VARA5/verified-agronomy-17k/blob/main/evidence/trainer_state.json)
is byte identical to it. Every number in this section comes out of that one file.

## Intended use and limits

**Use it for**

- Reproducing and inspecting this regression. That is the honest primary use.
- Studying what three epochs at 1e-4 does to a small instruct model on a corpus with one
  rigid output contract.
- A control arm against a one-epoch, 1e-5 run on the same data.
- Research on refusal behaviour: whether a model that has seen 1,796 unanswerable questions
  names the missing input or invents a number.

**Do not use it for**

- Anything that assumes it does agronomy arithmetic better than stock Gemma 3 4B. It does
  not, and the evaluation says so.
- Real fertiliser, irrigation, seeding or pesticide decisions. A wrong rate is a real cost
  and a real environmental consequence, and no model output here has been reviewed by an
  agronomist. Use a current soil test, the registered product label and local regulation.
- Anything safety, regulatory or financial where a confidently wrong number does damage.
- Any use that violates the [Gemma Terms of Use](https://ai.google.dev/gemma/terms) or the
  Gemma Prohibited Use Policy, which apply through the base model.

**Technical limits**

- The adapter targets the text tower only. If you attach it to the multimodal wrapper
  without rehosting the language model, it will bind nothing and behave exactly like the
  base. See [Quickstart](#quickstart).
- Trained on 100% generated data. Verified arithmetic, real citations, no field records and
  no real farmer language.
- Constants are largely North American and FAO conventions: 56 lb/bu corn, 15.5% market
  moisture, Fahrenheit degree-day bases alongside metric rates.
- English only.
- The declared base string was wrong until 2026-08-13. It resolves now, and the original is
  recorded in the config.
- Adapter weights are float32 while the base is bfloat16. PEFT casts on load, so expect the
  adapter's memory footprint to be double a bf16 export.
- One judge, one held-out split. Reported as-is, no reruns and no best-of selection.

## Dataset

Trained on Adaption dataset `5b8890d0-96eb-433a-9f70-ae1d3dc1743f`, built from
[`manifesta/verified-agronomy-17k`](https://huggingface.co/datasets/manifesta/verified-agronomy-17k)
(also on [Kaggle](https://www.kaggle.com/datasets/aivarasnavardauskas/verified-agronomy-17k)).

- 17,199 rows generated, **17,028 ingested** by Adaption, about **28,069** in the training
  set after the platform's domain and diversity expansion
- Ten calculation families: fertiliser rates and NPK blends, growing degree days, grain
  moisture and test weight, seeding rate, irrigation ETc, sprayer calibration, forage
  budgeting, farm economics, unit conversion, plus a refusal slice
- **1,796 rows (10.4%) are deliberately unanswerable.** The correct response names the
  missing input rather than estimating it, and those rows carry real but irrelevant detail
  (field size, previous crop, growth stage) on purpose, so plausible context cannot
  substitute for the required input
- 15,403 rows (89.6%) are exactly gradable against a known numeric answer
- Every formula is cited to a published source: FAO Irrigation and Drainage Paper 56, NDSU
  NDAWN, Cornell CSS412, WSU extension, Purdue, Ohio State, Pioneer
- Verified twice: 26 calculators checked against a worked example published by the source
  each one cites, and 25,324 arithmetic expressions inside the finished solutions
  re-evaluated with 0 inconsistent
- CC0-1.0. Every row was generated for this dataset, no third-party content, no scraping

Adaptive Data lifted the corpus quality before training:

| Adaptive Data metric | Before | After |
| --- | --- | --- |
| Quality score | 5.0 | **8.4 (+68.0% relative)** |
| Grade | C | B |
| Percentile | 7.2 | 31.5 |

**+68.0% is the improvement in the data, not in the model.** The data got substantially
better and the model got worse. Those are two different measurements and only one of them
is this card's headline. It is the largest data lift of the three entries in this series and
it is still not the number that matters here.

## Live interface

**MANIFESTA Agriculture** runs at
[manifestaagriculture.adaptionlabs.app](https://manifestaagriculture.adaptionlabs.app/).
Ask it for a fertiliser rate without giving it a soil test and see whether it names the
missing input or invents one.

## Verify all of this yourself

Every claim on this card is checked by a script that anyone can run, in
[github.com/A1VARA5/verified-agronomy-17k](https://github.com/A1VARA5/verified-agronomy-17k):

```bash
git clone https://github.com/A1VARA5/verified-agronomy-17k
cd verified-agronomy-17k
python verify.py
```

22 checks, standard library only, no install step. The ones that cover this model:

| | Check |
|---|---|
| 16 | the weights download, HTTP 200, and are exactly 119,273,568 bytes |
| 17 | `google/gemma-3-4b-it` resolves HTTP 200, and `togethercomputer/gemma-3-4b-it` still returns 401 |
| 18 | `adapter_config.json` declares the corrected base and preserves the original verbatim, r=16, alpha=32, 7 target modules |
| 19 | the published `trainer_state.json` is byte identical to the copy in the repo, 66 steps, peak gradient norm 2.6548, 5 clipped and all in the first five steps |
| 21 | the Kaggle model mirror answers HTTP 200 to a logged out fetch |
| 22 | the live interface answers HTTP 200 |

The same script runs in GitHub Actions on every push and once a day on a schedule, which is
what the badge at the top of this card reports.

## Related models

- [`manifesta/adaption_scientific_chart_qa_17k`](https://huggingface.co/manifesta/adaption_scientific_chart_qa_17k),
  chart QA on Gemma 3 27B. 50 against 50, a null result. Its vision tower was frozen, so the
  part of the model that reads a figure was never adapted.
- [`manifesta/adaption_verified_math_code_instruct`](https://huggingface.co/manifesta/adaption_verified_math_code_instruct),
  verified maths and code on Gemma 4 31B. 46 against 54, with 29 of 59 steps clipped and a
  peak gradient norm of 677.
- [`manifesta/scientific-chart-qa-lora-qwen3.5-9b`](https://huggingface.co/manifesta/scientific-chart-qa-lora-qwen3.5-9b),
  the same chart task on a 9B base and a smaller corpus. 51 against 49.

Four runs across the two challenge parts, and the pattern in them:

| Run | Base | Rows ingested | Epochs | LR | Steps | Clipped | Score vs base |
| --- | --- | ---: | ---: | ---: | ---: | ---: | --- |
| Chart QA, first build | Qwen3.5 9B | 6,976 | 1 | 1e-4 | 21 | not recorded | 51 vs 49 |
| Chart QA 17k | Gemma 3 27B | 17,070 | 1 | 1e-4 | 34 | 0 of 34 | 50 vs 50 |
| Verified math and code | Gemma 4 31B | 17,586 | 1 | 1e-4 | 59 | 29 of 59 | 46 vs 54 |
| **Agronomy (this model)** | **Gemma 3 4B** | **17,028** | **3** | **1e-4** | **66** | **5 of 66** | **46 vs 54** |

More steps did not help. This run has the most optimizer steps of the four and the joint
worst score. **Caveat, stated plainly:** four data points across four base models, four
corpora and four domains is not a controlled experiment and should not be read as one.

## License

- **Adapter weights:** released under the [Gemma Terms of Use](https://ai.google.dev/gemma/terms).
  Derivatives of Gemma inherit Gemma's terms, so the adapter travels with the same
  conditions as the base.
- **Base model:** `google/gemma-3-4b-it` is gated. Accept the licence and request access on
  the Hub before loading. The Gemma Prohibited Use Policy applies.
- **Training dataset:** [`manifesta/verified-agronomy-17k`](https://huggingface.co/datasets/manifesta/verified-agronomy-17k),
  released CC0-1.0. Every row was generated for that dataset, so there is nothing to
  attribute and no restriction to pass on.

## Citation

```bibtex
@misc{agronomy_calc_lora_2026,
  title  = {An agronomy calculation adapter for Gemma 3 4B, and why three epochs at 1e-4 made it worse},
  author = {Navardauskas, Aivaras},
  year   = {2026},
  note   = {MANIFESTA. Adapted with Adaptive Data by Adaption, AutoScientist Challenge, Agriculture.
            AutoScientist score 46 against a base of 54. Recipe hand-edited to 3 epochs at 1e-4.},
  howpublished = {\url{https://huggingface.co/manifesta/adaption_agronomy_calc_problems}}
}
```

Built with [Adaptive Data by Adaption](https://adaptionlabs.ai). Platform documentation at
[docs.adaptionlabs.ai](https://docs.adaptionlabs.ai). Dataset, adapter and analysis by
MANIFESTA (Aivaras Navardauskas) for the AutoScientist Challenge.