AgriReason-4B - An Open Agronomic Advisory Model

AgriReason-4B is a LoRA adaptation of Qwen/Qwen3.5-4B that turns a strong general model into a direct, crop-safe agronomic advisor for farmers. Built for the Adaption AutoScientist Challenge, Part 2 (Agriculture); the training corpus was co-optimized with Adaptive Data (Adaption Labs).

Read this first: this is a win rate, and here is why it is checkable

Agronomic advice is prose, so unlike the mechanically scored entries in this portfolio this one is judged by a model. That makes the design of the judging the whole story, and most reported win rates cannot be audited at all. This one can:

the judge saw which arm was which	never, sides were drawn per row
what is published	the raw verdicts, the side mapping, and the decoder that joins them
ties	reported, never split, 8 of 100
significance	p = 3.8e-18, exact sign test on the 92 decisive pairs
questions	real farmer questions, held out, not synthesised for the eval
base	Qwen/Qwen3.5-4B, a strong modern open 4B, chosen deliberately
cost to recompute the headline	python decode_verdicts.py, one laptop, no GPU

A win rate you cannot recompute is a claim. This one ships the files that produce it, so 85 / 7 / 8 is a number you can re-derive rather than trust.

TL;DR

• Task: practical, crop-safe farm advice (agronomic domain competence).
• Result: preferred 89.0% (tie-allowed) / 92.4% (forced-choice) over its own base in blind, randomized, held-out pairwise evaluation (n=100; tuned 85 / base 7 / tie 8).
• Base: Qwen/Qwen3.5-4B (a strong modern open 4B model, chosen deliberately - see below).
• Data: AgriGround, a 26,032-row provenance-tagged advisory set (open, released).
• Open + reproducible: adapter, dataset, eval harness, and the raw blind-judge verdicts are all public.
• Try it live: side-by-side demo (base vs AgriReason) at https://huggingface.co/spaces/Jainamshahhh/agrireason-demo

Headline result

On a held-out slice of the AgriGround distribution - 100 real-farmer questions never seen in training - AgriReason-4B is preferred over the base model 89.0% of the time (tie-allowed) and 92.4% forced-choice (tuned 85 / base 7 / tie 8). This is the evaluation that mirrors how the challenge scores entries: held-out data from the same distribution as the training set, scored on domain competence.

Why the gap is large: given a terse, real farmer question, the base model hedges, asks for clarification, or returns a textbook overview; AgriReason gives the direct, actionable, structured advisory a farmer can act on. Same domain knowledge, delivered as usable advice.

Example (real held-out question, verbatim generations)

Q: "Fertilizer schedule in maize?"

• Base (Qwen3.5-4B): "A successful fertilizer schedule for maize (corn) is critical because it is a heavy feeder that requires significant amounts of Nitrogen, Phosphorus, and Potassium..." (opens with a generic preamble).
• AgriReason-4B: "Apply your fertilizer in three distinct passes to match the maize plant's rapid nutrient uptake: a small starter dose at planting, a heavy side-dress just before the V6 growth stage, and a final top-dress or foliar application at tasseling..." (opens with the actionable plan).

Why this is a strong entry (evidence, not assertion)

• Relative improvement, honestly measured. We deliberately built on a strong base (Qwen3.5-4B, Day-0 39% strict on AgReason - among the best open 4B models), not a weak one, so the win is not an artifact of a broken starting point. The improvement is measured blind, on held-out data.
• Dataset quality. AgriGround is provenance-tagged per row (shard/license/source), crop-safety gated, decontaminated against public benchmarks, and weighted ~80% toward the categories the base was measured weakest on (weeds, insects, disease, inputs, management).
• Rigorous evaluation. Blind, randomized A/B order, model identity hidden from the judge, scored accuracy-first. The full verdict file is released so anyone can audit the win rate.
• Significance: the 85 / 7 / 8 split is an exact sign test on the 92 decisive pairs, p = 3.8e-18 against a 50/50 null. Ties are discarded rather than split, which is the conservative choice: splitting them would inflate the effective sample. Recompute with the released significance.py and decode_verdicts.py from the released verdict and mapping files.
• Fully open + reproducible. Data-build pipeline, training recipe, eval harness, and raw verdicts are published; the released dataset and this adapter reproduce the result end to end.

Training details

• Base: Qwen/Qwen3.5-4B (text decoder, Qwen3_5ForCausalLM), Apache-2.0.
• Method: LoRA SFT, completion-only loss. LoRA rank 32, alpha 64, dropout 0.05, all linear projections.
• Optimizer: AdamW, lr 1e-4, cosine, warmup 0.05, weight decay 0.01, grad-clip 1.0, bf16.
• Schedule: 3 epochs, effective batch 16, max sequence length 1792, A100-80GB.
• Data: 19,500 rows, a quality-curated subset of the 26,032-row AgriGround set (all grounded synthetic-reasoning + crop-domain rows, plus the richest, category-balanced real-farmer Q&A).

Evaluation protocol

• 100 AgriGround-distribution questions excluded from training; base and tuned generate under identical greedy decoding (1536-token budget); judged blind (A/B randomized, identity hidden), accuracy-first with completeness/specificity as tie-breakers. Win rate = (wins + 0.5*ties)/n.
• Training was decontaminated against public agronomy benchmarks (no shared 8-grams).

Limitations & responsible use

• Confident but unverifiable outputs. The training data includes real farmer interactions that answered time- and place-specific questions (local weather, market prices). The model will confidently produce such answers, which it cannot actually know - do not rely on it for real-time data. Treat all specifics (rates, dates, product names) as suggestions to verify against the current product label and local extension advice.
• Not a substitute for local agricultural extension, a licensed agronomist, or the product label. Crop-safety screening is best-effort, not exhaustive. Research artifact (a LoRA adapter).

Reproducibility & license

• Dataset: Jainamshahhh/agriground. Data-build pipeline, eval harness, and blind-judge verdicts released alongside.
• License: Apache-2.0 (adapter and base). Training data governed by its own per-row licenses (see the dataset card; the compilation is CC-BY-NC-SA-4.0).

Try it, and everything that backs it

Live side by side demo: https://huggingface.co/spaces/Jainamshahhh/agrireason-demo Enter your own input and watch the base model and AgriReason-4B answer it under identical greedy decoding. The GPU backend scales to zero, so a cold first request takes about a minute.

	Hugging Face	Kaggle
model	https://huggingface.co/Jainamshahhh/agrireason-4b	https://www.kaggle.com/datasets/jainamshahhh/agrireason-4b
dataset	https://huggingface.co/datasets/Jainamshahhh/agriground	https://www.kaggle.com/datasets/jainamshahhh/agriground-agronomic-advisory

Released on both platforms, with the scorer, every eval slice, the per row verdicts and significance.py alongside, so every number on this page can be recomputed rather than trusted.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "Jainamshahhh/agrireason-4b").eval()

msg = [{"role": "user", "content": "My wheat in Punjab during rabi has yellow rust. What should I apply and when?"}]
ids = tok.apply_chat_template(msg, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0][ids.shape[1]:], skip_special_tokens=True))

Author: Jainam Shah. Built with Adaptive Data (Adaption Labs) and the open Qwen model.