reference-material/roadmap.md

# FarmRL Round-1 Fast Development Roadmap

## Reference Materials

### Introduction

FarmRL is a reinforcement learning project that trains an agent to manage crop farming decisions. Given observable farm conditions such as soil properties, weather, and crop type, the agent learns to control irrigation, fertilizer application, and pesticide use in order to maximise crop yield while maintaining a healthy sustainability score.

The project is grounded in a tabular agricultural dataset and draws conceptual inspiration from the FarmGym simulation framework. Two training paradigms are supported: a classic RL agent via a custom OpenEnv environment, and an optional text-framing path using TRL for language-model-based decision making.

The raw CSV dataset is preprocessed once. The preprocessing adds the Water\_mm column (drawn uniformly from [20, min(Rainfall\_mm, 200)]) and subtracts that value from Rainfall\_mm to preserve water-balance invariance. A lightweight regression model (XGBoost) is then trained on the processed data to serve as the environment's transition model.

---

## Dataset preprocessing requirement

Add a preprocessing script that creates a new variable Water\_mm such that:

Rainfall\_original = Rainfall\_new + Water\_mm

This prevents bias by conserving total water availability.

Script file:

scripts/add\_water\_variable.py

```
"""
add_water_variable.py

Adds a Water_mm column to the farm dataset.
Water is drawn uniformly from [WATER_MIN, Rainfall_mm].
Rainfall_mm is reduced by the water drawn to prevent bias.
"""

import pandas as pd
import numpy as np
import sys

WATER_MIN = 20   # minimum meaningful irrigation (mm)
WATER_MAX = 200  # hard ceiling - avoids flooding; also capped at rainfall

def add_water(df: pd.DataFrame, seed: int = 42) -> pd.DataFrame:
    rng = np.random.default_rng(seed)
    df = df.copy()

    # Upper bound: rainfall itself, capped at WATER_MAX
    upper = df["Rainfall_mm"].clip(upper=WATER_MAX)

    # Where rainfall < WATER_MIN we can't irrigate meaningfully — set 0
    can_irrigate = upper >= WATER_MIN
    water = np.where(
        can_irrigate,
        rng.uniform(WATER_MIN, upper.where(can_irrigate, WATER_MIN)),
        0.0
    )

    df["Water_mm"] = np.round(water, 2)
    df["Rainfall_mm"] = np.round(df["Rainfall_mm"] - df["Water_mm"], 2)
    return df


def main():
    path = sys.argv[1] if len(sys.argv) > 1 else "farm_data.csv"
    out  = sys.argv[2] if len(sys.argv) > 2 else path.replace(".csv", "_watered.csv")

    df = pd.read_csv(path)
    required = {"Rainfall_mm"}
    missing = required - set(df.columns)
    if missing:
        raise ValueError(f"Missing columns: {missing}")

    df_out = add_water(df)

    print(f"Water_mm  — min: {df_out['Water_mm'].min():.1f}  "
          f"max: {df_out['Water_mm'].max():.1f}  "
          f"mean: {df_out['Water_mm'].mean():.1f}")
    print(f"Rainfall_mm after subtraction — min: {df_out['Rainfall_mm'].min():.1f}  "
          f"mean: {df_out['Rainfall_mm'].mean():.1f}")

    df_out.to_csv(out, index=False)
    print(f"Saved → {out}")


if __name__ == "__main__":
    main()

```

Purpose:

• introduces irrigation variable • prevents data leakage • preserves statistical consistency • improves realism of agent decisions

---

# 3-Phase Fast Development Plan (3–4 hours)

Goal: produce validator-compliant submission with improved reward design.

Scope limitations:

• simple environment dynamics • minimal dataset preprocessing • basic transition model • improved reward shaping only

---

# Phase 1 — OpenEnv Environment (Core functionality)

**Goal:** produce a valid OpenEnv-compliant environment that passes schema and endpoint checks.

Estimated time: **1.5 hours**

---

## Tasks

### 1. Define typed state model (Pydantic)

Keep small but realistic.

Example variables:

```
soil_moisture : float
soil_ph       : float
temperature   : float
rainfall      : float
crop_stage    : int
day           : int

```

Requirements satisfied:

- typed models required by OpenEnv spec
- deterministic state structure

---

### 2. Define typed action model

Discrete actions simplify LLM reliability:

```
water       : float   (0–50)
fertilizer  : float   (0–20)
pesticide   : float   (0–10)

```

Keep ranges bounded to stabilize scoring.

---

### 3. Implement environment class

File:

```
env/farm_env.py

```

Must implement:

```
reset()
step(action)
state()

```

---

### 4. Implement improved reward design (only sophistication added)

Reward must reflect:

- yield improvement
- sustainability balance
- penalty for overuse of chemicals

Example reward:

```
yield_score =
    0.4 * soil_moisture
  + 0.3 * temperature_factor
  + 0.3 * rainfall_factor

resource_penalty =
    0.03 * fertilizer^1.2
  + 0.04 * pesticide^1.3

sustainability_bonus =
    0.2 * exp(-fertilizer/20)
  + 0.2 * exp(-pesticide/10)

reward =
    yield_score
  + sustainability_bonus
  - resource_penalty

```

Characteristics:

- diminishing returns on fertilizer
- discourages excessive pesticide
- stable numeric range
- smooth gradients

---

### 5. Episode termination rule

```
max_days = 30

```

Short episodes ensure runtime < 20 min.

---

### 6. Create openenv.yaml

Define:

```
environment metadata
observation schema
action schema
reward schema
task definitions

```

Ensure field names exactly match Pydantic models.

---

### 7. Implement API wrapper (if required by spec)

Expose:

```
POST /reset
POST /step
GET /state

```

Ensure reset returns valid initial state.

Requirement satisfied:

HF Space ping must return 200.

---

# Phase 2 — inference pipeline + tasks + graders

**Goal:** produce valid evaluation run with structured logs and normalized scores.

Estimated time: **1.5 hours**

---

## Tasks

### 1. Create inference.py in root directory

File location:

```
/inference.py

```

Must:

- load environment
- call LLM via OpenAI client
- run episodes
- log structured output
- compute task scores

---

### 2. Implement OpenAI client usage

Must use env variables:

```
API_BASE_URL
MODEL_NAME
HF_TOKEN

```

LLM prompt format:

```
Farm state:
soil moisture: 34
temperature: 26
rainfall: 3
crop stage: 2

Choose action values:
water
fertilizer
pesticide

```

LLM output expected as JSON:

```
{
 "water": 20,
 "fertilizer": 5,
 "pesticide": 1
}

```

Add fallback defaults if parsing fails.

---

### 3. Define 3 tasks

Tasks must produce score ∈ [0,1].

---

#### Task 1 — yield performance

Measures productivity.

```
score =
normalized(total_reward)

```

---

#### Task 2 — chemical efficiency

Penalizes excessive fertilizer/pesticide.

```
score =
1 - normalized(total_chemical_use)

```

---

#### Task 3 — sustainability balance

Encourages moderate actions.

```
score =
yield / (fertilizer + pesticide + 1)
normalized to 0–1

```

---

### 4. Implement graders

Each grader returns:

```
{
 "task_id": "...",
 "score": float
}

```

Ensure:

```
0 ≤ score ≤ 1

```

Validator requirement.

---

### 5. Implement structured logs

Strict format:

```
[START]
model: MODEL_NAME

[STEP]
step: 1
action: {...}
reward: ...

[STEP]
step: 2
...

[END]
task_scores:
task1: 0.63
task2: 0.71
task3: 0.59

```

Formatting must match specification exactly.

---

### 6. Runtime optimization

Keep small:

```
episodes = 3
steps per episode = 20–30

```

Ensures runtime well below 20 minutes.

---

# Phase 3 — packaging, docker, validation

**Goal:** ensure infrastructure compatibility and reproducibility.

Estimated time: **1 hour**

---

## Tasks

### 1. requirements.txt

Minimal dependencies:

```
pydantic
numpy
pyyaml
openai
fastapi (optional)
uvicorn (optional)

```

Avoid heavy ML libraries.

---

### 2. Dockerfile

Must build automatically.

Example flow:

```
FROM python:3.11-slim

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD ["python", "inference.py"]

```

Validator requirement satisfied.

---

### 3. environment variables support

Ensure inference.py reads:

```
API_BASE_URL
MODEL_NAME
HF_TOKEN

```

No hardcoding.

---

### 4. basic local tests

Run:

```
python inference.py

```

Verify:

- no crashes
- scores generated
- logs formatted correctly

---

### 5. validation checklist

Confirm:

HF Space can call:

```
reset()
step()
state()

```

Ensure:

- numeric reward returned
- valid JSON outputs
- docker build successful

---

# Final deliverable structure

```
project/
│
├── openenv.yaml
├── inference.py
├── Dockerfile
├── requirements.txt
│
├── env/
│   └── farm_env.py
│
└── tasks/
    └── graders.py

```

---

# Expected outcome

Submission will pass:

- OpenEnv compliance
- structured logging requirement
- 3 task requirement
- reproducibility requirement
- runtime constraint
- docker build requirement
- HF space endpoint validation

---