# FarmRL Round-1 Fast Development Roadmap ## Reference Materials ### Introduction FarmRL is a reinforcement learning project that trains an agent to manage crop farming decisions. Given observable farm conditions such as soil properties, weather, and crop type, the agent learns to control irrigation, fertilizer application, and pesticide use in order to maximise crop yield while maintaining a healthy sustainability score. The project is grounded in a tabular agricultural dataset and draws conceptual inspiration from the FarmGym simulation framework. Two training paradigms are supported: a classic RL agent via a custom OpenEnv environment, and an optional text-framing path using TRL for language-model-based decision making. The raw CSV dataset is preprocessed once. The preprocessing adds the Water\_mm column (drawn uniformly from [20, min(Rainfall\_mm, 200)]) and subtracts that value from Rainfall\_mm to preserve water-balance invariance. A lightweight regression model (XGBoost) is then trained on the processed data to serve as the environment's transition model. --- ## Dataset preprocessing requirement Add a preprocessing script that creates a new variable Water\_mm such that: Rainfall\_original = Rainfall\_new + Water\_mm This prevents bias by conserving total water availability. Script file: scripts/add\_water\_variable.py ``` """ add_water_variable.py Adds a Water_mm column to the farm dataset. Water is drawn uniformly from [WATER_MIN, Rainfall_mm]. Rainfall_mm is reduced by the water drawn to prevent bias. """ import pandas as pd import numpy as np import sys WATER_MIN = 20 # minimum meaningful irrigation (mm) WATER_MAX = 200 # hard ceiling - avoids flooding; also capped at rainfall def add_water(df: pd.DataFrame, seed: int = 42) -> pd.DataFrame: rng = np.random.default_rng(seed) df = df.copy() # Upper bound: rainfall itself, capped at WATER_MAX upper = df["Rainfall_mm"].clip(upper=WATER_MAX) # Where rainfall < WATER_MIN we can't irrigate meaningfully — set 0 can_irrigate = upper >= WATER_MIN water = np.where( can_irrigate, rng.uniform(WATER_MIN, upper.where(can_irrigate, WATER_MIN)), 0.0 ) df["Water_mm"] = np.round(water, 2) df["Rainfall_mm"] = np.round(df["Rainfall_mm"] - df["Water_mm"], 2) return df def main(): path = sys.argv[1] if len(sys.argv) > 1 else "farm_data.csv" out = sys.argv[2] if len(sys.argv) > 2 else path.replace(".csv", "_watered.csv") df = pd.read_csv(path) required = {"Rainfall_mm"} missing = required - set(df.columns) if missing: raise ValueError(f"Missing columns: {missing}") df_out = add_water(df) print(f"Water_mm — min: {df_out['Water_mm'].min():.1f} " f"max: {df_out['Water_mm'].max():.1f} " f"mean: {df_out['Water_mm'].mean():.1f}") print(f"Rainfall_mm after subtraction — min: {df_out['Rainfall_mm'].min():.1f} " f"mean: {df_out['Rainfall_mm'].mean():.1f}") df_out.to_csv(out, index=False) print(f"Saved → {out}") if __name__ == "__main__": main() ``` Purpose: • introduces irrigation variable • prevents data leakage • preserves statistical consistency • improves realism of agent decisions --- # 3-Phase Fast Development Plan (3–4 hours) Goal: produce validator-compliant submission with improved reward design. Scope limitations: • simple environment dynamics • minimal dataset preprocessing • basic transition model • improved reward shaping only --- # Phase 1 — OpenEnv Environment (Core functionality) **Goal:** produce a valid OpenEnv-compliant environment that passes schema and endpoint checks. Estimated time: **1.5 hours** --- ## Tasks ### 1. Define typed state model (Pydantic) Keep small but realistic. Example variables: ``` soil_moisture : float soil_ph : float temperature : float rainfall : float crop_stage : int day : int ``` Requirements satisfied: - typed models required by OpenEnv spec - deterministic state structure --- ### 2. Define typed action model Discrete actions simplify LLM reliability: ``` water : float (0–50) fertilizer : float (0–20) pesticide : float (0–10) ``` Keep ranges bounded to stabilize scoring. --- ### 3. Implement environment class File: ``` env/farm_env.py ``` Must implement: ``` reset() step(action) state() ``` --- ### 4. Implement improved reward design (only sophistication added) Reward must reflect: - yield improvement - sustainability balance - penalty for overuse of chemicals Example reward: ``` yield_score = 0.4 * soil_moisture + 0.3 * temperature_factor + 0.3 * rainfall_factor resource_penalty = 0.03 * fertilizer^1.2 + 0.04 * pesticide^1.3 sustainability_bonus = 0.2 * exp(-fertilizer/20) + 0.2 * exp(-pesticide/10) reward = yield_score + sustainability_bonus - resource_penalty ``` Characteristics: - diminishing returns on fertilizer - discourages excessive pesticide - stable numeric range - smooth gradients --- ### 5. Episode termination rule ``` max_days = 30 ``` Short episodes ensure runtime < 20 min. --- ### 6. Create openenv.yaml Define: ``` environment metadata observation schema action schema reward schema task definitions ``` Ensure field names exactly match Pydantic models. --- ### 7. Implement API wrapper (if required by spec) Expose: ``` POST /reset POST /step GET /state ``` Ensure reset returns valid initial state. Requirement satisfied: HF Space ping must return 200. --- # Phase 2 — inference pipeline + tasks + graders **Goal:** produce valid evaluation run with structured logs and normalized scores. Estimated time: **1.5 hours** --- ## Tasks ### 1. Create inference.py in root directory File location: ``` /inference.py ``` Must: - load environment - call LLM via OpenAI client - run episodes - log structured output - compute task scores --- ### 2. Implement OpenAI client usage Must use env variables: ``` API_BASE_URL MODEL_NAME HF_TOKEN ``` LLM prompt format: ``` Farm state: soil moisture: 34 temperature: 26 rainfall: 3 crop stage: 2 Choose action values: water fertilizer pesticide ``` LLM output expected as JSON: ``` { "water": 20, "fertilizer": 5, "pesticide": 1 } ``` Add fallback defaults if parsing fails. --- ### 3. Define 3 tasks Tasks must produce score ∈ [0,1]. --- #### Task 1 — yield performance Measures productivity. ``` score = normalized(total_reward) ``` --- #### Task 2 — chemical efficiency Penalizes excessive fertilizer/pesticide. ``` score = 1 - normalized(total_chemical_use) ``` --- #### Task 3 — sustainability balance Encourages moderate actions. ``` score = yield / (fertilizer + pesticide + 1) normalized to 0–1 ``` --- ### 4. Implement graders Each grader returns: ``` { "task_id": "...", "score": float } ``` Ensure: ``` 0 ≤ score ≤ 1 ``` Validator requirement. --- ### 5. Implement structured logs Strict format: ``` [START] model: MODEL_NAME [STEP] step: 1 action: {...} reward: ... [STEP] step: 2 ... [END] task_scores: task1: 0.63 task2: 0.71 task3: 0.59 ``` Formatting must match specification exactly. --- ### 6. Runtime optimization Keep small: ``` episodes = 3 steps per episode = 20–30 ``` Ensures runtime well below 20 minutes. --- # Phase 3 — packaging, docker, validation **Goal:** ensure infrastructure compatibility and reproducibility. Estimated time: **1 hour** --- ## Tasks ### 1. requirements.txt Minimal dependencies: ``` pydantic numpy pyyaml openai fastapi (optional) uvicorn (optional) ``` Avoid heavy ML libraries. --- ### 2. Dockerfile Must build automatically. Example flow: ``` FROM python:3.11-slim WORKDIR /app COPY . . RUN pip install -r requirements.txt CMD ["python", "inference.py"] ``` Validator requirement satisfied. --- ### 3. environment variables support Ensure inference.py reads: ``` API_BASE_URL MODEL_NAME HF_TOKEN ``` No hardcoding. --- ### 4. basic local tests Run: ``` python inference.py ``` Verify: - no crashes - scores generated - logs formatted correctly --- ### 5. validation checklist Confirm: HF Space can call: ``` reset() step() state() ``` Ensure: - numeric reward returned - valid JSON outputs - docker build successful --- # Final deliverable structure ``` project/ │ ├── openenv.yaml ├── inference.py ├── Dockerfile ├── requirements.txt │ ├── env/ │ └── farm_env.py │ └── tasks/ └── graders.py ``` --- # Expected outcome Submission will pass: - OpenEnv compliance - structured logging requirement - 3 task requirement - reproducibility requirement - runtime constraint - docker build requirement - HF space endpoint validation ---