Add v4b step600 baseline300 full eval

eval/reports/sft_v4b_step600_baseline300_full_eval/SUMMARY.md

@@ -0,0 +1,42 @@
+# SFT v1 Eval Summary — sft_v4b_step600_baseline300_full_eval
+
+Date: 2026-05-04 05:22:53
+
+Base: `google/gemma-4-E2B-it`
+Adapter: `/home/ubuntu/checkpoints/sft_v4b_e2b_seed42/step600`
+
+
+## 1. Full Gold (167 items)
+- grounded n=73  chrF=22.81  url_recall=0.75  wrongly_refused=0  (0.0%)
+- refusal  n=91  correct_pct=83.5%  hallucinated=15
+
+## 2. LLM-as-judge (DeepSeek)
+- skipped: no data
+
+## 3. Belebele (50 NE MC) — regression check
+- accuracy: 58.0%  (29/50)
+- Gemma 4 IT baseline: see `eval/gemma3_nepali_baseline.md`
+
+## 4. GSM8K-en (30) — English replay regression
+- accuracy: 53.3%  (16/30)
+
+## 5. Roman-NE qualitative (10 prompts)
+- n_degen: 0/10
+- loops: 0, mojibake: 0, empty: 0
+- target: ≤1 degen (was 3 with base Gemma 4 IT)
+
+## 6. Side-by-side vs baseline
+- skipped: no baseline file
+
+## Demoability call
+
+**PASS signals**: 4
+
+- ✓ url_recall 0.75 ≥ 0.70
+- ✓ wrongly_refused 0% ≤ 10%
+- ✓ roman_degen 0/10 ≤ 1
+- ✓ belebele 0.58 ≥ 0.55
+
+**FAIL signals**: 1
+
+- ✗ refusal_correct 84% < 90%

eval/reports/sft_v4b_step600_baseline300_full_eval/belebele.json

@@ -0,0 +1,310 @@
+{
+  "summary": {
+    "n": 50,
+    "correct": 29,
+    "accuracy": 0.58,
+    "elapsed_sec": 17.7
+  },
+  "items": [
+    {
+      "idx": 654,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 114,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 25,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 759,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 281,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 250,
+      "gold": "D",
+      "model": "A",
+      "ok": false
+    },
+    {
+      "idx": 228,
+      "gold": "D",
+      "model": "D",
+      "ok": true
+    },
+    {
+      "idx": 142,
+      "gold": "A",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 754,
+      "gold": "A",
+      "model": "A",
+      "ok": true
+    },
+    {
+      "idx": 104,
+      "gold": "A",
+      "model": "A",
+      "ok": true
+    },
+    {
+      "idx": 692,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 758,
+      "gold": "D",
+      "model": "D",
+      "ok": true
+    },
+    {
+      "idx": 558,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 89,
+      "gold": "A",
+      "model": "A",
+      "ok": true
+    },
+    {
+      "idx": 604,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 432,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 32,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 30,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 95,
+      "gold": "A",
+      "model": "A",
+      "ok": true
+    },
+    {
+      "idx": 223,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 238,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 517,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 616,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 27,
+      "gold": "A",
+      "model": "C",
+      "ok": false
+    },
+    {
+      "idx": 574,
+      "gold": "A",
+      "model": "C",
+      "ok": false
+    },
+    {
+      "idx": 203,
+      "gold": "B",
+      "model": "A",
+      "ok": false
+    },
+    {
+      "idx": 733,
+      "gold": "A",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 665,
+      "gold": "D",
+      "model": "C",
+      "ok": false
+    },
+    {
+      "idx": 718,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 429,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 225,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 459,
+      "gold": "A",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 603,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 284,
+      "gold": "D",
+      "model": "D",
+      "ok": true
+    },
+    {
+      "idx": 828,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 890,
+      "gold": "D",
+      "model": "C",
+      "ok": false
+    },
+    {
+      "idx": 6,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 777,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 825,
+      "gold": "A",
+      "model": "C",
+      "ok": false
+    },
+    {
+      "idx": 163,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 714,
+      "gold": "C",
+      "model": "A",
+      "ok": false
+    },
+    {
+      "idx": 348,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 159,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 220,
+      "gold": "D",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 781,
+      "gold": "B",
+      "model": "D",
+      "ok": false
+    },
+    {
+      "idx": 344,
+      "gold": "C",
+      "model": "C",
+      "ok": true
+    },
+    {
+      "idx": 94,
+      "gold": "B",
+      "model": "C",
+      "ok": false
+    },
+    {
+      "idx": 389,
+      "gold": "C",
+      "model": "B",
+      "ok": false
+    },
+    {
+      "idx": 99,
+      "gold": "B",
+      "model": "B",
+      "ok": true
+    },
+    {
+      "idx": 367,
+      "gold": "D",
+      "model": "D",
+      "ok": true
+    }
+  ]
+}

eval/reports/sft_v4b_step600_baseline300_full_eval/gsm8k.json

@@ -0,0 +1,220 @@
+{
+  "summary": {
+    "n": 30,
+    "correct": 16,
+    "accuracy": 0.5333,
+    "elapsed_sec": 708.7
+  },
+  "items": [
+    {
+      "idx": 1309,
+      "gold": "2280",
+      "pred": "1880",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Find Alexandra's total amount raised:**\n   Alexandra raises $430.\n\n2. **Find Kim's total amount raised:**\n   Kim raises $320 more than Alexandra"
+    },
+    {
+      "idx": 228,
+      "gold": "1",
+      "pred": "1",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the puzzle problem:\n\n1. **Determine the total number of pieces in the puzzle:** The puzzle has 360 pieces.\n2. **Calculate the combined rate at which Kalinda and he"
+    },
+    {
+      "idx": 51,
+      "gold": "5",
+      "pred": "0",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the time Tom takes to travel from 1 PM to 4 PM.**\n   - Distance = 4 miles - 1 mile = 3 miles.\n   - Time = Distance / Speed = 3 miles /"
+    },
+    {
+      "idx": 563,
+      "gold": "12",
+      "pred": "3",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Determine the ages of James' two sons:**\n   - The older son is 12 years old.\n   - The younger son is 4 years younger than his brother.\n   - Youn"
+    },
+    {
+      "idx": 501,
+      "gold": "273",
+      "pred": "36",
+      "ok": false,
+      "resp": "To solve this problem, we need to find the total amount of yarn Mariah and her grandma used together.\n\n1. Calculate the amount of yarn Mariah used:\n   Mariah used $\\frac{1}{4}$ of a skein.\n\n2. Calcula"
+    },
+    {
+      "idx": 457,
+      "gold": "45",
+      "pred": "45",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Initial number of fairies:** Katelyn initially saw 50 fairies.\n2. **Fairies from the east:** Half as many fairies as Katelyn saw came from the e"
+    },
+    {
+      "idx": 285,
+      "gold": "21",
+      "pred": "21",
+      "ok": true,
+      "resp": "Here is the step-by-step solution:\n\n1. **Determine Ann's brother's current age:** Ann is 9 years old, and her brother is twice her age. So, Ann's brother is $2 \\times 9 = 18$ years old.\n2. **Determine"
+    },
+    {
+      "idx": 209,
+      "gold": "145",
+      "pred": "3",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the total cost of buying twenty dozen cups:**\n   - There are 12 cups in a dozen.\n   - Twenty dozen cups is $20 \\times 12 = 240$ cups.\n"
+    },
+    {
+      "idx": 1116,
+      "gold": "60",
+      "pred": "60",
+      "ok": true,
+      "resp": "To solve this problem, we need to calculate a 20% increase in the number of students enrolled in the calligraphy class.\n\nHere is the step-by-step reasoning:\n\n1. **Identify the initial number of studen"
+    },
+    {
+      "idx": 178,
+      "gold": "122",
+      "pred": "122",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Find the number of crabs Monic has:**\n   Monic has 4 fewer crabs than Bo.\n   Number of crabs Monic has = Number of crabs Bo has - 4\n   Number of"
+    },
+    {
+      "idx": 1209,
+      "gold": "29",
+      "pred": "29",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Determine the water removed by each setting per day:**\n   - Low setting: 1 liter/day\n   - Medium setting: 2 liters/day (twice the low setting)\n "
+    },
+    {
+      "idx": 864,
+      "gold": "80",
+      "pred": "580",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the total number of burritos ordered:**\n   George ordered 600 burritos.\n\n2. **Calculate the total number of burritos eaten by Mr. Geor"
+    },
+    {
+      "idx": 65,
+      "gold": "36",
+      "pred": "7",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the total time Jim spends watching TV in one week:**\n   Jim watches TV for 2 hours per day.\n   Total time watching TV in one week: $2 "
+    },
+    {
+      "idx": 61,
+      "gold": "1430",
+      "pred": "1430",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the total cost of the brooch:**\n   Cost of material + Cost of construction = Total cost\n   $\\$500 + \\$800 = \\$1300$\n\n2. **Calculate th"
+    },
+    {
+      "idx": 191,
+      "gold": "5",
+      "pred": "3",
+      "ok": false,
+      "resp": "To solve this problem, we need to determine the total weight of insects the ducks need per week and then divide that by the number of ducks to find the amount of insects they need per day.\n\nHere is th"
+    },
+    {
+      "idx": 447,
+      "gold": "5",
+      "pred": "5",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Joey's new score:** Joey started with 214 points and scored 26 points.\n   Joey's new total is $214 + 26 = 240$ points.\n\n2. **Marcy's new score:*"
+    },
+    {
+      "idx": 476,
+      "gold": "5",
+      "pred": "5",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the total number of books Sarah bought:**\n   Divide the total amount she spent by the cost of one book.\n   $$\\text{Total books} = \\fra"
+    },
+    {
+      "idx": 1034,
+      "gold": "66",
+      "pred": "66",
+      "ok": true,
+      "resp": "To solve this problem, we need to calculate the total amount Ted spent on books and pencils.\n\n1. Calculate the total cost of the 3 books:\n   Cost of 3 books = 3 books $\\times$ \\$16/book\n\n2. Calculate "
+    },
+    {
+      "idx": 1232,
+      "gold": "15",
+      "pred": "4",
+      "ok": false,
+      "resp": "Let $P$ be the original number of pomelos Eve had.\nLet $G$ be the number of pomelos Eve gave to her friend.\n\nThe problem states that after giving her friend some pomelos, Eve is left with $\\frac{1}{4}"
+    },
+    {
+      "idx": 54,
+      "gold": "40",
+      "pred": "12",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Identify the total number of kittens:** The Doubtfire sisters have 7 kittens in total.\n2. **Determine the number of kittens Patchy has:** Patchy"
+    },
+    {
+      "idx": 1149,
+      "gold": "93",
+      "pred": "93",
+      "ok": true,
+      "resp": "Here is the step-by-step solution:\n\n1. **Determine the total number of fish Jen has:** Jen has 3 fish.\n2. **Calculate the total amount of food needed:** Each fish needs $1 worth of food per day. So, $"
+    },
+    {
+      "idx": 407,
+      "gold": "2000",
+      "pred": "1000",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Elliott's total planned steps:** 10,000 steps.\n2. **Steps taken to and from school:** Half of his total planned steps.\n3. **Steps taken during h"
+    },
+    {
+      "idx": 859,
+      "gold": "1520",
+      "pred": "1520",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate Rafaela's total tip:** Rafaela received a \\$20 tip from each of the forty customers.\n   Rafaela's total tip = $20 \\times 40 = \\$800$.\n"
+    },
+    {
+      "idx": 451,
+      "gold": "11050",
+      "pred": "113",
+      "ok": false,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Define variables:**\n   - Let $C_1$ be the cost of filling one balloon on the first day.\n   - Let $C_2$ be the cost of filling one balloon on the"
+    },
+    {
+      "idx": 919,
+      "gold": "90",
+      "pred": "90",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the total number of lollipops sold by the class:**\n   Number of students $\\times$ Lollipops sold per student\n   $30 \\text{ students} \\"
+    },
+    {
+      "idx": 1206,
+      "gold": "40000",
+      "pred": "40000",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Calculate the amount of the loan John receives:**\n   The loan is 40% of the house's value.\n   Loan Amount = $250,000 \\times 0.40 = $100,000\n\n2. "
+    },
+    {
+      "idx": 569,
+      "gold": "21",
+      "pred": "21",
+      "ok": true,
+      "resp": "Here is the step-by-step solution to the problem:\n\n1. **Determine Liam's total earnings:** Liam made $63 one weekend.\n2. **Determine Mitchell's total earnings:** Mitchell sold 21 lemonades at $4 each."
+    },
+    {
+      "idx": 13,
+      "gold": "18",
+      "pred": "5",
+      "ok": false,
+      "resp": "Let $W$ be the initial number of vacuum cleaners Melanie had.\n\n1. She sold a third of her vacuum cleaners at the green house.\n   Number sold at the green house: $\\frac{1}{3}W$\n\n2. She sold 2 more to t"
+    },
+    {
+      "idx": 326,
+      "gold": "14",
+      "pred": "14",
+      "ok": true,
+      "resp": "Let $B$ be the initial number of people on the bus at the terminal.\n\nAt the first bus stop, 5 more people got on the bus. The number of people on the bus is now $B + 5$.\n\nAt the second bus stop, 7 peo"
+    },
+    {
+      "idx": 865,
+      "gold": "23",
+      "pred": "33",
+      "ok": false,
+      "resp": "Let $R$ be the number of red cars on the rink.\nLet $G$ be the number of green cars on the rink.\nLet $B$ be the number of blue cars on the rink.\nLet $Y$ be the number of yellow cars on the rink.\n\nFrom "
+    }
+  ]
+}

eval/reports/sft_v4b_step600_baseline300_full_eval/llm_judge.json

@@ -0,0 +1,208 @@
+{
+  "summary": {
+    "n": 0,
+    "n_errors": 50
+  },
+  "results": [
+    {
+      "id": "u_grnd_057",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_023",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_061",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_034",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_009",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_035",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_019",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_025",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_053",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_065",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_018",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_083",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_094",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_055",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_049",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_048",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_052",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_038",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_086",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_045",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_056",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_004",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_084",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_098",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_046",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_054",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_028",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_062",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_030",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_010",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_024",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_067",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_027",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_041",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_029",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_071",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_008",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_058",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_070",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_011",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_059",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_026",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_032",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_012",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_001",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_091",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_042",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_092",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_069",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    },
+    {
+      "id": "u_grnd_031",
+      "error": "RuntimeError: HTTP 402: {\"error\":{\"message\":\"Insufficient Balance\",\"type\":\"unknown_error\",\"param\":null,\"code\":\"invalid_request_error\"}}"
+    }
+  ]
+}

eval/reports/sft_v4b_step600_baseline300_full_eval/roman_ne.json

@@ -0,0 +1,82 @@
+{
+  "summary": {
+    "n": 10,
+    "n_degen": 0,
+    "n_loop": 0,
+    "n_mojibake": 0,
+    "n_empty": 0,
+    "elapsed_sec": 32.3
+  },
+  "items": [
+    {
+      "q": "mero nagarikta banauna ko lagi kun office janu parcha?",
+      "a": "Yo prashnako adhikarik srot bhetina. Nagarikta sambandhi thap jankari ko lagi Hello Sarkar 1111 ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "passport renew garna kaha janu parcha?",
+      "a": "Yo prashnako adhikarik srot bhetina. Passport renewal ko lagi Department of Passports ko website ma jane, athawa passport office ma sampark garnu hola.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "company registration kasari garne?",
+      "a": "Yo prashnako adhikarik srot bhetina. Company registration ko lagi Nepal ko Office of the Company Registrar ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "PAN number kasari banaune?",
+      "a": "Yo prashnako adhikarik srot bhetina. PAN number banaune lagi aafno tax office ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "driving license ko lagi k k chaine?",
+      "a": "Yo prashnako adhikarik srot bhetina. Driving license ko lagi Department of Transport Management, City Traffic Management, and Parking Services ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "VAT ra PAN ma k farak cha?",
+      "a": "Yo prashnako adhikarik srot bhetina. VAT ra PAN ko barema thap jankari ko lagi aantarik kar karyalaya ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "nagarikta certificate hareyo, kaha janu parcha?",
+      "a": "Yo prashnako adhikarik srot bhetina. Nagarikta certificate harayo bhane, tapaile nagarikta karyalaya ma sampark garnu parcha.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "jagga ko malpot kaha tirne?",
+      "a": "Yo prashnako adhikarik srot bhetina. Malpot karyalaya ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "bachhako janmadarta kasari garne?",
+      "a": "Yo prashnako adhikarik srot bhetina. Janmadarta sambandhi thap jankari ko lagi local ward office ma sampark garnuhos.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    },
+    {
+      "q": "online tax file kasari garne?",
+      "a": "Yo prashnako adhikarik srot bhetina. Online tax filing ko lagi Inland Revenue Department ko website ma jane ma saknu huncha.",
+      "loop": false,
+      "mojibake": false,
+      "empty": false
+    }
+  ]
+}

eval/reports/sft_v4b_step600_baseline300_full_eval/side_by_side.json

@@ -0,0 +1,45 @@
+{
+  "sft_results": [
+    {
+      "id": "u_grnd_057",
+      "model_output": "राष्ट्रिय कार्यमञ्चको विधान, २०६५ बमोजिम यसले आफ्ना उद्देश्यहरू कार्यान्वयन गर्न प्रचलित नेपाल कानुनबमोजिम सम्बन्धित निकायबाट अनुमति वा सोको प्रतिलिपि प्राप्त गरेर मात्र आफ्ना उद्देश्यहरू कार्यान्वयन गर्ने [https://moha.gov.np/upload/e66443e81e8cc9c4fa5c099a1fb1bb87/files/National_20Platform_20on_20DRR_20Bidhan_202065_52_20110916032012_1_(1).pdf]। यसले आफ्ना उद्देश्यहरू कार्यान्वयन गर्न आवश्यक पर्ने भवन निर्माण गर्ने वा भाडामा वा लिजमा लिने [https://moha.gov.np/upload/e66443e81e8cc9c4fa5c099a1fb1bb87/files/National_20Platform_20on_20DRR_20Bidhan_202065_52_20110916032012_1_(1).pdf]। यसले आवश्यक पर्ने कर्मचारी नियुक्त गर्ने तथा तिनको सेवाका सर्तहरू निर्धारण गर्ने [https://moha.gov.np/upload/e664"
+    },
+    {
+      "id": "u_grnd_023",
+      "model_output": "जिरी नगरपालिकामा अहिले ७ वटा माध्यमिक विद्यालय सञ्चालनमा रहेका छन् [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/%E0%A4%A8%E0%A4%97%E0%A4%B0%20%E0%A4%AA%E0%A5%8D%E0%A4%B0%E0%A5%8B%E0%A4%AB%E0%A4%BE%E0%A4%88%E0%A4%B2%20%E0%A5%A8%E0%A5%A6%E0%A5%AE%E0%A5%A6.pdf]। तीमध्ये ७ वटा सार्वजनिक र ३ वटा संस्थागत हुन् [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/%E0%A4%A8%E0%A4%97%E0%A4%B0%20%E0%A4%AA%E0%A5%8D%E0%A4%B0%E0%A5%8B%E0%A4%AB%E0%A4%"
+    },
+    {
+      "id": "u_grnd_061",
+      "model_output": "Pestivirus virus group of Flaviviridae family is found in pigs, hogs, and wild boars [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf]. It can be transmitted from pig to pig through direct contact, contaminated feed or water, or through the respiratory tract [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf]. The virus can survive in the pig's intestines for up to 27 days [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf].\nThe virus can be transmitted from pig to pig through contact with infected feces or milk [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf].\nThe virus can be transmitted from pig to pig through contact with infected meat [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-"
+    },
+    {
+      "id": "u_grnd_034",
+      "model_output": "Registration number 13906 ma darta SAROJ KUMAR YADAV le 537.97 quintal ukhu supply gareko thiyo [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf]. Unle 37657.90 rupiya incentive paaune ho [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_13896_13906.pdf]."
+    },
+    {
+      "id": "u_grnd_009",
+      "model_output": "भैँसीले पाडो फाल्यो भन्दैमा ब्रुसेलोसिस भनी रिपोर्ट गर्ने मिल्दैन जबसम्म प्रयोगशाला प्रविधिबाट रोग निदान गरिदैन [https://www.moha.gov.np/downloadsfiles/Manual-for-Epidemiological-Reporting-and-Animal-Disease-Recognition-1770710135.pdf]। रोग निदान कार्यमा सजिलो होस् / सबैले अपनाउने रोग निदानका आधारहरूमा एकरुपता आओस भनेर रोग विवरण तथा रोग निदान चार्ट यस पुस्तिकामा समाबेश गरीएको छ [https://www.moha.gov.np/downloadsfiles/Manual-for-Epidemiological-Reporting-and-Animal-Disease-Recognition-1770710135.pdf]।"
+    },
+    {
+      "id": "u_grnd_035",
+      "model_output": "Rautahat Akar Pather (112.76 quintal) supply gareko cha [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf]. Uslai 810 Sugar Mill bata NPR 18916.10 incentive mileko cha [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf]."
+    },
+    {
+      "id": "u_grnd_019",
+      "model_output": "क्यासिनोमा खेल्न आउने खेलाडीले अधिकतम पाँच हजार (५,०००) अमेरिकी डलर नगद विदेशी मुद्���ा ल्याउन सक्नेछ [https://www.nrb.org.np/fxm/%e0%a5%a7%e0%a5%a7-%e0%a5%a8%e0%a5%a6%e0%a5%ae%e0%a5%a8-%e0%a5%ae%e0%a5%a9-%e0%a4%8f%e0%a4%95%e0%a5%80%e0%a4%95%e0%a5%83%e0%a4%a4-%e0%a4%aa%e0%a4%b0%e0%a4%bf%e0%a4%aa%e0%a4%a4%e0%a5%8d/]। सोभन्दा बढी रकमको हकमा भन्सार घोषणा गरी प्रमाणित गरेको हुनुपर्नेछ वा बैंकिङ्ग प्रणालीमार्फत् रकम प्राप्त गरेको हुनुपर्नेछ [https://www.nrb.org.np/fxm/%e0%a5%a7%e0%a5%a7-%e0%a5%"
+    },
+    {
+      "id": "u_grnd_025",
+      "model_output": "नेपालमा विदेशी नागरिकको प्रवेश, बसाइ र प्रस्थानको व्यवस्थापन, नियमन, नियन्त्रण तथा अभिलेखीकरण गर्ने जिम्मेवारी अध्यागमन विभागको हो [https://moha.gov.np/page/main-functions]।"
+    },
+    {
+      "id": "u_grnd_053",
+      "model_output": "MAHABIR MAHATO (Mahabir Mahato ko chhora) lai ESCI LTD le unkako ukhu ko lagi 140.18 rupaiya diyeko chha [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf].\n[1] https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf"
+    },
+    {
+      "id": "u_grnd_065",
+      "model_output": "स्थानीय सरकार सञ्चालन ऐन, २०७४ को दफा ७१ अनुसार स्थानीय सरकारले आगामी आर्थिक वर्षको राजस्व र व्ययको अनुमान कार्यक्रम तथा बजेट प्रस्ताव नगरपालिकाबाट स्वीकृति गराई असार १० गतेसम्म पेश गर्नुपर्छ [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/mSsj01630472169%E0%A4%9C%E0%A4%BF%E0%A4%B0%E0%A5%80%20%E0%A4%A8%E0%A4%97%E0%A4%B0%E0%A4%AA%E0%A4%BE%E0%A4%B2%E0%A4%BF%E0%A4%95%E0%A4%BE%2C%20%E0%A5%A8%E0%A5%A6%E0%A5%AD%E0%A5%AE.pdf]। बजेट सभामा पेश भएको मितिले २०७६।३।११ मा पारित भएको हो [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/mSsj01"
+    }
+  ],
+  "pairwise": []
+}

eval/reports/sft_v4b_step600_baseline300_full_eval/side_by_side.md

@@ -0,0 +1,253 @@
+# Side-by-side: SFT vs sonnet-4-6-baseline
+
+
+---
+
+## 1. u_grnd_057  (`other` / `devanagari`)
+
+**Question**: विपद् जोखिम न्यूनीकरण राष्ट्रिय कार्यमञ्चले आफ्ना उद्देश्यहरू कार्यान्वयन गर्न के–के गर्न पाउँछ र त्यसका लागि कुनै अनुमति लिनुपर्छ?
+
+
+### Gold
+```
+The excerpt states the platform must first obtain permission or approval from the relevant authority under prevailing Nepal law before implementing its objectives, and may also construct or lease premises, purchase or hire office equipment and vehicles, and appoint staff with defined service conditions.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+राष्ट्रिय कार्यमञ्चको विधान, २०६५ बमोजिम यसले आफ्ना उद्देश्यहरू कार्यान्वयन गर्न प्रचलित नेपाल कानुनबमोजिम सम्बन्धित निकायबाट अनुमति वा सोको प्रतिलिपि प्राप्त गरेर मात्र आफ्ना उद्देश्यहरू कार्यान्वयन गर्ने [https://moha.gov.np/upload/e66443e81e8cc9c4fa5c099a1fb1bb87/files/National_20Platform_20on_20DRR_20Bidhan_202065_52_20110916032012_1_(1).pdf]। यसले आफ्ना उद्देश्यहरू कार्यान्वयन गर्न आवश्यक पर्ने भवन निर्माण गर्ने वा भाडामा वा लिजमा लिने [https://moha.gov.np/upload/e66443e81e8cc9c4fa5c099a1fb1bb87/files/National_20Platform_20on_20DRR_20Bidhan_202065_52_20110916032012_1_(1).pdf]। यसले आवश्यक
+```
+
+
+---
+
+## 2. u_grnd_023  (`education` / `devanagari`)
+
+**Question**: जिरी नगरपालिकामा अहिले कति माध्यमिक विद्यालय सञ्चालनमा छन् र ती मध्ये कति सार्वजनिक र कति संस्थागत हुन्?
+
+
+### Gold
+```
+The excerpt states that Jiri Municipality currently has 10 secondary schools in operation — 7 public and 3 institutional — along with 1 technical school and 2 bachelor-level colleges.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+जिरी नगरपालिकामा अहिले ७ वटा माध्यमिक विद्यालय सञ्चालनमा रहेका छन् [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/%E0%A4%A8%E0%A4%97%E0%A4%B0%20%E0%A4%AA%E0%A5%8D%E0%A4%B0%E0%A5%8B%E0%A4%AB%E0%A4%BE%E0%A4%88%E0%A4%B2%20%E0%A5%A8%E0%A5%A6%E0%A5%AE%E0%A5%A6.pdf]। तीमध्ये ७ वटा सार्वजनिक र ३ वटा संस्थागत हुन् [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/%E0%A4%A8%E0%A4%97%E0%A4%B0%20%E0%A4%AA%E0%A5%8D%E0%A4%B0%E0%A5%8B%E0%A4%AB%E0%A4%
+```
+
+
+---
+
+## 3. u_grnd_061  (`agriculture` / `roman_nepali`)
+
+**Question**: Swine fever lageko bangurko maasu maa yo virus kati din samma bachcha rahanchha?
+
+
+### Gold
+```
+The excerpt states the virus survives up to 27 days in infected pig meat and from several months to years in frozen meat, making disease control extremely difficult.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+Pestivirus virus group of Flaviviridae family is found in pigs, hogs, and wild boars [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf]. It can be transmitted from pig to pig through direct contact, contaminated feed or water, or through the respiratory tract [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf]. The virus can survive in the pig's intestines for up to 27 days [https://dls.gov.np/progressfiles/TADS-Manual-Nepali-Final-Revised-2-Sept-2024-1760434631.pdf].
+The virus can be transmitted from pig to
+```
+
+
+---
+
+## 4. u_grnd_034  (`agriculture` / `roman_nepali`)
+
+**Question**: Registration number 13906 ma darta SAROJ KUMAR YADAV le kati quintal ukhu supply gareko thiyo ra unle kati rupiya incentive paaune ho?
+
+
+### Gold
+```
+The excerpt's tabular register shows S.N. 13906 Saroj Kumar Yadav supplied 537.97 quintals of sugarcane and is entitled to Rs. 37,657.90 in production incentive from MS PVT.LTD.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+Registration number 13906 ma darta SAROJ KUMAR YADAV le 537.97 quintal ukhu supply gareko thiyo [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf]. Unle 37657.90 rupiya incentive paaune ho [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_13896_13906.pdf].
+```
+
+
+---
+
+## 5. u_grnd_009  (`agriculture` / `devanagari`)
+
+**Question**: भैँसीले पाडो फाल्यो भन्दैमा सोझै ब्रुसेलोसिस भनी रिपोर्ट गर्न मिल्छ कि मिल्दैन?
+
+
+### Gold
+```
+मिल्दैन — प्रयोगशाला प्रविधिबाट रोग निदान नगरिकन गर्भपतनको आधारमा मात्र ब्रुसेलोसिस रिपोर्ट गर्न पाइँदैन भनी म्यानुअलमा स्पष्ट उल्लेख गरिएको छ।
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+भैँसीले पाडो फाल्यो भन्दैमा ब्रुसेलोसिस भनी रिपोर्ट गर्ने मिल्दैन जबसम्म प्रयोगशाला प्रविधिबाट रोग निदान गरिदैन [https://www.moha.gov.np/downloadsfiles/Manual-for-Epidemiological-Reporting-and-Animal-Disease-Recognition-1770710135.pdf]। रोग निदान कार्यमा सजिलो होस् / सबैले अपनाउने रोग निदानका आधारहरूमा एकरुपता आओस भनेर रोग विवरण तथा रोग निदान चार्ट यस पुस्तिकामा समाबेश गरीएको छ [https://www.moha.gov.np/downloadsfiles/Manual-for-Epidemiological-Reporting-and-Animal-Disease-Recognition-1770710135.pdf]।
+```
+
+
+---
+
+## 6. u_grnd_035  (`agriculture` / `roman_nepali`)
+
+**Question**: Rautahat Akar Pather ka HARENDRA PATEL le kati quintal ukhu supply gareko cha ra uslai kun sugar mill bata kati paisa incentive mileko cha?
+
+
+### Gold
+```
+The excerpt's register shows S.N. 815 Harendra Patel of Rautahat, Akar Pather supplied 410.45 quintals of sugarcane and received Rs. 28,731.50 in incentive from SBBSCI PVT.LTD.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+Rautahat Akar Pather (112.76 quintal) supply gareko cha [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf]. Uslai 810 Sugar Mill bata NPR 18916.10 incentive mileko cha [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf].
+```
+
+
+---
+
+## 7. u_grnd_019  (`banking` / `devanagari`)
+
+**Question**: क्यासिनोमा खेल्न आउने खेलाडीले कति अमेरिकी डलर नगद विदेशी मुद्रा ल्याउन पाउँछ, र त्योभन्दा बढी रकम ल्याउनु परेमा के गर्नुपर्छ?
+
+
+### Gold
+```
+Players may bring up to USD 5,000 (or equivalent convertible foreign currency) in cash; any amount above that must either be declared and verified at customs or received through the banking system.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+क्यासिनोमा खेल्न आउने खेलाडीले अधिकतम पाँच हजार (५,०००) अमेरिकी डलर नगद विदेशी मुद्रा ल्याउन सक्नेछ [https://www.nrb.org.np/fxm/%e0%a5%a7%e0%a5%a7-%e0%a5%a8%e0%a5%a6%e0%a5%ae%e0%a5%a8-%e0%a5%ae%e0%a5%a9-%e0%a4%8f%e0%a4%95%e0%a5%80%e0%a4%95%e0%a5%83%e0%a4%a4-%e0%a4%aa%e0%a4%b0%e0%a4%bf%e0%a4%aa%e0%a4%a4%e0%a5%8d/]। सोभन्दा बढी रकमको हकमा भन्सार घोषणा गरी प्रमाणित गरेको हुनुपर्नेछ वा बैंकिङ्ग प्रणालीमार्फत् रकम प्राप्त गरेको हुनुपर्नेछ [https://www.nrb.org.np/fxm/%e0%a5%a7%e0%a5%a7-%e0%a5%
+```
+
+
+---
+
+## 8. u_grnd_025  (`visa_immigration` / `devanagari`)
+
+**Question**: नेपालमा विदेशी नागरिकको प्रवेश, बसाइ र प्रस्थानको व्यवस्थापन तथा अभिलेखीकरण गर्ने जिम्मेवारी कुन निकायको हो?
+
+
+### Gold
+```
+The excerpt lists this as item 30 under the Ministry of Home Affairs' main functions: management, regulation, control, and recording of foreign nationals' entry, presence, activities, and departure in Nepal.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+नेपालमा विदेशी नागरिकको प्रवेश, बसाइ र प्रस्थानको व्यवस्थापन, नियमन, नियन्त्रण तथा अभिलेखीकरण गर्ने जिम्मेवारी अध्यागमन विभागको हो [https://moha.gov.np/page/main-functions]।
+```
+
+
+---
+
+## 9. u_grnd_053  (`agriculture` / `roman_nepali`)
+
+**Question**: Mahottari Ramnagar ko MANGAL MAHATO (Mahabir Mahato ko chhora) lai ESCI LTD le unkako ukhu ko lagi kati rupiya diyeko chha?
+
+
+### Gold
+```
+The excerpt lists Mangal Mahato (son of Mahabir Mahato) of Mahottari Ramnagar with 140.18 quintals of sugarcane and a payment of Rs 9,812.6 from ESCI LTD.
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+MAHABIR MAHATO (Mahabir Mahato ko chhora) lai ESCI LTD le unkako ukhu ko lagi 140.18 rupaiya diyeko chha [https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf].
+[1] https://doanepal.gov.np/downloadfile/Sugarcane Book_2082_1752400803.pdf
+```
+
+
+---
+
+## 10. u_grnd_065  (`local_services` / `devanagari`)
+
+**Question**: स्थानीय सरकार सञ्चालन ऐन अनुसार नगरपालिकाले बजेट सभामा कुन मितिसम्म पेश गर्नुपर्छ र कहिलेसम्म पारित गर्नुपर्छ?
+
+
+### Gold
+```
+स्थानीय सरकार सञ्चालन ऐन, २०७४ को दफा ७१ अनुसार कार्यपालिकाबाट स्वीकृत बजेट असार १० गतेभित्र सभामा पेश गर्नुपर्छ र असार मसान्तभित्र सभाबाट पारित गर्नुपर्ने व्यवस्था छ।
+```
+
+
+### Baseline (sonnet-4-6-baseline)
+```
+
+```
+
+
+### SFT
+```
+स्थानीय सरकार सञ्चालन ऐन, २०७४ को दफा ७१ अनुसार स्थानीय सरकारले आगामी आर्थिक वर्षको राजस्व र व्ययको अनुमान कार्यक्रम तथा बजेट प्रस्ताव नगरपालिकाबाट स्वीकृति गराई असार १० गतेसम्म पेश गर्नुपर्छ [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/mSsj01630472169%E0%A4%9C%E0%A4%BF%E0%A4%B0%E0%A5%80%20%E0%A4%A8%E0%A4%97%E0%A4%B0%E0%A4%AA%E0%A4%BE%E0%A4%B2%E0%A4%BF%E0%A4%95%E0%A4%BE%2C%20%E0%A5%A8%E0%A5%A6%E0%A5%AD%E0%A5%AE.pdf]। बजेट सभामा पेश भएको मितिले २०७६।३।११ मा पारित भएको हो [https://jirimun.gov.np/sites/jirimun.gov.np/files/documents/mSsj01
+```