dagger-12B_SFT_GRPO / README.md

dipta007

updated readme (#1)

0f07817 verified 2 days ago

preview code

raw

history blame contribute delete

12.2 kB

metadata

library_name: transformers
license: gemma
license_link: https://ai.google.dev/gemma/terms
pipeline_tag: text-generation
tags:
  - math
  - reasoning
  - computational-graph
  - bangla
  - low-resource
  - distractor-aware
base_model:
  - google/gemma-3-12b-it
language:
  - bn
  - en
datasets:
  - dipta007/dagger
  - dipta007/DistractMath-Bn
model-index:
  - name: dagger-12B_SFT_GRPO
    results:
      - task:
          type: question-answering
          name: Math Word Problems
        dataset:
          name: MGSM-BN
          type: mgsm
        metrics:
          - type: accuracy
            value: 78.4
            name: Original Accuracy
          - type: accuracy
            value: 64
            name: Distractor Accuracy
      - task:
          type: question-answering
          name: Math Word Problems
        dataset:
          name: MSVAMP-BN
          type: msvamp
        metrics:
          - type: accuracy
            value: 78.8
            name: Original Accuracy
          - type: accuracy
            value: 66.8
            name: Distractor Accuracy

DAGGER-12B-SFT-GRPO

Highlights

DAGGER-12B-SFT-GRPO is our best-performing model for distractor-aware mathematical reasoning in Bangla. Key features:

89% fewer tokens than reasoning models while achieving comparable accuracy
Robust to distractors: Only 12-14 point accuracy drop under distractor augmentation (vs. 14-20 for reasoning models, 18-41 for standard CoT)
Executable outputs: Generates computational graphs that can be deterministically executed
Explicit distractor modeling: Identifies irrelevant information as distractor nodes

Model Overview

Attribute	Value
Base Model	Gemma-3-12B-Instruct
Training	SFT → GRPO
Parameters	12B
LoRA Rank	64
Max Sequence Length	4096
Output Format	JSON Computational Graph

Performance

Accuracy Comparison

Model	MGSM	MSVAMP	MGSM (+D)	MSVAMP (+D)	Weighted Avg	Tokens
Qwen 3-8B (Reasoning)	88.0	81.1	70.5	66.9	71.4	3,128
DAGGER-12B (Ours)	78.4	78.8	64.0	66.8	69.4	359
Gemma 3-12B (CoT)	76.8	72.3	54.3	48.7	55.7	599

(+D) = with distractors

Robustness (Accuracy Drop)

Distractor Type	Error Rate
Related Entity (RED)	36%
Orthogonal Attribute (OAD)	34%
Null-Effect Event (NEED)	33%

Output Format

The model generates computational graphs in JSON format:

{
  "nodes": [
    {"id": "n1", "op": "const", "val": 122195, "distractor": false, "label": "মিনার কলম"},
    {"id": "n2", "op": "const", "val": 25084, "distractor": true, "label": "রাজুর কলম"},
    {"id": "n3", "op": "const", "val": 45.6, "distractor": false, "label": "প্রতিটি কলমের দাম"},
    {"id": "total", "op": "mul", "args": ["n1", "n3"], "distractor": false, "label": "মোট টাকা"},
    {"id": "final_result", "op": "identity", "args": ["total"], "distractor": false}
  ]
}

Supported Operations: const, add, sub, mul, div, sum, mean, min, max, floor, ceil, round, sqrt, pow, mod, gcd, lcm, identity

Quickstart

Using Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "dipta007/dagger-12B_SFT_GRPO"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

USER_PROMPT_TEMPLATE = """You are an expert Bengali Math Reasoner. Your task is to solve mathematical problems by constructing a "Computational Graph".

### Graph Rules:
- `id`: Unique identifier (e.g., "n1", "n2").
- `val`: The raw number extracted from text (for input nodes).
- `op`: The operation (`add`, `sub`, `mul`, `div`, `round`, `sqrt`, `floor`, `sum`, `mean`, `ratio_split`). Use `const` for input numbers.
- `args`: List of input node IDs.
- `distractor`: Boolean (`true` / `false`). Set to `true` if the node is NOT used in the final calculation path.
- `label`: Label for the node.

### Available Operations:
- Input: `const` (Use this for all numbers found in text or constants).
- Arithmetic: `add`, `sub`, `mul`, `div`, `abs` (absolute difference).
- Logic/Stats: `sum`, `mean`, `min` (minimum), `max` (maximum).
- Rounding: `round` (nearest int), `floor` (round down), `ceil` (round up).
- Advanced: `sqrt`, `pow`, `mod` (remainder), `gcd`, `lcm`.
- Output: `identity` ("final_result" points to the answer node)

Only output a JSON graph representing the solution, nothing else. Nodes must be topologically sorted, and there must be exactly one "final_result" node that represents the final answer. One example is provided below.

### Example:
Question:
মিনার কাছে ১২২১৯৫ টা কলম আছে। রাজুর কাছে ২৫০৮৪ টা কলম আছে। মিনা রাজুর কাছে ১১২৬ টি কলম চাইল। রাজু ১০০০ টি কলম দিতে রাজি হল, কিন্তু পরে আর দিলেনা। প্রতিটি কলমের দাম ৪৫.৬ টাকা। মিনা যদি কলমগুলো বিক্রি করতে চায়, সে কত টাকা পাবে?

Output:
```json
{{
  "nodes": [
    {{"id": "n1", "op": "const", "val": 122195, "distractor": false, "label": "মিনার কলম"}},
    {{"id": "n2", "op": "const", "val": 25084, "distractor": true, "label": "রাজুর কলম"}},
    {{"id": "n3", "op": "const", "val": 1126, "distractor": true, "label": "মিনা রাজুর কাছে চাইল"}},
    {{"id": "n4", "op": "const", "val": 1000, "distractor": true, "label": "রাজু দিতে রাজি হল"}},
    {{"id": "n5", "op": "const", "val": 45.6, "distractor": false, "label": "প্রতিটি কলমের দাম"}},
    {{"id": "total_money", "op": "mul", "args": ["n1", "n5"], "distractor": false, "label": "মিনার মোট টাকা"}},
    {{"id": "final_result", "op": "identity", "args": ["total_money"], "distractor": false, "label": "চূড়ান্ত উত্তর"}}
  ]
}}```

### Your Task:

Question:
{question}

Output:
"""

question = "রজারের 5টি টেনিস বল আছে। সে আরও 2 ক্যান টেনিস বল কিনেছে। প্রতিটি ক্যানে 3টি করে টেনিস বল আছে। তার কাছে এখন কতগুলি টেনিস বল আছে?"
prompt = USER_PROMPT_TEMPLATE.format(question=question)

messages = [
  {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

# Generate
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, top_p=0.8)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)

print(response)

Using vLLM

vllm serve dipta007/dagger-12B_SFT_GRPO --max-model-len 4096

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="dipta007/dagger-12B_SFT_GRPO",
    messages=[
        {"role": "system", "content": "You are an expert Bangla Math Reasoner..."},
        {"role": "user", "content": "মিনার কাছে ১০০টি কলম আছে..."}
    ],
    max_tokens=1024
)

Graph Execution

import json

def execute_graph(graph_json):
    """Execute a computational graph and return the final result."""
    nodes = {n["id"]: n for n in graph_json["nodes"]}
    cache = {}

    def compute(node_id):
        if node_id in cache:
            return cache[node_id]

        node = nodes[node_id]
        op = node["op"]

        if op == "const":
            result = node["val"]
        elif op == "add":
            result = sum(compute(arg) if isinstance(arg, str) else arg for arg in node["args"])
        elif op == "sub":
            args = [compute(arg) if isinstance(arg, str) else arg for arg in node["args"]]
            result = args[0] - args[1]
        elif op == "mul":
            result = 1
            for arg in node["args"]:
                result *= compute(arg) if isinstance(arg, str) else arg
        elif op == "div":
            args = [compute(arg) if isinstance(arg, str) else arg for arg in node["args"]]
            result = args[0] / args[1]
        elif op == "identity":
            result = compute(node["args"][0])
        # ... add other operations

        cache[node_id] = result
        return result

    return compute("final_result")

# Parse and execute
graph = json.loads(response)
answer = execute_graph(graph)
print(f"Answer: {answer}")

Training Details

Stage 1: Supervised Fine-Tuning (SFT)

Parameter	Value
Base Model	Gemma-3-12B-Instruct
LoRA Rank / Alpha	64 / 128
Global Batch Size	256
Epochs	4
Learning Rate	1e-5 → 1e-6 (cosine)
Training Data	3,000 examples

Stage 2: Group Relative Policy Optimization (GRPO)

Parameter	Value
Base Model	SFT Checkpoint
LoRA Rank / Alpha	64 / 128
Global Batch Size	32
Generations per Prompt	8
Epochs	4
Loss Type	BNPO
β / ε / ε_high	0.0 / 0.2 / 0.28

Reward Function:

R(g, y) = 0.5 * I_fmt + 0.5 * I_exec + I_acc(exec(g), y)

I_fmt: Valid JSON format (+0.5)
I_exec: Successful execution (+0.5)
I_acc: Correct answer (+1.0)

Best Practices

Temperature: Use temperature=0.7 with top_p=0.8 for best results
Max Tokens: 1024 tokens is sufficient for most problems
System Prompt: Include the graph generation instructions in system message
Post-processing: Parse JSON and execute graph for final numeric answer

Limitations

Designed for arithmetic word problems; may not generalize to algebra, geometry, or calculus
Primarily trained on Bangla; English performance not evaluated
Requires JSON parsing and graph execution for final answers
4B variant shows lower performance, suggesting capacity requirements

Related Models

Model	Training	Weighted Avg
dagger-12B_SFT_GRPO	SFT → GRPO	69.4
dagger-12B_SFT	SFT only	66.7
dagger-12B_GRPO	Base → GRPO	69.4
dagger-4B_SFT_GRPO	SFT → GRPO	47.3

Citation

@misc{nazi2026dagdaggerdistractorawaregraphgeneration,
      title={{\dag}DAGGER: Distractor-Aware Graph Generation for Executable Reasoning in Math Problems}, 
      author={Zabir Al Nazi and Shubhashis Roy Dipta and Sudipta Kar},
      year={2026},
      eprint={2601.06853},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2601.06853}, 
}

Acknowledgments

Google Gemma for the base model
Unsloth for efficient fine-tuning
TRL for GRPO implementation