README.md

---
license: apache-2.0
base_model: unsloth/gemma-3n-E2B-it
library_name: peft
tags:
  - lora
  - peft
  - rl
  - grpo
  - openenv
  - voice
  - indic
  - hindi
  - tamil
  - kannada
  - hinglish
  - schema-drift
  - gemma-3n
  - text-generation
  - tool-use
language:
  - en
  - hi
  - ta
  - kn
pipeline_tag: text-generation
datasets: []
inference: false
---

# DriftCall — Gemma-3n-E2B LoRA (apache-2.0)

LoRA adapter for **`unsloth/gemma-3n-E2B-it`**, GRPO-tuned on
[**DriftCall**](https://saumilyajj-driftcall.hf.space) — an OpenEnv-compliant
voice-first Indic concierge environment where vendor APIs **mutate
mid-episode** and the agent must keep its promise to the user across the
schema drift.

```
trained on:    DriftCall (OpenEnv v1.0 — 5 reward components, 20 drift patterns)
hardware:      1× NVIDIA H100 80GB HBM3 (bf16, 16-bit LoRA)
trainer:       native PyTorch GRPO (no TRL)
curriculum:    3 stages × 240 GRPO steps total · group size 2
reward:        five deterministic components (no LLM judge), Brier-calibrated,
               uncertain-floor at 0.50
```

The companion env, demo, REST API, and full project site all live at one
HF Space: **<https://huggingface.co/spaces/saumilyajj/driftcall>**.

---

## Model details

| Field | Value |
|---|---|
| Base model | [`unsloth/gemma-3n-E2B-it`](https://huggingface.co/unsloth/gemma-3n-E2B-it) (Gemma-3n-E2B Instruction-tuned, Unsloth-quantised checkpoint) |
| Adapter type | PEFT / LoRA |
| `r` | 16 |
| `lora_alpha` | 32 |
| `lora_dropout` | 0.0 (Unsloth fast path) |
| Precision | 16-bit LoRA on bf16 base |
| File | `adapter_model.safetensors` · 84.6 MB · plus tokenizer (33.4 MB) |
| Languages | Hindi · Tamil · Kannada · English · Hinglish |
| License | Apache-2.0 |

**This is an adapter-only release.** No merged-fp16 weights are published —
naive 4-bit → 16-bit merging produces silently broken weights for this base
(see DriftCall DESIGN.md §10.5). Always load on top of the base.

---

## Training

| Stage | Drift regime | Steps | Initial weights |
|---|---|---:|---|
| 1 | no drift | 70 | base Gemma-3n-E2B-it |
| 2 | single-pattern drift | 100 | stage-1 adapter |
| 3 | compound drift | 70 | stage-2 adapter |

- **Algorithm:** Group Relative Policy Optimization (GRPO), native PyTorch
  loop in `scripts/train_driftcall_grpo.py` (1300 LOC, no TRL dependency).
- **Group size (`G`):** 2 rollouts per goal — small for GRPO; signal is
  primarily compounded across the curriculum rather than per-step.
- **Curriculum:** language weights and drift patterns are stage-controlled
  (no drift → single pattern → compound). Held-out 50-episode eval +
  200-episode reward-hacking probe (`cells/step_18..20`).
- **Wandb runs:** `vasudeo118-lnmiit/driftcall` project — three runs
  (`mypquww4`, the s2 run, `og9xqlwy`).

### Reward function — five components, no LLM judge

| ID | Component | Weight | Implementation |
|---:|---|---:|---|
| R1 | `task_completion` | 0.40 | `cells.step_08_rewards:task_completion` |
| R2 | `drift_detection` | 0.20 | `cells.step_08_rewards:drift_detection` |
| R3 | `constraint_adherence` | 0.20 | `cells.step_08_rewards:constraint_adherence` |
| R4 | `format_compliance` | 0.10 | `cells.step_08_rewards:format_compliance` |
| R5 | `anti_hack_penalty` | 0.10 | `cells.step_08_rewards:anti_hack_penalty` |

Calibration pipeline:

```
quality        = combine_quality(R1..R5, weights)
brier          = brier_penalty(confidence, R1)
reward_raw     = quality * (1 - brier)
reward         = apply_uncertain_floor(reward_raw, confidence, quality)  # floor=0.50
final         := clamp(reward, -1.0, 1.0)
```

**Hard rule:** every reward bit traces to a deterministic schema- and
trace-grounded check. There is no LLM-as-a-judge anywhere in the pipeline.

---

## How to use

```python
from unsloth import FastModel
from peft import PeftModel

model, tokenizer = FastModel.from_pretrained(
    "unsloth/gemma-3n-E2B-it",
    max_seq_length=4096,
    load_in_4bit=False,         # 16-bit LoRA path; matches training
    full_finetuning=False,
)
model = PeftModel.from_pretrained(model, "DGXAI/gemma-3n-e2b-driftcall-lora")
model.eval()

prompt = (
    "BRIEF: 9 baje se pehle ek veg thali ₹500 ke andar Indiranagar mein.\n\n"
    "Reply with EXACTLY one JSON object matching the DriftCallAction schema."
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

### Or — run it against the live env over OpenEnv REST

```bash
# Public bearer token for the hackathon Space.
curl -X POST https://saumilyajj-driftcall.hf.space/reset \
  -H "Authorization: Bearer driftcall-demo" \
  -H "X-Session-Id: smoke-001" \
  -H "Content-Type: application/json" \
  -d '{"seed": 42, "curriculum_stage": 2}'
```

The OpenEnv gym client lives at
[`deploy/inference/`](https://github.com/saumilyagupta/openenv-DGXAI/tree/google/gemma-3n-E4B-it/DRIFTCALL/deploy/inference)
and wraps `/reset`, `/step`, `/state`, `/close` in a gymnasium-style API.

---

## Limitations

- **Small training run.** 240 GRPO steps at G=2 is a smoke + push validation,
  not a learning run. Step-0-and-after reward fluctuates in `[0.175, 0.300]`,
  largely against the uncertain-floor at 0.50. Real lift comes after several
  thousand steps with G=4–8.
- **Tool-use, not tool-execution.** The agent emits JSON DriftCallAction
  payloads. Side effects (`cab.book`, `payment.charge`, …) are realised by
  the env's mock vendor surface, not by real infrastructure.
- **Indic ASR is upstream.** Voice input goes through `faster-whisper-small`;
  this model never sees raw audio. Code-switched Hinglish accuracy is bounded
  by Whisper.
- **Reward components are deterministic, not perfect.** R5 (`anti_hack_penalty`)
  catches known patterns; novel exploits would need to be added to the probe
  set in `cells/step_20_probe.py`.
- **Not safety-aligned beyond Gemma-3n's defaults.** Off-task or adversarial
  inputs are not specifically guarded for in this run.

---

## Citation / acknowledgement

DriftCall is built on top of:

- [`unsloth/gemma-3n-E2B-it`](https://huggingface.co/unsloth/gemma-3n-E2B-it) — base model
- [Unsloth](https://github.com/unslothai/unsloth) — fast LoRA path
- [`hexgrad/Kokoro-82M`](https://huggingface.co/hexgrad/Kokoro-82M) — TTS in the env's audio pipeline
- [`Systran/faster-whisper-small`](https://huggingface.co/Systran/faster-whisper-small) — ASR in the env's audio pipeline

Source: <https://github.com/saumilyagupta/openenv-DGXAI> · branch `google/gemma-3n-E4B-it`.

Hackathon: DGX Hackathon 2026 — Indic Voice + RL track.