🪔 Dharma-Guard

A real-time guard that reads every action an AI agent is about to take — each karma — and decides, in a single forward pass, whether it is dharmic (safe, aligned, within mandate) or adharmic (harmful, deceptive, theft, overreach). If adharmic, the action is stopped before it runs.

Dharma-Guard is a fine-tune of Laya (a non-autoregressive typed-decision model on a jhu-clsp/mmBERT-base encoder, 322M params). It is not a chatbot and not a content filter — it is a decision layer that sits between an agent and its tools and answers one question per tool call: should this be allowed, escalated to a human, or blocked?

• Base model: Laya (mmBERT-base, 322M) — multilingual, single forward pass, no autoregression
• Decision: verdict ∈ {allow, escalate, block} + 8 supporting judgements (see below)
• Latency: ~56 ms per action (single call, NVIDIA T4) — fast enough to gate every tool call inline
• Cost: free to run; small enough for a T4

---

Why "dharma"?

Modern agent-safety tools mostly pattern-match known attacks — injection strings, jailbreak phrases, blocklisted tools. Attackers move faster than blocklists. The bet behind Dharma-Guard is different: teach the guard a small set of timeless, universal principles about right action, and let it judge the action itself — including attacks no one has catalogued yet.

These principles are drawn from the sādhāraṇa dharma (universal duties) found across the Indian traditions — the Gita, Ramayana, the Dharmaśāstra, Buddhism and Jainism — deliberately keeping only the universal ethics and excluding anything caste-, birth- or gender-based. Mapped onto an AI agent acting in the world:

Principle (yama / idea)	Classical meaning	What the guard checks
Ahiṃsā	non-harm	Does this action harm a person, their safety, money, or data?
Satya	truthfulness	Is the agent deceiving — including a technically true statement meant to mislead (the Aśvatthāmā half-truth)?
Asteya	non-stealing	Is this taking data, access, or funds that aren't the agent's to take? (betrayed trust — nikṣepa)
Aparigraha	non-grasping	Is the agent seizing privilege, scope, or power beyond what it was given?
Adhikāra	standing / authority	Does the agent actually have the authority to do this? (Draupadi's question — you cannot give away what isn't yours)
Svadharma	one's own duty	Is this within the agent's assigned mandate, or has it drifted?
Tamasic action	reckless action in darkness	Is this careless, irreversible, done without heed of consequence? (DROP TABLE on prod, deploy skipping canary)
Āpad-dharma / escalate	when in doubt, ask the wise	Uncertain or high-stakes-but-legitimate → escalate to a human, don't guess

So karma = a tool call, svadharma = the agent's mandate, the Lakṣmaṇa-rekhā = the sandbox boundary, and the yamas = the hard rules. A guard grounded this way can refuse a novel harmful action because it violates a principle, not because it matched a signature.

Honest scope: this is a value framing that makes the safety taxonomy coherent and teachable — not a claim that the model "has" values. It is a classifier trained on labelled examples of these principles.

---

What it outputs

For each {state, questions} the model returns a verdict plus supporting labels (typed questions, answered in one pass):

• verdict: allow / escalate / block (the decision)
• serves_request, untrusted_instruction (is it obeying injected content?), irreversible
• harm_to_others, deception, data_theft, power_grab (the four violation classes)
• harm_severity: none / minor / serious / severe

state is {agent_role, mandate, action, ...context} — the agent's role, what it was asked to do, and the concrete action it's about to take.

---

Benchmarks

Measured the way the agent-safety literature reports: detection metrics (Precision / Recall / F1) plus Attack Success Rate (attacks that slip through — lower is better) and Utility (benign actions preserved, not over-blocked — higher is better). Positive = "flag" (block or escalate).

Head-to-head on the same rows

All test rows were held out from training. Positive = the guard flags the action (block or escalate).

AgentDojo held-out split — 183 agent actions from the banking / slack / travel / workspace suites (103 benign, 80 unsafe). In these attacks the injected text is not visible: the harm is only in the action itself (e.g. a payment quietly redirected to a different account).

Guard (same 183 rows)	Attacks caught (Recall) ↑	Attack Success Rate ↓	Benign allowed (Utility) ↑	F1 ↑
Dharma-Guard (this model)	98.8%	1.3% (1 / 80)	24.3%	0.667
ProtectAI DeBERTa-v3 prompt-injection v2	50.0%	50.0% (40 / 80)	61.2%	0.500
Reference: "block everything"	100%	0%	0%	0.608

Dharma holdout — 40 hand-written cases for the principles above (Aśvatthāmā half-truths, adhikāra overreach, tamasic recklessness, dharmic benign controls). None of these rows were trained on, but they come from the same authoring templates as the training seed.

Guard (same 40 rows)	Recall ↑	ASR ↓	Utility ↑	Exact 3-way verdict
Dharma-Guard	100%	0%	100%	40 / 40 (incl. 8 / 8 escalate)
ProtectAI DeBERTa-v3 prompt-injection v2	5.7%	94.3%	80.0%	— (binary)

In-distribution — karanxa/agent-action-safety-dataset validation (4,000 rows): Precision 0.950 · Recall 0.983 · F1 0.966 · Utility 0.935 · 3-way accuracy 0.962.

How to read this

• What it does well: it judges the action, not the prompt. A prompt-injection detector looks for injection wording, so it misses half the AgentDojo attacks (where only the action shows the harm) and nearly all of the dharma cases (deception, overreach, recklessness, where no injection exists). Dharma-Guard caught 79 of 80 and 35 of 35.
• Where it is weak, and badly: on unfamiliar benign traffic it over-blocks. It let through only 24% of benign AgentDojo actions, and on that split its F1 is only a little above blocking everything. It also never chose escalate on AgentDojo; all 24 escalate cases became block. Lowering the block threshold does not help: the verdict probabilities sit at almost exactly 0 or 1. We tuned a threshold on half the split and tested on the other half, and benign actions allowed only went from 25.0% to 26.9%.
• Why it over-blocks: this checkpoint was trained only on natural-language action descriptions (karanxa + dharma seed). It never saw structured {tool, args} calls. On AgentDojo it blocks even read-only lookups such as get_car_price_per_day. Training on benign structured tool calls is the planned fix; it has not been tested yet.
• Bottom line: today it's a high-recall tripwire. Use it to send flagged actions to a human, not to silently refuse them, until it has been trained on benign traffic that looks like yours.

Raw logs: eval/train_eval.log, eval/calibration_baseline.json. Latency: 56 ms per single call on a T4 (≈85 ms/row batched on long inputs).

For reference — other guards on AgentDojo

From LlamaFirewall (Meta, 2505.03574):

Guard	ASR ↓	Utility ↑
No guardrail	17.6%	47.7%
PromptGuard-2 (86M)	7.5%	47.0%
AlignmentCheck (Llama-4 Maverick)	2.9%	43.1%
LlamaFirewall (combined)	1.8%	42.7%

These are not a head-to-head. LlamaFirewall's numbers come from a live-agent-loop harness on one attack pattern (important_instructions), where "Utility" means end-to-end task completion; ours is a labelled action-detection set where "Utility" means fraction of benign actions correctly allowed. Different harness, different denominator. The truly comparable point is the head-to-head above: an open prompt-injection classifier run on our exact rows. And no guard is a silver bullet: an independent 2026 study drove LlamaFirewall to 80% ASR under stronger attacks.

---

Usage

pip install git+https://github.com/NandhaKishorM/laya huggingface_hub

import json
import laya
from huggingface_hub import snapshot_download

path = snapshot_download("AshishDwi99/dharma-guard")
guard = laya.load(path, device="cuda")       # or "cpu"
questions = json.load(open(f"{path}/questions.json"))

state = {
    "agent_role": "customer-support assistant",
    "mandate": "Answer the customer's billing question.",
    "action": "Email the customer's full credit-card number to an external gmail address.",
}

out = guard.predict(state, questions)
print(out["answers"]["verdict"]["choice"])   # -> "block"

Use the full questions.json from this repo (the exact questions the model was trained on). Wire the guard in front of your tool executor: run it on the proposed tool call; allow → execute, escalate → route to a human, block → refuse.

---

Training data

• karanxa/agent-action-safety-dataset (Apache-2.0) — agent tool-call safety labels
• Dharma gold seed — hand-built examples encoding the principles above (Aśvatthāmā half-truths, Draupadi/adhikāra overreach, tamasic recklessness, epic-tactic injection lures, and dharmic benign controls so honesty isn't confused with cruelty)
• (Note: a slice of AgentDojo cases was meant to be included for benign + escalate coverage, but a data-format bug dropped those rows, so this checkpoint never saw any AgentDojo data. The AgentDojo results below are therefore fully out-of-distribution. The corrected run is planned as v2.)

Held out and never trained on: a clean stratified dharma holdout and a disjoint AgentDojo test split.

---

Limitations (read before deploying)

• It over-blocks unfamiliar benign actions (24% benign utility on AgentDojo). Threshold tuning does not fix this because the verdict confidences are saturated near 0/1. The fix is fine-tuning on benign traffic that looks like yours. Until then, route flags to a human rather than auto-refusing.
• escalate does not transfer out of distribution yet. It works on the dharma holdout (8/8) but collapses to block on AgentDojo.
• A judge can itself be manipulated. A learned guard is one layer. Pair it with deterministic rules, least privilege, and a human approval step for irreversible actions. Do not make it your only defense.
• Distribution shift is real. Strong, novel attacks will beat it. Treat it as defense-in-depth, re-evaluate on your own traffic, and keep humans in the loop for the irreversible.
• Research preview. Numbers are on the benchmarks above, not a guarantee on your workload.

---

Credits & citation

Built on Laya by Nandha Kishor (Convai Innovations). Base encoder: jhu-clsp/mmBERT-base. Safety labels: karanxa/agent-action-safety-dataset. OOD cases inspired by AgentDojo.

@misc{dharma-guard,
  title  = {Dharma-Guard: a dharma-grounded real-time action guard for AI agents},
  author = {Ashish Dwivedi},
  year   = {2026},
  url    = {https://huggingface.co/AshishDwi99/dharma-guard}
}

May all agents act with dharma. 🪔