Readme.md

CACE — Cultural Context Arbitration Environment

RL-trained content moderation agent · Meta Oversight Board oracle · OpenEnv spec

---

What is CACE?

CACE is a reinforcement learning environment for training content moderation agents on culturally ambiguous social media content. It uses Meta's Oversight Board rulings as a verifiable ground truth oracle and implements a three-track deterministic reward across cultural meaning, harm detection, and policy calibration.

---

Repositories

Repo	Type	Description
hsr99/cace-env	Space (Docker)	OpenEnv environment server
hsr99/cace-final-model	Model	Merged Llama 3.1 8B (SFT + GRPO, FP16)
hsr99/cace-grpo-model	Model	GRPO LoRA adapters
hsr99/cace-sft-model	Model	SFT LoRA adapters
hsr99/cace-data	Dataset	master_dataset.json · reward_curve.json · meta_graph.json
hsr99/cace-demo	Space (Gradio)	Live content spread monitor demo

---

Environment API

The environment follows the OpenEnv spec. Base URL: https://hsr99-cace-env.hf.space

Endpoints

GET  /health          → {"status": "healthy"}
POST /reset           → observation
POST /step            → {observation, reward, done, info}

Reset

curl -X POST https://hsr99-cace-env.hf.space/reset

{
  "observation": {
    "post_text": "Tugeges wote ni hao",
    "language": "Swahili",
    "region": "KE",
    "cultural_context": "...",
    "adversarial_challenge": "...",
    "policy_anchor": "..."
  }
}

Step

curl -X POST https://hsr99-cace-env.hf.space/step \
  -H "Content-Type: application/json" \
  -d '{"action": {"action_int": 0, "selected_indices": [0]}}'

{
  "observation": { ... },
  "reward": 0.37,
  "done": false,
  "info": {
    "decision": "ALLOW",
    "ground_truth": "ALLOW",
    "reward_breakdown": {
      "t1_cultural": 72.0,
      "t2_harm": 45.0,
      "t3_policy": 60.0,
      "combined": 0.37
    }
  }
}

Action Space

action_int	Decision
0	ALLOW
1	ALLOW_WITH_LABEL
2	RESTRICT_DISTRIBUTION
3	ESCALATE
4	REMOVE

---

Reward Function

R = 0.40 × T1  +  0.35 × T2  +  0.25 × T3     ∈ [-1.0, +1.0]

Track	Weight	Signal
T1 — Cultural Meaning	40%	Correct identification of cultural legitimacy vs. weaponisation
T2 — Harm Detection	35%	Correct harm assessment — asymmetric penalty for missed removals
T3 — Policy Calibration	25%	Right level of intervention — partial credit within ±1 severity step

Fully deterministic. No LLM judge.

---

Dataset

960 unified cases across three sources:

Source	Cases	Notes
Meta Oversight Board	160	Manually extracted, enriched via Azure→Cerebras pipeline
tweet_eval (hate speech)	400	Multilingual
HatEval	400	English + Spanish

Enrichment pipeline: GPT-3.5-turbo (Azure) → llama3.1-8b (Cerebras) → cached to pipeline_cache.json (1,452 entries).

Dataset files on hsr99/cace-data:

master_dataset.json       — 960 unified cases
final_sft_dataset.jsonl   — 1,052 SFT training examples  
pipeline_cache.json       — enrichment cache
reward_curve.json         — 120 parsed GRPO training steps
meta_graph.json           — Facebook WOSN subgraph (3,000 nodes, 288K edges)

---

Training

Stage 1 — SFT

Base model : Llama 3.1 8B
Examples   : 1,052
Loss       : 1.897 → 0.469
Adapter    : hsr99/cace-sft-model

Stage 2 — GRPO

Steps      : 300
LR         : 5e-5
Max steps  : 300
Env        : hsr99/cace-env (OpenEnv, A10G)
Adapter    : hsr99/cace-grpo-model
Merged     : hsr99/cace-final-model (FP16, save_pretrained_merged)

Run Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "hsr99/cace-final-model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.float16
)
device = next(model.parameters()).device

prompt = """You are a content moderation decision agent.
===== CASE =====
POST: Tugeges wote ni hao
LANGUAGE: Swahili | REGION: KE
===== DECISION =====
Choose ONE: ALLOW | REMOVE | ALLOW_WITH_LABEL | ESCALATE | RESTRICT_DISTRIBUTION
Decision:"""

inputs = tokenizer(prompt, return_tensors="pt").to(device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=20, temperature=0.1, do_sample=True)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

File Structure (training)

Master Codebase -> https://github.com/sannidhayj20/Meta-finale.git

cace/
├── data/
│   ├── oversight_cases_1.json     # 160 OB cases
│   ├── master_dataset.json        # 960 unified cases
│   ├── pipeline_cache.json        # enrichment cache
│   └── final_sft_dataset.jsonl    # SFT examples
├── scripts/
│   ├── precompute_pipeline.py     # Azure → Cerebras enrichment
│   └── build_unified_dataset.py
├── training/
│   ├── train_sft.py
│   ├── train_grpo_openenv.py
│   └── merge_and_push.py
└── cace_env/
    ├── server.py                  # OpenEnv server
    ├── pipeline.py                # inference pipeline
    ├── reward.py                  # 3-track reward
    ├── dataset.py
    └── models.py