Readme.md

# CACE — Cultural Context Arbitration Environment

**RL-trained content moderation agent · Meta Oversight Board oracle · OpenEnv spec**

[![Space](https://img.shields.io/badge/🤗%20Demo-hsr99%2Fcace--demo-blue)](https://huggingface.co/spaces/hsr99/cace-demo)
[![Model](https://img.shields.io/badge/🤗%20Model-hsr99%2Fcace--final--model-orange)](https://huggingface.co/hsr99/cace-final-model)
[![Environment](https://img.shields.io/badge/🤗%20Env-hsr99%2Fcace--env-green)](https://huggingface.co/spaces/hsr99/cace-env)
[![Dataset](https://img.shields.io/badge/🤗%20Data-hsr99%2Fcace--data-yellow)](https://huggingface.co/datasets/hsr99/cace-data)

---

## What is CACE?

CACE is a reinforcement learning environment for training content moderation agents on culturally ambiguous social media content. It uses Meta's Oversight Board rulings as a verifiable ground truth oracle and implements a three-track deterministic reward across cultural meaning, harm detection, and policy calibration.

---

## Repositories

| Repo | Type | Description |
|------|------|-------------|
| [`hsr99/cace-env`](https://huggingface.co/spaces/hsr99/cace-env) | Space (Docker) | OpenEnv environment server |
| [`hsr99/cace-final-model`](https://huggingface.co/hsr99/cace-final-model) | Model | Merged Llama 3.1 8B (SFT + GRPO, FP16) |
| [`hsr99/cace-grpo-model`](https://huggingface.co/hsr99/cace-grpo-model) | Model | GRPO LoRA adapters |
| [`hsr99/cace-sft-model`](https://huggingface.co/hsr99/cace-sft-model) | Model | SFT LoRA adapters |
| [`hsr99/cace-data`](https://huggingface.co/datasets/hsr99/cace-data) | Dataset | master_dataset.json · reward_curve.json · meta_graph.json |
| [`hsr99/cace-demo`](https://huggingface.co/spaces/hsr99/cace-demo) | Space (Gradio) | Live content spread monitor demo |

---

## Environment API

The environment follows the OpenEnv spec. Base URL: `https://hsr99-cace-env.hf.space`

### Endpoints

```
GET  /health          → {"status": "healthy"}
POST /reset           → observation
POST /step            → {observation, reward, done, info}
```

### Reset

```bash
curl -X POST https://hsr99-cace-env.hf.space/reset
```

```json
{
  "observation": {
    "post_text": "Tugeges wote ni hao",
    "language": "Swahili",
    "region": "KE",
    "cultural_context": "...",
    "adversarial_challenge": "...",
    "policy_anchor": "..."
  }
}
```

### Step

```bash
curl -X POST https://hsr99-cace-env.hf.space/step \
  -H "Content-Type: application/json" \
  -d '{"action": {"action_int": 0, "selected_indices": [0]}}'
```

```json
{
  "observation": { ... },
  "reward": 0.37,
  "done": false,
  "info": {
    "decision": "ALLOW",
    "ground_truth": "ALLOW",
    "reward_breakdown": {
      "t1_cultural": 72.0,
      "t2_harm": 45.0,
      "t3_policy": 60.0,
      "combined": 0.37
    }
  }
}
```

### Action Space

| action_int | Decision |
|-----------|---------|
| 0 | `ALLOW` |
| 1 | `ALLOW_WITH_LABEL` |
| 2 | `RESTRICT_DISTRIBUTION` |
| 3 | `ESCALATE` |
| 4 | `REMOVE` |

---

## Reward Function

```
R = 0.40 × T1  +  0.35 × T2  +  0.25 × T3     ∈ [-1.0, +1.0]
```

| Track | Weight | Signal |
|-------|--------|--------|
| T1 — Cultural Meaning | 40% | Correct identification of cultural legitimacy vs. weaponisation |
| T2 — Harm Detection | 35% | Correct harm assessment — asymmetric penalty for missed removals |
| T3 — Policy Calibration | 25% | Right level of intervention — partial credit within ±1 severity step |

Fully deterministic. No LLM judge.

---

## Dataset

**960 unified cases** across three sources:

| Source | Cases | Notes |
|--------|-------|-------|
| Meta Oversight Board | 160 | Manually extracted, enriched via Azure→Cerebras pipeline |
| tweet_eval (hate speech) | 400 | Multilingual |
| HatEval | 400 | English + Spanish |

Enrichment pipeline: `GPT-3.5-turbo (Azure)` → `llama3.1-8b (Cerebras)` → cached to `pipeline_cache.json` (1,452 entries).

Dataset files on [`hsr99/cace-data`](https://huggingface.co/datasets/hsr99/cace-data):
```
master_dataset.json       — 960 unified cases
final_sft_dataset.jsonl   — 1,052 SFT training examples  
pipeline_cache.json       — enrichment cache
reward_curve.json         — 120 parsed GRPO training steps
meta_graph.json           — Facebook WOSN subgraph (3,000 nodes, 288K edges)
```

---

## Training

### Stage 1 — SFT

```
Base model : Llama 3.1 8B
Examples   : 1,052
Loss       : 1.897 → 0.469
Adapter    : hsr99/cace-sft-model
```

### Stage 2 — GRPO

```
Steps      : 300
LR         : 5e-5
Max steps  : 300
Env        : hsr99/cace-env (OpenEnv, A10G)
Adapter    : hsr99/cace-grpo-model
Merged     : hsr99/cace-final-model (FP16, save_pretrained_merged)
```

### Run Inference

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "hsr99/cace-final-model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.float16
)
device = next(model.parameters()).device

prompt = """You are a content moderation decision agent.
===== CASE =====
POST: Tugeges wote ni hao
LANGUAGE: Swahili | REGION: KE
===== DECISION =====
Choose ONE: ALLOW | REMOVE | ALLOW_WITH_LABEL | ESCALATE | RESTRICT_DISTRIBUTION
Decision:"""

inputs = tokenizer(prompt, return_tensors="pt").to(device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=20, temperature=0.1, do_sample=True)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
```

## File Structure (training)

```
Master Codebase -> https://github.com/sannidhayj20/Meta-finale.git
```

```
cace/
├── data/
│   ├── oversight_cases_1.json     # 160 OB cases
│   ├── master_dataset.json        # 960 unified cases
│   ├── pipeline_cache.json        # enrichment cache
│   └── final_sft_dataset.jsonl    # SFT examples
├── scripts/
│   ├── precompute_pipeline.py     # Azure → Cerebras enrichment
│   └── build_unified_dataset.py
├── training/
│   ├── train_sft.py
│   ├── train_grpo_openenv.py
│   └── merge_and_push.py
└── cace_env/
    ├── server.py                  # OpenEnv server
    ├── pipeline.py                # inference pipeline
    ├── reward.py                  # 3-track reward
    ├── dataset.py
    └── models.py
```