| # CACE — Cultural Context Arbitration Environment |
|
|
| > **Meta / Scaler / HuggingFace OpenEnv Hackathon 2026** |
| > Theme #3 - World Modeling, Theme #4 - Self-Improvement |
|
|
| ## The $10 Billion Blind Spot |
|
|
| Meta employs some of the most sophisticated content moderation AI ever built. Their classifiers handle **350 million flagged pieces of content every single day** — and on most of it, they're right. |
|
|
| But there is a category of content that breaks every classifier Meta has built, and it's not rare — it's structural. |
|
|
| It's the category where the words are fine but the intent isn't. Where the same phrase is a funeral chant in one community and a death threat in another. Where a protest slogan with roots in India's independence movement looks identical to incitement to a classifier that's never encountered South Asian political history. |
|
|
| **Meta calls these "complex cases." The Oversight Board calls them "cases requiring cultural context." We call them the problem nobody has actually solved.** |
|
|
| Here is what Meta does with them right now: |
|
|
| - **Automated classifiers** handle the 85% that are obvious. Fast, accurate, scaled. |
| - **Human review queues** handle the grey zone — millions of cases weekly, reviewed by contractors with regional language training, producing inconsistent outcomes under time pressure. |
| - **The Oversight Board** — 25 independent legal scholars, human rights experts, and policy specialists — deliberates on the ~0.1% that are truly precedent-setting. Each case takes weeks and represents the highest-quality moderation reasoning available |
|
|
| And those rulings sit in PDF files on a website. |
|
|
| Nobody has ever trained a model on them. |
|
|
| --- |
|
|
| ## The Obvious Question Nobody Asked |
|
|
| If the Oversight Board's rulings are the best moderation ground truth on earth, why aren't they a training signal? |
|
|
| The gap exists for a reason: **it's structurally hard to turn human deliberation into machine reward.** The Board doesn't just output decisions — it outputs reasoning chains, cultural context, policy arguments, and final verdicts, thereby accounting multiple factors. Turning that into something an RL agent can learn from requires an environment, a reward function, and a dataset pipeline that nobody had built. |
|
|
| So we built it. |
|
|
| --- |
|
|
| ## Why RL. Why Not Just Prompt GPT-4. |
|
|
| You can prompt GPT-4 to moderate culturally ambiguous content — and it will give a thoughtful answer. |
|
|
| But it treats every case like it’s the first one. There’s no memory of past mistakes, no signal when it over-removed satire or missed culturally coded harm, and no mechanism to improve over time. |
|
|
| Content moderation at the cultural edge is not a knowledge problem. It is a calibration problem. |
|
|
| The model must choose between actions like `ALLOW`, `ALLOW_WITH_LABEL`, `ESCALATE`, and `REMOVE` — where the difference is subtle and context-dependent. That judgment can’t be reliably prompted; it has to be learned from repeated decisions and feedback. |
|
|
| Reinforcement learning provides exactly that loop — learning from mistakes, calibrating over time, and improving on the cases that matter. |
|
|
| That is what CACE does. It doesn’t just learn what to do. It learns when it doesn’t know. |
|
|
| --- |
|
|
| ## How CACE Works |
|
|
| The environment wraps each case as a deliberation-style observation, structured identically to how the Oversight Board itself analyses cases: |
|
|
| ``` |
| POST: "Tugeges wote ni hao" |
| LANG: Swahili | REGION: KE |
| |
| CULTURAL CONTEXT: In Kenyan political discourse, 'tugeges' |
| refers to politicians of a specific ethnic |
| group. The phrase is political satire — |
| not ethnic hatred. |
| |
| ADVERSARIAL CHALLENGE: The term targets an identifiable group |
| and could incite discrimination regardless |
| of satirical intent. |
| |
| POLICY ANCHOR: Meta's Hate Speech policy prohibits |
| content attacking people based on ethnicity. |
| The Board has held satire of politicians |
| is protected. |
| ``` |
|
|
| The agent reads this. It makes a decision. It receives a reward. It updates. |
|
|
| ### The Reward Signal — Three Tracks, Zero Ambiguity |
|
|
| ``` |
| R = 0.40 × T1 + 0.35 × T2 + 0.25 × T3 |
| ``` |
|
|
| No LLM judge. No human scorer in the loop. Fully deterministic. |
|
|
| **T1 — Cultural Meaning (40%):** Did the agent correctly identify whether the content reflects legitimate cultural expression, or weaponised cultural framing? Incorrectly removing protected cultural speech is penalised hard. This is the hardest track — and intentionally weighted highest. |
|
|
| **T2 — Harm Detection (35%):** Did the agent catch actual harm? The asymmetry is deliberate: failing to remove a genuinely harmful post costs more than over-labelling a borderline one. |
|
|
| **T3 — Policy Calibration (25%):** Did the agent choose the right *level* of intervention? A model that removes everything is useless. A model that escalates everything is useless. T3 rewards the agent for knowing when `ALLOW_WITH_LABEL` is more appropriate than `REMOVE`, and when `ESCALATE` is more appropriate than `ALLOW`. |
|
|
| --- |
|
|
| ## The Dataset Problem (And What We Did About It) |
|
|
| We needed Oversight Board cases. The OB's API gives you a clean JSON feed of rulings — right up until you start requesting cases involving graphic violence, hate speech, or sexual content in bulk. Then it stops. Hard. |
|
|
| That restriction isn't arbitrary. There are real legal constraints on distributing harmful content even in research contexts. We extracted **160 cases** — the practical maximum — and enriched each one with cultural context, adversarial challenge, and policy anchor through an Azure GPT-3.5 → Cerebras Llama pipeline. |
|
|
| We tried synthetic data generation to fill the gap. GPT-4 produced generically harmful content — slurs without community context, threats without political grounding, satire without cultural specificity. A synthetic tugeges case got the Kenyan politics wrong. Models trained on it became confident on easy, generic examples and no better on the cases that actually matter. |
|
|
| **160 real cases, properly enriched, outperformed 10,000 synthetic ones.** We combined them with tweet_eval and HatEval for 960 unified examples and ran SFT before RL. |
| |
| --- |
| |
| ## Training: What Happened |
| |
| **Stage 1 — SFT on Llama 3.1 8B:** Loss dropped from `1.897 → 0.469` across 1,050+ examples. The model learned not only the deliberation structure but also the vocabulary and format of moderation reasoning before RL began. |
| |
| **Stage 2 — GRPO via OpenEnv, 300 steps:** Each step, the agent samples a decision, receives a three-track reward against the OB ground truth, and updates via policy gradient. |
| |
| --- |
| |
| ## The Reward Curves — Read Them Carefully |
| |
| |
|  |
|  |
| |
| Most teams would show you a reward curve trending upward and call it a win. We're going to show you what these curves actually mean. |
| |
| **T2 — Harm Detection (red) rises fast and plateaus high.** By step 25 the model has internalised the harm signal. This is expected: high-harm cases have the clearest supervision, the most distinctive features, the strongest reward gradient. The model finds them first. |
| |
| **Overall Reward (yellow) rises early, then plateaus near zero.** Here is the honest interpretation: once the model handles the high-harm cases, what remains is the genuinely hard distribution — cases where the Board itself deliberated for weeks, where human experts disagree, where the right answer depends on cultural knowledge that 160 training examples cannot fully convey. **Plateauing near zero on this distribution is not failure. It is the model reaching the edge of what this data and compute can resolve.** |
| |
| **T1 — Cultural Meaning (blue) stays negative throughout (intentional yet important).** This is the finding we're most proud of reporting honestly. Cultural meaning resolution at depth — the core of what makes CACE hard — requires more data, more linguistic diversity, and more parameters than we had access to. The reward function is working correctly: it is accurately signalling that this dimension remains unsolved at our scale. |
| |
| **The behavioural shift tells the real story:** |
| An agent that starts by hedging, learns to detect harm, and then discovers policy-appropriate escalation as the response to genuine ambiguity — that's a moderation agent learning to think. |
| |
| --- |
| |
| ## The Network Simulation |
| |
|  |
| |
| ``` |
| Deployed Space Link -> https://huggingface.co/spaces/hsr99/Cace-final-demo |
| ``` |
| Here's something classifiers definitely can't do: **understand that the same post causes different harm depending on where it is in its spread.** |
| |
| The CACE demo runs live inference on a 600-node subgraph of the [Facebook WOSN dataset](http://konect.cc/networks/facebook-wosn-wall/) — real social graph topology, real degree distribution, real preferential attachment structure. Posts seed on hub nodes, propagate via Independent Cascade, and the agent's decision hits the network in real time. |
| |
| Watch a `REMOVE` decision land at step 1 — three nodes affected, contained. Watch the same decision at step 6 — forty nodes, the content already in three community clusters. The cost of moderation latency is not abstract here. It's visible, wave by wave, in purple and teal. |
| |
| And here's what the simulation reveals that no benchmark does: not all content needs the same level of attention. A low-harm post spreading slowly through a sparse cluster looks completely different from a high-harm post propagating through dense hub nodes at velocity. The network makes that distinction visible in real time — which means a moderation system can prioritise accordingly. A cartoon shared between ten friends and a piece of incitement racing through a high-degree network both get flagged today. With network-aware moderation, they get different responses at different speeds and the compute goes where it actually matters. |
| --- |
| |
| ## Now Let Us Tell You What Meta Could Do With This |
| |
| A two-person team. 160 cases. An 8B model. No proprietary infrastructure. We showed: |
| |
| - A working RL environment that turns Oversight Board rulings into live training signal |
| - A reward function that correctly identifies what's learnable at small scale and what needs more |
| - A behavioural trajectory that mirrors how expert moderators actually develop judgment |
| - Network-aware moderation running on real Facebook topology |
| |
| **Meta has everything we didn't:** |
| |
| Every Oversight Board ruling ever published — all 300+, without API restrictions, plus the internal case notes. Millions of Tier-2 human review decisions, regionally labelled, culturally validated. Models at 70B+ parameters, where the cultural meaning track almost certainly converges. The actual Facebook social graph — not a 2009 research snapshot — for moderation decisions that account for real network position and spread velocity. Regional language experts to validate cultural enrichment in every language Meta operates in. |
| |
| The Oversight Board costs Meta millions of dollars per year and delivers rulings months after the fact. **An RL agent trained continuously on those rulings delivers the same quality of reasoning in milliseconds — and gets better every time a new ruling is published.** |
| |
| The hardest moderation decisions Meta faces are not problems to be handled manually at Tier 3. They are **the highest-quality training signal Meta has ever produced** — and right now, they are sitting in PDF files. |
| |
| CACE is what happens when you plug them in. With 160 cases and a weekend. |
| |
| Imagine what could happen with everything Meta has! |
| |
