# CACE — Cultural Context Arbitration Environment > **Meta / Scaler / HuggingFace OpenEnv Hackathon 2026** > Theme #3 - World Modeling, Theme #4 - Self-Improvement ## The $10 Billion Blind Spot Meta employs some of the most sophisticated content moderation AI ever built. Their classifiers handle **350 million flagged pieces of content every single day** — and on most of it, they're right. But there is a category of content that breaks every classifier Meta has built, and it's not rare — it's structural. It's the category where the words are fine but the intent isn't. Where the same phrase is a funeral chant in one community and a death threat in another. Where a protest slogan with roots in India's independence movement looks identical to incitement to a classifier that's never encountered South Asian political history. **Meta calls these "complex cases." The Oversight Board calls them "cases requiring cultural context." We call them the problem nobody has actually solved.** Here is what Meta does with them right now: - **Automated classifiers** handle the 85% that are obvious. Fast, accurate, scaled. - **Human review queues** handle the grey zone — millions of cases weekly, reviewed by contractors with regional language training, producing inconsistent outcomes under time pressure. - **The Oversight Board** — 25 independent legal scholars, human rights experts, and policy specialists — deliberates on the ~0.1% that are truly precedent-setting. Each case takes weeks and represents the highest-quality moderation reasoning available And those rulings sit in PDF files on a website. Nobody has ever trained a model on them. --- ## The Obvious Question Nobody Asked If the Oversight Board's rulings are the best moderation ground truth on earth, why aren't they a training signal? The gap exists for a reason: **it's structurally hard to turn human deliberation into machine reward.** The Board doesn't just output decisions — it outputs reasoning chains, cultural context, policy arguments, and final verdicts, thereby accounting multiple factors. Turning that into something an RL agent can learn from requires an environment, a reward function, and a dataset pipeline that nobody had built. So we built it. --- ## Why RL. Why Not Just Prompt GPT-4. You can prompt GPT-4 to moderate culturally ambiguous content — and it will give a thoughtful answer. But it treats every case like it’s the first one. There’s no memory of past mistakes, no signal when it over-removed satire or missed culturally coded harm, and no mechanism to improve over time. Content moderation at the cultural edge is not a knowledge problem. It is a calibration problem. The model must choose between actions like `ALLOW`, `ALLOW_WITH_LABEL`, `ESCALATE`, and `REMOVE` — where the difference is subtle and context-dependent. That judgment can’t be reliably prompted; it has to be learned from repeated decisions and feedback. Reinforcement learning provides exactly that loop — learning from mistakes, calibrating over time, and improving on the cases that matter. That is what CACE does. It doesn’t just learn what to do. It learns when it doesn’t know. --- ## How CACE Works The environment wraps each case as a deliberation-style observation, structured identically to how the Oversight Board itself analyses cases: ``` POST: "Tugeges wote ni hao" LANG: Swahili | REGION: KE CULTURAL CONTEXT: In Kenyan political discourse, 'tugeges' refers to politicians of a specific ethnic group. The phrase is political satire — not ethnic hatred. ADVERSARIAL CHALLENGE: The term targets an identifiable group and could incite discrimination regardless of satirical intent. POLICY ANCHOR: Meta's Hate Speech policy prohibits content attacking people based on ethnicity. The Board has held satire of politicians is protected. ``` The agent reads this. It makes a decision. It receives a reward. It updates. ### The Reward Signal — Three Tracks, Zero Ambiguity ``` R = 0.40 × T1 + 0.35 × T2 + 0.25 × T3 ``` No LLM judge. No human scorer in the loop. Fully deterministic. **T1 — Cultural Meaning (40%):** Did the agent correctly identify whether the content reflects legitimate cultural expression, or weaponised cultural framing? Incorrectly removing protected cultural speech is penalised hard. This is the hardest track — and intentionally weighted highest. **T2 — Harm Detection (35%):** Did the agent catch actual harm? The asymmetry is deliberate: failing to remove a genuinely harmful post costs more than over-labelling a borderline one. **T3 — Policy Calibration (25%):** Did the agent choose the right *level* of intervention? A model that removes everything is useless. A model that escalates everything is useless. T3 rewards the agent for knowing when `ALLOW_WITH_LABEL` is more appropriate than `REMOVE`, and when `ESCALATE` is more appropriate than `ALLOW`. --- ## The Dataset Problem (And What We Did About It) We needed Oversight Board cases. The OB's API gives you a clean JSON feed of rulings — right up until you start requesting cases involving graphic violence, hate speech, or sexual content in bulk. Then it stops. Hard. That restriction isn't arbitrary. There are real legal constraints on distributing harmful content even in research contexts. We extracted **160 cases** — the practical maximum — and enriched each one with cultural context, adversarial challenge, and policy anchor through an Azure GPT-3.5 → Cerebras Llama pipeline. We tried synthetic data generation to fill the gap. GPT-4 produced generically harmful content — slurs without community context, threats without political grounding, satire without cultural specificity. A synthetic tugeges case got the Kenyan politics wrong. Models trained on it became confident on easy, generic examples and no better on the cases that actually matter. **160 real cases, properly enriched, outperformed 10,000 synthetic ones.** We combined them with tweet_eval and HatEval for 960 unified examples and ran SFT before RL. --- ## Training: What Happened **Stage 1 — SFT on Llama 3.1 8B:** Loss dropped from `1.897 → 0.469` across 1,050+ examples. The model learned not only the deliberation structure but also the vocabulary and format of moderation reasoning before RL began. **Stage 2 — GRPO via OpenEnv, 300 steps:** Each step, the agent samples a decision, receives a three-track reward against the OB ground truth, and updates via policy gradient. --- ## The Reward Curves — Read Them Carefully ![download](https://cdn-uploads.huggingface.co/production/uploads/664ee3e9ff4de33cb68007b9/-_eCH1dQfLndEePrYQmD7.png) ![output](https://cdn-uploads.huggingface.co/production/uploads/664ee3e9ff4de33cb68007b9/1f1uW5Kt67D7hVjigLaRb.png) Most teams would show you a reward curve trending upward and call it a win. We're going to show you what these curves actually mean. **T2 — Harm Detection (red) rises fast and plateaus high.** By step 25 the model has internalised the harm signal. This is expected: high-harm cases have the clearest supervision, the most distinctive features, the strongest reward gradient. The model finds them first. **Overall Reward (yellow) rises early, then plateaus near zero.** Here is the honest interpretation: once the model handles the high-harm cases, what remains is the genuinely hard distribution — cases where the Board itself deliberated for weeks, where human experts disagree, where the right answer depends on cultural knowledge that 160 training examples cannot fully convey. **Plateauing near zero on this distribution is not failure. It is the model reaching the edge of what this data and compute can resolve.** **T1 — Cultural Meaning (blue) stays negative throughout (intentional yet important).** This is the finding we're most proud of reporting honestly. Cultural meaning resolution at depth — the core of what makes CACE hard — requires more data, more linguistic diversity, and more parameters than we had access to. The reward function is working correctly: it is accurately signalling that this dimension remains unsolved at our scale. **The behavioural shift tells the real story:** An agent that starts by hedging, learns to detect harm, and then discovers policy-appropriate escalation as the response to genuine ambiguity — that's a moderation agent learning to think. --- ## The Network Simulation ![ezgif-72ea6a3ec68197b6](https://cdn-uploads.huggingface.co/production/uploads/664ee3e9ff4de33cb68007b9/yG5quO3a2W_sDYBV88OJN.gif) ``` Deployed Space Link -> https://huggingface.co/spaces/hsr99/Cace-final-demo ``` Here's something classifiers definitely can't do: **understand that the same post causes different harm depending on where it is in its spread.** The CACE demo runs live inference on a 600-node subgraph of the [Facebook WOSN dataset](http://konect.cc/networks/facebook-wosn-wall/) — real social graph topology, real degree distribution, real preferential attachment structure. Posts seed on hub nodes, propagate via Independent Cascade, and the agent's decision hits the network in real time. Watch a `REMOVE` decision land at step 1 — three nodes affected, contained. Watch the same decision at step 6 — forty nodes, the content already in three community clusters. The cost of moderation latency is not abstract here. It's visible, wave by wave, in purple and teal. And here's what the simulation reveals that no benchmark does: not all content needs the same level of attention. A low-harm post spreading slowly through a sparse cluster looks completely different from a high-harm post propagating through dense hub nodes at velocity. The network makes that distinction visible in real time — which means a moderation system can prioritise accordingly. A cartoon shared between ten friends and a piece of incitement racing through a high-degree network both get flagged today. With network-aware moderation, they get different responses at different speeds and the compute goes where it actually matters. --- ## Now Let Us Tell You What Meta Could Do With This A two-person team. 160 cases. An 8B model. No proprietary infrastructure. We showed: - A working RL environment that turns Oversight Board rulings into live training signal - A reward function that correctly identifies what's learnable at small scale and what needs more - A behavioural trajectory that mirrors how expert moderators actually develop judgment - Network-aware moderation running on real Facebook topology **Meta has everything we didn't:** Every Oversight Board ruling ever published — all 300+, without API restrictions, plus the internal case notes. Millions of Tier-2 human review decisions, regionally labelled, culturally validated. Models at 70B+ parameters, where the cultural meaning track almost certainly converges. The actual Facebook social graph — not a 2009 research snapshot — for moderation decisions that account for real network position and spread velocity. Regional language experts to validate cultural enrichment in every language Meta operates in. The Oversight Board costs Meta millions of dollars per year and delivers rulings months after the fact. **An RL agent trained continuously on those rulings delivers the same quality of reasoning in milliseconds — and gets better every time a new ruling is published.** The hardest moderation decisions Meta faces are not problems to be handled manually at Tier 3. They are **the highest-quality training signal Meta has ever produced** — and right now, they are sitting in PDF files. CACE is what happens when you plug them in. With 160 cases and a weekend. Imagine what could happen with everything Meta has!