README.md

---
language:
- en
- zh
tags:
- text-classification
- safety
- education
- guardrails
- bert
- response-moderation
pipeline_tag: text-classification
library_name: transformers
license: apache-2.0
---

# Educational Guardrail System - Holistic Response Auditor

## Model Details

- **Model Name:** Holistic Response Auditor
- **Model Architecture:** BERT sequence classification model
- **Task:** Response Moderation
- **Domain:** K-12 Education & General AI Safety
- **Language:** English and Chinese

## Model Description

This model serves as the **Holistic Response Auditor**, the third and final stage of the **Educational Guardrail System (EGS)**. Unlike input filters, this model is specifically fine-tuned to audit the **output streams** generated by Large Language Models (LLMs) to ensure pedagogical safety and compliance.

### Key Mechanism: Sliding Window Auditing
As described in the EGS design, this model uses a **sliding window approach** (e.g., 512 tokens) to process long generated texts. It is designed to be a "Fail-Fast" auditor:
* The auditor evaluates the generated text stream in discrete segments.
* If any single segment is flagged as toxic or non-compliant, the generation can be stopped or routed for secondary handling before the learner is exposed to the content.

## Label Mapping

- **UNSAFE:** High-risk response segment
- **SAFE:** Low-risk response segment
- **AMBIGUOUS:** Medium-risk or unclear response segment

## Intended Use

This model is intended to be used as a post-generation filter for Educational LLMs.

* **Input:** The text generated by an LLM (Draft Response).
* **Output:** Three-way classification (UNSAFE / SAFE / AMBIGUOUS).
* **Application:** Detecting residual toxicity, hallucinations, or age-inappropriate content that bypassed input moderation.

### Usage Example (Sliding Window)

To replicate the paper's logic, you should apply the model over segments of the response:

```python
from transformers import pipeline

# Load the auditor
model_id = "ZeroLoss-Lab/egs-bert-response-auditor"
auditor = pipeline("text-classification", model=model_id)

def audit_stream(text_stream, window_size=512):
    """
    Audits an LLM response stream using the Holistic Response Auditor.
    """
    chunks = [text_stream[i:i+window_size] for i in range(0, len(text_stream), window_size)]

    for chunk in chunks:
        result = auditor(chunk)[0]
        if result["label"] == "UNSAFE":
            return False, "Blocked: Unsafe content detected."

        if result["label"] == "AMBIGUOUS":
            return False, "Needs secondary review: Ambiguous content detected."

    return True, "Passed"
```

## Limitations

This model is intended as a post-generation safety component, not as a complete safety solution. It should be combined with input filtering, policy logic, human review pathways where appropriate, and task-specific evaluation before deployment.