| --- |
| language: |
| - en |
| pipeline_tag: text-classification |
| tags: |
| - eu-ai-act |
| - risk-classification |
| - distilbert |
| - text-classification |
| base_model: |
| - distilbert/distilbert-base-uncased |
| --- |
| |
| # EU AI Act Risk Classifier |
|
|
| Fine-tuned DistilBERT model that classifies AI system descriptions into EU AI Act risk tiers. |
|
|
| ## Intended Use |
|
|
| Classifies a natural-language description of an AI system into one of five categories: |
| - `minimal_risk` |
| - `limited_risk` |
| - `high_risk` |
| - `prohibited_risk` |
| - `review` (flagged for human review — description is ambiguous or spans multiple tiers) |
|
|
| **Out of scope:** This model does not provide legal advice and should not be used as the sole basis for a compliance decision. |
|
|
| ## Training Data |
|
|
| - Dataset size: 1,538 labeled examples |
| - Labels sourced via a dual-labeling pipeline with human review on flagged conflicts |
| - Ambiguous/boundary-testing "review" cases which require human review. |
|
|
| ## Training Procedure |
|
|
| - Base model: distilbert-base-uncased |
| - Learning rate: 3e-5 |
| - Epochs: 10 - 20 |
| - Batch size: 8 |
| - Max sequence length: 256 |
|
|
| ## Evaluation Results |
|
|
| | Metric | Score | |
| |---|---| |
| | Accuracy | 92.08% | |
| | Macro F1 | 0.9209 | |
|
|
| **Per-class F1:** |
|
|
| | Class | F1 | |
| |---|---| |
| | minimal_risk | 0.97 | |
| | limited_risk | 0.93 | |
| | high_risk | 0.91 | |
| | prohibited_risk | 0.93 | |
| | review | 0.84 | |
|
|
| ## Limitations |
|
|
| - The "review" class has the lowest F1, the model is least confident distinguishing genuinely ambiguous cases, which is expected since these cases are excluded from training by design. |
| - High Risk and Limited Risk are the most frequently confused pair among the four primary tiers, particularly at the boundary between Annex III decision-influencing systems and Article 50 transparency-only systems. |
|
|
| ## Ethical Considerations / Risks |
| - The training labels came from an LLM-assisted first pass, then a decision tree based second pass, then a human reviewer. |
| - The results of the model are not intended to provide legal advice |
|
|
|
|
| ## Version History/Release Notes |
|
|
| ### v1.0 — 2026-08-14 |
| - Initial release |
| - Trained on 1,538 labeled examples across 5 tiers: minimal_risk, limited_risk, |
| high_risk, prohibited_risk, review |
| - Labels produced via a dual-pipeline: AI-assisted first pass → decision-tree |
| second pass → human review, with conflicts between the two automated passes |
| flagged for review |
| - Accuracy: 92.08% · Macro F1: 0.9209 |
| - Per-class F1: minimal_risk 0.97, limited_risk 0.93, high_risk 0.91, |
| prohibited_risk 0.93, review 0.84 |
| - Known limitation: review has the lowest F1 of the five classes — expected, |
| since ambiguous cases are excluded from confident training signal by design. |
| High Risk / Limited Risk is the most frequently confused pair among the four |
| primary tiers, particularly at the Annex III (decision-influencing) vs. |
| Article 50 (transparency-only) boundary. |
| |
