| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: Qwen/Qwen3-4B |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - lora |
| - regulatory |
| - compliance |
| - escalation |
| - decision-gate |
| - mlx |
| - gguf |
| - flowx |
| model-index: |
| - name: sentinel-gate |
| results: |
| - task: |
| type: text-generation |
| name: Escalation gate decision (ESCALATE vs DECIDE) on regulated-decision cases |
| dataset: |
| name: FlowX Sentinel held-out cases |
| type: flowxai/sentinel-gate |
| metrics: |
| - type: false_negative_rate |
| name: false-negative rate (missed escalations, the safety metric) |
| value: 0.000 |
| - type: accuracy |
| name: action accuracy (ESCALATE vs DECIDE) |
| value: 1.000 |
| - type: false_positive_rate |
| name: false-positive rate (over-escalation) |
| value: 0.000 |
| - type: accuracy |
| name: raw JSON validity (deploy with deterministic repair) |
| value: 0.89 |
| - type: accuracy |
| name: category accuracy on true-escalate (routing hint) |
| value: 0.61 |
| --- |
| |
| ## Inference contract |
|
|
| Running this model correctly requires its **frozen inference contract** - the exact |
| system prompt, output JSON schema, user-turn format, and decode spec it was trained |
| against. See [`inference_contract/`](./inference_contract): |
|
|
| - [`INFERENCE.md`](./inference_contract/INFERENCE.md) - wiring guide: system prompt, user turn `Case:\n<case JSON + "policy_schema">\n\nDecide: ESCALATE or DECIDE?`, decode settings (`enable_thinking=False`, temp 0, `max_new_tokens` ~1200), the six categories, and the **deterministic JSON repair step** (raw JSON validity 0.89) the deployed pipeline pairs with the model. |
| - [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt) - the system prompt, verbatim. |
| - [`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json) - output JSON Schema for the oracle decision. |
|
|
| Prompt version `sentinel_sys_v1`. Do not edit the prompt/schema; the weights are trained against them. |
|
|
| # FlowX Sentinel Gate (4B) - the escalation decision gate |
|
|
| **An on-device escalation gate for regulated decisions: given a complete case, it decides ESCALATE (route to a human) vs DECIDE (safe to automate), with a category, rationale, calibrated confidence, and an audit trail.** |
|
|
| Sentinel Gate is a **LoRA fine-tune of Qwen3-4B** (Apache-2.0), built by FlowX.AI. It reads |
| a complete regulated-decision case - the domain facts plus the applicable `policy_schema` |
| (a `PDP...` policy id) - and returns a single strict JSON decision. It sits **after |
| [`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper)** in a compliance |
| pipeline: the Mapper tags knowledge-base chunks, a policy layer assembles a case, and the |
| Sentinel Gate decides what is safe to automate versus what must be routed to a human. |
|
|
| The product insight that shapes everything: **the gate's job is the ESCALATE/DECIDE |
| decision, and that decision is safety-critical, so it is the metric we optimize and the field |
| you gate on.** A compliance automation gate that misses an escalation is dangerous in a way |
| that a wrong routing label is not. So we lead with the false-negative rate (missed |
| escalations), hold it at **0.000**, and treat the escalation *category* as a secondary |
| routing hint (~0.61) rather than a second gate. The model never over-escalates either |
| (false-positive rate 0.000), which is what makes the automation worth having. |
|
|
| Part of the **FlowX on-device model family**: |
| [`flowxai/caveat`](https://huggingface.co/flowxai/caveat), |
| [`flowxai/scam-guard-qwen06b`](https://huggingface.co/flowxai/scam-guard-qwen06b), |
| [`flowxai/scam-guard-qwen17b`](https://huggingface.co/flowxai/scam-guard-qwen17b), |
| [`flowxai/semantic-mapper`](https://huggingface.co/flowxai/semantic-mapper). |
|
|
| > **Not legal advice - decision-support only.** Sentinel Gate is a compliance automation |
| > gate, not a legal opinion and not a substitute for a compliance officer. It decides |
| > *whether a case is safe to automate or must go to a human*; a human owns every escalated |
| > case. Deploy with the confidence threshold your risk posture requires. |
|
|
| English at v1. Domains span **banking, insurance, logistics, and labor** (cases are EN). |
| On-device formats: **fp16 safetensors** (transformers/CUDA/vLLM), **MLX-quantized** (int4 + |
| int8, Apple Silicon), and **GGUF** (llama.cpp/CPU/Ollama, `Q8_0` + `Q4_K_M`). |
|
|
| --- |
|
|
| ## How do I use it? |
|
|
| Three copy-pasteable ways to turn a case into a decision. The system prompt is the exact |
| two-liner from [`prompt_sentinel_sys_v1.txt`](./inference_contract/prompt_sentinel_sys_v1.txt). |
| The user turn is `Case:\n<case JSON>\n\nDecide: ESCALATE or DECIDE?`. |
|
|
| Real example input (an OFAC-screening wire transfer - a `POLICY_VIOLATION` escalation): |
|
|
| ``` |
| Case: |
| { |
| "transaction_facts": { |
| "product": "international wire transfer", |
| "originator": "Crestwood Imports Inc", |
| "beneficiary_name": "Volna Trading LLC", |
| "beneficiary_country": "Cyprus", |
| "amount_usd": 118000, |
| "purpose": "machinery purchase", |
| "screening_result": "potential match - beneficiary owner on SDN-adjacent watchlist (50% rule concern)" |
| }, |
| "documents_provided": ["wire request form", "commercial invoice", "originator KYC on file"], |
| "documents_missing": [], |
| "policy_schema": "PDP.lending.payments.ofac_sanctions_block" |
| } |
| |
| Decide: ESCALATE or DECIDE? |
| ``` |
|
|
| ### (a) transformers / CUDA (fp16 safetensors) |
|
|
| The root of the repo is fp16 safetensors for transformers / vLLM. |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| SYSTEM = ("You are an escalation gate for regulated decisions.\n" |
| "Determine: ESCALATE or DECIDE? Output ONLY JSON.") |
| tok = AutoTokenizer.from_pretrained("flowxai/sentinel-gate") |
| model = AutoModelForCausalLM.from_pretrained("flowxai/sentinel-gate", torch_dtype="float16", device_map="cuda") |
| |
| case = open("wire_case.json").read() # the case JSON above |
| user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?" |
| |
| prompt = tok.apply_chat_template( |
| [{"role": "system", "content": SYSTEM}, |
| {"role": "user", "content": user}], |
| add_generation_prompt=True, enable_thinking=False, return_tensors="pt", |
| ).to("cuda") |
| out = model.generate(prompt, max_new_tokens=1200, do_sample=False, temperature=0.0) |
| raw = tok.decode(out[0][prompt.shape[-1]:], skip_special_tokens=True) |
| # then: deterministic JSON repair -> validate against schema_sentinel_v1.json |
| ``` |
|
|
| ### (b) MLX (Apple Silicon) |
|
|
| Off the MLX-quantized weights (int4/int8): |
|
|
| ```bash |
| mlx_lm.generate --model flowxai/sentinel-gate-mlx-int4 --temp 0 --max-tokens 1200 \ |
| --system-prompt "$(cat inference_contract/prompt_sentinel_sys_v1.txt)" \ |
| --prompt "$(printf 'Case:\n%s\n\nDecide: ESCALATE or DECIDE?' "$(cat wire_case.json)")" |
| ``` |
|
|
| ```python |
| from mlx_lm import load, generate |
| |
| SYSTEM = open("inference_contract/prompt_sentinel_sys_v1.txt").read() |
| model, tok = load("flowxai/sentinel-gate-mlx-int4") |
| case = open("wire_case.json").read() |
| user = f"Case:\n{case}\n\nDecide: ESCALATE or DECIDE?" |
| prompt = tok.apply_chat_template( |
| [{"role": "system", "content": SYSTEM}, |
| {"role": "user", "content": user}], |
| add_generation_prompt=True, enable_thinking=False, |
| ) |
| raw = generate(model, tok, prompt=prompt, max_tokens=1200, verbose=False) |
| ``` |
|
|
| ### (c) Short note - decode discipline |
|
|
| Two things are non-negotiable: **`enable_thinking=False`** (the adapter was trained on pure |
| JSON, so the default thinking template yields empty/degraded output) and **temp 0** (the gate |
| must be reproducible for audit). Give it **~1200 new tokens** - the oracle JSON can run long |
| and truncation is the main cause of invalid JSON. Always run the output through the |
| deterministic JSON repair step (see [`INFERENCE.md`](./inference_contract/INFERENCE.md)). |
| |
| All three backends return the **same strict JSON** for the wire-transfer case above: |
| |
| ```json |
| { |
| "action": "ESCALATE", |
| "escalation_category": "POLICY_VIOLATION", |
| "policy_violations": { |
| "ofac_potential_match": { |
| "triggered": true, |
| "policy": "PDP.lending.payments.ofac_sanctions_block", |
| "regulation": "31 CFR 501 / OFAC sanctions regulations; OFAC 50 Percent Rule; 31 CFR 1010 (BSA recordkeeping for funds transfers)", |
| "restriction": "Beneficiary ownership screening returned a potential SDN-adjacent match invoking the OFAC 50% Rule; payment is automatically held pending sanctions determination.", |
| "consequence": "Releasing the wire without OFAC clearance risks a sanctions violation, blocking/rejection obligations, and civil penalties; the transaction must be blocked from auto-release." |
| } |
| }, |
| "confidence_score": 0.33, |
| "confidence_reasoning": "Transaction documentation is complete, but a potential OFAC match on beneficiary ownership is a hard stop; only a sanctions analyst can clear, reject, or block the wire.", |
| "human_action_required": "OFAC/sanctions analyst to investigate the potential match, apply the 50% Rule analysis, and clear, reject, or block the wire per OFAC procedures.", |
| "audit_trail": [ |
| "Ran beneficiary screening; returned potential SDN-adjacent ownership match", |
| "Triggered OFAC 50 Percent Rule review condition", |
| "Matched POLICY_VIOLATION on sanctions block policy", |
| "Held wire and routed to sanctions analyst; auto-release prohibited" |
| ] |
| } |
| ``` |
| |
| For a case with no blocking condition and complete documentation, the gate returns |
| `{"action": "DECIDE", "escalation_category": null, "decision": "...", ...}` with a high |
| `confidence_score` and `human_action_required: "NONE"`. |
| |
| --- |
| |
| ## How it works |
| |
| ``` |
| +----------------------------------+ |
| case JSON (facts + | Sentinel Gate 4B | |
| policy_schema: "PDP...") ---> | Qwen3-4B (LoRA), thinking off | ---> strict JSON |
| Decide: ESCALATE or DECIDE? | greedy decode -> JSON repair | |
| +----------------------------------+ |
| | |
| + action (ESCALATE | DECIDE) <- the gate |
| + escalation_category <- routing hint |
| + category-specific block |
| + confidence_score / reasoning |
| + human_action_required |
| + audit_trail |
| ``` |
| |
| In the pipeline: `semantic-mapper` tags KB chunks -> a policy layer assembles a case -> |
| **Sentinel Gate decides automate vs route-to-human**. Under the hood, one turn is: |
| `apply_chat_template(enable_thinking=False) -> Qwen3-4B (LoRA fine-tuned) -> greedy decode -> |
| deterministic JSON repair -> validate against schema_sentinel_v1.json`. |
| |
| **The `action` field is the safety-critical output.** You branch your automation off it and |
| nothing else. The escalation_category, block, and confidence enrich the human hand-off but |
| never decide it. |
| |
| --- |
| |
| ## Output schema |
| |
| A single strict JSON object. Full schema: |
| [`schema_sentinel_v1.json`](./inference_contract/schema_sentinel_v1.json). |
| |
| - `action` - `ESCALATE` | `DECIDE`. **The gate.** `ESCALATE` routes the case to a human; |
| `DECIDE` marks it safe to automate. |
| - `escalation_category` - one of the six ids when `action=ESCALATE`, else `null`. A |
| **routing hint** (~0.61), not a second gate. |
| - **category-specific block** - one object whose key depends on the category: |
| - `policy_violations` (POLICY_VIOLATION) - keyed by violation; each entry names `policy`, |
| `regulation`, `restriction`, `consequence`. |
| - `missing_preconditions` (MISSING_REQUIRED_DOCUMENTATION) - each entry names `required_by`, |
| `regulation`, `severity`, `reason`. |
| - `boundary_analysis` (BOUNDARY_CONDITION) - `policy_threshold`, the case value, |
| `distance_from_threshold`, `assessment`. |
| - `confidence_factors` (INSUFFICIENT_CONFIDENCE) - `ambiguous_signals[]`, `why_uncertain`. |
| - `conflicting_signals` (CONFLICTING_SIGNALS) - the competing `{source, value}` items. |
| - `external_dependency` (EXTERNAL_DEPENDENCY) - `awaiting`, `blocking_gate`. |
| - `confidence_score` - number 0.0–1.0. Calibrated; apply your own threshold. Advisory. |
| - `confidence_reasoning` - one to three sentences explaining the score and the decision. |
| - `human_action_required` - the concrete next step a human must take, or `"NONE"` for DECIDE. |
| - `audit_trail` - ordered, append-only list of reasoning steps and policy gates, for review. |
| - (DECIDE cases also carry `decision` / `selected_route` / `policy_gates_passed`.) |
|
|
| **The six escalation categories:** `MISSING_REQUIRED_DOCUMENTATION`, `POLICY_VIOLATION`, |
| `BOUNDARY_CONDITION`, `INSUFFICIENT_CONFIDENCE`, `CONFLICTING_SIGNALS`, `EXTERNAL_DEPENDENCY`. |
|
|
| --- |
|
|
| ## Evaluation |
|
|
| Held-out set: **n=71 cases (46 escalate / 17 decide)**. Metrics in order of what they tell you. |
|
|
| ### The safety metric first: missed escalations |
|
|
| The dangerous failure for a compliance gate is **waving a case through that should have gone |
| to a human** (a missed escalation, i.e. a false negative). That is the number to read first. |
|
|
| | Metric | Result | What it means | |
| | --- | --- | --- | |
| | **False-negative rate (missed escalations)** | **0.000** (0/46) | **the safety metric** - every case that should escalate did | |
| | Action accuracy (ESCALATE vs DECIDE) | **1.000** | the gate decision is correct on every held-out case | |
| | False-positive rate (over-escalation) | **0.000** (0/17) | no safe-to-automate case was needlessly routed to a human | |
| | Raw JSON validity (no repair) | 0.89 | deploy with the deterministic repair step (see below) | |
| | Category accuracy (on true-escalate) | 0.61 | the human-routing label; categories legitimately overlap | |
|
|
| ### Decision confusion (ESCALATE vs DECIDE), n=71 |
|
|
| Rows = gold, cols = predicted: |
|
|
| | gold \ pred | ESCALATE | DECIDE | recall | |
| | --- | --- | --- | --- | |
| | ESCALATE | **46** | 0 | 1.000 (n=46) | |
| | DECIDE | 0 | **17** | 1.000 (n=17) | |
|
|
| The diagonal is complete: **no missed escalations, no over-escalation, action decided |
| correctly every time** on this held-out set. This is the gate's job, and on this set it is |
| perfect. |
|
|
| ### Honest reading |
|
|
| - **The decision is the product, and it is perfect on this set.** Gate on `action`. |
| - **The category is a secondary routing hint at ~0.61.** The six categories legitimately |
| overlap for some cases (a boundary case that is also a policy edge; a missing-doc case that |
| is also insufficient-confidence), so the label is genuinely ambiguous for a slice of |
| escalations. Use it to pick a specialist queue, not to make a correctness-critical branch, |
| and let a human re-label at intake. |
| - **Raw JSON validity is 0.89, so ship the deterministic repair step.** The deployed pipeline |
| pairs the model with a deterministic JSON repair/retry pass and schema validation; 0.89 is |
| the *raw* number before that pass, not the deployed one. |
| - **Home-field caveat.** These 71 cases are **realistic synthetic** - grounded in real |
| regulatory citations (31 CFR 1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul |
| muncii) but authored for the benchmark, not drawn from live traffic. The scenarios share the |
| distribution the model trained on. **Validate on your own case distribution before |
| production**; a perfect held-out gate is a necessary signal, not a promise for your traffic. |
|
|
| --- |
|
|
| ## Formats |
|
|
| | Path | Format | Runs on | |
| | --- | --- | --- | |
| | `/` (root) | fp16 safetensors | transformers / CUDA / vLLM | |
| | `mlx-int4/` | MLX int4 | Apple Silicon (smallest footprint) | |
| | `mlx-int8/` | MLX int8 | Apple Silicon (higher fidelity) | |
| | `gguf/*.gguf` | GGUF `Q8_0` / `Q4_K_M` | llama.cpp / CPU / Ollama | |
|
|
| `Q8_0` is the recommended GGUF quant; MLX int4 is the fastest quality-holding path on Apple |
| Silicon. All formats use the same frozen inference contract (`enable_thinking=False`, temp 0, |
| `max_new_tokens` ~1200). |
|
|
| --- |
|
|
| ## Intended use & limitations |
|
|
| **Intended use.** A **compliance automation gate**: it decides which regulated-decision cases |
| are safe to automate and which must be routed to a human, with a structured rationale and an |
| audit trail for the hand-off. It sits after `semantic-mapper` and before a human queue. |
|
|
| **Out of scope & limitations.** |
|
|
| - **Not legal advice.** A verdict is an automation-routing signal, not a legal opinion. A |
| human owns every escalated case, and the recommended action always routes to a human. |
| - **Category label is ~61% accurate.** Gate on `action` (ESCALATE/DECIDE); treat |
| `escalation_category` as a routing suggestion, not ground truth. |
| - **JSON validity is 0.89 raw.** Deploy with the deterministic repair/retry step; do not rely |
| on raw output being parseable. |
| - **Evaluated on 71 realistic-synthetic cases.** No real client data was used in training or |
| eval. Validate on your own case distribution before production. |
| - **English, four domains (banking / insurance / logistics / labor) at v1.** Cases outside |
| this distribution - other domains, other languages, malformed `policy_schema` - are out of |
| scope and should default to escalation, not automation. |
| - **Depends on the input case being complete and correct.** The gate reasons over the facts it |
| is given; a case assembled with wrong or missing facts can produce a wrong decision. The |
| upstream policy layer and `semantic-mapper` are part of the trust boundary. |
|
|
| --- |
|
|
| ## Training data |
|
|
| **471 realistic-synthetic escalation cases** (400 train / 71 held-out), balanced **~35% DECIDE |
| / 65% ESCALATE** across the six categories and four regulated domains (banking, insurance, |
| logistics, labor). Each case is **grounded in a real regulatory citation** (for example 31 CFR |
| 1010, Solvency II, 49 CFR 172, Directive 2003/88/EC, Codul muncii). The scenarios are |
| **realistic synthetic** - authored to reflect real regulatory conditions - with **no real |
| client data**. The assistant turn in each record is the exact oracle JSON the model must emit, |
| with thinking disabled. |
|
|
| ## Fine-tuning |
|
|
| LoRA (rank 32 / scale 16 / dropout 0.05), `num_layers -1` (all layers), from `Qwen/Qwen3-4B`. |
| Trained with **MLX-LM on Apple Silicon**: 400 iterations (~4 epochs), **cosine LR 5e-5 → |
| 5e-6** with a 40-step warmup, `max_seq_length 2048`, seed 42, thinking disabled. |
|
|
| > **Stability note.** An initial `1e-4` run **diverged**; `5e-5` with the 40-step warmup is |
| > the stable recipe. |
|
|
| --- |
|
|
| ## License |
|
|
| Apache-2.0 (weights and code). Copyright 2026 FlowX.AI. `NOTICE` present. Base model |
| `Qwen/Qwen3-4B` is Apache-2.0. Escalation scenarios are realistic synthetic, grounded in real |
| regulatory citations (no real client data). |
|
|
| _Author: Bogdan Răduță, Head of Research, FlowX.AI._ |
|
|
