caveat

caveat is a small, on-device model that reads a consumer contract and flags the clauses worth reading twice: auto-renewal traps, punitive early-termination, liability exclusions, data-selling, arbitration / class-action waivers, and the rest of a 34-type taxonomy. It runs privately on-device (no API, no data leaves the machine) in English, Romanian, Polish and Hungarian, and returns structured JSON, not prose.

caveat explains consequences. It is NOT legal advice and does not tell you whether to sign. A no-advice contract is enforced at generation time (the model is scanned for advice language; violations are rejected).

• Base model: Qwen/Qwen3-4B (Apache-2.0), LoRA fine-tuned.
• Deployment: on-device. Ship artifact is MLX int4 (mlx-int4/, ~2 GB, Apple Silicon); MLX int8 (mlx-int8/) and the standard fp16 merged model (repo root) are also provided.
• Full model card: see caveat_model_card.pdf.

What it does

Input: one contract chunk + a document-type hint. Output: a JSON ChunkResult:

{
  "summary": "<one plain-language sentence>",
  "red_flags": [
    {"clause_type": "auto_renewal", "severity": "warning",
     "span_text": "<verbatim clause text>", "explanation": "<consequence, no advice>",
     "truncated": false}
  ]
}

clause_type is one of 34 taxonomy ids across 6 categories (money, rights-waived, time/renewal, lease, insurance, data); severity is info | caution | warning.

Recommended inference: two passes, unioned

Real contracts pack many red flags into few paragraphs, and a whole-document pass and a per-paragraph pass surface different clauses (insurance detects better with full-document context; lease/loan clauses detect better isolated). Run both and union the results (de-duplicate by clause type + span). This roughly doubles real-world recall versus a single whole-document pass. See demo/explain.py in the project repo for a reference implementation.

Intended use

In scope: consumer-facing contracts (subscriptions / terms of service, insurance policies, loan / credit agreements, residential leases, utility / telecom) in EN/RO/PL/HU, as a triage aid that points a human to clauses worth a closer look.

Out of scope: legal advice or a decision on whether to sign; B2B / M&A / commercial contracts; jurisdictions or languages outside those trained; any use where a missed clause carries safety or legal risk without human review. Recall is partial (see Limitations).

Evaluation

Real-world recall (the headline). On five held-out real consumer contracts (gym EN, home-insurance EN, personal-loan RO, apartment-lease PL, mobile HU), scored with the two-pass inference, caveat detects 13 distinct red flags, up from 9 for a prior 4B and 7 for a 1.7B baseline, and it is the first version to catch flags on all five documents (the RO loan and PL lease were previously undetected).

Synthetic benchmark (RedFlag-Bench v0.1, 597 frozen held-out chunks). Overall clause-type F1 0.275 (+54% over the pre-coverage-fix 4B); by category: lease 0.33, money 0.32, data 0.32, rights-waived 0.23, time/renewal 0.20, insurance 0.00*; by language: EN 0.35, HU 0.29, PL 0.24, RO 0.14. Faithfulness: 2.6 span-verification failures per 100 flags; JSON validity 1.00; advice hits 0. (*The insurance bench F1 is a small-sample artifact of the frozen set; real-document insurance detection works, e.g. waiting-periods and buried-exclusions on the home-insurance doc.) The synthetic bench shares a generator family with the training data, so read it as a format-adherence / faithfulness ceiling, not a real-world recall measure.

Versus a frontier model (like-for-like slice). On a seeded 100-chunk slice, a frontier model (Claude Opus) leads the overall synthetic F1 (0.385 vs caveat 0.233); caveat wins specific cells (lease 0.80 vs 0.30, English 0.63 vs 0.27, severity agreement 0.43 vs 0.33). caveat's value is real-document recall and running privately, on-device, at no inference cost, not beating a frontier model on the synthetic benchmark.

Training

LoRA (rank 16 / alpha 32 / dropout 0.05, all-linear target modules), 2 epochs, cosine LR 2e-5 -> 2e-6, effective batch 4, max sequence length 2048, bf16, on an NVIDIA L4. Training data (5,263 train / 1,367 validation chunks, split by document): synthetic consumer contracts generated across five drafting registers with flag-dense examples, targeted insurance- and lease-coverage batches, and 450 real English clause spans from CUAD (train-only).

Attribution & licenses

• Model weights released under Apache-2.0 (inherited from the Qwen3-4B base).
• Training used the Contract Understanding Atticus Dataset (CUAD) v1, The Atticus Project, Inc. (CC-BY-4.0) — Hendrycks et al., CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review, arXiv:2103.06268. CUAD clause spans were used train-only for clause shapes.

Limitations & responsible use

• Not legal advice. A triage aid only; always have important contracts reviewed by a qualified professional.
• Partial recall. caveat surfaces many but not all red flags (~13 of 20+ on the real set); a clean result is not a guarantee the contract is safe.
• Uneven coverage. English is strongest; Romanian and Polish are weaker; insurance and lease phrasing vary in reliability. Use the two-pass inference for best recall.
• Small real-world eval. The real-document evidence is five contracts; treat the numbers as directional.

Citation

@misc{flowx_caveat_2026,
  title  = {caveat: on-device consumer-contract red-flag detection},
  author = {R\u{a}du\c{t}\u{a}, Bogdan},
  year   = {2026},
  note   = {FlowX.AI. LoRA fine-tune of Qwen3-4B.}
}

Author: Bogdan Răduță, Head of Research, FlowX.AI.