| --- |
| license: mit |
| base_model: zai-org/GLM-4.5-Air |
| tags: |
| - legal |
| - contract-review |
| - document-analysis |
| - lora |
| - glm |
| library_name: transformers |
| pipeline_tag: text-generation |
| --- |
| |
| # PolyClerk-12B |
|
|
| PolyClerk-12B (12B active / 106B total MoE parameters, GLM-4.5-Air derivative) is fine-tuned for legal work product: counterparty markup |
| analysis, redlining, contract drafting, and clause-level document review over |
| very long contexts (up to 128k tokens). |
|
|
| ## Lineage |
|
|
| 1. **Base**: [zai-org/GLM-4.5-Air](https://huggingface.co/zai-org/GLM-4.5-Air) (MIT) |
| 2. **Stage 1 (iter-2b)**: OAPL training on agentic tool-use trajectories |
| 3. **Stage 2 (this model)**: LoRA fine-tune (r=32, α=64) on legal-bench work-product |
| tasks, merged into the stage-1 weights. This repo contains the fully merged |
| weights — no adapter loading required. |
|
|
| ## Training |
|
|
| | | | |
| |---|---| |
| | Method | OAPL, LoRA r=32 / α=64 (merged) | |
| | Framework | ms-swift (Megatron backend), TP4 × CP2 | |
| | Hardware | 8× H200 | |
| | Sequence length | 131,072 | |
| | Epochs | 1 | |
|
|
| ## Evaluation |
|
|
| On a held-out legal work-product benchmark (whole-document mode, long-context |
| markup/review tasks), this model scores comparably to frontier closed models on |
| the small evaluated task set. Numbers are from a limited sample (N=3 task |
| families) — treat as indicative, not definitive. |
|
|
| ⚠️ **Contamination note**: this model was trained on tasks drawn from the LAB |
| legal benchmark family. Do not use LAB (or derivative benchmarks) to evaluate |
| this model. |
|
|
| ## Usage |
|
|
| Requires ~200GB of weights (bf16). Serve with vLLM: |
|
|
| ```bash |
| vllm serve polygramme/PolyClerk-12B --tensor-parallel-size 4 --max-model-len 131072 |
| ``` |
|
|
| Or load with transformers (multi-GPU required): |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| tok = AutoTokenizer.from_pretrained("polygramme/PolyClerk-12B") |
| model = AutoModelForCausalLM.from_pretrained("polygramme/PolyClerk-12B", device_map="auto", torch_dtype="bfloat16") |
| ``` |
|
|
| The chat template is included (`chat_template.jinja`). |
|
|
| ## Intended use & limitations |
|
|
| Intended for legal document analysis workflows (markup review, redline |
| drafting, provision-level analysis). Outputs are not legal advice; a qualified |
| lawyer must review all work product. The model may hallucinate section |
| references or values on documents unlike its training distribution — verify |
| against source documents. |
|
|
| ## Training data attribution |
|
|
| Fine-tuned on tasks from [harvey-labs](https://github.com/harveyai/harvey-labs) |
| (MIT License, © 2026 Harvey AI). The MIT permission notice is reproduced here in |
| accordance with the license: |
|
|
| > Permission is hereby granted, free of charge, to any person obtaining a copy |
| > of this software and associated documentation files (the "Software"), to deal |
| > in the Software without restriction [...] subject to inclusion of the above |
| > copyright notice and this permission notice in all copies or substantial |
| > portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY |
| > OF ANY KIND. |
|
|
| ## License |
|
|
| MIT, following the GLM-4.5-Air base license. © the model authors. |
|
|
