docs-ai documentation quality classifier
A frozen GLiNER encoder with a learned logistic head for documentation quality. The recall-first policy flags potentially weak pages for editorial review. This is a research model, with provisional labels and uncalibrated scores.
Download and run on Linux
Download quality-harness.zip, extract it, then follow its README. The small harness downloads the complete model archive from this repository at a pinned commit and checks SHA-256 before loading it. No Mac, MLX, Claude installation or Diátaxis model is needed. CPU dependencies are pinned separately for Linux x86-64 and ARM64, using Python3.12.
uv venv --python 3.12 .venv
uv pip sync --python .venv/bin/python --torch-backend cpu requirements-linux-cpu.lock
# ARM64: substitute requirements-linux-arm64-cpu.lock above.
.venv/bin/python quality_harness.py example.md > result.json
The first run downloads approximately1.95GB, plus Python dependencies. Later runs reuse the cache; --offline supports an already downloaded release. Page text is processed locally. Hosting here distributes the weights; it does not create a hosted inference endpoint or send document text to Hugging Face.
Model and method
- Base: fastino/GLiNER2.5-Decide at9d1bfb848cd16d93ee56ccbc59cd92bf0f284d90.
- Unchanged encoder; no backbone fine-tuning or quantization.
- Every native word window yields no/yes option representations and their difference. Word-weighted mean pooling precedes the learned3072-feature head.
- Learned head SHA-256: f93bcf4886640b05e1c8f5830aa455049c614e1fc3aa88a6d58c804ed70492cf.
- Operating policy SHA-256: b3301ca3fadf0a52be2a6ccfaf4bc5d70115c434d6a10411268e6946edb29c56.
- Flag an issue when good_documentation is below0.71. The cutoff favors issue recall over precision; it does not calibrate the score into a probability.
- The archive contains the encoder, tokenizers, original model card, license, learned head, policy and per-file hashes. It contains no training documents.
- The loader uses the installed GLiNER library and safetensors, with no downloaded Python execution or trust_remote_code. Use this harness rather than a generic Transformers pipeline, which does not implement the learned document head.
Evidence and limitations
Training uses62 provisional examples across22 source groups: nominated original documentation, deliberate Haiku degradations and authored editorial controls. Regularization was selected on development; the original head weights are frozen. The current cutoff was selected with training cross-fit and development results.
| Population | Weak pages flagged | Good pages falsely flagged |
|---|---|---|
| Development, selected policy | 12/12 | 2/12 |
| Previously opened exploratory Rust test, retrospective policy | 4/4 | 2/4 |
These small, source-dependent results are not independent expert validation. The Rust test was already inspected before this cutoff change; it cannot validate the new policy independently. Ordinary “clearer/improve” rewrites are unlabeled. Generator/source shortcuts are possible. Pooling can dilute a localized defect. The model does not establish factual correctness, command safety, security, compliance or suitability for publication. Every output retains review_required. High scores are not measured probabilities of correctness.
Linux ARM64 CPU execution was verified on all24 existing development pages: maximum score difference from the prior CPU runtime is0.00000356, with zero cutoff decision changes and identical full-window coverage. A separate long input covers12 windows. Linux x86-64 dependencies resolve, but x86-64 execution and CUDA have not been separately measured. These checks establish portability; they do not train a better model or create a new independent holdout.
Attribution
The base encoder is published by Fastino under Apache2.0; see BASE-LICENSE.txt and the upstream README preserved in the archive. Its checkpoint bytes are unchanged. The head, policy and harness are docs-ai project artifacts; no additional license for those project artifacts is asserted by the upstream license notice.
Source base: https://huggingface.co/fastino/GLiNER2.5-Decide/tree/9d1bfb848cd16d93ee56ccbc59cd92bf0f284d90