| --- |
| language: |
| - en |
| pipeline_tag: text-classification |
| base_model: fastino/GLiNER2.5-Decide |
| tags: |
| - documentation |
| - quality-classification |
| - gliner2 |
| - research |
| - cpu |
| --- |
| |
| # docs-ai documentation quality classifier |
|
|
| A frozen GLiNER encoder with a learned logistic head for documentation quality. |
| The recall-first policy flags potentially weak pages for editorial review. |
| This is a research model, with provisional labels and uncalibrated scores. |
|
|
| ## Download and run on Linux |
|
|
| Download [quality-harness.zip](https://huggingface.co/brenorb/doc-quality/resolve/main/quality-harness.zip), extract it, |
| then follow its README. The small harness downloads the complete model archive |
| from this repository at a pinned commit and checks SHA-256 before loading it. |
| No Mac, MLX, Claude installation or Diátaxis model is needed. CPU dependencies |
| are pinned separately for Linux x86-64 and ARM64, using Python3.12. |
|
|
| ```sh |
| uv venv --python 3.12 .venv |
| uv pip sync --python .venv/bin/python --torch-backend cpu requirements-linux-cpu.lock |
| # ARM64: substitute requirements-linux-arm64-cpu.lock above. |
| .venv/bin/python quality_harness.py example.md > result.json |
| ``` |
|
|
| The first run downloads approximately1.95GB, plus Python dependencies. Later |
| runs reuse the cache; --offline supports an already downloaded release. |
| Page text is processed locally. Hosting here distributes the weights; it does |
| not create a hosted inference endpoint or send document text to Hugging Face. |
|
|
| ## Model and method |
|
|
| - Base: fastino/GLiNER2.5-Decide at9d1bfb848cd16d93ee56ccbc59cd92bf0f284d90. |
| - Unchanged encoder; no backbone fine-tuning or quantization. |
| - Every native word window yields no/yes option representations and their |
| difference. Word-weighted mean pooling precedes the learned3072-feature head. |
| - Learned head SHA-256: |
| f93bcf4886640b05e1c8f5830aa455049c614e1fc3aa88a6d58c804ed70492cf. |
| - Operating policy SHA-256: |
| b3301ca3fadf0a52be2a6ccfaf4bc5d70115c434d6a10411268e6946edb29c56. |
| - Flag an issue when good_documentation is below0.71. The cutoff favors issue |
| recall over precision; it does not calibrate the score into a probability. |
| - The archive contains the encoder, tokenizers, original model card, license, |
| learned head, policy and per-file hashes. It contains no training documents. |
| - The loader uses the installed GLiNER library and safetensors, with no downloaded |
| Python execution or trust_remote_code. Use this harness rather than a generic |
| Transformers pipeline, which does not implement the learned document head. |
| |
| ## Evidence and limitations |
| |
| Training uses62 provisional examples across22 source groups: nominated original |
| documentation, deliberate Haiku degradations and authored editorial controls. |
| Regularization was selected on development; the original head weights are frozen. |
| The current cutoff was selected with training cross-fit and development results. |
| |
| | Population | Weak pages flagged | Good pages falsely flagged | |
| | --- | ---: | ---: | |
| | Development, selected policy | 12/12 | 2/12 | |
| | Previously opened exploratory Rust test, retrospective policy | 4/4 | 2/4 | |
| |
| These small, source-dependent results are not independent expert validation. |
| The Rust test was already inspected before this cutoff change; it cannot validate |
| the new policy independently. Ordinary “clearer/improve” rewrites are unlabeled. |
| Generator/source shortcuts are possible. Pooling can dilute a localized defect. |
| The model does not establish factual correctness, command safety, security, |
| compliance or suitability for publication. Every output retains review_required. |
| High scores are not measured probabilities of correctness. |
|
|
| Linux ARM64 CPU execution was verified on all24 existing development pages: |
| maximum score difference from the prior CPU runtime is0.00000356, with zero |
| cutoff decision changes and identical full-window coverage. A separate long |
| input covers12 windows. Linux x86-64 dependencies resolve, but x86-64 execution |
| and CUDA have not been separately measured. These checks establish portability; |
| they do not train a better model or create a new independent holdout. |
|
|
| ## Attribution |
|
|
| The base encoder is published by Fastino under Apache2.0; see BASE-LICENSE.txt |
| and the upstream README preserved in the archive. Its checkpoint bytes are |
| unchanged. The head, policy and harness are docs-ai project artifacts; no additional |
| license for those project artifacts is asserted by the upstream license notice. |
|
|
| Source base: https://huggingface.co/fastino/GLiNER2.5-Decide/tree/9d1bfb848cd16d93ee56ccbc59cd92bf0f284d90 |
|
|
