publish README: 2 files (10.7 KiB)
Browse filessha256 of every file is in MANIFEST.json
- MANIFEST.json +13 -0
- README.md +82 -29
MANIFEST.json
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"generated_at": "2026-08-07T09:09:59+00:00",
|
| 3 |
+
"repo_id": "digitable-lol/README",
|
| 4 |
+
"repo_type": "model",
|
| 5 |
+
"note": "sha256 of every file as published. The hash is the record, not the filename: a repository is the same artefact as this manifest only if every digest matches.",
|
| 6 |
+
"files": [
|
| 7 |
+
{
|
| 8 |
+
"path_in_repo": "README.md",
|
| 9 |
+
"sha256": "9d960acba49b6387c926ad6ce034b60d52efe5d38defe3c89fd170212daf1975",
|
| 10 |
+
"size_bytes": 10491
|
| 11 |
+
}
|
| 12 |
+
]
|
| 13 |
+
}
|
README.md
CHANGED
|
@@ -4,46 +4,81 @@ We build **Digit**, a local agent for the Russian-language course platform
|
|
| 4 |
[courses.digitable.life](https://courses.digitable.life). Its design constraint is
|
| 5 |
unusual: the language model is not allowed to be a source of facts. Every fact in an
|
| 6 |
answer comes from a deterministic utility, a verbatim quote from the course corpus, or a
|
| 7 |
-
formal certificate — the model only chooses which of those to invoke
|
| 8 |
-
|
| 9 |
-
The models published here are the piece that does the choosing.
|
| 10 |
|
| 11 |
## What is in this organisation
|
| 12 |
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
| Repository | What it is |
|
| 19 |
|---|---|
|
| 20 |
| [`digit-router-0.6b`](https://huggingface.co/digitable-lol/digit-router-0.6b) | The shipping router. LoRA adapters (v1/v2/v3) over `Qwen/Qwen3-0.6B` plus merged GGUF quantisations. 424 MiB at Q5_K_M, ~500 ms per full two-step routing cycle on 8 CPU threads. |
|
| 21 |
| [`digit-router-1.7b`](https://huggingface.co/digitable-lol/digit-router-1.7b) | The same training run at 1.7B. Higher routing accuracy; measurably not worth 3× the parameters for this task. |
|
| 22 |
| [`digit-router-experiments`](https://huggingface.co/digitable-lol/digit-router-experiments) | Three adapters that lost — Vikhr, ruadapt, QVikhr-3. Published so the negative result stays reproducible instead of becoming folklore. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
## How to read our numbers
|
| 25 |
|
| 26 |
-
|
| 27 |
report these numbers is misleading:
|
| 28 |
|
| 29 |
* **A counted refusal is not a conscious refusal.** An eval harness scores an unparseable
|
| 30 |
answer as a refusal, so a model that merely breaks scores like a model that knows when
|
| 31 |
to decline. We always report both columns. The untuned 0.6B base scores 75.3 % counted
|
| 32 |
against 9.3 % conscious — a 66-point gap that is entirely broken output.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
* **Known defects are stated before the good tables, not in a footnote.** The imatrix
|
| 34 |
-
quantisation of the 0.6B
|
| 35 |
-
do-not-deploy warning next to the file. At Q4 the routers
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
Every published file's sha256 is recorded in a `MANIFEST.json` in its repository. We track
|
| 41 |
runs by weight hash rather than by tag, because a tag was once re-created from a different
|
| 42 |
build while a 250-task evaluation was in flight.
|
| 43 |
|
| 44 |
-
Base models are `Qwen/Qwen3-*` under Apache-2.0. The training data is derived from a
|
| 45 |
GPL-3.0 utility catalogue; we state that provenance on every page and do not claim to have
|
| 46 |
-
resolved what it means for weights.
|
|
|
|
|
|
|
| 47 |
|
| 48 |
---
|
| 49 |
|
|
@@ -53,20 +88,38 @@ resolved what it means for weights.
|
|
| 53 |
[courses.digitable.life](https://courses.digitable.life). Ограничение архитектуры
|
| 54 |
необычное: языковой модели запрещено быть источником фактов. Содержание ответа даёт
|
| 55 |
детерминированная утилита, дословная цитата из корпуса курсов или формальный сертификат.
|
| 56 |
-
Модель выбирает, что вызвать, — и т
|
| 57 |
-
|
| 58 |
-
**Здесь лежат маршрутизаторы, а не универсальные ассистенты.**
|
| 59 |
-
категории инструментов и извлекает аргументы либо
|
| 60 |
-
|
| 61 |
-
и
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
sha256 каждого опубликованного файла записан в `MANIFEST.json` соответствующего
|
| 72 |
репозитория.
|
|
|
|
| 4 |
[courses.digitable.life](https://courses.digitable.life). Its design constraint is
|
| 5 |
unusual: the language model is not allowed to be a source of facts. Every fact in an
|
| 6 |
answer comes from a deterministic utility, a verbatim quote from the course corpus, or a
|
| 7 |
+
formal certificate — the model only chooses which of those to invoke, or writes the
|
| 8 |
+
artefact that a checker then verifies.
|
|
|
|
| 9 |
|
| 10 |
## What is in this organisation
|
| 11 |
|
| 12 |
+
Two lines of work, and neither produces a general-purpose assistant.
|
| 13 |
+
|
| 14 |
+
**Routers** map a user query to a tool category and emit a tool call with extracted
|
| 15 |
+
arguments — or refuse. They never write the content of an answer. Load one as a chat
|
| 16 |
+
model and you will get nonsense, and none of our published metrics describe that use.
|
| 17 |
+
|
| 18 |
+
**The specification generator** writes executable
|
| 19 |
+
[FTS](https://github.com/digitable-lol/flang) specifications from a task stated in
|
| 20 |
+
ordinary Russian. Its outputs are not trusted either: they are compiled, type checked,
|
| 21 |
+
executed and proved by a real toolchain, and only what survives counts.
|
| 22 |
+
|
| 23 |
+
### Models
|
| 24 |
|
| 25 |
| Repository | What it is |
|
| 26 |
|---|---|
|
| 27 |
| [`digit-router-0.6b`](https://huggingface.co/digitable-lol/digit-router-0.6b) | The shipping router. LoRA adapters (v1/v2/v3) over `Qwen/Qwen3-0.6B` plus merged GGUF quantisations. 424 MiB at Q5_K_M, ~500 ms per full two-step routing cycle on 8 CPU threads. |
|
| 28 |
| [`digit-router-1.7b`](https://huggingface.co/digitable-lol/digit-router-1.7b) | The same training run at 1.7B. Higher routing accuracy; measurably not worth 3× the parameters for this task. |
|
| 29 |
| [`digit-router-experiments`](https://huggingface.co/digitable-lol/digit-router-experiments) | Three adapters that lost — Vikhr, ruadapt, QVikhr-3. Published so the negative result stays reproducible instead of becoming folklore. |
|
| 30 |
+
| [`specgen-qwen3-1.7b`](https://huggingface.co/digitable-lol/specgen-qwen3-1.7b) | Writes executable FTS specifications from Russian prose. 99.4 % of its output passes the real compiler on a held-out set whose templates and subject areas it never saw; the untuned base scores 0. 133 MiB of LoRA, trained in 57 minutes inside 10 GiB. |
|
| 31 |
+
|
| 32 |
+
### Datasets
|
| 33 |
+
|
| 34 |
+
| Repository | What it is |
|
| 35 |
+
|---|---|
|
| 36 |
+
| [`digit-router-dataset`](https://huggingface.co/datasets/digitable-lol/digit-router-dataset) | 34 709 Russian rows for the two-step router over a 95-utility catalogue. 23.8 % are refusals, and that fraction is the point. |
|
| 37 |
+
| [`hard-negatives-ru`](https://huggingface.co/datasets/digitable-lol/hard-negatives-ru) | 8 470 machine-verified near-miss question–chunk pairs, 2 967 paraphrases, and a 498-item twins set on which a zero-false-accept threshold was measured to cost 94 % of recall. |
|
| 38 |
+
| [`digit-eval-tasks`](https://huggingface.co/datasets/digitable-lol/digit-eval-tasks) | 400 Russian evaluation tasks, 250 main and 150 red-team, for a system required to answer only from a tool, a quote or a certificate. |
|
| 39 |
+
| [`fts-specgen-dataset`](https://huggingface.co/datasets/digitable-lol/fts-specgen-dataset) | 10 500 "Russian task → executable FTS specification" pairs, every one compiler-checked — plus a **negative control of 4 400 deliberately corrupted documents** measuring what the checker misses. |
|
| 40 |
|
| 41 |
## How to read our numbers
|
| 42 |
|
| 43 |
+
Three conventions run through every card, and each exists because the obvious way to
|
| 44 |
report these numbers is misleading:
|
| 45 |
|
| 46 |
* **A counted refusal is not a conscious refusal.** An eval harness scores an unparseable
|
| 47 |
answer as a refusal, so a model that merely breaks scores like a model that knows when
|
| 48 |
to decline. We always report both columns. The untuned 0.6B base scores 75.3 % counted
|
| 49 |
against 9.3 % conscious — a 66-point gap that is entirely broken output.
|
| 50 |
+
|
| 51 |
+
* **A filter that never rejects anything is not evidence.** "All 10 500 documents passed
|
| 52 |
+
the compiler" is worth nothing until somebody shows the compiler can fail things, so
|
| 53 |
+
`fts-specgen-dataset` ships 4 400 documents corrupted on purpose in eleven ways and the
|
| 54 |
+
verdict on each. The gate catches 91.6 %; the card names the three corruptions it
|
| 55 |
+
catches worst and why. That measurement also *changed the work*: one class was caught
|
| 56 |
+
5.0 % of the time until the generator was taught to emit boundary examples, after which
|
| 57 |
+
it was 72.8 %.
|
| 58 |
+
|
| 59 |
* **Known defects are stated before the good tables, not in a footnote.** The imatrix
|
| 60 |
+
quantisation of the 0.6B router measurably breaks its ability to refuse and carries a
|
| 61 |
+
do-not-deploy warning next to the file. At Q4 the routers emit structurally flawless
|
| 62 |
+
tool calls with invented arguments, and a GBNF grammar does not catch that. Our v3
|
| 63 |
+
adapters are a routing regression against v2 on a single seed. The spec generator omits
|
| 64 |
+
the theorem in 127 of 1 491 otherwise-valid documents, and its own training data turns
|
| 65 |
+
out never to have shown it a utility and a theorem in the same document — a hole found
|
| 66 |
+
only after the model made it visible. All of this is on the model pages.
|
| 67 |
+
|
| 68 |
+
A fourth, specific to the verified work: **compiling is not meaning.** Our gate proves a
|
| 69 |
+
specification parses, type checks, runs its own examples and proves its own theorem. It
|
| 70 |
+
does not prove the document says what the request asked for. We have not measured that,
|
| 71 |
+
and we say so rather than letting a 99.4 % imply it.
|
| 72 |
|
| 73 |
Every published file's sha256 is recorded in a `MANIFEST.json` in its repository. We track
|
| 74 |
runs by weight hash rather than by tag, because a tag was once re-created from a different
|
| 75 |
build while a 250-task evaluation was in flight.
|
| 76 |
|
| 77 |
+
Base models are `Qwen/Qwen3-*` under Apache-2.0. The router training data is derived from a
|
| 78 |
GPL-3.0 utility catalogue; we state that provenance on every page and do not claim to have
|
| 79 |
+
resolved what it means for weights. The FTS work has no such question — the language, the
|
| 80 |
+
compiler and the gate are ours under BSD-2-Clause — and the cards say so plainly instead of
|
| 81 |
+
reusing the cautious wording where it is not needed.
|
| 82 |
|
| 83 |
---
|
| 84 |
|
|
|
|
| 88 |
[courses.digitable.life](https://courses.digitable.life). Ограничение архитектуры
|
| 89 |
необычное: языковой модели запрещено быть источником фактов. Содержание ответа даёт
|
| 90 |
детерминированная утилита, дословная цитата из корпуса курсов или формальный сертификат.
|
| 91 |
+
Модель выбирает, что вызвать, — либо пишет артефакт, который затем проверяет программа.
|
| 92 |
+
|
| 93 |
+
**Здесь лежат маршрутизаторы и генератор спецификаций, а не универсальные ассистенты.**
|
| 94 |
+
Маршрутизатор относит запрос к категории инструментов и извлекает аргументы либо
|
| 95 |
+
отказывается; ответ он не пишет. Генератор спецификаций пишет исполняемые документы
|
| 96 |
+
[FTS](https://github.com/digitable-lol/flang) — и им тоже не доверяют: их компилируют,
|
| 97 |
+
типизируют, исполняют и доказывают настоящим инструментарием.
|
| 98 |
+
|
| 99 |
+
Три правила чтения наших чисел.
|
| 100 |
+
|
| 101 |
+
**Первое: засчитанный отказ ≠ ос��знанный** — харнесс считает отказом любой неразбираемый
|
| 102 |
+
ответ, поэтому сломанная модель выглядит как осторожная; мы всегда печатаем обе колонки.
|
| 103 |
+
|
| 104 |
+
**Второе: фильтр, который никогда ничего не отверг, — не доказательство.** Фраза «все
|
| 105 |
+
10 500 документов прошли компилятор» ничего не стоит, пока не показано, что компилятор
|
| 106 |
+
вообще способен что-то отвергнуть. Поэтому в `fts-specgen-dataset` опубликованы 4 400
|
| 107 |
+
намеренно испорченных документов (одиннадцать видов порчи) и вердикт по каждому. Гейт
|
| 108 |
+
ловит 91,6 %, и в карточке названы три вида порчи, которые он ловит хуже всего, и почему.
|
| 109 |
+
Этот замер ещё и изменил работу: один класс ловился на 5,0 %, пока генератор не научили
|
| 110 |
+
добавлять граничные примеры, — после этого 72,8 %.
|
| 111 |
+
|
| 112 |
+
**Третье: известные дефекты стоят до таблиц с хорошими числами, а не в примечаниях.**
|
| 113 |
+
imatrix-квант 0.6B ломает способность отказываться и помечен как непригодный к поставке;
|
| 114 |
+
при Q4 модель выдаёт структурно безупречные вызовы с выдуманными аргументами, и грамматика
|
| 115 |
+
этого не ловит; адаптеры v3 — измеренный регресс против v2 на одном seed; генератор
|
| 116 |
+
спецификаций теряет теорему в 127 документах из 1 491 формально валидных — и его обучающие
|
| 117 |
+
данные, как выяснилось, **ни разу** не показали ему утилиту и теорему в одном документе.
|
| 118 |
+
|
| 119 |
+
И четвёртое, для верифицированной части: **компилируется — не значит «означает».** Гейт
|
| 120 |
+
доказывает, что документ разбирается, типизируется, исполняет свои примеры и доказывает
|
| 121 |
+
свою теорему. Что он означает ровно то, что просили словами, — не доказывает. Мы это не
|
| 122 |
+
измеряли и пишем об этом прямо, а не даём числу 99,4 % намекнуть на большее.
|
| 123 |
|
| 124 |
sha256 каждого опубликованного файла записан в `MANIFEST.json` соответствующего
|
| 125 |
репозитория.
|