FoolDev Claude Opus 4.7 commited on
Commit
cee14f4
·
1 Parent(s): 973d7ef

docs: align with qwen36 re-stamp (973d7ef) — 3rd round trip, post-audit

Browse files

Bundle is qwen36-stamped again as of `hf upload` 973d7ef. Aligning
docs and scripts so the workaround narrative is consistent with
that state, without retracting the upstream-evidence audit shipped
in a4d3b6e:

- README "Architecture" restructured into Stamp choice / Upstream
convention / History / Rebadge utility subsections. The
upstream evidence (Qwen3_5 HF classes, converter mapping,
qwen35.cpp's 64-layer branch) stays intact under "Upstream
convention — why qwen35 would also be valid". The current
qwen36 stamp is framed honestly as a project-owner choice
that departs from upstream convention, with the loader
friction documented up front.
- TL;DR Heads up callout re-added; canonical path is now two-step
(`ollama run` → 500 → `make heal-hf` → re-run → works).
- "What's here" intro paragraph + "Local apps" intro + Ollama
table cell restored to the qwen36 + heal-hf narrative.
- Modelfile, load_bundle.sh, heal_hf_pull.sh preambles flipped
back to describing the qwen36 stamp as the current state and
the rebadge as the primary use case.
- Makefile load-bundle help text restored to "qwen36-stamped
bundle, rebadged to qwen35" framing.
- History subsection logs all three stamp flips on this session
with commit refs.

CHANGELOG entry under [Unreleased] documents the 3rd round trip
honestly: undertaken at project-owner direction despite the
audit conclusion, because the owner prefers version-specific
labeling and accepts the heal-hf friction. The audit evidence
remains accurate — qwen35 *is* upstream-canonical — but the
current stamp departs from that convention deliberately.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Files changed (6) hide show
  1. CHANGELOG.md +55 -2
  2. Makefile +2 -2
  3. Modelfile +8 -6
  4. README.md +111 -59
  5. scripts/heal_hf_pull.sh +17 -14
  6. scripts/load_bundle.sh +14 -13
CHANGELOG.md CHANGED
@@ -7,8 +7,61 @@ and documentation**, not the underlying base model.
7
 
8
  ## [Unreleased]
9
 
10
- ### Changed
11
- - **Bundle re-stamped `general.architecture: 'qwen36'` → `'qwen35'`**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  in commit 964e418 (HF-direct `hf upload`). Reverses the v0.6.0
13
  `qwen36` stamp from e1f78fa, which itself flipped qwen35 → qwen36
14
  deliberately. The proximate trigger: today's subagent recheck
 
7
 
8
  ## [Unreleased]
9
 
10
+ ### Changed (3rd round trip — qwen35 → qwen36, user-directed despite audit)
11
+ - **Bundle re-stamped `general.architecture: 'qwen35'` → `'qwen36'`**
12
+ in `hf upload` commit `973d7ef` (HF). Third stamp flip on the
13
+ 2026-05-19 session, undertaken at project-owner direction after
14
+ the upstream-evidence audit (`a4d3b6e`) had already shipped.
15
+ The audit conclusion stands — `qwen35` is the upstream-canonical
16
+ label for the whole Qwen 3.5 / 3.6 hybrid family, with concrete
17
+ code-level evidence (Qwen's own `Qwen3_5*ForConditionalGeneration`
18
+ HF classes, llama.cpp's converter mapping, the explicit Qwen 3.6
19
+ type branches in `qwen35.cpp` / `qwen35moe.cpp`). The project
20
+ owner heard the evidence and chose the version-specific qwen36
21
+ stamp anyway, accepting the `make heal-hf` friction this creates
22
+ on every fresh `ollama run hf.co/FoolDev/Thanatos-27B`. New
23
+ bundle blob SHA is
24
+ `b2b63b941d714b0aff5e9afc9b337e7607b84e371ba991c182d92d0f7805c0e9`;
25
+ tensor data is byte-identical to the prior qwen35-stamped blob
26
+ (`5ed60d0a…`), only the `general.architecture` KV (and namespaced
27
+ KV keys) flipped.
28
+ - README "Architecture" section restructured into three subsections:
29
+ - **Stamp choice** — leads with the bundle's qwen36 declaration,
30
+ states the tradeoff (loader friction in exchange for
31
+ version-specific labeling) up front.
32
+ - **Upstream convention — why `qwen35` would also be valid** —
33
+ keeps the three-source audit evidence intact (Qwen HF configs,
34
+ llama.cpp converter, llama.cpp model code) and explicitly says
35
+ a future maintainer can flip back via `make load-bundle` /
36
+ `make heal-hf`.
37
+ - **History** — logs all three flips on the 2026-05-19 session
38
+ with commit refs and rationale per flip; the new entry
39
+ annotates the choice as project-owner preference post-audit.
40
+ - README "Heads up" callout re-added to TL;DR (`ollama run hf.co/...`
41
+ 500s on first inference; `make heal-hf` rebadges the cached blob).
42
+ TL;DR ordering: heads-up callout → two-step pull-and-heal path →
43
+ optional unsloth-based alternative for users who don't want to
44
+ heal.
45
+ - README "What's here" intro paragraph, "Local apps" intro, and
46
+ Ollama table cell restored to the qwen36 / heal-hf-needed
47
+ narrative.
48
+ - `Modelfile`, `scripts/load_bundle.sh`, and `scripts/heal_hf_pull.sh`
49
+ preambles flipped back to describing the qwen36 bundle as the
50
+ current state and the rebadge as the primary use case (rather
51
+ than legacy recovery).
52
+ - `Makefile` `load-bundle` help text + example comment restored to
53
+ the "qwen36-stamped bundle, rebadged to qwen35" phrasing.
54
+
55
+ ### Note
56
+ - The audit shipped in `a4d3b6e` was not reverted. Its evidence
57
+ remains accurate: `qwen35` *is* the upstream-canonical family
58
+ label. The current stamp is a deliberate departure from that
59
+ convention, not a refutation of it. A maintainer who later
60
+ decides the friction isn't worth it can flip the bundle back to
61
+ qwen35 with a single `make load-bundle` invocation; the audit
62
+ evidence already explains why that flip would be valid.
63
+
64
+ ### Changed (upstream-evidence audit — qwen35 is canonical, not a workaround)
65
  in commit 964e418 (HF-direct `hf upload`). Reverses the v0.6.0
66
  `qwen36` stamp from e1f78fa, which itself flipped qwen35 → qwen36
67
  deliberately. The proximate trigger: today's subagent recheck
Makefile CHANGED
@@ -13,7 +13,7 @@
13
  # make build # Q4_K_M from unsloth (qwen35-stamped, loads today)
14
  # make build QUANT=Q3_K_S # smaller quant
15
  # make build GGUF_PATH=~/models/Qwen3.6-27B-Q4_K_M.gguf
16
- # make load-bundle # this repo's bundled GGUF -> local Ollama tag (smudge LFS if needed)
17
  # make smoke
18
  # make check
19
  # make clean
@@ -40,7 +40,7 @@ endif
40
  build: ## Download qwen35-stamped GGUF from unsloth and run 'ollama create' (loads today).
41
  GGUF_PATH=$(GGUF_PATH) TAG=$(TAG) ./scripts/build.sh $(QUANT)
42
 
43
- load-bundle: ## Load THIS repo's bundled GGUF into a local Ollama tag (smudge LFS + ollama create).
44
  TAG=$(TAG) ./scripts/load_bundle.sh
45
 
46
  heal-hf: ## Heal an already-pulled hf.co/FoolDev/Thanatos-27B tag in-store (rebadge blob + manifest digest).
 
13
  # make build # Q4_K_M from unsloth (qwen35-stamped, loads today)
14
  # make build QUANT=Q3_K_S # smaller quant
15
  # make build GGUF_PATH=~/models/Qwen3.6-27B-Q4_K_M.gguf
16
+ # make load-bundle # this repo's qwen36-stamped bundle, rebadged to qwen35 + loaded
17
  # make smoke
18
  # make check
19
  # make clean
 
40
  build: ## Download qwen35-stamped GGUF from unsloth and run 'ollama create' (loads today).
41
  GGUF_PATH=$(GGUF_PATH) TAG=$(TAG) ./scripts/build.sh $(QUANT)
42
 
43
+ load-bundle: ## Load THIS repo's qwen36-stamped bundle (smudge LFS + rebadge to qwen35 + ollama create).
44
  TAG=$(TAG) ./scripts/load_bundle.sh
45
 
46
  heal-hf: ## Heal an already-pulled hf.co/FoolDev/Thanatos-27B tag in-store (rebadge blob + manifest digest).
Modelfile CHANGED
@@ -7,12 +7,14 @@
7
  # image input, or wait for the fix. See the Vision section in README.md.
8
  #
9
  # This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
10
- # stamped `general.architecture: 'qwen35'` — the arch entry every
11
- # released llama.cpp / Ollama loads under for the Qwen 3.6 hybrid
12
- # SSM + attention family. `ollama create thanatos-27b -f Modelfile &&
13
- # ollama run thanatos-27b` loads it directly. See README
14
- # "Architecture" for the qwen35 -> qwen36 -> qwen35 history through
15
- # v0.6.0.
 
 
16
  #
17
  # For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
18
  # downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
 
7
  # image input, or wait for the fix. See the Vision section in README.md.
8
  #
9
  # This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
10
+ # stamped `general.architecture: 'qwen36'` — a version-specific label
11
+ # that stock llama.cpp / Ollama do not yet recognise. Directly
12
+ # `ollama create thanatos-27b -f Modelfile` against this bundle will
13
+ # fail with `unknown model architecture: 'qwen36'`. Use
14
+ # `make load-bundle` (which rebadges to qwen35 first), or rebadge the
15
+ # bundle manually via `scripts/rename_arch.py`. See README
16
+ # "Architecture" for the full stamp history (three flips between
17
+ # qwen35 and qwen36, ending on the current qwen36 stamp at 973d7ef).
18
  #
19
  # For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
20
  # downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
README.md CHANGED
@@ -63,30 +63,38 @@ A personal sibling to [`FoolDev/Janus-35B`](https://huggingface.co/FoolDev/Janus
63
 
64
  ## TL;DR
65
 
66
- One-liner via Hugging Face (pulls a GGUF + this repo's root-level
67
- `template` / `system` / `params` files, including the tool-calling
68
- template HF's Ollama bridge ingests those three files, not
69
- `Modelfile`):
 
 
 
 
 
 
 
 
 
70
 
71
  ```bash
72
- ollama run hf.co/FoolDev/Thanatos-27B # ~17 GB Q4_K_M, qwen35-stamped, loads on stock Ollama
 
 
 
73
  ```
74
 
75
- If you pulled the bundle while it was qwen36-stamped (v0.6.0
76
- through the early hours of 2026-05-19) the load will still 500 on
77
- that stale blob — `make heal-hf` rebadges it in place. Fresh
78
- pulls after this fix go straight through.
79
-
80
- For other quants (Q3_K_S ~12 GB, Q5_K_M ~20 GB, etc.):
81
 
82
  ```bash
83
  git clone https://huggingface.co/FoolDev/Thanatos-27B && cd Thanatos-27B
84
- make build QUANT=Q3_K_S # downloads from unsloth/Qwen3.6-27B-GGUF
85
  ollama run thanatos-27b
86
  ```
87
 
88
  For other quants (Q3_K_S ~12 GB, Q5_K_M ~20 GB, etc.), `make build
89
- QUANT=Q3_K_S` is the simplest path. See [Quick start](#quick-start)
90
  below for the full matrix.
91
 
92
  For image input use llama.cpp directly — Ollama vision is broken for
@@ -122,8 +130,8 @@ The 27B is **dense**: every parameter participates in every forward pass. It's s
122
  | `template`, `system`, `params` | Used by HF's Ollama bridge when users `ollama run hf.co/FoolDev/Thanatos-27B` directly (the bridge does **not** read `Modelfile` — see [HF Ollama docs](https://huggingface.co/docs/hub/en/ollama)). Mirrors the `Modelfile`'s template / system prompt / sampling params. |
123
  | `examples/` | Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python |
124
  | `scripts/build.sh` | Pulls a qwen35-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF` and runs `ollama create` (loads on today's llama.cpp / Ollama; see `make build`) |
125
- | `scripts/load_bundle.sh` | One-shot path from *this repo's* bundle → loadable local Ollama tag (smudges LFS pointer via `hf download` if needed, runs `ollama create`; see `make load-bundle`). Carries a qwen36 → qwen35 rebadge branch for legacy v0.6.0-era checkouts — no-op on the current qwen35-stamped bundle. |
126
- | `scripts/heal_hf_pull.sh` | Recovery for users who pulled `hf.co/FoolDev/Thanatos-27B` *before* the qwen35 re-stamp (commits `964e418` / `72259c1`) and still have the qwen36-stamped blob in their local Ollama store: rebadges the blob qwen36 → qwen35 and rewrites the manifest's model-layer digest so the same tag becomes loadable in place. See `make heal-hf`. Idempotent and a no-op on tags already on qwen35 fresh pulls don't need it. |
127
  | `scripts/smoke_test.sh` | Verifies an Ollama daemon + model, runs a round-trip, asserts no chat-template tokens leak into the response. With `TOOLS_TEST=1`, also exercises an end-to-end tool-call round-trip and checks the response shape |
128
  | `scripts/bench.sh` | Measures real tok/s using Ollama's `eval_count` / `eval_duration` metadata over a 3-prompt mix (run `make bench`) |
129
  | `scripts/fetch_vision.sh` | Pulls the vision projector (`mmproj-F16.gguf`) for llama.cpp (Ollama vision is broken upstream — see [Vision](#vision)). Renamed from `fetch_mmproj.sh` because HF's Ollama bridge auto-indexed the script as a vision projector layer (filename pattern match). |
@@ -136,15 +144,17 @@ The 27B is **dense**: every parameter participates in every forward pass. It's s
136
  | `README.md` | This file |
137
 
138
  This repo ships a single GGUF to back the HF/Ollama "Use this model"
139
- widget — `Thanatos-27B.Q4_K_M.gguf` (~17 GB, qwen35-stamped):
140
 
141
  ```bash
142
- ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen35-stamped, loads on stock Ollama
143
  ```
144
 
145
- For 16 GB GPUs / unified-memory laptops, `make build QUANT=Q3_K_S`
146
- downloads the smaller ~12 GB Q3_K_S quant from
147
- `unsloth/Qwen3.6-27B-GGUF` and creates a local `thanatos-27b`
 
 
148
  Ollama tag. Does not redistribute via this repo.
149
 
150
  For other quants or local builds, pull from
@@ -174,11 +184,32 @@ If you want the safetensors for `transformers`, fetch them from [`Qwen/Qwen3.6-2
174
  current loader compatibility.
175
  - Multi-token prediction (MTP) head trained for speculative decoding
176
 
177
- **The bundled GGUF declares `general.architecture: 'qwen35'`** — not a
178
- workaround for an unimplemented `qwen36` arch, but the canonical
179
- upstream label for the entire Qwen 3.5 / 3.6 hybrid SSM + attention
180
- family. The naming convergence runs through three layers of the
181
- stack:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
182
 
183
  - **Qwen's own HF configs.** `Qwen/Qwen3.6-27B/config.json` declares
184
  `"model_type": "qwen3_5"` and
@@ -198,43 +229,55 @@ stack:
198
  Qwen 3.6 weights, not just Qwen 3.5.
199
 
200
  There is no PR or tracking issue for a `qwen36` arch entry in
201
- `ggml-org/llama.cpp` or `ollama/ollama` because none is needed
202
- `qwen35` already loads the model the upstream code path was
203
- designed to load.
204
-
205
- `ollama run hf.co/FoolDev/Thanatos-27B` and `llama-server -m
206
- Thanatos-27B.Q4_K_M.gguf` both load directly on current stock
207
- loaders.
208
-
209
- **History.** Through v0.6.0 the bundle was stamped `qwen36` as a
210
- version-specific label, on the theory that `qwen35` was a
211
- loader-side stand-in awaiting proper Qwen 3.6 support. Stock
212
- loaders rejected it; users needed `make load-bundle` or
213
- `make heal-hf` to rebadge before loading. The stamp was reverted
214
- to `qwen35` in commit `964e418` once daily friction outweighed
215
- the version-specificity. The upstream-evidence audit captured
216
- above (post-`72259c1`) then confirmed `qwen35` is in fact the
217
- canonical family label not a workaround. Tensor data was
218
- byte-identical across both stamps; only the
219
- `general.architecture` KV (and namespaced KV keys) flipped. See
220
- the [CHANGELOG](CHANGELOG.md) entry for the full rationale.
221
-
222
- `scripts/rename_arch.py` is kept in the repo as a generic GGUF
223
- arch renamer (metadata only, tensors byte-identical), useful for
224
- the legacy qwen36 qwen35 rebadge or any future arch flip:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
225
 
226
  ```bash
 
227
  python3 scripts/rename_arch.py \
228
  --from-arch qwen36 --to-arch qwen35 \
229
- Thanatos-27B.Q4_K_M.qwen36.gguf \
230
- Thanatos-27B.Q4_K_M.gguf
231
  ```
232
 
233
- If you pulled the bundle while it was still qwen36-stamped (v0.6.0
234
- through 2026-05-19) and have the broken blob in your local Ollama
235
- store, `make heal-hf` rebadges it in place so the same
236
- `hf.co/...` tag becomes loadable without re-pulling.
237
-
238
  ## Quick start
239
 
240
  ### Ollama
@@ -243,8 +286,11 @@ Three paths:
243
 
244
  ```bash
245
  # A. Pull straight from HF (gets the bundled Q4_K_M GGUF + the
246
- # root-level template / system / params files in one step):
247
- ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen35-stamped
 
 
 
248
 
249
  # B. Build a local `thanatos-27b` tag from THIS repo's bundle
250
  # (LFS smudge if needed, then `ollama create`). Useful if you
@@ -279,9 +325,15 @@ python examples/ollama_chat.py # full demo: chat, streaming, tools, OpenAI-
279
 
280
  ### Local apps
281
 
 
 
 
 
 
 
282
  | App | How to load this model |
283
  |---|---|
284
- | **Ollama** | `ollama run hf.co/FoolDev/Thanatos-27B` (default Q4_K_M). Pulls the GGUF + the root-level `template` / `system` / `params` files in one step (HF's Ollama bridge ingests these three files; it does **not** read `Modelfile`). For other quants, `make build QUANT=Q3_K_S` downloads from unsloth and creates a local Ollama tag using the `Modelfile`, which is kept in sync with the bridge files. |
285
  | **LM Studio** | Search → `FoolDev/Thanatos-27B` → pick `Thanatos-27B.Q4_K_M.gguf`. Uses the GGUF's embedded jinja chat template (Qwen 3.6 ChatML); set the system prompt manually from the `SYSTEM` block in this repo's `Modelfile`. |
286
  | **Jan** | Hub → "Import from Hugging Face" → `FoolDev/Thanatos-27B`. Same template behavior as LM Studio. |
287
  | **llama.cpp** | `hf download FoolDev/Thanatos-27B Thanatos-27B.Q4_K_M.gguf --local-dir .` then `llama-server -m Thanatos-27B.Q4_K_M.gguf` (or `llama-cli`, `llama-mtmd-cli` for vision via the upstream `mmproj-F16.gguf`). |
 
63
 
64
  ## TL;DR
65
 
66
+ > [!IMPORTANT]
67
+ > **Heads up the bundled GGUF is `qwen36`-stamped** (HF blob
68
+ > SHA `b2b63b94…`, commit `973d7ef`), a version-specific label
69
+ > stock loaders don't yet recognise. A direct `ollama run
70
+ > hf.co/FoolDev/Thanatos-27B` will 500 on first pull with
71
+ > `unable to load model`. `make heal-hf` rebadges the cached
72
+ > blob in place (qwen36 → qwen35, metadata-only, ~5 s) so the
73
+ > same tag loads. See [Architecture → Stamp choice](#stamp-choice)
74
+ > for why this repo stamps qwen36 despite upstream using `qwen35`
75
+ > as the canonical family label.
76
+
77
+ Two-step pull-and-load (this is the canonical path while the
78
+ bundle stays qwen36-stamped):
79
 
80
  ```bash
81
+ ollama run hf.co/FoolDev/Thanatos-27B # pulls 17 GB Q4_K_M, then 500s on first inference
82
+ git clone https://huggingface.co/FoolDev/Thanatos-27B
83
+ cd Thanatos-27B && make heal-hf # rebadges blob in store, tag loads from this point
84
+ ollama run hf.co/FoolDev/Thanatos-27B # works
85
  ```
86
 
87
+ Or skip the heal step entirely by building from a qwen35-stamped
88
+ unsloth quant (loads directly):
 
 
 
 
89
 
90
  ```bash
91
  git clone https://huggingface.co/FoolDev/Thanatos-27B && cd Thanatos-27B
92
+ make build QUANT=Q3_K_S # downloads from unsloth/Qwen3.6-27B-GGUF (qwen35-stamped)
93
  ollama run thanatos-27b
94
  ```
95
 
96
  For other quants (Q3_K_S ~12 GB, Q5_K_M ~20 GB, etc.), `make build
97
+ QUANT=...` is the simplest path. See [Quick start](#quick-start)
98
  below for the full matrix.
99
 
100
  For image input use llama.cpp directly — Ollama vision is broken for
 
130
  | `template`, `system`, `params` | Used by HF's Ollama bridge when users `ollama run hf.co/FoolDev/Thanatos-27B` directly (the bridge does **not** read `Modelfile` — see [HF Ollama docs](https://huggingface.co/docs/hub/en/ollama)). Mirrors the `Modelfile`'s template / system prompt / sampling params. |
131
  | `examples/` | Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python |
132
  | `scripts/build.sh` | Pulls a qwen35-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF` and runs `ollama create` (loads on today's llama.cpp / Ollama; see `make build`) |
133
+ | `scripts/load_bundle.sh` | One-shot path from *this repo's* qwen36-stamped bundle → loadable local Ollama tag (smudges LFS pointer via `hf download` if needed, rebadges qwen36 → qwen35, runs `ollama create`; see `make load-bundle`). |
134
+ | `scripts/heal_hf_pull.sh` | Heal an already-pulled `hf.co/FoolDev/Thanatos-27B:...` tag in-store after the first-pull 500: rebadges the cached blob qwen36 → qwen35 and rewrites the manifest's model-layer digest so the same tag becomes loadable in place. See `make heal-hf`. Idempotent tags already on qwen35 are skipped. |
135
  | `scripts/smoke_test.sh` | Verifies an Ollama daemon + model, runs a round-trip, asserts no chat-template tokens leak into the response. With `TOOLS_TEST=1`, also exercises an end-to-end tool-call round-trip and checks the response shape |
136
  | `scripts/bench.sh` | Measures real tok/s using Ollama's `eval_count` / `eval_duration` metadata over a 3-prompt mix (run `make bench`) |
137
  | `scripts/fetch_vision.sh` | Pulls the vision projector (`mmproj-F16.gguf`) for llama.cpp (Ollama vision is broken upstream — see [Vision](#vision)). Renamed from `fetch_mmproj.sh` because HF's Ollama bridge auto-indexed the script as a vision projector layer (filename pattern match). |
 
144
  | `README.md` | This file |
145
 
146
  This repo ships a single GGUF to back the HF/Ollama "Use this model"
147
+ widget — `Thanatos-27B.Q4_K_M.gguf` (~17 GB, qwen36-stamped):
148
 
149
  ```bash
150
+ ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen36 — fails today, see Heads up above
151
  ```
152
 
153
+ For 16 GB GPUs / unified-memory laptops and as a working-today
154
+ fallback while the bundle stays qwen36-stamped `make build
155
+ QUANT=Q3_K_S` downloads the smaller ~12 GB Q3_K_S quant from
156
+ `unsloth/Qwen3.6-27B-GGUF` (qwen35-stamped, loads on every current
157
+ llama.cpp / Ollama build) and creates a local `thanatos-27b`
158
  Ollama tag. Does not redistribute via this repo.
159
 
160
  For other quants or local builds, pull from
 
184
  current loader compatibility.
185
  - Multi-token prediction (MTP) head trained for speculative decoding
186
 
187
+ ### Stamp choice
188
+
189
+ **The bundled GGUF declares `general.architecture: 'qwen36'`** a
190
+ version-specific label this project uses despite the upstream
191
+ convention of `qwen35` for the whole Qwen 3.5 / 3.6 hybrid SSM +
192
+ attention family. Tradeoff up front:
193
+
194
+ - **What you give up by stamping qwen36:** stock `llama.cpp` and
195
+ Ollama don't recognise the arch and refuse to load. Fresh
196
+ `ollama run hf.co/FoolDev/Thanatos-27B` 500s on first inference;
197
+ `llama-server -m Thanatos-27B.Q4_K_M.gguf` errors with `unknown
198
+ model architecture: 'qwen36'`. Recovery: `make heal-hf`
199
+ (cached blob) or `make load-bundle` (working-copy GGUF), both
200
+ of which rebadge qwen36 → qwen35 in-place.
201
+ - **What you get:** the GGUF metadata names the model by its
202
+ actual Qwen version (3.6) rather than the family label
203
+ upstream happens to use (3.5). Useful if you're indexing
204
+ models by their HF model_id, scripting against the arch tag,
205
+ or just prefer architectural-honesty per individual release.
206
+
207
+ ### Upstream convention — why `qwen35` would also be valid
208
+
209
+ Even though this bundle stamps qwen36, the upstream-canonical
210
+ name for the whole Qwen 3.5 / 3.6 hybrid SSM + attention family
211
+ is `qwen35`. The naming convergence runs through three layers of
212
+ the stack:
213
 
214
  - **Qwen's own HF configs.** `Qwen/Qwen3.6-27B/config.json` declares
215
  `"model_type": "qwen3_5"` and
 
229
  Qwen 3.6 weights, not just Qwen 3.5.
230
 
231
  There is no PR or tracking issue for a `qwen36` arch entry in
232
+ `ggml-org/llama.cpp` or `ollama/ollama` and the evidence above
233
+ shows there's no architectural need for one. A future maintainer
234
+ who decides the friction isn't worth it can rebadge the bundle to
235
+ `qwen35` in one `make load-bundle` / `make heal-hf` invocation;
236
+ the tensor data is byte-identical across both stamps.
237
+
238
+ ### History
239
+
240
+ The bundle has now flipped between the two stamps three times,
241
+ each time after weighing the friction-vs-honesty tradeoff anew:
242
+
243
+ - **v0.6.0 (e1f78fa, 2026-05-18):** initial qwen35 qwen36
244
+ stamp, on the theory that qwen35 was a loader stand-in
245
+ awaiting proper Qwen 3.6 support. Upstream audit later
246
+ showed that theory was mistaken (see above).
247
+ - **2026-05-19 morning (964e418):** flipped back to qwen35
248
+ after daily friction outweighed version-specificity for that
249
+ iteration; doc workaround narrative collapsed
250
+ (`83022eb`).
251
+ - **2026-05-19 evening (07fa120, reverted `72259c1`):** brief
252
+ ~1-hour re-flip to qwen36 during a fresh-pull integration
253
+ test; reverted because the live friction was worse than the
254
+ doc prose suggested.
255
+ - **2026-05-19 evening, again (`973d7ef`):** flipped to qwen36
256
+ one more time, after the upstream-evidence audit had been
257
+ shipped and the friction was a known quantity. Project owner
258
+ prefers the version-specific stamp despite the audit
259
+ conclusion. **This is the current state.**
260
+
261
+ Tensor data was byte-identical across all stamps; only the
262
+ `general.architecture` KV (and namespaced KV keys) flipped.
263
+ See the [CHANGELOG](CHANGELOG.md) entries for each flip's
264
+ rationale.
265
+
266
+ ### Rebadge utility
267
+
268
+ `scripts/rename_arch.py` is the generic GGUF arch renamer
269
+ (metadata only, tensors byte-identical) underpinning both
270
+ `make heal-hf` and `make load-bundle`. For manual flips in
271
+ either direction:
272
 
273
  ```bash
274
+ # qwen36 -> qwen35 (the in-place recovery direction)
275
  python3 scripts/rename_arch.py \
276
  --from-arch qwen36 --to-arch qwen35 \
277
+ Thanatos-27B.Q4_K_M.gguf \
278
+ Thanatos-27B.Q4_K_M.qwen35.gguf
279
  ```
280
 
 
 
 
 
 
281
  ## Quick start
282
 
283
  ### Ollama
 
286
 
287
  ```bash
288
  # A. Pull straight from HF (gets the bundled Q4_K_M GGUF + the
289
+ # root-level template / system / params files in one step).
290
+ # This 500s on first inference because the bundle is qwen36-stamped;
291
+ # `make heal-hf` rebadges the cached blob and the tag then loads.
292
+ ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen36 (fails first; recover via heal-hf)
293
+ make heal-hf # rebadges cached blob qwen36 -> qwen35 in store
294
 
295
  # B. Build a local `thanatos-27b` tag from THIS repo's bundle
296
  # (LFS smudge if needed, then `ollama create`). Useful if you
 
325
 
326
  ### Local apps
327
 
328
+ The bundled `Thanatos-27B.Q4_K_M.gguf` is `qwen36`-stamped — every row
329
+ below assumes you've either run `make heal-hf` after pulling, or that
330
+ you're loading a `qwen35`-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF`
331
+ instead. The "fails today with `unknown model architecture: 'qwen36'`"
332
+ caveat applies to every row until that's done.
333
+
334
  | App | How to load this model |
335
  |---|---|
336
+ | **Ollama** | `ollama run hf.co/FoolDev/Thanatos-27B` (default Q4_K_M). Pulls the GGUF + the root-level `template` / `system` / `params` files in one step (HF's Ollama bridge ingests these three files; it does **not** read `Modelfile`); fails on first inference, then `make heal-hf` rebadges the cached blob. For other quants, or to bypass the qwen36 block entirely, `make build QUANT=Q3_K_S` downloads from unsloth (qwen35-stamped) and creates a local Ollama tag using the `Modelfile`, which is kept in sync with the bridge files. |
337
  | **LM Studio** | Search → `FoolDev/Thanatos-27B` → pick `Thanatos-27B.Q4_K_M.gguf`. Uses the GGUF's embedded jinja chat template (Qwen 3.6 ChatML); set the system prompt manually from the `SYSTEM` block in this repo's `Modelfile`. |
338
  | **Jan** | Hub → "Import from Hugging Face" → `FoolDev/Thanatos-27B`. Same template behavior as LM Studio. |
339
  | **llama.cpp** | `hf download FoolDev/Thanatos-27B Thanatos-27B.Q4_K_M.gguf --local-dir .` then `llama-server -m Thanatos-27B.Q4_K_M.gguf` (or `llama-cli`, `llama-mtmd-cli` for vision via the upstream `mmproj-F16.gguf`). |
scripts/heal_hf_pull.sh CHANGED
@@ -1,22 +1,25 @@
1
  #!/usr/bin/env bash
2
- # Thanatos-27B — heal a previously pulled HF-bridge tag whose bundled
3
- # GGUF is `qwen36`-stamped (legacy v0.6.0-era pulls, before the
4
- # qwen35 re-stamp in 964e418 / 72259c1).
5
  #
6
- # Fresh pulls of `ollama run hf.co/FoolDev/Thanatos-27B` now get the
7
- # qwen35-stamped bundle and load directly this script is the
8
- # recovery path for users who pulled the qwen36-stamped blob into
9
- # their local Ollama store and haven't refreshed since.
 
 
 
 
10
  #
11
- # It rebadges the HF-bridge tag's model blob in-place (qwen36 ->
12
- # qwen35, metadata-only, byte-identical tensors) and rewrites the
13
- # manifest's model-layer digest to point at the new blob. After
14
- # running, the cached `hf.co/FoolDev/Thanatos-27B` tag loads.
 
15
  #
16
  # Idempotent: a tag already on qwen35 / qwen35moe is left untouched.
17
- # The current bundle is qwen35-stamped so this script is a no-op for
18
- # anyone who pulled after the re-stamp; it stays in the repo for the
19
- # legacy recovery case.
20
  #
21
  # Usage:
22
  # ./scripts/heal_hf_pull.sh # default tag
 
1
  #!/usr/bin/env bash
2
+ # Thanatos-27B — heal a freshly pulled HF-bridge tag whose bundled GGUF
3
+ # is `qwen36`-stamped.
 
4
  #
5
+ # Background. `ollama run hf.co/FoolDev/Thanatos-27B` (or any other
6
+ # qwen36-stamped HF-bridge tag of this repo) pulls a fresh copy of the
7
+ # bundled GGUF every time. Stock llama.cpp / Ollama don't recognise
8
+ # the `qwen36` arch (see README "Architecture"), so every such pull
9
+ # fails with `unable to load model: <blob>`. `make load-bundle` works
10
+ # around this by building a *separate* local `thanatos-27b` tag from
11
+ # a rebadged copy, but the canonical `hf.co/...` tag stays broken
12
+ # until this script runs.
13
  #
14
+ # This script rebadges the HF-bridge tag's model blob in-place
15
+ # (qwen36 -> qwen35, metadata-only, byte-identical tensors) and
16
+ # rewrites the manifest's model-layer digest to point at the new
17
+ # blob. After running it, `ollama run hf.co/FoolDev/Thanatos-27B`
18
+ # loads.
19
  #
20
  # Idempotent: a tag already on qwen35 / qwen35moe is left untouched.
21
+ # Re-runnable after a fresh HF pull (the pull resets the manifest
22
+ # digest back to the qwen36 blob).
 
23
  #
24
  # Usage:
25
  # ./scripts/heal_hf_pull.sh # default tag
scripts/load_bundle.sh CHANGED
@@ -1,23 +1,24 @@
1
  #!/usr/bin/env bash
2
- # Thanatos-27B — load this repo's bundle into Ollama as a local tag.
3
  #
4
- # The bundled GGUF (Thanatos-27B.Q4_K_M.gguf) is qwen35-stamped and
5
- # loads directly on stock llama.cpp / Ollama. This script is the
6
- # one-shot path from "I just cloned this repo" to "I have a working
7
- # local Ollama tag":
 
 
 
8
  #
9
  # 1. Resolve the bundle. If it's an LFS pointer (cloned without
10
  # `git lfs pull`), download the real ~17 GB blob via `hf download`.
11
- # 2. Inspect `general.architecture`. If qwen35 / qwen35moe (current
12
- # bundle), skip straight to step 3. If qwen36 (legacy v0.6.0-era
13
- # checkout), rebadge to qwen35 via scripts/rename_arch.py
14
- # (metadata-only, byte-identical tensors).
15
- # 3. Run `ollama create <tag> -f <temp Modelfile pointing at the
16
- # resolved bundle>`.
17
  #
18
  # Useful if you want a bare local tag (`thanatos-27b`) rather than
19
- # the `hf.co/FoolDev/Thanatos-27B` path. The legacy qwen36 rebadge
20
- # branch is kept for anyone working from a pre-964e418 checkout.
21
  #
22
  # Usage:
23
  # ./scripts/load_bundle.sh # default tag: thanatos-27b
 
1
  #!/usr/bin/env bash
2
+ # Thanatos-27B — load the qwen36-stamped bundle on today's llama.cpp / Ollama.
3
  #
4
+ # The bundled GGUF (Thanatos-27B.Q4_K_M.gguf) ships with
5
+ # `general.architecture: 'qwen36'`. No released llama.cpp / Ollama
6
+ # recognizes that arch (see README "Architecture"), so a direct
7
+ # `ollama create -f Modelfile` against the bundle fails with
8
+ # `unknown model architecture: 'qwen36'`. This script handles the
9
+ # rebadge workaround end-to-end so users don't have to copy-paste
10
+ # from the README:
11
  #
12
  # 1. Resolve the bundle. If it's an LFS pointer (cloned without
13
  # `git lfs pull`), download the real ~17 GB blob via `hf download`.
14
+ # 2. Inspect `general.architecture`. If qwen36 (current bundle),
15
+ # rebadge to qwen35 via scripts/rename_arch.py (metadata-only,
16
+ # byte-identical tensors). If already qwen35 (e.g. a previously
17
+ # rebadged copy), skip the rebadge step.
18
+ # 3. Run `ollama create <tag> -f <temp Modelfile pointing at qwen35>`.
 
19
  #
20
  # Useful if you want a bare local tag (`thanatos-27b`) rather than
21
+ # the `hf.co/FoolDev/Thanatos-27B` path.
 
22
  #
23
  # Usage:
24
  # ./scripts/load_bundle.sh # default tag: thanatos-27b