Instructions to use FoolDev/Thanatos-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FoolDev/Thanatos-27B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="FoolDev/Thanatos-27B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FoolDev/Thanatos-27B", dtype="auto") - llama-cpp-python
How to use FoolDev/Thanatos-27B with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="FoolDev/Thanatos-27B", filename="Thanatos-27B.Q4_K_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use FoolDev/Thanatos-27B with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Use Docker
docker model run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use FoolDev/Thanatos-27B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FoolDev/Thanatos-27B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Thanatos-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- SGLang
How to use FoolDev/Thanatos-27B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FoolDev/Thanatos-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Thanatos-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FoolDev/Thanatos-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Thanatos-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use FoolDev/Thanatos-27B with Ollama:
ollama run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- Unsloth Studio new
How to use FoolDev/Thanatos-27B with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Thanatos-27B to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Thanatos-27B to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for FoolDev/Thanatos-27B to start chatting
- Pi new
How to use FoolDev/Thanatos-27B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FoolDev/Thanatos-27B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use FoolDev/Thanatos-27B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FoolDev/Thanatos-27B:Q4_K_M
Run Hermes
hermes
- Docker Model Runner
How to use FoolDev/Thanatos-27B with Docker Model Runner:
docker model run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- Lemonade
How to use FoolDev/Thanatos-27B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FoolDev/Thanatos-27B:Q4_K_M
Run and chat with the model
lemonade run user.Thanatos-27B-Q4_K_M
List all available models
lemonade list
docs: align with qwen36 re-stamp (973d7ef) — 3rd round trip, post-audit
Browse filesBundle is qwen36-stamped again as of `hf upload` 973d7ef. Aligning
docs and scripts so the workaround narrative is consistent with
that state, without retracting the upstream-evidence audit shipped
in a4d3b6e:
- README "Architecture" restructured into Stamp choice / Upstream
convention / History / Rebadge utility subsections. The
upstream evidence (Qwen3_5 HF classes, converter mapping,
qwen35.cpp's 64-layer branch) stays intact under "Upstream
convention — why qwen35 would also be valid". The current
qwen36 stamp is framed honestly as a project-owner choice
that departs from upstream convention, with the loader
friction documented up front.
- TL;DR Heads up callout re-added; canonical path is now two-step
(`ollama run` → 500 → `make heal-hf` → re-run → works).
- "What's here" intro paragraph + "Local apps" intro + Ollama
table cell restored to the qwen36 + heal-hf narrative.
- Modelfile, load_bundle.sh, heal_hf_pull.sh preambles flipped
back to describing the qwen36 stamp as the current state and
the rebadge as the primary use case.
- Makefile load-bundle help text restored to "qwen36-stamped
bundle, rebadged to qwen35" framing.
- History subsection logs all three stamp flips on this session
with commit refs.
CHANGELOG entry under [Unreleased] documents the 3rd round trip
honestly: undertaken at project-owner direction despite the
audit conclusion, because the owner prefers version-specific
labeling and accepts the heal-hf friction. The audit evidence
remains accurate — qwen35 *is* upstream-canonical — but the
current stamp departs from that convention deliberately.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- CHANGELOG.md +55 -2
- Makefile +2 -2
- Modelfile +8 -6
- README.md +111 -59
- scripts/heal_hf_pull.sh +17 -14
- scripts/load_bundle.sh +14 -13
|
@@ -7,8 +7,61 @@ and documentation**, not the underlying base model.
|
|
| 7 |
|
| 8 |
## [Unreleased]
|
| 9 |
|
| 10 |
-
### Changed
|
| 11 |
-
- **Bundle re-stamped `general.architecture: '
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
in commit 964e418 (HF-direct `hf upload`). Reverses the v0.6.0
|
| 13 |
`qwen36` stamp from e1f78fa, which itself flipped qwen35 → qwen36
|
| 14 |
deliberately. The proximate trigger: today's subagent recheck
|
|
|
|
| 7 |
|
| 8 |
## [Unreleased]
|
| 9 |
|
| 10 |
+
### Changed (3rd round trip — qwen35 → qwen36, user-directed despite audit)
|
| 11 |
+
- **Bundle re-stamped `general.architecture: 'qwen35'` → `'qwen36'`**
|
| 12 |
+
in `hf upload` commit `973d7ef` (HF). Third stamp flip on the
|
| 13 |
+
2026-05-19 session, undertaken at project-owner direction after
|
| 14 |
+
the upstream-evidence audit (`a4d3b6e`) had already shipped.
|
| 15 |
+
The audit conclusion stands — `qwen35` is the upstream-canonical
|
| 16 |
+
label for the whole Qwen 3.5 / 3.6 hybrid family, with concrete
|
| 17 |
+
code-level evidence (Qwen's own `Qwen3_5*ForConditionalGeneration`
|
| 18 |
+
HF classes, llama.cpp's converter mapping, the explicit Qwen 3.6
|
| 19 |
+
type branches in `qwen35.cpp` / `qwen35moe.cpp`). The project
|
| 20 |
+
owner heard the evidence and chose the version-specific qwen36
|
| 21 |
+
stamp anyway, accepting the `make heal-hf` friction this creates
|
| 22 |
+
on every fresh `ollama run hf.co/FoolDev/Thanatos-27B`. New
|
| 23 |
+
bundle blob SHA is
|
| 24 |
+
`b2b63b941d714b0aff5e9afc9b337e7607b84e371ba991c182d92d0f7805c0e9`;
|
| 25 |
+
tensor data is byte-identical to the prior qwen35-stamped blob
|
| 26 |
+
(`5ed60d0a…`), only the `general.architecture` KV (and namespaced
|
| 27 |
+
KV keys) flipped.
|
| 28 |
+
- README "Architecture" section restructured into three subsections:
|
| 29 |
+
- **Stamp choice** — leads with the bundle's qwen36 declaration,
|
| 30 |
+
states the tradeoff (loader friction in exchange for
|
| 31 |
+
version-specific labeling) up front.
|
| 32 |
+
- **Upstream convention — why `qwen35` would also be valid** —
|
| 33 |
+
keeps the three-source audit evidence intact (Qwen HF configs,
|
| 34 |
+
llama.cpp converter, llama.cpp model code) and explicitly says
|
| 35 |
+
a future maintainer can flip back via `make load-bundle` /
|
| 36 |
+
`make heal-hf`.
|
| 37 |
+
- **History** — logs all three flips on the 2026-05-19 session
|
| 38 |
+
with commit refs and rationale per flip; the new entry
|
| 39 |
+
annotates the choice as project-owner preference post-audit.
|
| 40 |
+
- README "Heads up" callout re-added to TL;DR (`ollama run hf.co/...`
|
| 41 |
+
500s on first inference; `make heal-hf` rebadges the cached blob).
|
| 42 |
+
TL;DR ordering: heads-up callout → two-step pull-and-heal path →
|
| 43 |
+
optional unsloth-based alternative for users who don't want to
|
| 44 |
+
heal.
|
| 45 |
+
- README "What's here" intro paragraph, "Local apps" intro, and
|
| 46 |
+
Ollama table cell restored to the qwen36 / heal-hf-needed
|
| 47 |
+
narrative.
|
| 48 |
+
- `Modelfile`, `scripts/load_bundle.sh`, and `scripts/heal_hf_pull.sh`
|
| 49 |
+
preambles flipped back to describing the qwen36 bundle as the
|
| 50 |
+
current state and the rebadge as the primary use case (rather
|
| 51 |
+
than legacy recovery).
|
| 52 |
+
- `Makefile` `load-bundle` help text + example comment restored to
|
| 53 |
+
the "qwen36-stamped bundle, rebadged to qwen35" phrasing.
|
| 54 |
+
|
| 55 |
+
### Note
|
| 56 |
+
- The audit shipped in `a4d3b6e` was not reverted. Its evidence
|
| 57 |
+
remains accurate: `qwen35` *is* the upstream-canonical family
|
| 58 |
+
label. The current stamp is a deliberate departure from that
|
| 59 |
+
convention, not a refutation of it. A maintainer who later
|
| 60 |
+
decides the friction isn't worth it can flip the bundle back to
|
| 61 |
+
qwen35 with a single `make load-bundle` invocation; the audit
|
| 62 |
+
evidence already explains why that flip would be valid.
|
| 63 |
+
|
| 64 |
+
### Changed (upstream-evidence audit — qwen35 is canonical, not a workaround)
|
| 65 |
in commit 964e418 (HF-direct `hf upload`). Reverses the v0.6.0
|
| 66 |
`qwen36` stamp from e1f78fa, which itself flipped qwen35 → qwen36
|
| 67 |
deliberately. The proximate trigger: today's subagent recheck
|
|
@@ -13,7 +13,7 @@
|
|
| 13 |
# make build # Q4_K_M from unsloth (qwen35-stamped, loads today)
|
| 14 |
# make build QUANT=Q3_K_S # smaller quant
|
| 15 |
# make build GGUF_PATH=~/models/Qwen3.6-27B-Q4_K_M.gguf
|
| 16 |
-
# make load-bundle # this repo's
|
| 17 |
# make smoke
|
| 18 |
# make check
|
| 19 |
# make clean
|
|
@@ -40,7 +40,7 @@ endif
|
|
| 40 |
build: ## Download qwen35-stamped GGUF from unsloth and run 'ollama create' (loads today).
|
| 41 |
GGUF_PATH=$(GGUF_PATH) TAG=$(TAG) ./scripts/build.sh $(QUANT)
|
| 42 |
|
| 43 |
-
load-bundle: ## Load THIS repo's
|
| 44 |
TAG=$(TAG) ./scripts/load_bundle.sh
|
| 45 |
|
| 46 |
heal-hf: ## Heal an already-pulled hf.co/FoolDev/Thanatos-27B tag in-store (rebadge blob + manifest digest).
|
|
|
|
| 13 |
# make build # Q4_K_M from unsloth (qwen35-stamped, loads today)
|
| 14 |
# make build QUANT=Q3_K_S # smaller quant
|
| 15 |
# make build GGUF_PATH=~/models/Qwen3.6-27B-Q4_K_M.gguf
|
| 16 |
+
# make load-bundle # this repo's qwen36-stamped bundle, rebadged to qwen35 + loaded
|
| 17 |
# make smoke
|
| 18 |
# make check
|
| 19 |
# make clean
|
|
|
|
| 40 |
build: ## Download qwen35-stamped GGUF from unsloth and run 'ollama create' (loads today).
|
| 41 |
GGUF_PATH=$(GGUF_PATH) TAG=$(TAG) ./scripts/build.sh $(QUANT)
|
| 42 |
|
| 43 |
+
load-bundle: ## Load THIS repo's qwen36-stamped bundle (smudge LFS + rebadge to qwen35 + ollama create).
|
| 44 |
TAG=$(TAG) ./scripts/load_bundle.sh
|
| 45 |
|
| 46 |
heal-hf: ## Heal an already-pulled hf.co/FoolDev/Thanatos-27B tag in-store (rebadge blob + manifest digest).
|
|
@@ -7,12 +7,14 @@
|
|
| 7 |
# image input, or wait for the fix. See the Vision section in README.md.
|
| 8 |
#
|
| 9 |
# This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
|
| 10 |
-
# stamped `general.architecture: '
|
| 11 |
-
#
|
| 12 |
-
#
|
| 13 |
-
#
|
| 14 |
-
#
|
| 15 |
-
#
|
|
|
|
|
|
|
| 16 |
#
|
| 17 |
# For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
|
| 18 |
# downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
|
|
|
|
| 7 |
# image input, or wait for the fix. See the Vision section in README.md.
|
| 8 |
#
|
| 9 |
# This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
|
| 10 |
+
# stamped `general.architecture: 'qwen36'` — a version-specific label
|
| 11 |
+
# that stock llama.cpp / Ollama do not yet recognise. Directly
|
| 12 |
+
# `ollama create thanatos-27b -f Modelfile` against this bundle will
|
| 13 |
+
# fail with `unknown model architecture: 'qwen36'`. Use
|
| 14 |
+
# `make load-bundle` (which rebadges to qwen35 first), or rebadge the
|
| 15 |
+
# bundle manually via `scripts/rename_arch.py`. See README
|
| 16 |
+
# "Architecture" for the full stamp history (three flips between
|
| 17 |
+
# qwen35 and qwen36, ending on the current qwen36 stamp at 973d7ef).
|
| 18 |
#
|
| 19 |
# For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
|
| 20 |
# downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
|
|
@@ -63,30 +63,38 @@ A personal sibling to [`FoolDev/Janus-35B`](https://huggingface.co/FoolDev/Janus
|
|
| 63 |
|
| 64 |
## TL;DR
|
| 65 |
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
```bash
|
| 72 |
-
ollama run hf.co/FoolDev/Thanatos-27B
|
|
|
|
|
|
|
|
|
|
| 73 |
```
|
| 74 |
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
that stale blob — `make heal-hf` rebadges it in place. Fresh
|
| 78 |
-
pulls after this fix go straight through.
|
| 79 |
-
|
| 80 |
-
For other quants (Q3_K_S ~12 GB, Q5_K_M ~20 GB, etc.):
|
| 81 |
|
| 82 |
```bash
|
| 83 |
git clone https://huggingface.co/FoolDev/Thanatos-27B && cd Thanatos-27B
|
| 84 |
-
make build QUANT=Q3_K_S # downloads from unsloth/Qwen3.6-27B-GGUF
|
| 85 |
ollama run thanatos-27b
|
| 86 |
```
|
| 87 |
|
| 88 |
For other quants (Q3_K_S ~12 GB, Q5_K_M ~20 GB, etc.), `make build
|
| 89 |
-
QUANT=
|
| 90 |
below for the full matrix.
|
| 91 |
|
| 92 |
For image input use llama.cpp directly — Ollama vision is broken for
|
|
@@ -122,8 +130,8 @@ The 27B is **dense**: every parameter participates in every forward pass. It's s
|
|
| 122 |
| `template`, `system`, `params` | Used by HF's Ollama bridge when users `ollama run hf.co/FoolDev/Thanatos-27B` directly (the bridge does **not** read `Modelfile` — see [HF Ollama docs](https://huggingface.co/docs/hub/en/ollama)). Mirrors the `Modelfile`'s template / system prompt / sampling params. |
|
| 123 |
| `examples/` | Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python |
|
| 124 |
| `scripts/build.sh` | Pulls a qwen35-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF` and runs `ollama create` (loads on today's llama.cpp / Ollama; see `make build`) |
|
| 125 |
-
| `scripts/load_bundle.sh` | One-shot path from *this repo's* bundle → loadable local Ollama tag (smudges LFS pointer via `hf download` if needed, runs `ollama create`; see `make load-bundle`).
|
| 126 |
-
| `scripts/heal_hf_pull.sh` |
|
| 127 |
| `scripts/smoke_test.sh` | Verifies an Ollama daemon + model, runs a round-trip, asserts no chat-template tokens leak into the response. With `TOOLS_TEST=1`, also exercises an end-to-end tool-call round-trip and checks the response shape |
|
| 128 |
| `scripts/bench.sh` | Measures real tok/s using Ollama's `eval_count` / `eval_duration` metadata over a 3-prompt mix (run `make bench`) |
|
| 129 |
| `scripts/fetch_vision.sh` | Pulls the vision projector (`mmproj-F16.gguf`) for llama.cpp (Ollama vision is broken upstream — see [Vision](#vision)). Renamed from `fetch_mmproj.sh` because HF's Ollama bridge auto-indexed the script as a vision projector layer (filename pattern match). |
|
|
@@ -136,15 +144,17 @@ The 27B is **dense**: every parameter participates in every forward pass. It's s
|
|
| 136 |
| `README.md` | This file |
|
| 137 |
|
| 138 |
This repo ships a single GGUF to back the HF/Ollama "Use this model"
|
| 139 |
-
widget — `Thanatos-27B.Q4_K_M.gguf` (~17 GB,
|
| 140 |
|
| 141 |
```bash
|
| 142 |
-
ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M,
|
| 143 |
```
|
| 144 |
|
| 145 |
-
For 16 GB GPUs / unified-memory laptops
|
| 146 |
-
|
| 147 |
-
`
|
|
|
|
|
|
|
| 148 |
Ollama tag. Does not redistribute via this repo.
|
| 149 |
|
| 150 |
For other quants or local builds, pull from
|
|
@@ -174,11 +184,32 @@ If you want the safetensors for `transformers`, fetch them from [`Qwen/Qwen3.6-2
|
|
| 174 |
current loader compatibility.
|
| 175 |
- Multi-token prediction (MTP) head trained for speculative decoding
|
| 176 |
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
|
| 183 |
- **Qwen's own HF configs.** `Qwen/Qwen3.6-27B/config.json` declares
|
| 184 |
`"model_type": "qwen3_5"` and
|
|
@@ -198,43 +229,55 @@ stack:
|
|
| 198 |
Qwen 3.6 weights, not just Qwen 3.5.
|
| 199 |
|
| 200 |
There is no PR or tracking issue for a `qwen36` arch entry in
|
| 201 |
-
`ggml-org/llama.cpp` or `ollama/ollama`
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
|
| 211 |
-
|
| 212 |
-
|
| 213 |
-
|
| 214 |
-
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
|
| 218 |
-
|
| 219 |
-
`
|
| 220 |
-
|
| 221 |
-
|
| 222 |
-
|
| 223 |
-
|
| 224 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 225 |
|
| 226 |
```bash
|
|
|
|
| 227 |
python3 scripts/rename_arch.py \
|
| 228 |
--from-arch qwen36 --to-arch qwen35 \
|
| 229 |
-
Thanatos-27B.Q4_K_M.
|
| 230 |
-
Thanatos-27B.Q4_K_M.gguf
|
| 231 |
```
|
| 232 |
|
| 233 |
-
If you pulled the bundle while it was still qwen36-stamped (v0.6.0
|
| 234 |
-
through 2026-05-19) and have the broken blob in your local Ollama
|
| 235 |
-
store, `make heal-hf` rebadges it in place so the same
|
| 236 |
-
`hf.co/...` tag becomes loadable without re-pulling.
|
| 237 |
-
|
| 238 |
## Quick start
|
| 239 |
|
| 240 |
### Ollama
|
|
@@ -243,8 +286,11 @@ Three paths:
|
|
| 243 |
|
| 244 |
```bash
|
| 245 |
# A. Pull straight from HF (gets the bundled Q4_K_M GGUF + the
|
| 246 |
-
# root-level template / system / params files in one step)
|
| 247 |
-
|
|
|
|
|
|
|
|
|
|
| 248 |
|
| 249 |
# B. Build a local `thanatos-27b` tag from THIS repo's bundle
|
| 250 |
# (LFS smudge if needed, then `ollama create`). Useful if you
|
|
@@ -279,9 +325,15 @@ python examples/ollama_chat.py # full demo: chat, streaming, tools, OpenAI-
|
|
| 279 |
|
| 280 |
### Local apps
|
| 281 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 282 |
| App | How to load this model |
|
| 283 |
|---|---|
|
| 284 |
-
| **Ollama** | `ollama run hf.co/FoolDev/Thanatos-27B` (default Q4_K_M). Pulls the GGUF + the root-level `template` / `system` / `params` files in one step (HF's Ollama bridge ingests these three files; it does **not** read `Modelfile`). For other quants, `make build QUANT=Q3_K_S` downloads from unsloth and creates a local Ollama tag using the `Modelfile`, which is kept in sync with the bridge files. |
|
| 285 |
| **LM Studio** | Search → `FoolDev/Thanatos-27B` → pick `Thanatos-27B.Q4_K_M.gguf`. Uses the GGUF's embedded jinja chat template (Qwen 3.6 ChatML); set the system prompt manually from the `SYSTEM` block in this repo's `Modelfile`. |
|
| 286 |
| **Jan** | Hub → "Import from Hugging Face" → `FoolDev/Thanatos-27B`. Same template behavior as LM Studio. |
|
| 287 |
| **llama.cpp** | `hf download FoolDev/Thanatos-27B Thanatos-27B.Q4_K_M.gguf --local-dir .` then `llama-server -m Thanatos-27B.Q4_K_M.gguf` (or `llama-cli`, `llama-mtmd-cli` for vision via the upstream `mmproj-F16.gguf`). |
|
|
|
|
| 63 |
|
| 64 |
## TL;DR
|
| 65 |
|
| 66 |
+
> [!IMPORTANT]
|
| 67 |
+
> **Heads up — the bundled GGUF is `qwen36`-stamped** (HF blob
|
| 68 |
+
> SHA `b2b63b94…`, commit `973d7ef`), a version-specific label
|
| 69 |
+
> stock loaders don't yet recognise. A direct `ollama run
|
| 70 |
+
> hf.co/FoolDev/Thanatos-27B` will 500 on first pull with
|
| 71 |
+
> `unable to load model`. `make heal-hf` rebadges the cached
|
| 72 |
+
> blob in place (qwen36 → qwen35, metadata-only, ~5 s) so the
|
| 73 |
+
> same tag loads. See [Architecture → Stamp choice](#stamp-choice)
|
| 74 |
+
> for why this repo stamps qwen36 despite upstream using `qwen35`
|
| 75 |
+
> as the canonical family label.
|
| 76 |
+
|
| 77 |
+
Two-step pull-and-load (this is the canonical path while the
|
| 78 |
+
bundle stays qwen36-stamped):
|
| 79 |
|
| 80 |
```bash
|
| 81 |
+
ollama run hf.co/FoolDev/Thanatos-27B # pulls 17 GB Q4_K_M, then 500s on first inference
|
| 82 |
+
git clone https://huggingface.co/FoolDev/Thanatos-27B
|
| 83 |
+
cd Thanatos-27B && make heal-hf # rebadges blob in store, tag loads from this point
|
| 84 |
+
ollama run hf.co/FoolDev/Thanatos-27B # works
|
| 85 |
```
|
| 86 |
|
| 87 |
+
Or skip the heal step entirely by building from a qwen35-stamped
|
| 88 |
+
unsloth quant (loads directly):
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
|
| 90 |
```bash
|
| 91 |
git clone https://huggingface.co/FoolDev/Thanatos-27B && cd Thanatos-27B
|
| 92 |
+
make build QUANT=Q3_K_S # downloads from unsloth/Qwen3.6-27B-GGUF (qwen35-stamped)
|
| 93 |
ollama run thanatos-27b
|
| 94 |
```
|
| 95 |
|
| 96 |
For other quants (Q3_K_S ~12 GB, Q5_K_M ~20 GB, etc.), `make build
|
| 97 |
+
QUANT=...` is the simplest path. See [Quick start](#quick-start)
|
| 98 |
below for the full matrix.
|
| 99 |
|
| 100 |
For image input use llama.cpp directly — Ollama vision is broken for
|
|
|
|
| 130 |
| `template`, `system`, `params` | Used by HF's Ollama bridge when users `ollama run hf.co/FoolDev/Thanatos-27B` directly (the bridge does **not** read `Modelfile` — see [HF Ollama docs](https://huggingface.co/docs/hub/en/ollama)). Mirrors the `Modelfile`'s template / system prompt / sampling params. |
|
| 131 |
| `examples/` | Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python |
|
| 132 |
| `scripts/build.sh` | Pulls a qwen35-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF` and runs `ollama create` (loads on today's llama.cpp / Ollama; see `make build`) |
|
| 133 |
+
| `scripts/load_bundle.sh` | One-shot path from *this repo's* qwen36-stamped bundle → loadable local Ollama tag (smudges LFS pointer via `hf download` if needed, rebadges qwen36 → qwen35, runs `ollama create`; see `make load-bundle`). |
|
| 134 |
+
| `scripts/heal_hf_pull.sh` | Heal an already-pulled `hf.co/FoolDev/Thanatos-27B:...` tag in-store after the first-pull 500: rebadges the cached blob qwen36 → qwen35 and rewrites the manifest's model-layer digest so the same tag becomes loadable in place. See `make heal-hf`. Idempotent — tags already on qwen35 are skipped. |
|
| 135 |
| `scripts/smoke_test.sh` | Verifies an Ollama daemon + model, runs a round-trip, asserts no chat-template tokens leak into the response. With `TOOLS_TEST=1`, also exercises an end-to-end tool-call round-trip and checks the response shape |
|
| 136 |
| `scripts/bench.sh` | Measures real tok/s using Ollama's `eval_count` / `eval_duration` metadata over a 3-prompt mix (run `make bench`) |
|
| 137 |
| `scripts/fetch_vision.sh` | Pulls the vision projector (`mmproj-F16.gguf`) for llama.cpp (Ollama vision is broken upstream — see [Vision](#vision)). Renamed from `fetch_mmproj.sh` because HF's Ollama bridge auto-indexed the script as a vision projector layer (filename pattern match). |
|
|
|
|
| 144 |
| `README.md` | This file |
|
| 145 |
|
| 146 |
This repo ships a single GGUF to back the HF/Ollama "Use this model"
|
| 147 |
+
widget — `Thanatos-27B.Q4_K_M.gguf` (~17 GB, qwen36-stamped):
|
| 148 |
|
| 149 |
```bash
|
| 150 |
+
ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen36 — fails today, see Heads up above
|
| 151 |
```
|
| 152 |
|
| 153 |
+
For 16 GB GPUs / unified-memory laptops — and as a working-today
|
| 154 |
+
fallback while the bundle stays qwen36-stamped — `make build
|
| 155 |
+
QUANT=Q3_K_S` downloads the smaller ~12 GB Q3_K_S quant from
|
| 156 |
+
`unsloth/Qwen3.6-27B-GGUF` (qwen35-stamped, loads on every current
|
| 157 |
+
llama.cpp / Ollama build) and creates a local `thanatos-27b`
|
| 158 |
Ollama tag. Does not redistribute via this repo.
|
| 159 |
|
| 160 |
For other quants or local builds, pull from
|
|
|
|
| 184 |
current loader compatibility.
|
| 185 |
- Multi-token prediction (MTP) head trained for speculative decoding
|
| 186 |
|
| 187 |
+
### Stamp choice
|
| 188 |
+
|
| 189 |
+
**The bundled GGUF declares `general.architecture: 'qwen36'`** — a
|
| 190 |
+
version-specific label this project uses despite the upstream
|
| 191 |
+
convention of `qwen35` for the whole Qwen 3.5 / 3.6 hybrid SSM +
|
| 192 |
+
attention family. Tradeoff up front:
|
| 193 |
+
|
| 194 |
+
- **What you give up by stamping qwen36:** stock `llama.cpp` and
|
| 195 |
+
Ollama don't recognise the arch and refuse to load. Fresh
|
| 196 |
+
`ollama run hf.co/FoolDev/Thanatos-27B` 500s on first inference;
|
| 197 |
+
`llama-server -m Thanatos-27B.Q4_K_M.gguf` errors with `unknown
|
| 198 |
+
model architecture: 'qwen36'`. Recovery: `make heal-hf`
|
| 199 |
+
(cached blob) or `make load-bundle` (working-copy GGUF), both
|
| 200 |
+
of which rebadge qwen36 → qwen35 in-place.
|
| 201 |
+
- **What you get:** the GGUF metadata names the model by its
|
| 202 |
+
actual Qwen version (3.6) rather than the family label
|
| 203 |
+
upstream happens to use (3.5). Useful if you're indexing
|
| 204 |
+
models by their HF model_id, scripting against the arch tag,
|
| 205 |
+
or just prefer architectural-honesty per individual release.
|
| 206 |
+
|
| 207 |
+
### Upstream convention — why `qwen35` would also be valid
|
| 208 |
+
|
| 209 |
+
Even though this bundle stamps qwen36, the upstream-canonical
|
| 210 |
+
name for the whole Qwen 3.5 / 3.6 hybrid SSM + attention family
|
| 211 |
+
is `qwen35`. The naming convergence runs through three layers of
|
| 212 |
+
the stack:
|
| 213 |
|
| 214 |
- **Qwen's own HF configs.** `Qwen/Qwen3.6-27B/config.json` declares
|
| 215 |
`"model_type": "qwen3_5"` and
|
|
|
|
| 229 |
Qwen 3.6 weights, not just Qwen 3.5.
|
| 230 |
|
| 231 |
There is no PR or tracking issue for a `qwen36` arch entry in
|
| 232 |
+
`ggml-org/llama.cpp` or `ollama/ollama` — and the evidence above
|
| 233 |
+
shows there's no architectural need for one. A future maintainer
|
| 234 |
+
who decides the friction isn't worth it can rebadge the bundle to
|
| 235 |
+
`qwen35` in one `make load-bundle` / `make heal-hf` invocation;
|
| 236 |
+
the tensor data is byte-identical across both stamps.
|
| 237 |
+
|
| 238 |
+
### History
|
| 239 |
+
|
| 240 |
+
The bundle has now flipped between the two stamps three times,
|
| 241 |
+
each time after weighing the friction-vs-honesty tradeoff anew:
|
| 242 |
+
|
| 243 |
+
- **v0.6.0 (e1f78fa, 2026-05-18):** initial qwen35 → qwen36
|
| 244 |
+
stamp, on the theory that qwen35 was a loader stand-in
|
| 245 |
+
awaiting proper Qwen 3.6 support. Upstream audit later
|
| 246 |
+
showed that theory was mistaken (see above).
|
| 247 |
+
- **2026-05-19 morning (964e418):** flipped back to qwen35
|
| 248 |
+
after daily friction outweighed version-specificity for that
|
| 249 |
+
iteration; doc workaround narrative collapsed
|
| 250 |
+
(`83022eb`).
|
| 251 |
+
- **2026-05-19 evening (07fa120, reverted `72259c1`):** brief
|
| 252 |
+
~1-hour re-flip to qwen36 during a fresh-pull integration
|
| 253 |
+
test; reverted because the live friction was worse than the
|
| 254 |
+
doc prose suggested.
|
| 255 |
+
- **2026-05-19 evening, again (`973d7ef`):** flipped to qwen36
|
| 256 |
+
one more time, after the upstream-evidence audit had been
|
| 257 |
+
shipped and the friction was a known quantity. Project owner
|
| 258 |
+
prefers the version-specific stamp despite the audit
|
| 259 |
+
conclusion. **This is the current state.**
|
| 260 |
+
|
| 261 |
+
Tensor data was byte-identical across all stamps; only the
|
| 262 |
+
`general.architecture` KV (and namespaced KV keys) flipped.
|
| 263 |
+
See the [CHANGELOG](CHANGELOG.md) entries for each flip's
|
| 264 |
+
rationale.
|
| 265 |
+
|
| 266 |
+
### Rebadge utility
|
| 267 |
+
|
| 268 |
+
`scripts/rename_arch.py` is the generic GGUF arch renamer
|
| 269 |
+
(metadata only, tensors byte-identical) underpinning both
|
| 270 |
+
`make heal-hf` and `make load-bundle`. For manual flips in
|
| 271 |
+
either direction:
|
| 272 |
|
| 273 |
```bash
|
| 274 |
+
# qwen36 -> qwen35 (the in-place recovery direction)
|
| 275 |
python3 scripts/rename_arch.py \
|
| 276 |
--from-arch qwen36 --to-arch qwen35 \
|
| 277 |
+
Thanatos-27B.Q4_K_M.gguf \
|
| 278 |
+
Thanatos-27B.Q4_K_M.qwen35.gguf
|
| 279 |
```
|
| 280 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 281 |
## Quick start
|
| 282 |
|
| 283 |
### Ollama
|
|
|
|
| 286 |
|
| 287 |
```bash
|
| 288 |
# A. Pull straight from HF (gets the bundled Q4_K_M GGUF + the
|
| 289 |
+
# root-level template / system / params files in one step).
|
| 290 |
+
# This 500s on first inference because the bundle is qwen36-stamped;
|
| 291 |
+
# `make heal-hf` rebadges the cached blob and the tag then loads.
|
| 292 |
+
ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen36 (fails first; recover via heal-hf)
|
| 293 |
+
make heal-hf # rebadges cached blob qwen36 -> qwen35 in store
|
| 294 |
|
| 295 |
# B. Build a local `thanatos-27b` tag from THIS repo's bundle
|
| 296 |
# (LFS smudge if needed, then `ollama create`). Useful if you
|
|
|
|
| 325 |
|
| 326 |
### Local apps
|
| 327 |
|
| 328 |
+
The bundled `Thanatos-27B.Q4_K_M.gguf` is `qwen36`-stamped — every row
|
| 329 |
+
below assumes you've either run `make heal-hf` after pulling, or that
|
| 330 |
+
you're loading a `qwen35`-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF`
|
| 331 |
+
instead. The "fails today with `unknown model architecture: 'qwen36'`"
|
| 332 |
+
caveat applies to every row until that's done.
|
| 333 |
+
|
| 334 |
| App | How to load this model |
|
| 335 |
|---|---|
|
| 336 |
+
| **Ollama** | `ollama run hf.co/FoolDev/Thanatos-27B` (default Q4_K_M). Pulls the GGUF + the root-level `template` / `system` / `params` files in one step (HF's Ollama bridge ingests these three files; it does **not** read `Modelfile`); fails on first inference, then `make heal-hf` rebadges the cached blob. For other quants, or to bypass the qwen36 block entirely, `make build QUANT=Q3_K_S` downloads from unsloth (qwen35-stamped) and creates a local Ollama tag using the `Modelfile`, which is kept in sync with the bridge files. |
|
| 337 |
| **LM Studio** | Search → `FoolDev/Thanatos-27B` → pick `Thanatos-27B.Q4_K_M.gguf`. Uses the GGUF's embedded jinja chat template (Qwen 3.6 ChatML); set the system prompt manually from the `SYSTEM` block in this repo's `Modelfile`. |
|
| 338 |
| **Jan** | Hub → "Import from Hugging Face" → `FoolDev/Thanatos-27B`. Same template behavior as LM Studio. |
|
| 339 |
| **llama.cpp** | `hf download FoolDev/Thanatos-27B Thanatos-27B.Q4_K_M.gguf --local-dir .` then `llama-server -m Thanatos-27B.Q4_K_M.gguf` (or `llama-cli`, `llama-mtmd-cli` for vision via the upstream `mmproj-F16.gguf`). |
|
|
@@ -1,22 +1,25 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
-
# Thanatos-27B — heal a
|
| 3 |
-
#
|
| 4 |
-
# qwen35 re-stamp in 964e418 / 72259c1).
|
| 5 |
#
|
| 6 |
-
#
|
| 7 |
-
#
|
| 8 |
-
#
|
| 9 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
#
|
| 11 |
-
#
|
| 12 |
-
# qwen35, metadata-only, byte-identical tensors) and
|
| 13 |
-
# manifest's model-layer digest to point at the new
|
| 14 |
-
# running,
|
|
|
|
| 15 |
#
|
| 16 |
# Idempotent: a tag already on qwen35 / qwen35moe is left untouched.
|
| 17 |
-
#
|
| 18 |
-
#
|
| 19 |
-
# legacy recovery case.
|
| 20 |
#
|
| 21 |
# Usage:
|
| 22 |
# ./scripts/heal_hf_pull.sh # default tag
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
+
# Thanatos-27B — heal a freshly pulled HF-bridge tag whose bundled GGUF
|
| 3 |
+
# is `qwen36`-stamped.
|
|
|
|
| 4 |
#
|
| 5 |
+
# Background. `ollama run hf.co/FoolDev/Thanatos-27B` (or any other
|
| 6 |
+
# qwen36-stamped HF-bridge tag of this repo) pulls a fresh copy of the
|
| 7 |
+
# bundled GGUF every time. Stock llama.cpp / Ollama don't recognise
|
| 8 |
+
# the `qwen36` arch (see README "Architecture"), so every such pull
|
| 9 |
+
# fails with `unable to load model: <blob>`. `make load-bundle` works
|
| 10 |
+
# around this by building a *separate* local `thanatos-27b` tag from
|
| 11 |
+
# a rebadged copy, but the canonical `hf.co/...` tag stays broken
|
| 12 |
+
# until this script runs.
|
| 13 |
#
|
| 14 |
+
# This script rebadges the HF-bridge tag's model blob in-place
|
| 15 |
+
# (qwen36 -> qwen35, metadata-only, byte-identical tensors) and
|
| 16 |
+
# rewrites the manifest's model-layer digest to point at the new
|
| 17 |
+
# blob. After running it, `ollama run hf.co/FoolDev/Thanatos-27B`
|
| 18 |
+
# loads.
|
| 19 |
#
|
| 20 |
# Idempotent: a tag already on qwen35 / qwen35moe is left untouched.
|
| 21 |
+
# Re-runnable after a fresh HF pull (the pull resets the manifest
|
| 22 |
+
# digest back to the qwen36 blob).
|
|
|
|
| 23 |
#
|
| 24 |
# Usage:
|
| 25 |
# ./scripts/heal_hf_pull.sh # default tag
|
|
@@ -1,23 +1,24 @@
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
-
# Thanatos-27B — load
|
| 3 |
#
|
| 4 |
-
# The bundled GGUF (Thanatos-27B.Q4_K_M.gguf)
|
| 5 |
-
#
|
| 6 |
-
#
|
| 7 |
-
#
|
|
|
|
|
|
|
|
|
|
| 8 |
#
|
| 9 |
# 1. Resolve the bundle. If it's an LFS pointer (cloned without
|
| 10 |
# `git lfs pull`), download the real ~17 GB blob via `hf download`.
|
| 11 |
-
# 2. Inspect `general.architecture`. If
|
| 12 |
-
#
|
| 13 |
-
#
|
| 14 |
-
#
|
| 15 |
-
# 3. Run `ollama create <tag> -f <temp Modelfile pointing at
|
| 16 |
-
# resolved bundle>`.
|
| 17 |
#
|
| 18 |
# Useful if you want a bare local tag (`thanatos-27b`) rather than
|
| 19 |
-
# the `hf.co/FoolDev/Thanatos-27B` path.
|
| 20 |
-
# branch is kept for anyone working from a pre-964e418 checkout.
|
| 21 |
#
|
| 22 |
# Usage:
|
| 23 |
# ./scripts/load_bundle.sh # default tag: thanatos-27b
|
|
|
|
| 1 |
#!/usr/bin/env bash
|
| 2 |
+
# Thanatos-27B — load the qwen36-stamped bundle on today's llama.cpp / Ollama.
|
| 3 |
#
|
| 4 |
+
# The bundled GGUF (Thanatos-27B.Q4_K_M.gguf) ships with
|
| 5 |
+
# `general.architecture: 'qwen36'`. No released llama.cpp / Ollama
|
| 6 |
+
# recognizes that arch (see README "Architecture"), so a direct
|
| 7 |
+
# `ollama create -f Modelfile` against the bundle fails with
|
| 8 |
+
# `unknown model architecture: 'qwen36'`. This script handles the
|
| 9 |
+
# rebadge workaround end-to-end so users don't have to copy-paste
|
| 10 |
+
# from the README:
|
| 11 |
#
|
| 12 |
# 1. Resolve the bundle. If it's an LFS pointer (cloned without
|
| 13 |
# `git lfs pull`), download the real ~17 GB blob via `hf download`.
|
| 14 |
+
# 2. Inspect `general.architecture`. If qwen36 (current bundle),
|
| 15 |
+
# rebadge to qwen35 via scripts/rename_arch.py (metadata-only,
|
| 16 |
+
# byte-identical tensors). If already qwen35 (e.g. a previously
|
| 17 |
+
# rebadged copy), skip the rebadge step.
|
| 18 |
+
# 3. Run `ollama create <tag> -f <temp Modelfile pointing at qwen35>`.
|
|
|
|
| 19 |
#
|
| 20 |
# Useful if you want a bare local tag (`thanatos-27b`) rather than
|
| 21 |
+
# the `hf.co/FoolDev/Thanatos-27B` path.
|
|
|
|
| 22 |
#
|
| 23 |
# Usage:
|
| 24 |
# ./scripts/load_bundle.sh # default tag: thanatos-27b
|