Instructions to use FoolDev/Thanatos-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FoolDev/Thanatos-27B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="FoolDev/Thanatos-27B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FoolDev/Thanatos-27B", dtype="auto") - llama-cpp-python
How to use FoolDev/Thanatos-27B with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="FoolDev/Thanatos-27B", filename="Thanatos-27B.Q4_K_M.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use FoolDev/Thanatos-27B with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf FoolDev/Thanatos-27B:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf FoolDev/Thanatos-27B:Q4_K_M
Use Docker
docker model run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use FoolDev/Thanatos-27B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FoolDev/Thanatos-27B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Thanatos-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- SGLang
How to use FoolDev/Thanatos-27B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FoolDev/Thanatos-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Thanatos-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FoolDev/Thanatos-27B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Thanatos-27B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Ollama
How to use FoolDev/Thanatos-27B with Ollama:
ollama run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- Unsloth Studio new
How to use FoolDev/Thanatos-27B with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Thanatos-27B to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for FoolDev/Thanatos-27B to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for FoolDev/Thanatos-27B to start chatting
- Pi new
How to use FoolDev/Thanatos-27B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FoolDev/Thanatos-27B:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use FoolDev/Thanatos-27B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf FoolDev/Thanatos-27B:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FoolDev/Thanatos-27B:Q4_K_M
Run Hermes
hermes
- Docker Model Runner
How to use FoolDev/Thanatos-27B with Docker Model Runner:
docker model run hf.co/FoolDev/Thanatos-27B:Q4_K_M
- Lemonade
How to use FoolDev/Thanatos-27B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FoolDev/Thanatos-27B:Q4_K_M
Run and chat with the model
lemonade run user.Thanatos-27B-Q4_K_M
List all available models
lemonade list
docs: collapse qwen36 workaround narrative now that bundle is qwen35-stamped
Browse filesFollow-up to 1c88b41, which deferred the deeper docs cleanup
("intentionally stale in this commit, follow-up will collapse").
This is that follow-up.
Touched three doc surfaces:
- README "Architecture" section: replaced the long
"qwen36-stamped, here's why upstream is the lagging side, here
are two one-liner workarounds, here's the manual rebadge
recipe, once-upstream-adds-qwen36-the-workaround-disappears"
block with a one-paragraph history note: bundle is qwen35,
loads directly, was qwen36 through v0.6.0, reverted in 964e418.
`scripts/rename_arch.py` example flipped from qwen36 -> qwen35
to qwen35 -> qwen36 to match its new "tool kept around for
future flips" framing.
- README Quick start Ollama option A: dropped the "fails today
with `unknown model architecture: 'qwen36'`" warning and the
`make heal-hf && ollama run ...` recovery line. A is now just
`ollama run hf.co/FoolDev/Thanatos-27B` — 17 GB Q4_K_M,
qwen35-stamped. B's comment lost the
"rebadges qwen36 -> qwen35" detail (irrelevant on a qwen35
bundle).
- examples/README.md Ollama setup: similar collapse — pull + run
is now the lead recipe with no intervening heal step. heal-hf
kept as a one-paragraph callout for anyone who pulled before
the 964e418 re-stamp.
- Modelfile preamble: shrunk from a 15-line qwen36-stamp +
rebadge-recipe block to a 6-line "bundle is qwen35, loads
directly, see README Architecture for history" note.
Net -39 lines across the three doc surfaces. No tooling change;
heal-hf, load-bundle, rename_arch.py all stay wired up exactly as
shipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- CHANGELOG.md +11 -5
- Modelfile +6 -15
- README.md +38 -64
- examples/README.md +12 -16
|
@@ -31,16 +31,22 @@ and documentation**, not the underlying base model.
|
|
| 31 |
stock Ollama. `make heal-hf` is mentioned as the recovery path
|
| 32 |
for anyone who pulled the v0.6.0 bundle and still has the qwen36
|
| 33 |
blob in their store.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
### Note
|
| 36 |
- `make heal-hf`, `make load-bundle`, and `scripts/rename_arch.py`
|
| 37 |
stay in the repo and stay useful. heal-hf is idempotent on qwen35
|
| 38 |
(skips early); load-bundle does a no-op rebadge on a qwen35 bundle
|
| 39 |
-
and runs `ollama create` as normal.
|
| 40 |
-
Quick start / examples-README narrative still talks about the
|
| 41 |
-
qwen36 stamp and the workaround dance; left intentionally stale
|
| 42 |
-
in this commit for follow-up cleanup, since the immediate user
|
| 43 |
-
pain (load 500 on fresh pull) is what the upload commit fixed.
|
| 44 |
|
| 45 |
### Added
|
| 46 |
- `scripts/check.sh` now greps for the `VAR="$(cmd 2>/dev/null | filter)"`
|
|
|
|
| 31 |
stock Ollama. `make heal-hf` is mentioned as the recovery path
|
| 32 |
for anyone who pulled the v0.6.0 bundle and still has the qwen36
|
| 33 |
blob in their store.
|
| 34 |
+
- README "Architecture" section, Quick start option A,
|
| 35 |
+
`examples/README.md` Ollama setup, and `Modelfile` preamble all
|
| 36 |
+
collapsed back to the pre-qwen36 single-path narrative. The
|
| 37 |
+
long "qwen36 is unloadable / here are the workarounds" prose
|
| 38 |
+
was replaced with a one-paragraph history note pointing at
|
| 39 |
+
commit 964e418 + the [CHANGELOG](CHANGELOG.md) entry. Net -39
|
| 40 |
+
lines across the three doc surfaces. `scripts/rename_arch.py`
|
| 41 |
+
example in Architecture flipped from qwen36 → qwen35 to qwen35
|
| 42 |
+
→ qwen36 to match its new "tool kept around for future flips"
|
| 43 |
+
framing.
|
| 44 |
|
| 45 |
### Note
|
| 46 |
- `make heal-hf`, `make load-bundle`, and `scripts/rename_arch.py`
|
| 47 |
stay in the repo and stay useful. heal-hf is idempotent on qwen35
|
| 48 |
(skips early); load-bundle does a no-op rebadge on a qwen35 bundle
|
| 49 |
+
and runs `ollama create` as normal.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
|
| 51 |
### Added
|
| 52 |
- `scripts/check.sh` now greps for the `VAR="$(cmd 2>/dev/null | filter)"`
|
|
@@ -7,21 +7,12 @@
|
|
| 7 |
# image input, or wait for the fix. See the Vision section in README.md.
|
| 8 |
#
|
| 9 |
# This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
|
| 10 |
-
# stamped `general.architecture: '
|
| 11 |
-
#
|
| 12 |
-
# `ollama create thanatos-27b -f Modelfile &&
|
| 13 |
-
#
|
| 14 |
-
#
|
| 15 |
-
#
|
| 16 |
-
# python3 scripts/rename_arch.py --from-arch qwen36 --to-arch qwen35 \
|
| 17 |
-
# Thanatos-27B.Q4_K_M.gguf Thanatos-27B.Q4_K_M.qwen35.gguf
|
| 18 |
-
# # then temporarily point FROM at the qwen35 file, or use:
|
| 19 |
-
# # echo "FROM $PWD/Thanatos-27B.Q4_K_M.qwen35.gguf" > /tmp/Modelfile.qwen35
|
| 20 |
-
# # ollama create thanatos-27b -f /tmp/Modelfile.qwen35
|
| 21 |
-
#
|
| 22 |
-
# Once upstream adds the qwen36 arch entry the workaround disappears
|
| 23 |
-
# and the FROM line below works as-is. See README "Architecture" for
|
| 24 |
-
# the full story.
|
| 25 |
#
|
| 26 |
# For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
|
| 27 |
# downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
|
|
|
|
| 7 |
# image input, or wait for the fix. See the Vision section in README.md.
|
| 8 |
#
|
| 9 |
# This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
|
| 10 |
+
# stamped `general.architecture: 'qwen35'` — the arch entry every
|
| 11 |
+
# released llama.cpp / Ollama loads under for the Qwen 3.6 hybrid
|
| 12 |
+
# SSM + attention family. `ollama create thanatos-27b -f Modelfile &&
|
| 13 |
+
# ollama run thanatos-27b` loads it directly. See README
|
| 14 |
+
# "Architecture" for the qwen35 -> qwen36 -> qwen35 history through
|
| 15 |
+
# v0.6.0.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
#
|
| 17 |
# For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
|
| 18 |
# downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
|
|
@@ -176,65 +176,43 @@ If you want the safetensors for `transformers`, fetch them from [`Qwen/Qwen3.6-2
|
|
| 176 |
current loader compatibility.
|
| 177 |
- Multi-token prediction (MTP) head trained for speculative decoding
|
| 178 |
|
| 179 |
-
**The bundled GGUF declares `general.architecture: '
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
stack
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
# (LFS smudge + rebadge + `ollama create`, idempotent):
|
| 205 |
-
make load-bundle
|
| 206 |
-
ollama run thanatos-27b
|
| 207 |
-
|
| 208 |
-
# Already pulled and hit the 500: rebadge the already-downloaded
|
| 209 |
-
# blob in place + rewrite the manifest digest, so the same
|
| 210 |
-
# `hf.co/...` tag becomes loadable without switching tag names:
|
| 211 |
-
make heal-hf
|
| 212 |
-
ollama run hf.co/FoolDev/Thanatos-27B
|
| 213 |
-
```
|
| 214 |
-
|
| 215 |
-
What that wraps, if you want to do it by hand
|
| 216 |
-
(`scripts/rename_arch.py` is metadata-only — tensors stay
|
| 217 |
-
byte-identical):
|
| 218 |
|
| 219 |
```bash
|
| 220 |
python3 scripts/rename_arch.py \
|
| 221 |
-
--from-arch
|
| 222 |
Thanatos-27B.Q4_K_M.gguf \
|
| 223 |
-
Thanatos-27B.Q4_K_M.
|
| 224 |
-
# Then either build a local Ollama tag (note absolute path —
|
| 225 |
-
# `ollama create` resolves a relative FROM against the Modelfile's
|
| 226 |
-
# directory, not your CWD):
|
| 227 |
-
echo "FROM $PWD/Thanatos-27B.Q4_K_M.qwen35.gguf" > /tmp/Modelfile.qwen35
|
| 228 |
-
ollama create thanatos-27b -f /tmp/Modelfile.qwen35
|
| 229 |
-
ollama run thanatos-27b hi
|
| 230 |
-
# …or point llama-server at the qwen35 file directly:
|
| 231 |
-
llama-server -m Thanatos-27B.Q4_K_M.qwen35.gguf -ngl 99 -c 8192
|
| 232 |
```
|
| 233 |
|
| 234 |
-
If
|
| 235 |
-
|
| 236 |
-
`make heal-hf`
|
| 237 |
-
|
| 238 |
|
| 239 |
## Quick start
|
| 240 |
|
|
@@ -243,17 +221,13 @@ works as-is.
|
|
| 243 |
Three paths:
|
| 244 |
|
| 245 |
```bash
|
| 246 |
-
# A. Pull straight from HF (
|
| 247 |
-
# template / system / params files
|
| 248 |
-
|
| 249 |
-
|
| 250 |
-
#
|
| 251 |
-
#
|
| 252 |
-
|
| 253 |
-
make heal-hf && ollama run hf.co/FoolDev/Thanatos-27B # heal + re-run after the first fails
|
| 254 |
-
|
| 255 |
-
# B. Load THIS repo's bundle via the rebadge helper (smudges LFS if
|
| 256 |
-
# needed, rebadges qwen36 → qwen35, runs `ollama create`):
|
| 257 |
make load-bundle # creates local tag thanatos-27b
|
| 258 |
ollama run thanatos-27b
|
| 259 |
|
|
|
|
| 176 |
current loader compatibility.
|
| 177 |
- Multi-token prediction (MTP) head trained for speculative decoding
|
| 178 |
|
| 179 |
+
**The bundled GGUF declares `general.architecture: 'qwen35'`** — the
|
| 180 |
+
arch entry every released llama.cpp / Ollama loads the Qwen 3.6
|
| 181 |
+
hybrid SSM + attention family under. Upstream `ggml-org/llama.cpp`
|
| 182 |
+
and `ollama/ollama` register the stack as `qwen35` / `qwen35moe`,
|
| 183 |
+
including for actual Qwen 3.6-family weights (the upstream
|
| 184 |
+
`Qwen3.6-35B-A3B_Q8_0` GGUFs load under `qwen35moe`); no `qwen36`
|
| 185 |
+
arch entry exists, and there's no open PR or tracking issue for
|
| 186 |
+
one — `qwen35` is the canonical load-time label.
|
| 187 |
+
|
| 188 |
+
`ollama run hf.co/FoolDev/Thanatos-27B` and `llama-server -m
|
| 189 |
+
Thanatos-27B.Q4_K_M.gguf` both load directly on current stock
|
| 190 |
+
loaders.
|
| 191 |
+
|
| 192 |
+
**History.** Through v0.6.0 the bundle was stamped `qwen36` — the
|
| 193 |
+
architecturally-honest label, but unloadable on stock loaders.
|
| 194 |
+
Users needed `make load-bundle` or `make heal-hf` to rebadge
|
| 195 |
+
before loading. The stamp was reverted to `qwen35` in commit
|
| 196 |
+
`964e418` to stop forcing the workaround on every fresh pull;
|
| 197 |
+
tensor data was byte-identical across both stamps, only the
|
| 198 |
+
`general.architecture` KV (and namespaced KV keys) flipped. See
|
| 199 |
+
the [CHANGELOG](CHANGELOG.md) entry for the full rationale.
|
| 200 |
+
|
| 201 |
+
`scripts/rename_arch.py` is kept in the repo as a generic GGUF
|
| 202 |
+
arch renamer (metadata only, tensors byte-identical) for future
|
| 203 |
+
flips:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
|
| 205 |
```bash
|
| 206 |
python3 scripts/rename_arch.py \
|
| 207 |
+
--from-arch qwen35 --to-arch qwen36 \
|
| 208 |
Thanatos-27B.Q4_K_M.gguf \
|
| 209 |
+
Thanatos-27B.Q4_K_M.qwen36.gguf
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 210 |
```
|
| 211 |
|
| 212 |
+
If you pulled the bundle while it was still qwen36-stamped (v0.6.0
|
| 213 |
+
through 2026-05-19) and have the broken blob in your local Ollama
|
| 214 |
+
store, `make heal-hf` rebadges it in place so the same
|
| 215 |
+
`hf.co/...` tag becomes loadable without re-pulling.
|
| 216 |
|
| 217 |
## Quick start
|
| 218 |
|
|
|
|
| 221 |
Three paths:
|
| 222 |
|
| 223 |
```bash
|
| 224 |
+
# A. Pull straight from HF (gets the bundled Q4_K_M GGUF + the
|
| 225 |
+
# root-level template / system / params files in one step):
|
| 226 |
+
ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen35-stamped
|
| 227 |
+
|
| 228 |
+
# B. Build a local `thanatos-27b` tag from THIS repo's bundle
|
| 229 |
+
# (LFS smudge if needed, then `ollama create`). Useful if you
|
| 230 |
+
# want a bare local tag rather than the `hf.co/...` path:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 231 |
make load-bundle # creates local tag thanatos-27b
|
| 232 |
ollama run thanatos-27b
|
| 233 |
|
|
@@ -21,37 +21,33 @@ from the `Modelfile` / bridge files.
|
|
| 21 |
|
| 22 |
Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's
|
| 23 |
root-level `template` / `system` / `params` files via HF's Ollama
|
| 24 |
-
bridge)
|
| 25 |
-
actually load it. The pull itself succeeds; the first inference
|
| 26 |
-
fails with `unable to load model` until heal — see parent README's
|
| 27 |
-
[Architecture](../README.md#architecture) for why:
|
| 28 |
|
| 29 |
```bash
|
| 30 |
ollama pull hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M (only bundled quant)
|
| 31 |
-
cd .. && make heal-hf && cd examples # rebadges the pulled blob qwen36 -> qwen35 in store
|
| 32 |
pip install requests
|
| 33 |
MODEL=hf.co/FoolDev/Thanatos-27B python ollama_chat.py
|
| 34 |
```
|
| 35 |
|
| 36 |
-
|
| 37 |
-
|
|
|
|
|
|
|
| 38 |
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
local `thanatos-27b` tag):
|
| 43 |
|
| 44 |
```bash
|
| 45 |
-
cd .. && make
|
| 46 |
MODEL=thanatos-27b python ollama_chat.py
|
| 47 |
```
|
| 48 |
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
creates the same local tag — bypasses the bundle and the heal entirely:
|
| 52 |
|
| 53 |
```bash
|
| 54 |
-
cd .. && make
|
| 55 |
MODEL=thanatos-27b python ollama_chat.py
|
| 56 |
```
|
| 57 |
|
|
|
|
| 21 |
|
| 22 |
Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's
|
| 23 |
root-level `template` / `system` / `params` files via HF's Ollama
|
| 24 |
+
bridge):
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
```bash
|
| 27 |
ollama pull hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M (only bundled quant)
|
|
|
|
| 28 |
pip install requests
|
| 29 |
MODEL=hf.co/FoolDev/Thanatos-27B python ollama_chat.py
|
| 30 |
```
|
| 31 |
|
| 32 |
+
If you pulled before commit `964e418` (the qwen35 re-stamp) and
|
| 33 |
+
still have the broken qwen36 blob in your Ollama store, run
|
| 34 |
+
`cd .. && make heal-hf` once to rebadge it in place. Fresh pulls
|
| 35 |
+
after the re-stamp go straight through.
|
| 36 |
|
| 37 |
+
For a non-bundled quant (e.g. Q3_K_S ~12 GB, Q5_K_M ~20 GB),
|
| 38 |
+
`make build QUANT=...` downloads from `unsloth/Qwen3.6-27B-GGUF`
|
| 39 |
+
and creates a local `thanatos-27b` tag:
|
|
|
|
| 40 |
|
| 41 |
```bash
|
| 42 |
+
cd .. && make build QUANT=Q3_K_S && cd examples
|
| 43 |
MODEL=thanatos-27b python ollama_chat.py
|
| 44 |
```
|
| 45 |
|
| 46 |
+
Or build a local tag from this repo's bundled GGUF without going
|
| 47 |
+
through the HF pull:
|
|
|
|
| 48 |
|
| 49 |
```bash
|
| 50 |
+
cd .. && make load-bundle && cd examples
|
| 51 |
MODEL=thanatos-27b python ollama_chat.py
|
| 52 |
```
|
| 53 |
|