--- license: apache-2.0 base_model: - llmfan46/Qwen3.6-35B-A3B-uncensored-heretic base_model_relation: finetune datasets: - crownelius/Creative_Writing_ShareGPT_Enhanced - microsoft/rStar-Coder - peteromallet/dataclaw-peteromallet - crownelius/Opus-4.7-Reasoning - openbmb/UltraData-Math - Crownelius/Crow-Heretic-TeichAI-Unified language: - en - zh - ru - es - fr - it - ja - ko - de - ar - tr - pl - sv - nl - he - id - uk - fa - pt - ms - fi - el tags: - qwen36 - moe - conversational - multimodal - agent - gguf - heretic - uncensored library_name: transformers pipeline_tag: image-text-to-text --- Janus-35B banner [![License](https://img.shields.io/badge/License-Apache_2.0-7aa2f7?style=flat&labelColor=1a1b26)](https://opensource.org/licenses/Apache-2.0) [![Base Model](https://img.shields.io/badge/Base-Heretic-bb9af7?style=flat&labelColor=1a1b26)](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) [![Architecture](https://img.shields.io/badge/Arch-MoE_35B/3B_active-ff9e64?style=flat&labelColor=1a1b26)](#architecture) [![Quant](https://img.shields.io/badge/GGUF-Q4__K__M-9ece6a?style=flat&labelColor=1a1b26)](#whats-here) [![Sibling](https://img.shields.io/badge/Sibling-Thanatos--27B-7dcfff?style=flat&labelColor=1a1b26)](https://huggingface.co/FoolDev/Thanatos-27B-HERETIC) Buy me a coffee — support Janus-35B # Janus-35B > **Flagship Reasoning. Sparse Footprint. Uncensored.** > *llmfan46's Heretic abliteration of Qwen 3.6 35B-A3B, repackaged with Claude Fable 5 in the teacher slot.* **`Architecture:`** `Qwen 3.6 35B-A3B (MoE)` | **`Total Params:`** `35B` | **`Active Params:`** `3B` | **`Base:`** `Heretic (llmfan46)` | **`Teacher:`** `Claude Fable 5` | **`Type:`** `Distilled + Abliterated MoE LLM` A personal fork of [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) — an uncensored Heretic-style abliteration of [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B), the 35B-total / 3B-active mixture-of-experts multimodal base — repackaged as Janus-35B with Claude Fable 5 reasoning data in the teacher slot. Refusal-trained behavior is dialed back at the base layer. ## TL;DR One-liner via Hugging Face (pulls a GGUF + this repo's root-level `template` / `system` / `params` files, including the tool-calling template — HF's Ollama bridge ingests those three files, not `Modelfile`): ```bash ollama run hf.co/FoolDev/Janus-35B-HERETIC # default ~19 GB Q4_K_M ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M # same blob, explicit tag ``` Or build locally (uses this repo's `Modelfile`, kept in sync with the three bridge files): ```bash git clone https://huggingface.co/FoolDev/Janus-35B-HERETIC && cd Janus-35B-HERETIC ollama create janus -f Modelfile && ollama run janus ``` After either path, `ollama show janus` lists `completion`, `tools`, and `thinking` under Capabilities. Hardware: ~39 GB RAM at default `num_ctx 32768`, or trim ctx + batch to fit 32 GB hosts (see [Hardware requirements](#hardware-requirements)). ## What's here | File | Use | |---|---| | `Janus-35B-A3B.Q4_K_M.gguf` | Recommended default, ~19 GB | | `Modelfile` | Ollama wrapper for **local** builds (`ollama create janus -f Modelfile`) — overrides the GGUF's embedded template with one that exposes `.Tools` / `.ToolCalls` to Ollama's capability detector. | | `template`, `system`, `params` | Used by HF's Ollama bridge when users `ollama run hf.co/FoolDev/Janus-35B-HERETIC` directly. The bridge does **not** read `Modelfile` (see [HF Ollama docs](https://huggingface.co/docs/hub/en/ollama)); it ingests these three root-level files instead. Kept in sync with the `Modelfile`'s `TEMPLATE` / `SYSTEM` / `PARAMETER` directives. | | `scripts/build.sh` | Pulls a GGUF from `llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF` (default Q4_K_M) and runs `ollama create janus`. The bundled Q4_K_M is already this Heretic quant; use this to build other quants locally. | | `scripts/check_bridge_sync.py` | Run before pushing a `Modelfile` / `template` / `system` / `params` edit to verify the four configurations remain in sync. Exits 0 if in sync, 1 with a per-key diff if not. | | `scripts/smoke_test.sh` | Integration smoke test against a running Ollama daemon: server reachable, model loaded, `tools` capability present, chat round-trip, and no control-token leakage. `TOOLS_TEST=1` adds a tool-call round-trip. Defaults to `MODEL=janus`. | | `scripts/bench.sh` | Measures tok/s from Ollama's `eval_count` / `eval_duration` over a short/medium/long prompt mix (with a discarded warmup). Defaults to `MODEL=janus`. | | `scripts/load_bundle.sh` | Loads the bundled `Janus-35B-A3B.Q4_K_M.gguf` into Ollama as a local `janus` tag without an upstream pull (smudges the LFS pointer via `hf download` if needed, checks the arch is `qwen35moe`). | | `scripts/fetch_vision.sh` | Downloads the vision projector (`Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf`) from the Heretic GGUF repo for llama.cpp image input (Ollama vision is broken upstream — see [Vision](#vision)). | | `examples/` | Ready-to-run Python clients for Ollama, Transformers, and llama-cpp-python (text, tools, and vision — see `examples/README.md`) | GGUF-only release. Pull the Heretic safetensors from [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) if you need the `transformers` tree (or the vanilla pre-Heretic base from [`Qwen/Qwen3.6-35B-A3B`](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)). > **Bundled blob status:** the bundled `Janus-35B-A3B.Q4_K_M.gguf` is the Heretic > Q4_K_M quant (from > [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF)), > `qwen35moe`-stamped and verified against the Architecture below (40 layers, 256 > experts, vocab 248,320). It serves the uncensored Heretic behavior directly; > `./scripts/build.sh` remains the path for other quants. ## Architecture

animated MoE routing visualization: 16x16 grid of 256 expert dots with 8 lit at any time, cycling through 8 routing patterns

- Qwen 3.6, 35B total / 3B active, MoE (256 experts, 8 activated per token) - 40 layers, 10 × (3 × DeltaNet → MoE / 1 × Gated Attention → MoE) - 262 144 native context, extensible to ~1 M with YaRN - Vision + video supported by upstream (mmproj not included in this release) - Vocab 248,320 ## Quick start ### llama.cpp / LM Studio Drop the GGUF into your loader of choice. The chat template is embedded in the GGUF metadata, so llama.cpp's `--chat-template auto` and LM Studio's GGUF auto-detection handle plain conversation correctly. ### Ollama The chat template baked into the GGUF is **not sufficient on Ollama** — it lacks the `.Tools` / `.ToolCalls` blocks Ollama's capability detector requires, so a naive `ollama pull` reports `does not support tools` and rejects any request carrying a `tools` array. Two paths fix this: ```bash # A. Pull straight from HF (uses the root-level template/system/params files): ollama run hf.co/FoolDev/Janus-35B-HERETIC # default tag, ~19 GB Q4_K_M ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M # same blob, explicit tag # Note: HF's Ollama bridge does NOT read Modelfile; it reads template/system/params. # B. Build locally (uses Modelfile, which is kept in sync with the three above): ollama create janus -f Modelfile && ollama run janus ``` After either path, `ollama show janus` should list `completion`, `tools`, and `thinking` under Capabilities. ### Inference examples Once the model is loaded (via `ollama run janus`, `lms server`, or `llama-server`), all the standard OpenAI-compatible clients work. Examples assume the loader is listening on `http://localhost:11434` (Ollama default) — adjust the port for LM Studio (`:1234`) or llama.cpp (`:8080`). Runnable versions of everything below live in [`examples/`](examples/README.md). #### curl ```bash curl -s http://localhost:11434/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "janus", "messages": [ {"role": "system", "content": "You are Janus, a precise reasoning assistant."}, {"role": "user", "content": "Sketch an algorithm to detect cycles in a directed graph."} ], "temperature": 0.6, "max_tokens": 800 }' | jq -r '.choices[0].message.content' ``` #### Python (openai-compat) ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ignored") resp = client.chat.completions.create( model="janus", messages=[ {"role": "user", "content": "Write a haiku about a stack overflow."} ], temperature=0.8, top_p=0.95, ) print(resp.choices[0].message.content) ``` #### Streaming ```python stream = client.chat.completions.create( model="janus", messages=[{"role": "user", "content": "Explain RoPE briefly."}], stream=True, ) for chunk in stream: delta = chunk.choices[0].delta.content or "" print(delta, end="", flush=True) ``` ### Recommended sampling | Use | temp | top_p | top_k | repeat_penalty | |---|---:|---:|---:|---:| | Reasoning / general | 0.6 | 0.95 | 20 | 1.05 | | Creative / RP | 0.8 | 0.95 | 40 | 1.02 | Lower temperature (0.4–0.6) and bump `repeat_penalty` to 1.08 if it loops inside `` tags. ### System prompt ```text You are Janus, a precise and capable assistant for reasoning, writing, coding, and long-form dialogue. Behavior rules: - Answer the user's actual request directly. - Be accurate, complete, and structured. - Think before answering, but do not get stuck in repetitive loops or meta-commentary. - If the request is ambiguous or incomplete, state what is missing and make the smallest reasonable assumption needed to continue. - If the user wants creative writing, preserve tone, continuity, and character consistency. - If the user wants analysis or technical help, prefer concrete steps, examples, and decisions over fluff. - Finish with a usable answer, not just planning. ``` ## Vision The Qwen 3.6 base supports image (and video) input via a separate `mmproj` projector. The full multimodal stack is: ``` Janus-35B-A3B.Q4_K_M.gguf (~19 GB, the text decoder) Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf (~903 MB, the vision projector) ``` The projector and other-quant text decoders live at [`llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF`](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF) (BF16 mmproj only). For the vanilla pre-Heretic projector in F16/F32, see [`unsloth/Qwen3.6-35B-A3B-GGUF`](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) (`mmproj-F16.gguf`). This repo intentionally does not redistribute either; `./scripts/fetch_vision.sh` pulls the projector into the repo root. ### Loader compatibility | Loader | Text | Vision (mmproj) | Notes | |---|---|---|---| | **llama.cpp** (`llama-mtmd-cli`, `llama-server --mmproj`) | ✅ | ✅ | Reference path. Upstream has the `qwen35moe` arch entry. | | **llama-cpp-python** | ✅ | ✅ | See `examples/llama_cpp_vision.py`. | | **Ollama 0.24** | ✅ | ❌ | Text inference works: Ollama's Go engine has the `qwen35` / `qwen35moe` arch entries. Vision (mmproj) is still broken: the C++ llama.cpp fallback that Ollama switches to when an mmproj is attached lacks those entries. `ollama create` accepts a dual-`FROM` (text + mmproj) and `ollama show` reports `vision` capability — but the **first inference request** fails with `error loading model architecture: unknown model architecture: 'qwen35moe'`, and once mmproj is attached this blocks text inference too. See [ollama/ollama#15898](https://github.com/ollama/ollama/issues/15898). | | **LM Studio** | ✅ | ✅ | Uses upstream llama.cpp directly. | ### Vision via llama.cpp ```bash # Fetch the projector first (into the repo root): ./scripts/fetch_vision.sh # Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf # A. HTTP via llama-server (the easiest path): llama-server \ -m Janus-35B-A3B.Q4_K_M.gguf \ --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \ --host 127.0.0.1 --port 8765 -c 8192 -ngl 99 # then POST OpenAI-style chat completions with an image_url content block — # e.g. {"type":"image_url","image_url":{"url":"data:image/jpeg;base64,..."}} # B. CLI via llama-mtmd-cli (one-shot). It's a separate cmake target, so a # selective build can skip it; a plain `cmake --build build` produces it. llama-mtmd-cli \ -m Janus-35B-A3B.Q4_K_M.gguf \ --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \ --image photo.jpg \ -p "Describe this image." # C. Python via llama-cpp-python: python examples/llama_cpp_vision.py \ --gguf Janus-35B-A3B.Q4_K_M.gguf \ --mmproj Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf \ --image /path/to/photo.jpg \ --prompt "What is in this image?" ``` Until the Ollama upstream issue is fixed, treat Ollama as **text-only** for this model. The bundled Q4_K_M decoder pairs with the projector directly — the mmproj is family-wide for Qwen 3.6 35B-A3B, so no separate text download is needed for vision. ## Hardware requirements This is a ~19 GB Q4_K_M GGUF. Ollama's runtime footprint at default settings is **roughly 2× the model file** (weights mmap + compute graph allocation), plus KV cache — so ~39 GB total memory at `num_ctx 32768` (KV cache ~2 GB at 32K with `OLLAMA_KV_CACHE_TYPE=q8_0`). The compute-graph allocation scales with context and batch size, so 32 GB hosts can fit the model by trimming both (see Z13 row in the table). | Hardware | Status | |---|---| | ≥48 GB RAM (CPU-only) | Works, ~3-6 tok/s | | Single H100/A100 80 GB | Works, full offload, ~30+ tok/s | | RTX 4090 24 GB / 5090 32 GB + 32 GB RAM | Works, partial offload, ~15-25 tok/s | | Mac Studio M2/M3 Ultra 64 GB+ unified | Works, ~20+ tok/s | | 32 GB unified-memory laptops (Ryzen AI Max+, Apple M-series) | Works with `num_ctx ≤ 4096` and `num_batch ≤ 256` to fit the compute graph; default 32K ctx OOMs. Measured 28.71 tok/s on ASUS ROG Flow Z13 GZ302EA at Q4_K_M (Radeon 8060S iGPU via ROCm gfx1151). | ## Chat template The model uses the standard Qwen 3.x ChatML format with `<|im_start|>` / `<|im_end|>` role markers. The template is embedded in the GGUF metadata for plain conversation use, but Ollama users should rely on the `TEMPLATE` block in the included `Modelfile` — that version exposes the tool-calling scaffolding Ollama's capability detector requires (the embedded template alone is insufficient; see [Ollama](#ollama) above). ### Plain conversation ```text <|im_start|>system You are Janus, a precise and capable assistant…<|im_end|> <|im_start|>user What is the time complexity of mergesort?<|im_end|> <|im_start|>assistant ``` ### With reasoning trace When the model decides to think, the assistant turn contains a `…` block followed by the visible answer: ```text <|im_start|>assistant The user is asking about mergesort. Mergesort divides the array, recursively sorts each half, then merges. The recurrence T(n) = 2T(n/2) + O(n) solves to O(n log n). Mergesort runs in **O(n log n)** time in the worst, average, and best cases. The recurrence is T(n) = 2T(n/2) + O(n), which solves to Θ(n log n) by the master theorem.<|im_end|> ``` Most clients (Open WebUI, LibreChat, etc.) hide the `` block by default and show only the final answer. If your client doesn't, set its "show reasoning" toggle off. ### Tool / function calling The wire format depends on which path you take. **Both are valid** — the model adapts to whichever format the system prompt specifies. **Ollama path** (this repo's `Modelfile`). The TEMPLATE advertises tools inside `…` and asks the model to reply in JSON-in-XML — the form Ollama's tool-call extractor parses into a structured `tool_calls` array on `/api/chat` and `/v1/chat/completions`: ```text {"name": "get_weather", "arguments": {"city": "Tokyo"}} ``` **Embedded-jinja path** (llama.cpp, llama-cpp-python, LM Studio). The Qwen 3.6 native chat template baked into the GGUF instructs the model to emit a more verbose XML form. This is the shape you'll see if you talk to `llama-server` or LM Studio directly: ```text Tokyo ``` Pick the parser shape that matches your loader. Don't mix. #### Example (Ollama, OpenAI-compatible API) ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ignored") resp = client.chat.completions.create( model="janus", messages=[ {"role": "user", "content": "Call get_weather for Tokyo. Respond ONLY with the tool call."} ], tools=[{ "type": "function", "function": { "name": "get_weather", "description": "Get current weather for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }], temperature=0.3, ) print(resp.choices[0].message.tool_calls) # [ToolCall(id='call_xxx', type='function', # function=Function(name='get_weather', arguments='{"city":"Tokyo"}'))] ``` #### Tips - Use direct prompts ("Call X for Y") rather than soft hints ("Use the tool"). The model thinks before committing to a call, and weak prompts can exhaust `num_predict` inside the `` block before the call is emitted. - Allow at least `num_predict: 1024` (or `max_tokens: 1024`) for tool-calling turns, more if the schemas are large. - The Modelfile's JSON-in-XML format is what Ollama's tool-call extractor understands; if you swap loaders, swap the parser to match (see "Embedded-jinja path" above). ## Known limitations - **No mmproj in this release.** The base Qwen3.6 supports image and video input via a separate `mmproj` file, which is not included here. Text-only inference works out of the box; multimodal inference requires fetching `Qwen3.6-35B-A3B-uncensored-heretic-mmproj-BF16.gguf` (or equivalent) from upstream — run `./scripts/fetch_vision.sh` and see [Vision](#vision) for the full path. - **Quantization-induced quality loss.** Q4_K_M is a strong general-purpose quant but does measurably degrade math and code accuracy compared to BF16. If you need maximum quality, run the upstream safetensors on a GPU that fits BF16 (~70 GB). - **MoE expert utilization is uneven.** Stock Qwen3.6-35B-A3B routes 8 of 256 experts per token. On narrow domains (e.g. only one programming language) a small subset of experts dominates; load-balance loss was a training-time concern, not a runtime guarantee. - **Thinking traces can loop.** Like most reasoning-distilled models, Janus-35B occasionally gets stuck repeating itself inside `` tags. Mitigations: lower temperature to 0.4-0.6, raise `repeat_penalty` to 1.08, or set a ``-token budget cap if your loader supports it. - **Uncensored base — not aligned with any specific safety policy.** This is a personal repackage of an open-weight base whose refusal behavior has been abliterated away (the llmfan46 Heretic base). There is no RLHF refusal layer; the model will attempt most requests, so downstream safety is entirely the operator's responsibility. - **No formal evaluation in this card.** Numbers in the hardware table are estimates, not measured. If you produce real benchmarks (MMLU, HumanEval, etc.) and want them included, file a PR. ## Related models | Model | Size | Notes | |---|---|---| | [llmfan46/Qwen3.6-35B-A3B-uncensored-heretic](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) | 35B / 3B active | **Immediate base.** Uncensored Heretic abliteration of Qwen 3.6 35B-A3B; `transformers`-native safetensors. | | [llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-GGUF) | 35B / 3B active | Heretic GGUFs — pull other quants here; the bundled Q4_K_M is already this Heretic quant. | | [llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved) | 35B / 3B active | Same Heretic base but keeps the MTP head for vLLM / SGLang speculative decoding. | | [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) | 35B / 3B active | Upstream pre-Heretic base model. `transformers`-native multimodal weights. | | [FoolDev/Thanatos-27B-HERETIC](https://huggingface.co/FoolDev/Thanatos-27B-HERETIC) | 27B dense | Dense sibling on the [`llmfan46/Qwen3.6-27B-uncensored-heretic-v2`](https://huggingface.co/llmfan46/Qwen3.6-27B-uncensored-heretic-v2) Heretic base. Same teacher (Fable 5), same dataset family, smaller memory footprint, no MoE quirks. (The older `FoolDev/Thanatos-27B` and `Thanatos-27B-Heretic` slugs now 307 to this path.) | | [Crownelius/Crow-9B-HERETIC-4.6](https://huggingface.co/Crownelius/Crow-9B-HERETIC-4.6) | 9B dense | Heretic-flavored fine-tune on a smaller 9B Qwen base. Useful as a fast first-pass model when 35B is too heavy for the host. | ## Credits - Immediate base: [llmfan46/Qwen3.6-35B-A3B-uncensored-heretic](https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) — Heretic-style abliteration of Qwen 3.6 35B-A3B - Upstream base: [Qwen/Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) (Alibaba) - Reasoning teacher: Claude Fable 5 (Anthropic) - Distillation lineage and dataset curation: [Crownelius](https://huggingface.co/Crownelius) License inherited from upstream: Apache-2.0.