Instructions to use FoolDev/Janus-35B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FoolDev/Janus-35B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="FoolDev/Janus-35B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("FoolDev/Janus-35B", dtype="auto")

llama-cpp-python

How to use FoolDev/Janus-35B with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="FoolDev/Janus-35B",
	filename="Janus-35B-A3B.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use FoolDev/Janus-35B with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Use Docker

docker model run hf.co/FoolDev/Janus-35B:Q4_K_M

LM Studio
Jan

vLLM

How to use FoolDev/Janus-35B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FoolDev/Janus-35B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/FoolDev/Janus-35B:Q4_K_M

SGLang

How to use FoolDev/Janus-35B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FoolDev/Janus-35B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FoolDev/Janus-35B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use FoolDev/Janus-35B with Ollama:
```
ollama run hf.co/FoolDev/Janus-35B:Q4_K_M
```

Unsloth Studio new

How to use FoolDev/Janus-35B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for FoolDev/Janus-35B to start chatting

Pi new

How to use FoolDev/Janus-35B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf FoolDev/Janus-35B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FoolDev/Janus-35B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use FoolDev/Janus-35B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf FoolDev/Janus-35B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FoolDev/Janus-35B:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use FoolDev/Janus-35B with Docker Model Runner:
```
docker model run hf.co/FoolDev/Janus-35B:Q4_K_M
```

Lemonade

How to use FoolDev/Janus-35B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FoolDev/Janus-35B:Q4_K_M

Run and chat with the model

lemonade run user.Janus-35B-Q4_K_M

List all available models

lemonade list

Janus-35B / CHANGELOG.md

FoolDev

Update sibling link: Thanatos-27B → Thanatos-27B-Heretic

52b76d1 6 days ago

preview code

raw

history blame contribute delete

6.63 kB

	# Changelog

	All notable changes to this repository. Format loosely follows
	[Keep a Changelog](https://keepachangelog.com/en/1.1.0/). This repo holds
	a model card, an Ollama Modelfile, the HF Ollama-bridge `template` /
	`system` / `params` files, and the bundled Q4_K_M GGUF, so versions
	track the tooling and documentation, not the underlying base model.

	## [Unreleased]

	### Changed (sibling rename — Thanatos-27B → Thanatos-27B-Heretic)
	- README "Related models" row updated to point at
	`FoolDev/Thanatos-27B-Heretic` (the dense sibling renamed from
	`FoolDev/Thanatos-27B` on 2026-05-23 along with a base swap to
	`llmfan46/Qwen3.6-27B-uncensored-heretic-v2`, an uncensored
	Heretic-style abliteration of Qwen 3.6 27B). Description now
	notes the Heretic base, the uncensored property, and the
	old-path 307 redirect HF serves.

	### Added
	- Root-level `template`, `system`, and `params` files for HF's Ollama
	bridge. The bridge generates Ollama manifests at request time from
	these three files (NOT from `Modelfile` — confirmed against
	https://huggingface.co/docs/hub/en/ollama). Without them, `ollama
	run hf.co/FoolDev/janus` got an auto-generated manifest with the
	broken `{{ if .Prompt }} .Prompt }}<\|im_end\|>` template (Ollama's
	faulty Go-template conversion of the GGUF's embedded jinja),
	corrupted stop tokens (`".Prompt }}<\|im_end\|>"` bleed), and no
	`.Tools` / `.ToolCalls` blocks — so the published Ollama tag
	advertised `completion` only and rejected any request with a
	`tools` array. The three files mirror the `Modelfile`'s `TEMPLATE`
	/ `SYSTEM` / `PARAMETER` directives; both routes wire tool calling
	correctly. Edit them together when changing one. Verified by
	re-pulling the fresh tag: `ollama show hf.co/FoolDev/janus` now
	reports `completion`, `tools`, `thinking` and tool calls round-trip
	end-to-end through `/api/chat`.

	### Changed
	- README "Tool / function calling" section: split into explicit
	Ollama-path and embedded-jinja-path subsections. Earlier wording
	conflated the two on-the-wire formats. The Ollama path (Modelfile
	`TEMPLATE` and the new `template` bridge file, both kept in sync)
	prompts JSON-in-XML — the form Ollama's tool-call extractor parses
	into a structured `tool_calls` array. The embedded-jinja path
	(llama.cpp, llama-cpp-python, LM Studio) reads the Qwen 3.6 native
	chat template baked into the GGUF, which prompts the verbose
	`<function=name>` / `<parameter=arg>` form the model was trained
	on. Both are valid; the model adapts to whichever shape the system
	prompt prescribes. README now shows both formats side by side.
	- README "Quick start / Ollama" section: documents both pull paths
	(`hf.co/...` via bridge files vs `make ... -f Modelfile` locally)
	and explicitly notes that HF's bridge does not read `Modelfile`.
	- README "Hardware requirements" intro: re-framed the "~38 GB
	minimum" claim as "~38 GB at default `num_ctx 16384`" and
	documented that 32 GB hosts can fit the model by trimming context
	and batch size.
	- README "Quick start / Ollama" snippet: show both
	`ollama run hf.co/FoolDev/janus` and the explicit-tag form
	`ollama run hf.co/FoolDev/janus:Q4_K_M`. Same blob (the default
	tag maps to Q4_K_M), but parity with the 27B sibling — which lists
	both `:latest` and `:Q3_K_S` — and removes ambiguity for users
	scripting against an explicit quant tag. Verified the explicit tag
	resolves to the same manifest (model SHA `a076aa0d3a1a`, bridge
	blobs `22c7ade72045` / `84a1a6ac580b` / `f7b1992cf9c1`).

	### Added (cont'd)
	- README `## TL;DR` section near the top of the model card, mirroring
	the 27B sibling. Two paths (HF Ollama bridge / local Modelfile
	build) with explicit tags and a one-line capability check. Notes
	the bridge ingests `template` / `system` / `params`, not
	`Modelfile`, so users skimming the top of the page won't form the
	wrong mental model of which file gets used when.
	- `CITATION.cff` for citation metadata (Apache-2.0, references the
	upstream Qwen3.6-35B-A3B base and the dense Janus-27B sibling).
	The 27B sibling has had this file since 0.5.0; adding here for
	parity so academic-style citations work across both repos.
	- `LICENSE` file containing the full Apache 2.0 text. The model card
	front-matter has always declared `license: apache-2.0` and the
	upstream Qwen 3.6 license inherits Apache-2.0, but until now the
	repo lacked the actual license text file. Same Apache 2.0 text
	shipped in the 27B sibling.
	- `scripts/check_bridge_sync.py` — regression guard for the
	`Modelfile` <-> `template` / `system` / `params` sync invariant.
	The two configurations are consumed by different code paths
	(`ollama create -f Modelfile` for local builds vs HF's Ollama
	bridge for `hf.co/...` pulls — HF does not read `Modelfile`), so
	drift between them re-introduces the bug fixed in commit 70ccef1
	where `hf.co/FoolDev/janus` shipped a broken auto-generated
	template while local builds had the correct one. Script parses
	the Modelfile's `TEMPLATE` / `SYSTEM` / `PARAMETER` directives,
	loads the three bridge files, and fails on any mismatch with a
	per-key diff. Run on demand before pushing edits to either side
	of the configuration. The 27B sibling wires an equivalent script
	into a pre-commit hook (commit 5c67b08); this repo stays leaner
	and runs it manually.

	### Fixed
	- README "Chat template" intro previously claimed all loaders handle
	the embedded jinja automatically. True for llama.cpp / LM Studio /
	llama-cpp-python; not true for Ollama, which needs an explicit
	override (the `Modelfile` TEMPLATE block locally, the root-level
	`template` file when serving via `hf.co/...`).
	- README "Tool / function calling" earlier said the XML form
	`<function=name><parameter=arg>` is "not what this model produces".
	That was wrong: the embedded GGUF jinja prompts exactly that form,
	and llama.cpp / LM Studio / llama-cpp-python users will see it.
	The "JSON-in-XML" claim only applies on the Ollama path because
	that's what the Modelfile TEMPLATE prompt instructs.

	## [0.1.0] — initial public release

	### Added
	- Model card with architecture overview, sampling defaults, hardware
	table, and `Modelfile` for `ollama create janus -f Modelfile`.
	- Bundled `Janus-35B-A3B.Q4_K_M.gguf` (~19 GB) via Git LFS so the HF
	"Use this model" widget surfaces a working `ollama run` snippet.
	- Tokyo Night themed banner (PNG sourced from the SVG).
	- Status badges for license, base model, architecture, quant.
	- Linked sibling `FoolDev/janus-27b` (dense Qwen 3.6 27B base) under
	Related models.