Instructions to use FoolDev/Janus-35B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FoolDev/Janus-35B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="FoolDev/Janus-35B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("FoolDev/Janus-35B", dtype="auto")

llama-cpp-python

How to use FoolDev/Janus-35B with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="FoolDev/Janus-35B",
	filename="Janus-35B-A3B.Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use FoolDev/Janus-35B with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FoolDev/Janus-35B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FoolDev/Janus-35B:Q4_K_M

Use Docker

docker model run hf.co/FoolDev/Janus-35B:Q4_K_M

LM Studio
Jan

vLLM

How to use FoolDev/Janus-35B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FoolDev/Janus-35B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/FoolDev/Janus-35B:Q4_K_M

SGLang

How to use FoolDev/Janus-35B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FoolDev/Janus-35B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FoolDev/Janus-35B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FoolDev/Janus-35B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use FoolDev/Janus-35B with Ollama:
```
ollama run hf.co/FoolDev/Janus-35B:Q4_K_M
```

Unsloth Studio new

How to use FoolDev/Janus-35B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FoolDev/Janus-35B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for FoolDev/Janus-35B to start chatting

Pi new

How to use FoolDev/Janus-35B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf FoolDev/Janus-35B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FoolDev/Janus-35B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use FoolDev/Janus-35B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf FoolDev/Janus-35B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FoolDev/Janus-35B:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use FoolDev/Janus-35B with Docker Model Runner:
```
docker model run hf.co/FoolDev/Janus-35B:Q4_K_M
```

Lemonade

How to use FoolDev/Janus-35B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FoolDev/Janus-35B:Q4_K_M

Run and chat with the model

lemonade run user.Janus-35B-Q4_K_M

List all available models

lemonade list

Janus-35B / CHANGELOG.md

FoolDev

Update sibling link: Thanatos-27B → Thanatos-27B-Heretic

52b76d1 6 days ago

preview code

raw

history blame contribute delete

6.63 kB

Changelog

All notable changes to this repository. Format loosely follows Keep a Changelog. This repo holds a model card, an Ollama Modelfile, the HF Ollama-bridge template / system / params files, and the bundled Q4_K_M GGUF, so versions track the tooling and documentation, not the underlying base model.

[Unreleased]

Changed (sibling rename — Thanatos-27B → Thanatos-27B-Heretic)

README "Related models" row updated to point at FoolDev/Thanatos-27B-Heretic (the dense sibling renamed from FoolDev/Thanatos-27B on 2026-05-23 along with a base swap to llmfan46/Qwen3.6-27B-uncensored-heretic-v2, an uncensored Heretic-style abliteration of Qwen 3.6 27B). Description now notes the Heretic base, the uncensored property, and the old-path 307 redirect HF serves.

Added

Root-level template, system, and params files for HF's Ollama bridge. The bridge generates Ollama manifests at request time from these three files (NOT from Modelfile — confirmed against https://huggingface.co/docs/hub/en/ollama). Without them, ollama run hf.co/FoolDev/janus got an auto-generated manifest with the broken {{ if .Prompt }} .Prompt }}<|im_end|> template (Ollama's faulty Go-template conversion of the GGUF's embedded jinja), corrupted stop tokens (".Prompt }}<|im_end|>" bleed), and no .Tools / .ToolCalls blocks — so the published Ollama tag advertised completion only and rejected any request with a tools array. The three files mirror the Modelfile's TEMPLATE / SYSTEM / PARAMETER directives; both routes wire tool calling correctly. Edit them together when changing one. Verified by re-pulling the fresh tag: ollama show hf.co/FoolDev/janus now reports completion, tools, thinking and tool calls round-trip end-to-end through /api/chat.

Changed

README "Tool / function calling" section: split into explicit Ollama-path and embedded-jinja-path subsections. Earlier wording conflated the two on-the-wire formats. The Ollama path (Modelfile TEMPLATE and the new template bridge file, both kept in sync) prompts JSON-in-XML — the form Ollama's tool-call extractor parses into a structured tool_calls array. The embedded-jinja path (llama.cpp, llama-cpp-python, LM Studio) reads the Qwen 3.6 native chat template baked into the GGUF, which prompts the verbose <function=name> / <parameter=arg> form the model was trained on. Both are valid; the model adapts to whichever shape the system prompt prescribes. README now shows both formats side by side.
README "Quick start / Ollama" section: documents both pull paths (hf.co/... via bridge files vs make ... -f Modelfile locally) and explicitly notes that HF's bridge does not read Modelfile.
README "Hardware requirements" intro: re-framed the "~~38 GB minimum" claim as "~~38 GB at default num_ctx 16384" and documented that 32 GB hosts can fit the model by trimming context and batch size.
README "Quick start / Ollama" snippet: show both ollama run hf.co/FoolDev/janus and the explicit-tag form ollama run hf.co/FoolDev/janus:Q4_K_M. Same blob (the default tag maps to Q4_K_M), but parity with the 27B sibling — which lists both :latest and :Q3_K_S — and removes ambiguity for users scripting against an explicit quant tag. Verified the explicit tag resolves to the same manifest (model SHA a076aa0d3a1a, bridge blobs 22c7ade72045 / 84a1a6ac580b / f7b1992cf9c1).

Added (cont'd)

README ## TL;DR section near the top of the model card, mirroring the 27B sibling. Two paths (HF Ollama bridge / local Modelfile build) with explicit tags and a one-line capability check. Notes the bridge ingests template / system / params, not Modelfile, so users skimming the top of the page won't form the wrong mental model of which file gets used when.
CITATION.cff for citation metadata (Apache-2.0, references the upstream Qwen3.6-35B-A3B base and the dense Janus-27B sibling). The 27B sibling has had this file since 0.5.0; adding here for parity so academic-style citations work across both repos.
LICENSE file containing the full Apache 2.0 text. The model card front-matter has always declared license: apache-2.0 and the upstream Qwen 3.6 license inherits Apache-2.0, but until now the repo lacked the actual license text file. Same Apache 2.0 text shipped in the 27B sibling.
scripts/check_bridge_sync.py — regression guard for the Modelfile <-> template / system / params sync invariant. The two configurations are consumed by different code paths (ollama create -f Modelfile for local builds vs HF's Ollama bridge for hf.co/... pulls — HF does not read Modelfile), so drift between them re-introduces the bug fixed in commit 70ccef1 where hf.co/FoolDev/janus shipped a broken auto-generated template while local builds had the correct one. Script parses the Modelfile's TEMPLATE / SYSTEM / PARAMETER directives, loads the three bridge files, and fails on any mismatch with a per-key diff. Run on demand before pushing edits to either side of the configuration. The 27B sibling wires an equivalent script into a pre-commit hook (commit 5c67b08); this repo stays leaner and runs it manually.

Fixed

README "Chat template" intro previously claimed all loaders handle the embedded jinja automatically. True for llama.cpp / LM Studio / llama-cpp-python; not true for Ollama, which needs an explicit override (the Modelfile TEMPLATE block locally, the root-level template file when serving via hf.co/...).
README "Tool / function calling" earlier said the XML form <function=name><parameter=arg> is "not what this model produces". That was wrong: the embedded GGUF jinja prompts exactly that form, and llama.cpp / LM Studio / llama-cpp-python users will see it. The "JSON-in-XML" claim only applies on the Ollama path because that's what the Modelfile TEMPLATE prompt instructs.

[0.1.0] — initial public release

Added

Model card with architecture overview, sampling defaults, hardware table, and Modelfile for ollama create janus -f Modelfile.
Bundled Janus-35B-A3B.Q4_K_M.gguf (~19 GB) via Git LFS so the HF "Use this model" widget surfaces a working ollama run snippet.
Tokyo Night themed banner (PNG sourced from the SVG).
Status badges for license, base model, architecture, quant.
Linked sibling FoolDev/janus-27b (dense Qwen 3.6 27B base) under Related models.