Text Generation
GGUF
qwen36
Mixture of Experts
conversational
multimodal
agent
ollama
heretic
uncensored
reasoning
distillation
Instructions to use FoolDev/Janus-35B-HERETIC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use FoolDev/Janus-35B-HERETIC with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: llama cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf FoolDev/Janus-35B-HERETIC:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use FoolDev/Janus-35B-HERETIC with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FoolDev/Janus-35B-HERETIC" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FoolDev/Janus-35B-HERETIC", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Ollama
How to use FoolDev/Janus-35B-HERETIC with Ollama:
ollama run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Unsloth Desktop
- Pi
How to use FoolDev/Janus-35B-HERETIC with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "FoolDev/Janus-35B-HERETIC:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use FoolDev/Janus-35B-HERETIC with Docker Model Runner:
docker model run hf.co/FoolDev/Janus-35B-HERETIC:Q4_K_M
- Lemonade
How to use FoolDev/Janus-35B-HERETIC with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull FoolDev/Janus-35B-HERETIC:Q4_K_M
Run and chat with the model
lemonade run user.Janus-35B-HERETIC-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use FoolDev/Janus-35B-HERETIC with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default FoolDev/Janus-35B-HERETIC:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use FoolDev/Janus-35B-HERETIC with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf FoolDev/Janus-35B-HERETIC:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "FoolDev/Janus-35B-HERETIC:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download scripts/check_tag_sync.sh from FoolDev/Janus-35B-HERETIC: direct link, hf CLI and curl.
- Browser
- Download file 3.71 kB
-
https://huggingface.co/FoolDev/Janus-35B-HERETIC/resolve/main/scripts/check_tag_sync.sh
- Command line
-
hf download hf://FoolDev/Janus-35B-HERETIC/scripts/check_tag_sync.sh
-
curl -L -o check_tag_sync.sh https://huggingface.co/FoolDev/Janus-35B-HERETIC/resolve/main/scripts/check_tag_sync.sh
3.71 kB
| # Janus-35B β does an installed Ollama tag still carry this repo's template? | |
| # | |
| # Ollama resolves a tag once, at pull or create time, and never re-checks this | |
| # repo afterwards. A tag made before a template/system/params change keeps | |
| # serving the old one indefinitely: nothing warns you, and `ollama list` shows | |
| # the pull date, not which release the tag carries. | |
| # | |
| # A local `janus` tag here served the 0.9.6 template for two days while this | |
| # repo was at 0.9.10, missing 0.9.7's tool-chain render β the only change in | |
| # that span to how tool calls are emitted β and 0.9.8's system-block fix, which | |
| # affects every request carrying no tools array. | |
| # | |
| # The static checks here all compare this repo's own files. `make smoke`, | |
| # `make smoke-tools` and `make bench` do hit the installed tag, but none of | |
| # them compares its template with this repo's. | |
| # | |
| # Usage: | |
| # ./scripts/check_tag_sync.sh # tag `janus`, or $TAG | |
| # ./scripts/check_tag_sync.sh <tag> | |
| # TAG=<tag> make check-tag | |
| # | |
| # Exit 0 in sync, 1 drifted, 2 unusable (no ollama; template missing, empty or | |
| # unreadable; tag unreadable). | |
| # | |
| # Requires: ollama. | |
| set -euo pipefail | |
| ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | |
| TAG="${1:-${TAG:-janus}}" | |
| TEMPLATE="${ROOT}/template" | |
| if ! command -v ollama >/dev/null 2>&1; then echo "[!] ollama not found in PATH" >&2; exit 2; fi | |
| # One cause per message, as the tag/daemon split above. | |
| if [[ ! -e "${TEMPLATE}" ]]; then | |
| echo "[!] ${TEMPLATE} does not exist" >&2; exit 2 | |
| elif [[ ! -r "${TEMPLATE}" ]]; then | |
| echo "[!] ${TEMPLATE} is not readable" >&2; exit 2 | |
| elif [[ ! -s "${TEMPLATE}" ]]; then | |
| echo "[!] ${TEMPLATE} is empty" >&2; exit 2 | |
| fi | |
| # Split the assignment from any filter. A direct | |
| # HAVE="$(ollama show ... 2>/dev/null | filter)" | |
| # aborts under set -e + pipefail when the tag is unreadable, and the emptiness | |
| # check below never runs β the footgun check.sh guards. | |
| if ! HAVE="$(ollama show --template "${TAG}" 2>/dev/null)"; then | |
| # One cause per message: unreachable daemon, absent tag, or a tag that is | |
| # present but unreadable. Consult the list rather than assuming. | |
| if ! LISTED="$(ollama list 2>/dev/null)"; then | |
| echo "[!] cannot reach the Ollama daemon β is \`ollama serve\` running?" >&2 | |
| else | |
| # `ollama list` prints a bare tag as name:latest. | |
| WANTED="${TAG}" | |
| [[ "${WANTED}" == *:* ]] || WANTED="${WANTED}:latest" | |
| if printf '%s\n' "${LISTED}" | awk 'NR>1{print $1}' | grep -qxF -- "${WANTED}"; then | |
| echo "[!] tag '${TAG}' is installed but its template could not be read" >&2 | |
| else | |
| echo "[!] tag '${TAG}' is not installed" >&2 | |
| fi | |
| fi | |
| exit 2 | |
| fi | |
| if [[ -z "${HAVE}" ]]; then | |
| echo "[!] tag '${TAG}' carries no template" >&2 | |
| exit 2 | |
| fi | |
| # `ollama show --template` emits the template verbatim, with no trailing | |
| # newline added, so this compares like for like against the file. | |
| if ! WANT="$(cat "${TEMPLATE}")"; then | |
| echo "[!] cannot read ${TEMPLATE}" >&2 | |
| exit 2 | |
| fi | |
| if [[ "${HAVE}" == "${WANT}" ]]; then | |
| echo "[ ok ] '${TAG}' carries this repo's template" | |
| exit 0 | |
| fi | |
| # Name the refresh for the tag actually given, not a hard-coded one. | |
| echo "[!] '${TAG}' has DRIFTED from this repo's template" >&2 | |
| if [[ "${TAG}" == hf.co/* ]]; then | |
| echo " refresh it: ollama pull ${TAG}" >&2 | |
| else | |
| echo " rebuild it: ollama create ${TAG} -f Modelfile" >&2 | |
| echo " Modelfile's FROM names the bundled Q4_K_M, so that" >&2 | |
| echo " swaps any other quant back. If you built another," >&2 | |
| echo " re-run that build: TAG=${TAG} ./scripts/build.sh" >&2 | |
| fi | |
| exit 1 | |