Instructions to use Phil2Sat/T5XXL-Unchained-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Phil2Sat/T5XXL-Unchained-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0
# Run inference directly in the terminal:
llama cli -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0
# Run inference directly in the terminal:
./llama-cli -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Phil2Sat/T5XXL-Unchained-GGUF:Q4_0

Use Docker

docker model run hf.co/Phil2Sat/T5XXL-Unchained-GGUF:Q4_0

LM Studio
Jan
Ollama
How to use Phil2Sat/T5XXL-Unchained-GGUF with Ollama:
```
ollama run hf.co/Phil2Sat/T5XXL-Unchained-GGUF:Q4_0
```

Unsloth Studio

How to use Phil2Sat/T5XXL-Unchained-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Phil2Sat/T5XXL-Unchained-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Phil2Sat/T5XXL-Unchained-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Phil2Sat/T5XXL-Unchained-GGUF to start chatting

Docker Model Runner
How to use Phil2Sat/T5XXL-Unchained-GGUF with Docker Model Runner:
```
docker model run hf.co/Phil2Sat/T5XXL-Unchained-GGUF:Q4_0
```

Lemonade

How to use Phil2Sat/T5XXL-Unchained-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Phil2Sat/T5XXL-Unchained-GGUF:Q4_0

Run and chat with the model

lemonade run user.T5XXL-Unchained-GGUF-Q4_0

List all available models

lemonade list

Atomic Chat

Quantized version of Kaoru8/T5XXL-Unchained.

This is a fork of the T5-XXL encoder that uses an extended tokenizer to facilitate Flux uncensoring and offer other benefits to tokenization. More info is on the project's GitHub page.

As of 17.3.2025, third-party tools like ComfyUI and Kohya's training scripts don't yet support the new architecture. Their respective developers have been notified of this release, and may or may not choose to officially support it in a future update to their code. In the meantime, temporary patches for both tools are available in the GitHub repo linked above.

Downloads last month: 1,067

GGUF

Model size

5B params

Architecture

t5encoder

Hardware compatibility

4-bit

8-bit

16-bit

Model tree for Phil2Sat/T5XXL-Unchained-GGUF

Base model

Kaoru8/T5XXL-Unchained

Quantized

(1)

this model