Instructions to use void-gryph/flux-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use void-gryph/flux-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="void-gryph/flux-GGUF",
	filename="flux.Q4_K_M.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use void-gryph/flux-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf void-gryph/flux-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf void-gryph/flux-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf void-gryph/flux-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf void-gryph/flux-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf void-gryph/flux-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf void-gryph/flux-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf void-gryph/flux-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf void-gryph/flux-GGUF:Q4_K_M

Use Docker

docker model run hf.co/void-gryph/flux-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use void-gryph/flux-GGUF with Ollama:
```
ollama run hf.co/void-gryph/flux-GGUF:Q4_K_M
```

Unsloth Studio new

How to use void-gryph/flux-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for void-gryph/flux-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for void-gryph/flux-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for void-gryph/flux-GGUF to start chatting

Docker Model Runner
How to use void-gryph/flux-GGUF with Docker Model Runner:
```
docker model run hf.co/void-gryph/flux-GGUF:Q4_K_M
```

Lemonade

How to use void-gryph/flux-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull void-gryph/flux-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.flux-GGUF-Q4_K_M

List all available models

lemonade list

FLUX - GGUF Ultimate Edition 🏭

Este repositorio ofrece la colección en formato GGUF del modelo original FLUX. Optimizados para inferencia con poca memoria.

� Tabla de Comparativa de Cuantizaciones

Versión	Tipo	Peso	Calidad	Uso Recomendado
Q4_K_M	Medium	~5-7GB	Basica	Balance / Velocidad
Q5_K_M	Large	~8-10GB	Alta	Calidad
Q8_0	Ultra	Original	Alta	Calidad

⚙️ Guía de Optimización y Comparativa de Parámetros

Para obtener los mejores resultados con esta versión GGUF, se recomiendan los siguientes ajustes:

1. 🎚️ Comparativa de CFG (Classifier Free Guidance)

CFG Scale	Efecto en GGUF	Resultado Visual
1.0 - 3.5	Suave / Realista	Menos contraste, ideal para estilos fotográficos.
4.0 - 6.5	Recomendado	Balance perfecto entre fidelidad al prompt y detalle.
7.0 - 9.0	Estilizado	Colores más saturados y bordes más definidos.

2. ⚡ Comparativa de Pasos (Sampling Steps)

Pasos	Rendimiento	Nivel de Detalle
15 - 20	Ultra Rápido	Bocetos rápidos o previsualizaciones.
25 - 35	Óptimo	El equilibio para GGUF con casi cero ruido.
40+	Estándar	Máximo refinamiento de texturas complejas.

3. 🌫️ Comparativa de Denoise (Solo para i2i / Hires Fix)

0.35 - 0.45: Mantiene la estructura original pero con limpieza de artefactos.
0.50 - 0.65: El rango ideal para añadir detalle sin deformar el sujeto.
0.70+: Cambio significativo de composición (usar con precaución).

👤 Créditos y Atribución

Autor Original: Unknown
Cuantización Experta: void-gryph

🚀 Despliegue en ComfyUI

Archivo GGUF: Mover a ComfyUI/models/unet/
Nodos Requeridos: Es necesario tener instalado ComfyUI-GGUF.
Componentes Originales: Use el CLIP y VAE incluidos en este repo para máxima fidelidad (extraídos sin prefijos de contenedor).

📝 Nota del Autor Original

Please check out the Quickstart Guide to Flux for all the info you need to get started!

FLUX.1 [dev] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions. For more information, please read our blog post.

Key FeaturesCutting-edge output quality, second only to our state-of-the-art model FLUX.1 [pro].

Competitive prompt following, matching the performance of closed source alternatives .

Trained using guidance distillation, making FLUX.1 [dev] more efficient.

Open weights to drive new scientific research, and empower artists to develop innovative workflows.

Generated outputs can be used for personal, scientific, and commercial purposes as described in the flux-1-dev-non-commercial-license.

UsageWe provide a reference implementation of FLUX.1 [dev], as well as sampling code, in a dedicated github repository. Developers and creatives looking to build on top of FLUX.1 [dev] are encouraged to use this as a starting point.

Learn More Here: https://huggingface.co/black-forest-labs/FLUX.1-dev

GGUF Quantizer - Comprimiendo la inteligencia artificial, píxel a píxel.

Downloads last month: 21

GGUF

Model size

12B params

Architecture

flux

Hardware compatibility

4-bit

5-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support