README.md

---
tags:
- gguf
- comfyui
- workflow-generation
- qwen
- text-generation
- sentence-transformers
library_name: gguf
base_model: Qwen/Qwen2.5-14B
license: gpl-3.0
language:
- en
pipeline_tag: text-generation
---

# ComfyUI-WorkflowGenerator Models

This repository contains the quantized GGUF models required for [ComfyUI-WorkflowGenerator](https://github.com/danielpflorian/ComfyUI-WorkflowGenerator), a custom node implementation that generates ComfyUI workflows from natural language descriptions.

## Models Included

### 1. Workflow Generator Model (Required)

- **File**: `workflow-generator-q8_0.gguf`
- **Tokenizer**: `workflow-generator/` directory
- **Purpose**: Generates workflow diagrams from natural language instructions
- **Base Model**: Qwen2.5-14B
- **Training**: Fine-tuned from Qwen2.5-14B using LLaMA-Factory (see [original ComfyGPT repository](https://github.com/comfygpt/comfygpt/tree/main))
- **Status**: **Required** - This model is always needed

### 2. Embedding Model (Required)

- **Directory**: `paraphrase-multilingual-MiniLM-L12-v2/`
- **Purpose**: Semantic search for node name matching and validation
- **Base Model**: [sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2)
- **Status**: **Required** - Always needed for semantic search in NodeValidator

### 3. Node Validator Model (Optional)

- **File**: `Qwen2.5-7B-Instruct-q8_0.gguf`
- **Tokenizer**: `Qwen2.5-7B-Instruct/` directory
- **Purpose**: Refines and corrects node names in workflow diagrams (LLM refinement mode)
- **Base Model**: Qwen2.5-7B-Instruct (base model, not fine-tuned)
- **Status**: **Optional** - Only needed if using LLM refinement (`use_llm_refinement=True`)

## Model Training Information

### Workflow Generator Model

The `workflow-generator-q8_0.gguf` model was trained based on the [ComfyGPT research](https://github.com/comfygpt/comfygpt/tree/main) methodology:

- **Original Model Source**: [xiatianzs/resources](https://huggingface.co/xiatianzs/resources/tree/main) - Original fine-tuned model from ComfyGPT research team
- **Base Model**: [Qwen/Qwen2.5-14B](https://huggingface.co/Qwen/Qwen2.5-14B) from HuggingFace
- **Training Method**: Full fine-tuning (Supervised Fine-Tuning / SFT)
- **Training Framework**: [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)
- **Training Dataset**: `FlowDataset.json` - Contains instruction-input-output pairs where:
  - Instruction: "Based on the description I provided, generate a JSON example of the required ComfyUi workflow."
  - Input: Natural language workflow descriptions
  - Output: JSON diagrams (list of edges representing workflow connections)
- **Training Hyperparameters**:
  - Learning rate: 1.0e-5
  - Epochs: 3.0
  - Batch size: 1 per device (gradient accumulation: 4 steps)
  - LR scheduler: Cosine with 0.1 warmup ratio
  - Precision: bf16
  - Cutoff length: 8,192 tokens (training cutoff; model architecture supports up to 131,072 tokens)
  - DeepSpeed: ZeRO-3 optimization
- **Quantization**: q8_0 (8-bit quantization for efficient inference)

For more details on the training process, see the [original ComfyGPT repository](https://github.com/comfygpt/comfygpt/tree/main) and [training configuration](https://github.com/comfygpt/comfygpt/tree/main/train/sft).

### Embedding Model

The `paraphrase-multilingual-MiniLM-L12-v2` model is a SentenceTransformer model used for semantic search in the NodeValidator. It encodes node names into embeddings and finds the most similar nodes when correcting invalid node names in workflow diagrams.

- **Original Model**: [sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2)
- **Type**: SentenceTransformer (HuggingFace format)
- **Size**: ~420 MB
- **Dimensions**: 384 (embedding vector size)
- **Use Case**: Semantic similarity search for node name matching

### Node Validator Model

The `Qwen2.5-7B-Instruct-q8_0.gguf` model is the base [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) model (not fine-tuned), used for its built-in instruction-following capabilities to select the best node from semantic search candidates.

## Quick Download

Download all models at once:

```bash
huggingface-cli download DanielPFlorian/comfyui-workflowgenerator-models \
    --local-dir ./ComfyUI/models/LLM/
```

Or download specific models:

```bash
# Download only the required models (workflow-generator + embedding model)
huggingface-cli download DanielPFlorian/comfyui-workflowgenerator-models \
    --include "workflow-generator-q8_0.gguf" "workflow-generator/*" "paraphrase-multilingual-MiniLM-L12-v2/*" \
    --local-dir ./ComfyUI/models/LLM/
```

## Installation and Setup

### Step 1: Download Models

Download the models to your ComfyUI models directory:

```bash
# Navigate to ComfyUI directory
cd /path/to/ComfyUI

# Download all models
huggingface-cli download DanielPFlorian/comfyui-workflowgenerator-models \
    --local-dir ./models/LLM/
```

### Step 2: Organize Files in LLM Directory

After downloading, organize the files in `ComfyUI/models/LLM/` as follows:

```
ComfyUI/models/LLM/
├── workflow-generator-q8_0.gguf          # Main model (required)
├── workflow-generator/                   # Main tokenizer (required)
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   ├── vocab.json
│   ├── merges.txt
│   ├── special_tokens_map.json
│   ├── added_tokens.json
│   ├── config.json
│   ├── generation_config.json
│   └── model.safetensors.index.json
├── paraphrase-multilingual-MiniLM-L12-v2/  # Embedding model (required)
│   ├── config.json
│   ├── model.safetensors
│   ├── modules.json
│   ├── sentence_bert_config.json
│   ├── config_sentence_transformers.json
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   ├── special_tokens_map.json
│   ├── sentencepiece.bpe.model
│   ├── unigram.json
│   └── 1_Pooling/
│       └── config.json
├── Qwen2.5-7B-Instruct-q8_0.gguf         # NodeValidator model (optional)
└── Qwen2.5-7B-Instruct/                   # NodeValidator tokenizer (optional)
    ├── tokenizer.json
    ├── tokenizer_config.json
    ├── vocab.json
    ├── merges.txt
    ├── config.json
    └── generation_config.json
```

**Important**: The tokenizer directory name must match the model name (without `.gguf` extension and quantization suffix). The auto-detection code looks for:
- `workflow-generator-q8_0.gguf` → `workflow-generator/` tokenizer
- `Qwen2.5-7B-Instruct-q8_0.gguf` → `Qwen2.5-7B-Instruct/` tokenizer

## Usage

### Required Models

- **workflow-generator-q8_0.gguf** + **workflow-generator/** tokenizer - Always needed
- **paraphrase-multilingual-MiniLM-L12-v2/** - Always needed for semantic search

### Optional Models

- **Qwen2.5-7B-Instruct-q8_0.gguf** + **Qwen2.5-7B-Instruct/** tokenizer - Only needed if using LLM refinement (`use_llm_refinement=True`)

### Model Usage in ComfyUI-WorkflowGenerator

1. **WorkflowGenerator Node**: Uses `workflow-generator-q8_0.gguf` to generate workflow diagrams from natural language
2. **NodeValidator Node**: 
   - Uses `paraphrase-multilingual-MiniLM-L12-v2` for semantic search (always)
   - Uses `Qwen2.5-7B-Instruct-q8_0.gguf` for LLM refinement (optional, when `use_llm_refinement=True`)
3. **WorkflowBuilder Node**: No models needed (deterministic code)

## Model Specifications

### Workflow Generator Model

- **Format**: GGUF (q8_0 quantization)
- **Base**: Qwen2.5-14B
- **Size**: ~8-9 GB (quantized)
- **Context Window**: 131,072 tokens (128K) - Model architecture supports up to 131K tokens, though training used 16,384 token cutoff
- **Quantization**: q8_0 (8-bit, good balance of quality and size)

### Embedding Model

- **Format**: SentenceTransformer (HuggingFace format)
- **Base**: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
- **Size**: ~420 MB
- **Dimensions**: 384 (embedding vector size)

### Node Validator Model

- **Format**: GGUF (q8_0 quantization)
- **Base**: Qwen2.5-7B-Instruct
- **Size**: ~4-5 GB (quantized)
- **Context Window**: 32,768 tokens
- **Quantization**: q8_0 (8-bit, good balance of quality and size)

## System Requirements

- **VRAM**: 
  - Minimum: 8 GB (for workflow-generator with CPU offloading)
  - Recommended: 16+ GB (for both models on GPU)
- **RAM**: 16+ GB recommended
- **Storage**: ~15-16 GB for all models, tokenizers, and embedding model

## Performance Tips

1. **Use GGUF models**: Smaller size and better VRAM efficiency than HuggingFace models
2. **GPU Layers**: Use "auto" setting for optimal GPU layer allocation
3. **LLM Refinement**: Only enable if you need higher accuracy (slower but more accurate)
4. **Semantic Search Only**: Faster execution, deterministic results (recommended for most use cases)

## Troubleshooting

### Model Not Found

- Verify models are in `ComfyUI/models/LLM/` directory
- Check tokenizer directory name matches model name (without `.gguf` and quantization suffix)
- Restart ComfyUI after moving files

### Tokenizer Not Found

- Ensure tokenizer directory exists with the correct name
- Verify tokenizer files (`tokenizer.json`, `tokenizer_config.json`, `vocab.json`) are present
- Check directory structure matches the expected format

### Out of Memory

- Reduce `n_gpu_layers` (try "auto" or lower number)
- Use smaller quantization (q4_0 instead of q8_0) - note: you'll need to re-quantize
- Set `device_preference` to "cpu" for some operations

## Related Resources

- **ComfyUI-WorkflowGenerator**: [GitHub Repository](https://github.com/danielpflorian/ComfyUI-WorkflowGenerator)
- **Original ComfyGPT Research**: [GitHub Repository](https://github.com/comfygpt/comfygpt)
- **Research Paper**: [arXiv:2503.17671](https://arxiv.org/abs/2503.17671)
- **Project Website**: [https://comfygpt.github.io/](https://comfygpt.github.io/)

## Citation

If you use these models in your research, please cite the original ComfyGPT paper:

```bibtex
@article{huang2025comfygpt,
  title={ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation},
  author={Huang, Oucheng and Ma, Yuhang and Zhao, Zeng and Wu, Mingrui and Ji, Jiayi and Zhang, Rongsheng and Hu, Zhipeng and Sun, Xiaoshuai and Ji, Rongrong},
  journal={arXiv preprint arXiv:2503.17671},
  year={2025}
}
```

## License

These models are provided for use with ComfyUI-WorkflowGenerator. Please refer to:
- Original ComfyGPT repository for model training details and licensing
- Qwen2.5 model licenses from HuggingFace
- ComfyUI-WorkflowGenerator repository for usage terms

## Support

For issues, questions, or contributions:
- **Issues**: [ComfyUI-WorkflowGenerator Issues](https://github.com/danielpflorian/ComfyUI-WorkflowGenerator/issues)
- **Documentation**: [ComfyUI-WorkflowGenerator Wiki](https://github.com/danielpflorian/ComfyUI-WorkflowGenerator/wiki)