Add HF README (model-card frontmatter, GitHub link)
Browse files
README.md
ADDED
|
@@ -0,0 +1,81 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: image-text-to-text
|
| 4 |
+
tags:
|
| 5 |
+
- captioning
|
| 6 |
+
- joycaption
|
| 7 |
+
- joytag
|
| 8 |
+
- dataset-preparation
|
| 9 |
+
- gradio
|
| 10 |
+
base_model:
|
| 11 |
+
- fancyfeast/llama-joycaption-beta-one-hf-llava
|
| 12 |
+
- fancyfeast/joytag
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# JoyCaption Batch WebUI
|
| 16 |
+
|
| 17 |
+
A single-window local Gradio UI for **[JoyCaption Beta One](https://huggingface.co/fancyfeast/llama-joycaption-beta-one-hf-llava)** (8B captioning VLM) and **[JoyTag](https://huggingface.co/fancyfeast/joytag)** (fast ViT Danbooru tagger), built for **LoRA / fine-tune dataset prep**:
|
| 18 |
+
|
| 19 |
+
- **Batch folder captioning** — point it at a dataset folder and it writes a `.txt` next to each image (kohya / diffusion-pipe style), with *skip-existing* and an optional *trigger-word prefix*.
|
| 20 |
+
- **All JoyCaption caption types** — Descriptive, Straightforward, Stable Diffusion prompt, Danbooru/e621/Rule34/booru tags, Art Critic, Product Listing, and more, plus length control and the full extra-options list, with a live editable prompt.
|
| 21 |
+
- **JoyTag tab** — near-instant Danbooru tagging (~0.4 GB VRAM), single image with confidence scores or whole folders, threshold slider, underscores→spaces toggle. Loads/unloads independently of the big model.
|
| 22 |
+
- **VRAM control** — Load/Unload buttons free VRAM without closing the app (handy for switching back to ComfyUI), and a 4-bit NF4 mode (~7 GB) lets it run alongside a diffusion UI on one GPU.
|
| 23 |
+
- **Streaming output**, copy-to-clipboard, LAN mode (`--listen`).
|
| 24 |
+
|
| 25 |
+
> 🖥️ **This is a local tool, not a hosted demo** — batch mode reads dataset folders on your own disk, so clone it and run it on your own GPU. Main repo: **https://github.com/justineightyone/joycaption-batch-webui** (this HF repo is a mirror).
|
| 26 |
+
|
| 27 |
+
## Requirements
|
| 28 |
+
|
| 29 |
+
- NVIDIA GPU: **~18 GB VRAM** in bf16 (RTX 3090/4090 class) or **~7 GB** with `--nf4` (RTX 3060 12GB and up work fine)
|
| 30 |
+
- Python 3.10+
|
| 31 |
+
- ~17 GB disk for the JoyCaption model (auto-downloaded on first caption into `./models`)
|
| 32 |
+
|
| 33 |
+
## Install & run (Windows)
|
| 34 |
+
|
| 35 |
+
```
|
| 36 |
+
git clone https://github.com/justineightyone/joycaption-batch-webui
|
| 37 |
+
cd joycaption-batch-webui
|
| 38 |
+
install_windows.bat
|
| 39 |
+
start_windows.bat (full quality, ~18 GB VRAM)
|
| 40 |
+
start_windows_lowvram.bat (4-bit NF4, ~7 GB VRAM)
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
## Install & run (Linux)
|
| 44 |
+
|
| 45 |
+
```
|
| 46 |
+
git clone https://github.com/justineightyone/joycaption-batch-webui
|
| 47 |
+
cd joycaption-batch-webui
|
| 48 |
+
bash install_linux.sh
|
| 49 |
+
./start_linux.sh # add --nf4 for the ~7 GB mode
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
The UI opens at http://127.0.0.1:7861 (port chosen to never collide with ComfyUI's 8188 or A1111's 7860).
|
| 53 |
+
|
| 54 |
+
Flags: `--nf4` (4-bit), `--port N`, `--listen` (LAN access), `--cache-dir PATH` (where models download; default `./models`).
|
| 55 |
+
|
| 56 |
+
## Usage notes
|
| 57 |
+
|
| 58 |
+
- The model loads on the **first caption** (or the Load button). The **Unload** button frees VRAM without closing the app.
|
| 59 |
+
- **Batch Folder** writes `image_name.txt` next to each image. *Skip existing* is on by default, so you can re-run a folder after adding images.
|
| 60 |
+
- The **prompt box is editable** — the dropdowns build the prompt, but you can hand-tweak it before captioning.
|
| 61 |
+
- JoyTag is a different model (a 91M-param ViT classifier, not the VLM) — use it when you want fast, consistent booru tags rather than prose.
|
| 62 |
+
|
| 63 |
+
## Performance
|
| 64 |
+
|
| 65 |
+
Defaults are tuned for Ampere+ GPUs: bf16 weights, SDPA attention, TF32 matmul, whole model on GPU. On an RTX 3090 that's roughly **6–10 s per caption** in bf16. JoyTag is near-instant per image.
|
| 66 |
+
|
| 67 |
+
## ⚠️ Gradio version
|
| 68 |
+
|
| 69 |
+
Pinned to **gradio==6.12.0**. Gradio 6.14.0 has a rendering regression that hides the Caption Type / Caption Length dropdowns and stops the prompt from updating. If the dropdowns ever look broken, you've been auto-upgraded — fix with:
|
| 70 |
+
|
| 71 |
+
```
|
| 72 |
+
pip install gradio==6.12.0
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
## Credits & license
|
| 76 |
+
|
| 77 |
+
- **JoyCaption** and **JoyTag** models and original code by [fancyfeast (fpgaminer)](https://github.com/fpgaminer/joycaption) — Apache-2.0. JoyCaption is free, open, and uncensored; if you find it useful, go star the upstream project.
|
| 78 |
+
- UI derived from the HF Spaces [fancyfeast/joy-caption-beta-one](https://huggingface.co/spaces/fancyfeast/joy-caption-beta-one) and [MegaTronX/JoyCaptionBetaOne](https://huggingface.co/spaces/MegaTronX/JoyCaptionBetaOne).
|
| 79 |
+
- `joytag_models.py` is from the official [fancyfeast/joytag](https://huggingface.co/spaces/fancyfeast/joytag) Space.
|
| 80 |
+
|
| 81 |
+
This repo is licensed **Apache-2.0** (see [LICENSE](LICENSE)).
|