I’ll fetch the requested HF Spaces playbook first, then I’ll read the relevant local skills for HF CLI and Gradio before touching the Space implementation.


/bin/zsh -lc 'curl -L --fail --silent https://gist.githubusercontent.com/gary149/37c955b832558837c40e1c14ff6d955d/raw/ad35807f8466378afd04d7653d53683a847b96c4/hf-spaces-agent-quickstart-compact.md'


# HF Spaces Agent Quickstart (Compact)

---

## DO / NEVER

**DO:** Iterate ON the Space. Push early, verify against live URL. Stream logs. Read logs first, act once, make one targeted fix. Use cheapest iteration rung. Set all three cache env vars at module top, before any import. Measure `@spaces.GPU(duration=)` empirically. Call `c.view_api()` before any `predict()` call. `python3 -m py_compile app.py` is the maximum local check before pushing.

**NEVER:** Sleep-poll. Build mock modes, `SKIP_MODEL_LOAD` env vars, Playwright harnesses, or local Gradio servers. Restart before reading the error. Issue concurrent uploads. Restart while uploading. Stack restarts while `runtime.sha` is still flipping. Use integer device IDs (`.to(0)`, `device_map={"": 0}`, `set_device(0)`). Pin `torch` or `torchaudio`. Proxy another community ZeroGPU Space via `gradio_client`. Wrap a `gradio_client` proxy call in `@spaces.GPU`. Stack `@spaces.GPU` and `@app.api` on the same function. Edit files in dev-mode SSH expecting them to survive restarts.

---

## §1 Fast Path

```bash
hf spaces search "<model name or task>" --sdk gradio --limit 10   # search reference Space first

hf repos create <ns>/<name> --type space \
  --space-sdk <gradio|streamlit|docker|static> [--flavor zero-a10g] --exist-ok

hf upload <ns>/<name> . . --type space \
  --exclude '.git/*' --exclude '__pycache__/*' --exclude '.venv/*' --exclude '*.pyc' \
  --commit-message 'init'

hf spaces logs <ns>/<name> --build --follow
hf spaces logs <ns>/<name> --follow
hf spaces logs <id> --build --tail 200    # BUILD_ERROR: find the FIRST error
hf spaces logs <id> --tail 200            # RUNTIME_ERROR or stuck APP_STARTING
```

README frontmatter (required fields):
```yaml
---
title: ...
emoji: 🚀
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.10.0        # omit for docker/static
app_file: app.py            # gradio/streamlit only
short_description: ...      # <= 60 chars
startup_duration_timeout: 1h  # default 30m; bump for big LLMs
---
```

`hardware:` in frontmatter is silently ignored. Set hardware via `--flavor` at create time OR `hf spaces settings <id> --hardware zero-a10g`. A silently wrong hardware shows up later as `RuntimeError: Found no NVIDIA driver`.

---

## §2 Iteration Ladder

| Rung | When | Command |
|---|---|---|
| 1. Hot-reload | Pure Python edit, Gradio >= 6.1, no new deps | `hf spaces hot-reload <id> -f app.py` |
| 2. Dev SSH | Diagnostics only (edits don't persist) | `ssh -i ~/.ssh/id_rsa <subdomain>@ssh.hf.space '<cmd>'` |
| 3. Targeted upload | Code-only, non-Gradio file, `gr.Server` | `hf upload <ns>/<name> --include '<file>' && hf spaces logs <id> --follow` |
| 4. Full rebuild | `requirements.txt`, Dockerfile, frontmatter, hardware | `hf spaces logs <id> --build --follow` |
| 5. Factory reboot | Container in inconsistent state, last resort | `hf spaces restart <id> --factory-reboot` |

Hot-reload poisoning: `--factory-reboot` after a hot-reload-only commit fails with `could not read Username for https://huggingface.co`. Push any normal `hf upload` commit first, then restart. After upload, `runtime.sha` lags; do NOT restart again until it flips.

States: `BUILDING -> APP_STARTING -> RUNNING`.

---

## §3 Dev Mode (SSH)

```bash
hf spaces dev-mode <ns>/<name>   # triggers rebuild; PRO/Team/Enterprise only
hf spaces info <ns>/<name> --format json \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['runtime']['raw']['domains'][0]['domain'])"
git add . && git commit && git push   # from inside SSH to persist edits
```

Treat all SSH edits as throwaway. Use dev mode for: reading logs, `pip list`, ad-hoc imports, `curl localhost:7860`. Not as an editor.

---

## §4 ZeroGPU Specifics

ZeroGPU is Gradio-only. Flavor: `zero-a10g`. Actual GPU: NVIDIA RTX PRO 6000 Blackwell.

**Stub probe (required for zero-a10g Spaces):**
```python
@spaces.GPU(duration=1)
def _zerogpu_probe(): return "ready"
```
Without at least one `@spaces.GPU` function, the Space `RUNTIME_ERROR`s immediately.

**Cache env vars + import order (set at top of app.py, before any import):**
```python
import os
os.environ.setdefault("HF_HOME", "/data/.cache/huggingface")  # or /tmp on non-bucket spaces
os.environ.setdefault("HF_MODULES_CACHE", "/tmp/hf_modules")
os.environ.setdefault("MPLCONFIGDIR", "/tmp/matplotlib")
import spaces  # before torch; avoids libcudart.so.X errors
```

**NeMo/RNNT exception (import order matters):**
```python
os.environ.setdefault("NUMBA_DISABLE_CUDA", "1")
import spaces  # immediately after NUMBA_DISABLE_CUDA, before any nemo/torch import
```
Load model on CPU at startup; inside `@spaces.GPU` do `model.to("cuda")` then infer then `model.to("cpu")`.

**Model loading:** Load at module level including `.to("cuda")`. Only compute goes inside `@spaces.GPU`.
```python
model = Model.from_pretrained(..., torch_dtype=torch.bfloat16).to("cuda")
# NOT device_map="cuda" for plain from_pretrained - crashes with "Found no NVIDIA driver"
# device_map="cuda" is OK only with bitsandbytes quantization_config (Pattern B)
```
Exception: frameworks that touch CUDA at import (pynvml, NCCL, vllm, NVFP4 quantizers) must defer construction into the first `@spaces.GPU` call with duration set high enough to cover load + inference.

**Multiple checkpoints:**
```python
BUNDLES = {}
for family in ("image", "video"):
    BUNDLES[family] = load(family).to("cuda")
```
Never unload/reload between requests.

**Duration:** Never hardcode without measuring. Ship placeholder, instrument with `time.perf_counter()`, run 2-3 calls (include at least one cold start, which is 1.5-3x slower), set `duration = round(measured_max * 1.4)`. Too-high surfaces as `"duration above maximum"` at call time. Too-low silently truncates. Neither shows at deploy.

Callable duration (use `*args, **kwargs` to absorb `gr.Progress` positional arg):
```python
def _estimate(prompt, history, max_new_tokens, *args, **kwargs):
    return min(240, 60 + int(max_new_tokens / 64))
@spaces.GPU(duration=_estimate)
def chat(prompt, history, max_new_tokens, ..., progress=gr.Progress(track_tqdm=True)): ...
```

**Two-ceiling:** `@spaces.GPU(duration=)` and `@app.api(time_limit=)` both apply; lower wins. Put them on SEPARATE functions:
```python
@spaces.GPU(duration=60)
def _run_gpu(prompt): return inference(prompt)

@app.api(name="generate", concurrency_limit=1, time_limit=180)
def generate(prompt: str) -> str: return _run_gpu(prompt)
```

**Size:** default `"large"` = 48 GB, 1x quota. Use `size="xlarge"` only when exceeding 48 GB:
```python
@spaces.GPU(duration=120, size="xlarge")   # xlarge = 96 GB, 2x quota
def heavy_fn(): ...
```
bf16 costs `params_B * 2` GB (27B=54 GB overflows large). NF4 4-bit costs `params_B * ~0.55` GB (27B=~15 GB fits large, 70B=~40 GB fits large).

**Flash-attn / xformers:** bitsandbytes works on ZeroGPU. Legacy flash-attn 1.x/2.x, triton, and xformers do not build cleanly; flash-attn 3 is officially recommended. Wrap unconditional top-level imports:
```python
try:
    from flash_attn import flash_attn_func; HAS_FA = True
except ImportError:
    flash_attn_func = None; HAS_FA = False
```
**xformers SDPA shim** (for models with hard `import xformers.ops`): install at boot before model imports.
```python
xformers.ops.memory_efficient_attention = _meff  # full shim code in §4 item 5 of full gist
```

**Misc:** `torch.compile` NOT supported on ZeroGPU; use AoTI (torch >= 2.8). Gemma2 / transformers >= 4.50: `attn_implementation="eager"`. Thinking models: `max_new_tokens >= 512`; suppress with `chat_template_kwargs={"enable_thinking": False}` (not `enable_thinking=` directly, raises `TypeError` in transformers >= 5). ONNX: use `onnxruntime-gpu`; rewrite custom ops to opset-20. TorchScript: `torch._C._set_graph_executor_optimize(False)`.

---

## §5 Per-SDK Gotchas

**Gradio slow startup:** set `GRADIO_SSR_MODE=false` via env var (NOT `launch(ssr_mode=False)`, ignored on HF) AND `startup_duration_timeout: 1h` in README YAML. Both required.
```bash
hf spaces variables add <id> --env GRADIO_SSR_MODE=false
```

**`gr.Examples`:** use `cache_examples=True, cache_mode="lazy"`. `"eager"` runs every example at startup and burns ZeroGPU daily quota. Cache is keyed by file path, not content hash; bump a `cache_version` constant to wipe stale cache.

**Streamlit:** port 8501 only. **Docker:** set `app_port:` in README YAML if not 7860 (`EXPOSE` not auto-read); build with `--platform=linux/amd64`. **Static:** set `app_build_command: npm run build` and `app_file: dist/index.html` in README YAML.

---

## §6 requirements.txt Rules

Pin everything except `torch`/`torchaudio` (base layer). Add `torchvision` unpinned if needed. Include `accelerate` whenever using `device_map=`. No local paths or editable installs. Non-pip-installable model code: include the directory in the upload; it lands in `/home/user/app/`. CUDA-extension build failures: vendor a same-named pure-PyTorch shim in the Space root. Verify with `python3 -m py_compile fast_hadamard_transform.py` before pushing. Build hangs = pip backtracking; read `--build` logs and pin the conflicting transitive dep.

---

## §8 Minimum Agent Loop

0. Search reference Space: `hf spaces search "<model name or task>" --sdk gradio --limit 10`. Read its `app.py` + `requirements.txt`. Cap pre-push research at one reference Space.
1. Decide SDK + hardware. Write minimal `app.py` + `requirements.txt` + README frontmatter.
2. Push immediately: `hf repos create --exist-ok` then `hf upload` then `hf spaces logs --build --follow`.
3. Once `RUNNING`: verify with `gradio_client` (see §11).
4. Iterate via cheapest rung in §2.
5. On error: read the FIRST error, make ONE targeted fix, use smallest rung.

---

## §9 Big LLM Cookbook

**Pattern A (Inference Provider proxy):** stateless chat, any model size, zero VRAM, no `@spaces.GPU`, `cpu-basic` hardware.
```python
gr.load("models/<org>/<model>", accept_token=button, provider="fireworks-ai")
```
README needs `hf_oauth: true` and `hf_oauth_scopes: [inference-api]`.

**Pattern B (NF4 4-bit on ZeroGPU):** custom inference, tool use, multimodal, stateful.
```python
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, trust_remote_code=True,
    device_map="cuda", torch_dtype=torch.bfloat16, quantization_config=bnb,
    attn_implementation="sdpa", low_cpu_mem_usage=True).eval()
```
Streaming (non-negotiable for chat UX):
```python
streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True, timeout=120)
Thread(target=model.generate, kwargs=dict(**inputs, streamer=streamer, ...)).start()
```
Pattern B minimum `requirements.txt`:
```
gradio>=6.10
spaces>=0.41
transformers>=4.57
accelerate>=1.10
bitsandbytes>=0.48
sentencepiece
```
Before deploying Pattern B: set `GRADIO_SSR_MODE=false` and `startup_duration_timeout: 1h`. Full Pattern B code in full gist §9.

---

## §10 Custom Frontend + Storage

```python
demo = app  # HF runtime expects this name

@spaces.GPU(duration=60)
def _run_gpu(prompt): return inference(prompt)   # separate function

@app.api(name="generate", concurrency_limit=1, time_limit=180)
def generate(prompt: str) -> str: return _run_gpu(prompt)  # never stack @app.api + @spaces.GPU
```
Hot-reload (rung 1) does NOT work with `gr.Server`. Files in `@app.api` routes are plain dicts: `isinstance(v, dict) and (v.get("path") or v.get("name"))`.

**Persistent storage (HF Buckets):**
```bash
hf buckets create <ns>/<bucket-name>
hf spaces volumes set <ns>/<space> -v hf://buckets/<ns>/<bucket-name>:/data
```
`/data` is ephemeral by default; bucket mount makes it durable. Do NOT load model weights from `/data`; bucket I/O is S3-paced and stalls past any `@spaces.GPU` duration cap. Use bucket for user-generated content only.

---

## §11 Verifying a Deployed Space

```bash
hf spaces info <id> --expand runtime --format json | python3 -c \
  "import json,sys; r=json.load(sys.stdin)['runtime']; print(r['stage'], r.get('hardware','?'), '| requested:', r.get('requested_hardware','?'))"
# expect: RUNNING zero-a10g | requested: zero-a10g
```

```python
from gradio_client import Client, handle_file
import os
c = Client("ns/name", token=os.environ["HF_TOKEN"],   # token=, not hf_token=
           httpx_kwargs={"timeout": 600})              # >= @spaces.GPU duration + 60
print(c.view_api())   # always call first; never guess api_name
result = c.predict(handle_file("input.png"), "prompt", api_name="/generate")
# Streaming:
job = c.submit("prompt", api_name="/chat")
for chunk in job: print(chunk, end="")
# Custom @app.get/post routes (not in view_api()):
import httpx
r = httpx.post(f"{base}/your_route", json={...}, headers=headers, timeout=600)
```

No Playwright. No mock-mode local servers. No anonymous calls to private Spaces (returns 404, not an app bug).

---

## §12 HF Jobs / AOTI

```bash
hf jobs logs <id> --follow   # stream; never sleep+poll
```
`zero-a10g` flavor name is historical. Actual GPU is RTX PRO 6000 Blackwell (`sm_120`). Compile AOTI artifacts for `sm_120`, not A10G `sm_86`.

---

## Stuck-State Cheatsheet

```
BUILDING > 5min                   -> logs --build --tail 500, find FIRST error
APP_STARTING 10-25min (big LLM)   -> NORMAL: model download/load; tail logs to see progress
APP_STARTING forever (small app)  -> logs --tail 500; usually missing import or model OOM
RUNTIME_ERROR right after long    -> startup timed out: (1) set startup_duration_timeout: 1h
  APP_STARTING, sparse logs          in README YAML AND (2) set GRADIO_SSR_MODE=false env var
RUNNING but 404 from public URL   -> Space is private; auth gradio_client with token=
SSH hangs / Permission denied     -> check ~/.ssh/id_rsa, key on /settings/keys, PRO/Team plan
@spaces.GPU "duration above max"  -> lower the value
ZeroGPU init spinner / GPU abort  -> CUDA touched before allocation; for NeMo/RNNT set
  before app logs anything           NUMBA_DISABLE_CUDA=1 then 'import spaces' immediately;
                                     for others reorder so 'import spaces' precedes torch
libcudart.so.X at startup         -> reorder: import spaces before import torch
Factory reboot fails "could not   -> previous commit was hot-reload-only; push any normal
  read Username..."                  hf upload commit first, then restart
runtime.sha stuck on old commit   -> container still loading; poll SHA, do NOT re-restart
Stale example after asset regen   -> gr.Examples cache keyed by path not hash; bump cache_version
```

---

## Footgun Lookup Table

| Symptom | Fix | Ref |
|---|---|---|
| Space on cpu-basic despite `hardware:` in README YAML | `--flavor` at create or `hf spaces settings <id> --hardware zero-a10g`; frontmatter silently ignored | §1 |
| `RuntimeError: Found no NVIDIA driver` inside `@spaces.GPU` | cpu-basic hardware (above) OR `device_map='cuda'` on plain `from_pretrained`; use `.to('cuda')` | §1/§4 |
| factory-reboot fails `could not read Username` | Push any normal `hf upload` commit first, then restart | §2 |
| `runtime.sha` stuck after upload | Container loading; do NOT issue another restart | §2 |
| SSH Permission denied or hangs | Key on `/settings/keys`, PRO/Team plan, subdomain looked up dynamically | §3 |
| SSH edits lost after restart | `git add . && git commit && git push` from inside SSH session | §3 |
| `RUNTIME_ERROR` after long `APP_STARTING` | `startup_duration_timeout: 1h` in README YAML AND `GRADIO_SSR_MODE=false` env var | §5/§7 |
| `RUNNING` but 404 | Private Space; `token=` (not `hf_token=`) in `gradio_client` | §11 |
| `duration above maximum` at call time | Lower `@spaces.GPU(duration=)` value; not visible at deploy time | §4 |
| ZeroGPU init spinner or aborted before app logs | NeMo: `NUMBA_DISABLE_CUDA=1` then `import spaces` immediately; others: `import spaces` before torch | §4 |
| `libcudart.so.X` at startup | `import spaces` before `import torch` | §4 |
| xformers CUDA extension mismatch | SDPA shim before model imports; full shim in full gist §4 item 5 | §4 |
| Gemma2 `sdpa_mask` / `_vmap_for_bhqkv` failures | `attn_implementation='eager'` in `from_pretrained` | §4 |
| `gr.Examples` stale output after in-place asset regen | Cache keyed by path not hash; bump `cache_version` | §5 |
| `gr.Examples` eager burns ZeroGPU quota at startup | `cache_examples=True, cache_mode='lazy'` | §5 |
| `TypeError: N+1 positional args` in callable duration | Add `*args, **kwargs` to duration function signature | §4 |
| Files in `@app.api` routes are plain dicts not FileData | `isinstance(v, dict) and (v.get('path') or v.get('name'))` | §10 |
| AOTI artifacts for `sm_86` fail on ZeroGPU | Compile for `sm_120`; `zero-a10g` name is historical | §12 |
| ONNX custom op errors | `onnxruntime-gpu`; rewrite to opset-20; dynamic batch dim | §4 |
| `enable_thinking=` raises `TypeError` in transformers >= 5 | `chat_template_kwargs={"enable_thinking": False}` in `apply_chat_template` | §4 |
| `gradio_client` proxy in `@spaces.GPU` burns quota | Never wrap HTTP-only proxy in `@spaces.GPU`; proxy Space uses cpu-basic | §9 |
| Hot-reload has no effect with `gr.Server` | Use `hf upload` (rung 3) or real commit | §10 |
| `HF_HOME` set but cache fails silently | Set all three: `HF_HOME`, `HF_MODULES_CACHE=/tmp/hf_modules`, `MPLCONFIGDIR=/tmp/matplotlib`, before any import | §4 |


I’ve read the requested playbook and will apply its deployment loop: reference search, early Space push, log-first debugging, and no local Gradio server or Playwright checks. Next I’m loading the HF CLI and Gradio skill instructions so the implementation and publish flow match the installed tooling.


/bin/zsh -lc "sed -n '1,240p' /root/.codex/plugins/cache/openai-curated-remote/hugging-face/1.0.0/skills/cli/SKILL.md"


---
name: hf-cli
description: "Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing repositories, models, datasets, and Spaces on the Hugging Face Hub. Replaces now deprecated `huggingface-cli` command."
---

Install: `curl -LsSf https://hf.co/cli/install.sh | bash -s`.

The Hugging Face Hub CLI tool `hf` is available. IMPORTANT: The `hf` command replaces the deprecated `huggingface-cli` command.

Use `hf --help` to view available functions. Note that auth commands are now all under `hf auth` e.g. `hf auth whoami`.

## Commands

- `hf download REPO_ID` — Download files from the Hub. `[--type CHOICE --revision TEXT --include TEXT --exclude TEXT --cache-dir TEXT --local-dir TEXT --force-download --dry-run --quiet --max-workers INTEGER]`
- `hf env` — Print information about the environment.
- `hf sync` — Sync files between local directory and a bucket. `[--delete --ignore-times --ignore-sizes --plan TEXT --apply TEXT --dry-run --include TEXT --exclude TEXT --filter-from TEXT --existing --ignore-existing --verbose --quiet]`
- `hf upload REPO_ID` — Upload a file or a folder to the Hub. Recommended for single-commit uploads. `[--type CHOICE --revision TEXT --private --include TEXT --exclude TEXT --delete TEXT --commit-message TEXT --commit-description TEXT --create-pr --every FLOAT --quiet]`
- `hf upload-large-folder REPO_ID LOCAL_PATH` — Upload a large folder to the Hub. Recommended for resumable uploads. `[--type CHOICE --revision TEXT --private --include TEXT --exclude TEXT --num-workers INTEGER --no-report --no-bars]`
- `hf version` — Print information about the hf version.

### `hf auth` — Manage authentication (login, logout, etc.).

- `hf auth list` — List all stored access tokens.
- `hf auth login` — Login using a token from huggingface.co/settings/tokens. `[--add-to-git-credential --force]`
- `hf auth logout` — Logout from a specific token. `[--token-name TEXT]`
- `hf auth switch` — Switch between access tokens. `[--token-name TEXT --add-to-git-credential]`
- `hf auth whoami` — Find out which huggingface.co account you are logged in as. `[--format CHOICE]`

### `hf buckets` — Commands to interact with buckets.

- `hf buckets cp SRC` — Copy a single file to or from a bucket. `[--quiet]`
- `hf buckets create BUCKET_ID` — Create a new bucket. `[--private --exist-ok --quiet]`
- `hf buckets delete BUCKET_ID` — Delete a bucket. `[--yes --missing-ok --quiet]`
- `hf buckets info BUCKET_ID` — Get info about a bucket. `[--quiet]`
- `hf buckets list` — List buckets or files in a bucket. `[--human-readable --tree --recursive --format CHOICE --quiet]`
- `hf buckets move FROM_ID TO_ID` — Move (rename) a bucket to a new name or namespace.
- `hf buckets remove ARGUMENT` — Remove files from a bucket. `[--recursive --yes --dry-run --include TEXT --exclude TEXT --quiet]`
- `hf buckets sync` — Sync files between local directory and a bucket. `[--delete --ignore-times --ignore-sizes --plan TEXT --apply TEXT --dry-run --include TEXT --exclude TEXT --filter-from TEXT --existing --ignore-existing --verbose --quiet]`

### `hf cache` — Manage local cache directory.

- `hf cache list` — List cached repositories or revisions. `[--cache-dir TEXT --revisions --filter TEXT --format CHOICE --quiet --sort CHOICE --limit INTEGER]`
- `hf cache prune` — Remove detached revisions from the cache. `[--cache-dir TEXT --yes --dry-run]`
- `hf cache rm TARGETS` — Remove cached repositories or revisions. `[--cache-dir TEXT --yes --dry-run]`
- `hf cache verify REPO_ID` — Verify checksums for a single repo revision from cache or a local directory. `[--type CHOICE --revision TEXT --cache-dir TEXT --local-dir TEXT --fail-on-missing-files --fail-on-extra-files]`

### `hf collections` — Interact with collections on the Hub.

- `hf collections add-item COLLECTION_SLUG ITEM_ID ITEM_TYPE` — Add an item to a collection. `[--note TEXT --exists-ok]`
- `hf collections create TITLE` — Create a new collection on the Hub. `[--namespace TEXT --description TEXT --private --exists-ok]`
- `hf collections delete COLLECTION_SLUG` — Delete a collection from the Hub. `[--missing-ok]`
- `hf collections delete-item COLLECTION_SLUG ITEM_OBJECT_ID` — Delete an item from a collection. `[--missing-ok]`
- `hf collections info COLLECTION_SLUG` — Get info about a collection on the Hub. Output is in JSON format.
- `hf collections list` — List collections on the Hub. `[--owner TEXT --item TEXT --sort CHOICE --limit INTEGER --format CHOICE --quiet]`
- `hf collections update COLLECTION_SLUG` — Update a collection's metadata on the Hub. `[--title TEXT --description TEXT --position INTEGER --private --theme TEXT]`
- `hf collections update-item COLLECTION_SLUG ITEM_OBJECT_ID` — Update an item in a collection. `[--note TEXT --position INTEGER]`

### `hf datasets` — Interact with datasets on the Hub.

- `hf datasets info DATASET_ID` — Get info about a dataset on the Hub. Output is in JSON format. `[--revision TEXT --expand TEXT]`
- `hf datasets list` — List datasets on the Hub. `[--search TEXT --author TEXT --filter TEXT --sort CHOICE --limit INTEGER --expand TEXT --format CHOICE --quiet]`
- `hf datasets parquet DATASET_ID` — List parquet file URLs available for a dataset. `[--subset TEXT --split TEXT --format CHOICE --quiet]`
- `hf datasets sql SQL` — Execute a raw SQL query with DuckDB against dataset parquet URLs. `[--format CHOICE]`

### `hf discussions` — Manage discussions and pull requests on the Hub.

- `hf discussions close REPO_ID NUM` — Close a discussion or pull request. `[--comment TEXT --yes --type CHOICE]`
- `hf discussions comment REPO_ID NUM` — Comment on a discussion or pull request. `[--body TEXT --body-file PATH --type CHOICE]`
- `hf discussions create REPO_ID --title TEXT` — Create a new discussion or pull request on a repo. `[--body TEXT --body-file PATH --pull-request --type CHOICE]`
- `hf discussions diff REPO_ID NUM` — Show the diff of a pull request. `[--type CHOICE]`
- `hf discussions info REPO_ID NUM` — Get info about a discussion or pull request. `[--comments --diff --no-color --type CHOICE --format CHOICE]`
- `hf discussions list REPO_ID` — List discussions and pull requests on a repo. `[--status CHOICE --kind CHOICE --author TEXT --limit INTEGER --type CHOICE --format CHOICE --quiet]`
- `hf discussions merge REPO_ID NUM` — Merge a pull request. `[--comment TEXT --yes --type CHOICE]`
- `hf discussions rename REPO_ID NUM NEW_TITLE` — Rename a discussion or pull request. `[--type CHOICE]`
- `hf discussions reopen REPO_ID NUM` — Reopen a closed discussion or pull request. `[--comment TEXT --yes --type CHOICE]`

### `hf endpoints` — Manage Hugging Face Inference Endpoints.

- `hf endpoints catalog deploy --repo TEXT` — Deploy an Inference Endpoint from the Model Catalog. `[--name TEXT --accelerator TEXT --namespace TEXT]`
- `hf endpoints catalog list` — List available Catalog models.
- `hf endpoints delete NAME` — Delete an Inference Endpoint permanently. `[--namespace TEXT --yes]`
- `hf endpoints deploy NAME --repo TEXT --framework TEXT --accelerator TEXT --instance-size TEXT --instance-type TEXT --region TEXT --vendor TEXT` — Deploy an Inference Endpoint from a Hub repository. `[--namespace TEXT --task TEXT --min-replica INTEGER --max-replica INTEGER --scale-to-zero-timeout INTEGER --scaling-metric CHOICE --scaling-threshold FLOAT]`
- `hf endpoints describe NAME` — Get information about an existing endpoint. `[--namespace TEXT]`
- `hf endpoints list` — Lists all Inference Endpoints for the given namespace. `[--namespace TEXT --format CHOICE --quiet]`
- `hf endpoints pause NAME` — Pause an Inference Endpoint. `[--namespace TEXT]`
- `hf endpoints resume NAME` — Resume an Inference Endpoint. `[--namespace TEXT --fail-if-already-running]`
- `hf endpoints scale-to-zero NAME` — Scale an Inference Endpoint to zero. `[--namespace TEXT]`
- `hf endpoints update NAME` — Update an existing endpoint. `[--namespace TEXT --repo TEXT --accelerator TEXT --instance-size TEXT --instance-type TEXT --framework TEXT --revision TEXT --task TEXT --min-replica INTEGER --max-replica INTEGER --scale-to-zero-timeout INTEGER --scaling-metric CHOICE --scaling-threshold FLOAT]`

### `hf extensions` — Manage hf CLI extensions.

- `hf extensions exec NAME` — Execute an installed extension.
- `hf extensions install REPO_ID` — Install an extension from a public GitHub repository. `[--force]`
- `hf extensions list` — List installed extension commands. `[--format CHOICE --quiet]`
- `hf extensions remove NAME` — Remove an installed extension.
- `hf extensions search` — Search extensions available on GitHub (tagged with 'hf-extension' topic). `[--format CHOICE --quiet]`

### `hf jobs` — Run and manage Jobs on the Hub.

- `hf jobs cancel JOB_ID` — Cancel a Job `[--namespace TEXT]`
- `hf jobs hardware` — List available hardware options for Jobs
- `hf jobs inspect JOB_IDS` — Display detailed information on one or more Jobs `[--namespace TEXT]`
- `hf jobs logs JOB_ID` — Fetch the logs of a Job. `[--follow --tail INTEGER --namespace TEXT]`
- `hf jobs ps` — List Jobs. `[--all --namespace TEXT --filter TEXT --format TEXT --quiet]`
- `hf jobs run IMAGE COMMAND` — Run a Job. `[--env TEXT --secrets TEXT --label TEXT --env-file TEXT --secrets-file TEXT --flavor CHOICE --timeout TEXT --detach --namespace TEXT]`
- `hf jobs scheduled delete SCHEDULED_JOB_ID` — Delete a scheduled Job. `[--namespace TEXT]`
- `hf jobs scheduled inspect SCHEDULED_JOB_IDS` — Display detailed information on one or more scheduled Jobs `[--namespace TEXT]`
- `hf jobs scheduled ps` — List scheduled Jobs `[--all --namespace TEXT --filter TEXT --format TEXT --quiet]`
- `hf jobs scheduled resume SCHEDULED_JOB_ID` — Resume (unpause) a scheduled Job. `[--namespace TEXT]`
- `hf jobs scheduled run SCHEDULE IMAGE COMMAND` — Schedule a Job. `[--suspend --concurrency --env TEXT --secrets TEXT --label TEXT --env-file TEXT --secrets-file TEXT --flavor CHOICE --timeout TEXT --namespace TEXT]`
- `hf jobs scheduled suspend SCHEDULED_JOB_ID` — Suspend (pause) a scheduled Job. `[--namespace TEXT]`
- `hf jobs scheduled uv run SCHEDULE SCRIPT` — Run a UV script (local file or URL) on HF infrastructure `[--suspend --concurrency --image TEXT --flavor CHOICE --env TEXT --secrets TEXT --label TEXT --env-file TEXT --secrets-file TEXT --timeout TEXT --namespace TEXT --with TEXT --python TEXT]`
- `hf jobs stats` — Fetch the resource usage statistics and metrics of Jobs `[--namespace TEXT]`
- `hf jobs uv run SCRIPT` — Run a UV script (local file or URL) on HF infrastructure `[--image TEXT --flavor CHOICE --env TEXT --secrets TEXT --label TEXT --env-file TEXT --secrets-file TEXT --timeout TEXT --detach --namespace TEXT --with TEXT --python TEXT]`

### `hf models` — Interact with models on the Hub.

- `hf models info MODEL_ID` — Get info about a model on the Hub. Output is in JSON format. `[--revision TEXT --expand TEXT]`
- `hf models list` — List models on the Hub. `[--search TEXT --author TEXT --filter TEXT --num-parameters TEXT --sort CHOICE --limit INTEGER --expand TEXT --format CHOICE --quiet]`

### `hf papers` — Interact with papers on the Hub.

- `hf papers list` — List daily papers on the Hub. `[--date TEXT --sort CHOICE --limit INTEGER --format CHOICE --quiet]`

### `hf repos` — Manage repos on the Hub.

- `hf repos branch create REPO_ID BRANCH` — Create a new branch for a repo on the Hub. `[--revision TEXT --type CHOICE --exist-ok]`
- `hf repos branch delete REPO_ID BRANCH` — Delete a branch from a repo on the Hub. `[--type CHOICE]`
- `hf repos create REPO_ID` — Create a new repo on the Hub. `[--type CHOICE --space-sdk TEXT --private --exist-ok --resource-group-id TEXT]`
- `hf repos delete REPO_ID` — Delete a repo from the Hub. This is an irreversible operation. `[--type CHOICE --missing-ok]`
- `hf repos delete-files REPO_ID PATTERNS` — Delete files from a repo on the Hub. `[--type CHOICE --revision TEXT --commit-message TEXT --commit-description TEXT --create-pr]`
- `hf repos duplicate FROM_ID` — Duplicate a repo on the Hub (model, dataset, or Space). `[--type CHOICE --private --exist-ok]`
- `hf repos move FROM_ID TO_ID` — Move a repository from a namespace to another namespace. `[--type CHOICE]`
- `hf repos settings REPO_ID` — Update the settings of a repository. `[--gated CHOICE --private --type CHOICE]`
- `hf repos tag create REPO_ID TAG` — Create a tag for a repo. `[--message TEXT --revision TEXT --type CHOICE]`
- `hf repos tag delete REPO_ID TAG` — Delete a tag for a repo. `[--yes --type CHOICE]`
- `hf repos tag list REPO_ID` — List tags for a repo. `[--type CHOICE]`

### `hf skills` — Manage skills for AI assistants.

- `hf skills add` — Download a skill and install it for an AI assistant. `[--claude --codex --cursor --opencode --global --dest PATH --force]`
- `hf skills preview` — Print the generated SKILL.md to stdout.

### `hf spaces` — Interact with spaces on the Hub.

- `hf spaces dev-mode SPACE_ID` — Enable or disable dev mode on a Space. `[--stop]`
- `hf spaces hot-reload SPACE_ID` — Hot-reload any Python file of a Space without a full rebuild + restart. `[--local-file TEXT --skip-checks --skip-summary]`
- `hf spaces info SPACE_ID` — Get info about a space on the Hub. Output is in JSON format. `[--revision TEXT --expand TEXT]`
- `hf spaces list` — List spaces on the Hub. `[--search TEXT --author TEXT --filter TEXT --sort CHOICE --limit INTEGER --expand TEXT --format CHOICE --quiet]`

### `hf webhooks` — Manage webhooks on the Hub.

- `hf webhooks create --watch TEXT` — Create a new webhook. `[--url TEXT --job-id TEXT --domain CHOICE --secret TEXT]`
- `hf webhooks delete WEBHOOK_ID` — Delete a webhook permanently. `[--yes]`
- `hf webhooks disable WEBHOOK_ID` — Disable an active webhook.
- `hf webhooks enable WEBHOOK_ID` — Enable a disabled webhook.
- `hf webhooks info WEBHOOK_ID` — Show full details for a single webhook as JSON.
- `hf webhooks list` — List all webhooks for the current user. `[--format CHOICE --quiet]`
- `hf webhooks update WEBHOOK_ID` — Update an existing webhook. Only provided options are changed. `[--url TEXT --watch TEXT --domain CHOICE --secret TEXT]`

## Common options

- `--format` — Output format: `--format json` (or `--json`) or `--format table` (default).
- `-q / --quiet` — Minimal output.
- `--revision` — Git revision id which can be a branch name, a tag, or a commit hash.
- `--token` — Use a User Access Token. Prefer setting `HF_TOKEN` env var instead of passing `--token`.
- `--type` — The type of repository (model, dataset, or space).

## Tips

- Use `hf <command> --help` for full options, descriptions, usage, and real-world examples
- Authenticate with `HF_TOKEN` env var (recommended) or with `--token`


/bin/zsh -lc "sed -n '1,260p' /root/.codex/plugins/cache/openai-curated-remote/hugging-face/1.0.0/skills/gradio/SKILL.md"


---
name: huggingface-gradio
description: Build Gradio web UIs and demos in Python. Use when creating or editing Gradio apps, components, event listeners, layouts, or chatbots.
---

# Gradio

Gradio is a Python library for building interactive web UIs and ML demos. This skill covers the core API, patterns, and examples.

## Guides

Detailed guides on specific topics (read these when relevant):

- [Quickstart](https://www.gradio.app/guides/quickstart)
- [The Interface Class](https://www.gradio.app/guides/the-interface-class)
- [Blocks and Event Listeners](https://www.gradio.app/guides/blocks-and-event-listeners)
- [Controlling Layout](https://www.gradio.app/guides/controlling-layout)
- [More Blocks Features](https://www.gradio.app/guides/more-blocks-features)
- [Custom CSS and JS](https://www.gradio.app/guides/custom-CSS-and-JS)
- [Streaming Outputs](https://www.gradio.app/guides/streaming-outputs)
- [Streaming Inputs](https://www.gradio.app/guides/streaming-inputs)
- [Sharing Your App](https://www.gradio.app/guides/sharing-your-app)
- [Custom HTML Components](https://www.gradio.app/guides/custom-HTML-components)
- [Getting Started with the Python Client](https://www.gradio.app/guides/getting-started-with-the-python-client)
- [Getting Started with the JS Client](https://www.gradio.app/guides/getting-started-with-the-js-client)

## Core Patterns

**Interface** (high-level): wraps a function with input/output components.

```python
import gradio as gr

def greet(name):
    return f"Hello {name}!"

gr.Interface(fn=greet, inputs="text", outputs="text").launch()
```

**Blocks** (low-level): flexible layout with explicit event wiring.

```python
import gradio as gr

with gr.Blocks() as demo:
    name = gr.Textbox(label="Name")
    output = gr.Textbox(label="Greeting")
    btn = gr.Button("Greet")
    btn.click(fn=lambda n: f"Hello {n}!", inputs=name, outputs=output)

demo.launch()
```

**ChatInterface**: high-level wrapper for chatbot UIs.

```python
import gradio as gr

def respond(message, history):
    return f"You said: {message}"

gr.ChatInterface(fn=respond).launch()
```

## Key Component Signatures

### `Textbox(value: str | I18nData | Callable | None = None, type: Literal['text', 'password', 'email'] = "text", lines: int = 1, max_lines: int | None = None, placeholder: str | I18nData | None = None, label: str | I18nData | None = None, info: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, autofocus: bool = False, autoscroll: bool = True, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", text_align: Literal['left', 'right'] | None = None, rtl: bool = False, buttons: list[Literal['copy'] | Button] | None = None, max_length: int | None = None, submit_btn: str | bool | None = False, stop_btn: str | bool | None = False, html_attributes: InputHTMLAttributes | None = None)`
Creates a textarea for user to enter string input or display string output..

### `Number(value: float | Callable | None = None, label: str | I18nData | None = None, placeholder: str | I18nData | None = None, info: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", buttons: list[Button] | None = None, precision: int | None = None, minimum: float | None = None, maximum: float | None = None, step: float = 1)`
Creates a numeric field for user to enter numbers as input or display numeric output..

### `Slider(minimum: float = 0, maximum: float = 100, value: float | Callable | None = None, step: float | None = None, precision: int | None = None, label: str | I18nData | None = None, info: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", randomize: bool = False, buttons: list[Literal['reset']] | None = None)`
Creates a slider that ranges from {minimum} to {maximum} with a step size of {step}..

### `Checkbox(value: bool | Callable = False, label: str | I18nData | None = None, info: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", buttons: list[Button] | None = None)`
Creates a checkbox that can be set to `True` or `False`.

### `Dropdown(choices: Sequence[str | int | float | tuple[str, str | int | float]] | None = None, value: str | int | float | Sequence[str | int | float] | Callable | DefaultValue | None = DefaultValue(), type: Literal['value', 'index'] = "value", multiselect: bool | None = None, allow_custom_value: bool = False, max_choices: int | None = None, filterable: bool = True, label: str | I18nData | None = None, info: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", buttons: list[Button] | None = None)`
Creates a dropdown of choices from which a single entry or multiple entries can be selected (as an input component) or displayed (as an output component)..

### `Radio(choices: Sequence[str | int | float | tuple[str, str | int | float]] | None = None, value: str | int | float | Callable | None = None, type: Literal['value', 'index'] = "value", label: str | I18nData | None = None, info: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", rtl: bool = False, buttons: list[Button] | None = None)`
Creates a set of (string or numeric type) radio buttons of which only one can be selected..

### `Image(value: str | PIL.Image.Image | np.ndarray | Callable | None = None, format: str = "webp", height: int | str | None = None, width: int | str | None = None, image_mode: Literal['1', 'L', 'P', 'RGB', 'RGBA', 'CMYK', 'YCbCr', 'LAB', 'HSV', 'I', 'F'] | None = "RGB", sources: list[Literal['upload', 'webcam', 'clipboard']] | Literal['upload', 'webcam', 'clipboard'] | None = None, type: Literal['numpy', 'pil', 'filepath'] = "numpy", label: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, buttons: list[Literal['download', 'share', 'fullscreen'] | Button] | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, streaming: bool = False, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", webcam_options: WebcamOptions | None = None, placeholder: str | None = None, watermark: WatermarkOptions | None = None)`
Creates an image component that can be used to upload images (as an input) or display images (as an output)..

### `Audio(value: str | Path | tuple[int, np.ndarray] | Callable | None = None, sources: list[Literal['upload', 'microphone']] | Literal['upload', 'microphone'] | None = None, type: Literal['numpy', 'filepath'] = "numpy", label: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, streaming: bool = False, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", format: Literal['wav', 'mp3'] | None = None, autoplay: bool = False, editable: bool = True, buttons: list[Literal['download', 'share'] | Button] | None = None, waveform_options: WaveformOptions | dict | None = None, loop: bool = False, recording: bool = False, subtitles: str | Path | list[dict[str, Any]] | None = None, playback_position: float = 0)`
Creates an audio component that can be used to upload/record audio (as an input) or display audio (as an output)..

### `Video(value: str | Path | Callable | None = None, format: str | None = None, sources: list[Literal['upload', 'webcam']] | Literal['upload', 'webcam'] | None = None, height: int | str | None = None, width: int | str | None = None, label: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", webcam_options: WebcamOptions | None = None, include_audio: bool | None = None, autoplay: bool = False, buttons: list[Literal['download', 'share'] | Button] | None = None, loop: bool = False, streaming: bool = False, watermark: WatermarkOptions | None = None, subtitles: str | Path | list[dict[str, Any]] | None = None, playback_position: float = 0)`
Creates a video component that can be used to upload/record videos (as an input) or display videos (as an output).

### `File(value: str | list[str] | Callable | None = None, file_count: Literal['single', 'multiple', 'directory'] = "single", file_types: list[str] | None = None, type: Literal['filepath', 'binary'] = "filepath", label: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, height: int | str | float | None = None, interactive: bool | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", allow_reordering: bool = False, buttons: list[Button] | None = None)`
Creates a file component that allows uploading one or more generic files (when used as an input) or displaying generic files or URLs for download (as output).

### `Chatbot(value: list[MessageDict | Message] | Callable | None = None, label: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, container: bool = True, scale: int | None = None, min_width: int = 160, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, autoscroll: bool = True, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", height: int | str | None = 400, resizable: bool = False, max_height: int | str | None = None, min_height: int | str | None = None, editable: Literal['user', 'all'] | None = None, latex_delimiters: list[dict[str, str | bool]] | None = None, rtl: bool = False, buttons: list[Literal['share', 'copy', 'copy_all'] | Button] | None = None, watermark: str | None = None, avatar_images: tuple[str | Path | None, str | Path | None] | None = None, sanitize_html: bool = True, render_markdown: bool = True, feedback_options: list[str] | tuple[str, ...] | None = ('Like', 'Dislike'), feedback_value: Sequence[str | None] | None = None, line_breaks: bool = True, layout: Literal['panel', 'bubble'] | None = None, placeholder: str | None = None, examples: list[ExampleMessage] | None = None, allow_file_downloads: <class 'inspect._empty'> = True, group_consecutive_messages: bool = True, allow_tags: list[str] | bool = True, reasoning_tags: list[tuple[str, str]] | None = None, like_user_message: bool = False)`
Creates a chatbot that displays user-submitted messages and responses.

### `Button(value: str | I18nData | Callable = "Run", every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, variant: Literal['primary', 'secondary', 'stop', 'huggingface'] = "secondary", size: Literal['sm', 'md', 'lg'] = "lg", icon: str | Path | None = None, link: str | None = None, link_target: Literal['_self', '_blank', '_parent', '_top'] = "_self", visible: bool | Literal['hidden'] = True, interactive: bool = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", scale: int | None = None, min_width: int | None = None)`
Creates a button that can be assigned arbitrary .click() events.

### `Markdown(value: str | I18nData | Callable | None = None, label: str | I18nData | None = None, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool | None = None, rtl: bool = False, latex_delimiters: list[dict[str, str | bool]] | None = None, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", sanitize_html: bool = True, line_breaks: bool = False, header_links: bool = False, height: int | str | None = None, max_height: int | str | None = None, min_height: int | str | None = None, buttons: list[Literal['copy']] | None = None, container: bool = False, padding: bool = False)`
Used to render arbitrary Markdown output.

### `HTML(value: Any | Callable | None = None, label: str | I18nData | None = None, html_template: str = "${value}", css_template: str = "", js_on_load: str | None = "element.addEventListener('click', function() { trigger('click') });", apply_default_css: bool = True, every: Timer | float | None = None, inputs: Component | Sequence[Component] | set[Component] | None = None, show_label: bool = False, visible: bool | Literal['hidden'] = True, elem_id: str | None = None, elem_classes: list[str] | str | None = None, render: bool = True, key: int | str | tuple[int | str, ...] | None = None, preserved_by_key: list[str] | str | None = "value", min_height: int | None = None, max_height: int | None = None, container: bool = False, padding: bool = False, autoscroll: bool = False, buttons: list[Button] | None = None, props: Any)`
Creates a component with arbitrary HTML.

## Custom HTML Components

If a task requires significant customization of an existing component or a component that doesn't exist in Gradio, you can create one with `gr.HTML`. It supports `html_template` (with `${}` JS expressions and `{{}}` Handlebars syntax), `css_template` for scoped styles, and `js_on_load` for interactivity — where `props.value` updates the component value and `trigger('event_name')` fires Gradio events. For reuse, subclass `gr.HTML` and define `api_info()` for API/MCP support. See the [full guide](https://www.gradio.app/guides/custom-HTML-components).

Here's an example that shows how to create and use these kinds of components:

```python
import gradio as gr

class StarRating(gr.HTML):
    def __init__(self, label, value=0, **kwargs):
        html_template = """
        <h2>${label} rating:</h2>
        ${Array.from({length: 5}, (_, i) => `<img class='${i < value ? '' : 'faded'}' src='https://upload.wikimedia.org/wikipedia/commons/d/df/Award-star-gold-3d.svg'>`).join('')}
        """
        css_template = """
            img { height: 50px; display: inline-block; cursor: pointer; }
            .faded { filter: grayscale(100%); opacity: 0.3; }
        """
        js_on_load = """
            const imgs = element.querySelectorAll('img');
            imgs.forEach((img, index) => {
                img.addEventListener('click', () => {
                    props.value = index + 1;
                });
            });
        """
        super().__init__(value=value, label=label, html_template=html_template, css_template=css_template, js_on_load=js_on_load, **kwargs)

    def api_info(self):
        return {"type": "integer", "minimum": 0, "maximum": 5}

with gr.Blocks() as demo:
    gr.Markdown("# Restaurant Review")
    food_rating = StarRating(label="Food", value=3)
    service_rating = StarRating(label="Service", value=3)
    ambience_rating = StarRating(label="Ambience", value=3)
    average_btn = gr.Button("Calculate Average Rating")
    rating_output = StarRating(label="Average", value=3)
    def calculate_average(food, service, ambience):
        return round((food + service + ambience) / 3)
    average_btn.click(
        fn=calculate_average,
        inputs=[food_rating, service_rating, ambience_rating],
        outputs=rating_output
    )

demo.launch()
```

## Event Listeners

All event listeners share the same signature:

```python
component.event_name(
    fn: Callable | None | Literal["decorator"] = "decorator",
    inputs: Component | Sequence[Component] | set[Component] | None = None,
    outputs: Component | Sequence[Component] | set[Component] | None = None,
    api_name: str | None = None,
    api_description: str | None | Literal[False] = None,
    scroll_to_output: bool = False,
    show_progress: Literal["full", "minimal", "hidden"] = "full",
    show_progress_on: Component | Sequence[Component] | None = None,
    queue: bool = True,
    batch: bool = False,
    max_batch_size: int = 4,
    preprocess: bool = True,
    postprocess: bool = True,
    cancels: dict[str, Any] | list[dict[str, Any]] | None = None,
    trigger_mode: Literal["once", "multiple", "always_last"] | None = None,
    js: str | Literal[True] | None = None,
    concurrency_limit: int | None | Literal["default"] = "default",
    concurrency_id: str | None = None,
    api_visibility: Literal["public", "private", "undocumented"] = "public",
    time_limit: int | None = None,
    stream_every: float = 0.5,
    key: int | str | tuple[int | str, ...] | None = None,
    validator: Callable | None = None,
) -> Dependency
```

Supported events per component:

- **AnnotatedImage**: select
- **Audio**: stream, change, clear, play, pause, stop, pause, start_recording, pause_recording, stop_recording, upload, input
- **BarPlot**: select, double_click
- **BrowserState**: change
- **Button**: click
- **Chatbot**: change, select, like, retry, undo, example_select, option_select, clear, copy, edit
- **Checkbox**: change, input, select
- **CheckboxGroup**: change, input, select
- **ClearButton**: click
- **Code**: change, input, focus, blur
- **ColorPicker**: change, input, submit, focus, blur
- **Dataframe**: change, input, select, edit
- **Dataset**: click, select
- **DateTime**: change, submit
- **DeepLinkButton**: click
- **Dialogue**: change, input, submit
- **DownloadButton**: click
- **Dropdown**: change, input, select, focus, blur, key_up
- **DuplicateButton**: click
- **File**: change, select, clear, upload, delete, download
- **FileExplorer**: change, input, select
- **Gallery**: select, upload, change, delete, preview_close, preview_open
- **HTML**: change, input, click, double_click, submit, stop, edit, clear, play, pause, end, start_recording, pause_recording, stop_recording, focus, blur, upload, release, select, stream, like, example_select, option_select, load, key_up, apply, delete, tick, undo, retry, expand, collapse, download, copy
- **HighlightedText**: change, select
- **Image**: clear, change, stream, select, upload, input
- **ImageEditor**: clear, change, input, select, upload, apply
- **ImageSlider**: clear, change, stream, select, upload, input
- **JSON**: change
- **Label**: change, select
- **LinePlot**: select, double_click
- **LoginButton**: click
- **Markdown**: change, copy
- **Model3D**: change, upload, edit, clear
- **MultimodalTextbox**: change, input, select, submit, focus, blur, stop
- **Navbar**: change
- **Number**: change, input, submit, focus, blur
- **ParamViewer**: change, upload
- **Plot**: change
- **Radio**: select, change, input
- **ScatterPlot**: select, double_click
- **SimpleImage**: clear, change, upload
- **Slider**: change, input, release
- **State**: change
- **Textbox**: change, input, select, submit, focus, blur, stop, copy
- **Timer**: tick
- **UploadButton**: click, upload
- **Video**: change, clear, start_recording, stop_recording, stop, play, pause, end, upload, input

## Additional Reference

- [End-to-End Examples](examples.md) — complete working apps


I’m checking the current workspace and HF account, and I’m doing the required reference Space search before deeper model research. That should tell me whether there’s an existing demo pattern worth borrowing, without spending much time on it.


/bin/zsh -lc "pwd && rg --files -g '"'!__pycache__'"' -g '"'!.git'"'"


/bin/zsh -lc 'hf auth whoami --format json'


/bin/zsh -lc 'hf spaces search "DeepSpec speculative decoding deepseek" --sdk gradio --limit 10'


/bin/zsh -lc 'hf collections info deepseek-ai/deepspec --format json'


fatal: not a git repository (or any of the parent directories): .git


DeepSpec


No results found.
Hint: Use --description to show AI-generated descriptions.


The collection resolves cleanly under `deepseek-ai/deepspec-6a410e3f1831ca8ca801b88b` and the authenticated HF namespace is `Mike0021`. I’m pulling the actual model-card content next; the reference Space search returned no existing DeepSpec Gradio Space to use as a template.


/bin/zsh -lc "sed -n '1,220p' PROMPT.txt"


/goal Research the DeepSpec collection by deepseek-ai on HuggingFace (https://huggingface.co/collections/deepseek-ai/deepspec) and build the most impressive HuggingFace Space possible from it.

The collection contains 12 models across three speculative decoding families:
- dSpark (qwen3_4b, qwen3_8b, qwen3_14b, gemma4_12b) - block7 variants
- dFlash (qwen3_4b, qwen3_8b, qwen3_14b, gemma4_12b) - block7 variants
- EAGLE3 (qwen3_4b, qwen3_8b, qwen3_14b, gemma4_12b) - ttt7 variants

Steps:
1. Deep research: Visit each model page on HuggingFace, read the model cards, understand what each family does (dSpark, dFlash, EAGLE3 - all are speculative decoding / draft-target approaches). Read any linked papers. Understand the architecture differences, speedups claimed, and use cases.
2. Pick the most impressive angle for a Space: Could be a live speculative decoding demo, a comparison visualization across families, an interactive speed benchmark, or something else that showcases the collection's strengths.
3. Implement the Space as a Gradio app. Make it visually impressive and functional.
4. Push to a HuggingFace Space repo.

Before doing anything else, fetch and follow these HF Spaces operating guidelines for the entire task:
curl -L --fail --silent https://gist.githubusercontent.com/gary149/37c955b832558837c40e1c14ff6d955d/raw/ad35807f8466378afd04d7653d53683a847b96c4/hf-spaces-agent-quickstart-compact.md
Read them fully first, then apply them throughout (iteration ladder, push early, ZeroGPU duration measurement, and the listed anti-patterns).


/bin/zsh -lc "wc -l rollout.jsonl && sed -n '1,5p' rollout.jsonl"


25 rollout.jsonl
Reading additional input from stdin...
{"type":"thread.started","thread_id":"019f125a-6d51-76d1-af24-6642c27f6f23"}
{"type":"turn.started"}
{"type":"item.completed","item":{"id":"item_0","type":"agent_message","text":"I’ll fetch the requested HF Spaces playbook first, then I’ll read the relevant local skills for HF CLI and Gradio before touching the Space implementation."}}
{"type":"item.started","item":{"id":"item_1","type":"command_execution","command":"/bin/zsh -lc 'curl -L --fail --silent https://gist.githubusercontent.com/gary149/37c955b832558837c40e1c14ff6d955d/raw/ad35807f8466378afd04d7653d53683a847b96c4/hf-spaces-agent-quickstart-compact.md'","aggregated_output":"","exit_code":null,"status":"in_progress"}}


The workspace is just the saved prompt and turn log, so I’m starting from a clean Space implementation. I’m going to gather model-card metadata and paper links now, then decide whether a live GPU demo is feasible or whether a research-backed visual simulator will showcase the collection better.


RuntimeException: Error calling MCP tool: [TextContent(type='text', text='MCP error -32602: Input validation error: Invalid arguments for tool hub_repo_details: [\n  {\n    "code": "too_big",\n    "maximum": 10,\n    "type": "array",\n    "inclusive": true,\n    "exact": false,\n    "message": "Provide at most 10 repo ids",\n    "path": [\n      "repo_ids"\n    ]\n  }\n]', annotations=None, meta=None)]

MCP error -32602: Input validation error: Invalid arguments for tool hub_repo_details: [
  {
    "code": "too_big",
    "maximum": 10,
    "type": "array",
    "inclusive": true,
    "exact": false,
    "message": "Provide at most 10 repo ids",
    "path": [
      "repo_ids"
    ]
  }
]


The repo-details tool only accepts up to 10 repos at once; I’m splitting the 12-model read into two batches.


# deepseek-ai/dflash_qwen3_14b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 2
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 3338.5M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/dflash_qwen3_14b_block7](https://hf.co/deepseek-ai/dflash_qwen3_14b_block7)

---

# deepseek-ai/dflash_gemma4_12b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 5
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 3296.2M
- **Architecture:** gemma4_text

## Tags
`safetensors` `gemma4_text` `region:us`

**Link:** [https://hf.co/deepseek-ai/dflash_gemma4_12b_block7](https://hf.co/deepseek-ai/dflash_gemma4_12b_block7)

---

# deepseek-ai/eagle3_qwen3_4b_ttt7

## Overview
- **Author:** deepseek-ai
- **Likes:** 1
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 927.3M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/eagle3_qwen3_4b_ttt7](https://hf.co/deepseek-ai/eagle3_qwen3_4b_ttt7)

---

# deepseek-ai/eagle3_qwen3_8b_ttt7

## Overview
- **Author:** deepseek-ai
- **Likes:** 1
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 1546.7M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/eagle3_qwen3_8b_ttt7](https://hf.co/deepseek-ai/eagle3_qwen3_8b_ttt7)

---

# deepseek-ai/eagle3_qwen3_14b_ttt7

## Overview
- **Author:** deepseek-ai
- **Likes:** 1
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 2053.9M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/eagle3_qwen3_14b_ttt7](https://hf.co/deepseek-ai/eagle3_qwen3_14b_ttt7)

---

# deepseek-ai/eagle3_gemma4_12b_ttt7

## Overview
- **Author:** deepseek-ai
- **Likes:** 5
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 2362.3M
- **Architecture:** gemma4_unified_text

## Tags
`safetensors` `gemma4_unified_text` `region:us`

**Link:** [https://hf.co/deepseek-ai/eagle3_gemma4_12b_ttt7](https://hf.co/deepseek-ai/eagle3_gemma4_12b_ttt7)


# deepseek-ai/dspark_qwen3_4b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 11
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 1393.1M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/dspark_qwen3_4b_block7](https://hf.co/deepseek-ai/dspark_qwen3_4b_block7)

---

# deepseek-ai/dspark_qwen3_8b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 2
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 2371.1M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/dspark_qwen3_8b_block7](https://hf.co/deepseek-ai/dspark_qwen3_8b_block7)

---

# deepseek-ai/dspark_qwen3_14b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 3
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 3416.3M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/dspark_qwen3_14b_block7](https://hf.co/deepseek-ai/dspark_qwen3_14b_block7)

---

# deepseek-ai/dspark_gemma4_12b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 8
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 3430.4M
- **Architecture:** gemma4_text

## Tags
`safetensors` `gemma4_text` `region:us`

**Link:** [https://hf.co/deepseek-ai/dspark_gemma4_12b_block7](https://hf.co/deepseek-ai/dspark_gemma4_12b_block7)

---

# deepseek-ai/dflash_qwen3_4b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 1
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 1315.3M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/dflash_qwen3_4b_block7](https://hf.co/deepseek-ai/dflash_qwen3_4b_block7)

---

# deepseek-ai/dflash_qwen3_8b_block7

## Overview
- **Author:** deepseek-ai
- **Likes:** 1
- **Updated:** 28 Jun, 2026

## Technical Details
- **Parameters:** 2293.3M
- **Architecture:** qwen3

## Tags
`safetensors` `qwen3` `region:us`

**Link:** [https://hf.co/deepseek-ai/dflash_qwen3_8b_block7](https://hf.co/deepseek-ai/dflash_qwen3_8b_block7)


Question: site:huggingface.co/deepseek-ai/dspark_qwen3_4b_block7 DeepSpec dSpark model card

Question: site:huggingface.co/deepseek-ai/dspark_qwen3_4b_block7 DeepSpec dSpark model card

DeepSpec DeepSeek dSpark dFlash EAGLE3 speculative decoding paper

dSpark dFlash DeepSpec DeepSeek speculative decoding


Question: DeepSeek DeepSpec GitHub DSpark paper

Question: DeepSeek DeepSpec GitHub DSpark paper

DeepSeek DSpark paper arxiv dSpark dFlash EAGLE3

DeepSpec deepseek-ai GitHub speculative decoding


The DeepSpec repo README is more informative than the individual HF model pages: the pages themselves report “No model card,” while the repo README maps all 12 released checkpoints to Table 1 in the paper and explains the train/eval workflow. I’m reading the linked DSpark PDF plus the DFlash and EAGLE-3 papers now so the app’s comparisons are grounded in the actual methods.


/bin/zsh -lc "curl -L --fail --silent https://raw.githubusercontent.com/deepseek-ai/DeepSpec/main/DSpark_paper.pdf -o /tmp/DSpark_paper.pdf && pdftotext /tmp/DSpark_paper.pdf - | sed -n '1,220p'"


/bin/zsh -lc "curl -L --fail --silent https://raw.githubusercontent.com/deepseek-ai/DeepSpec/main/README.md | sed -n '1,220p'"


/bin/zsh -lc "curl -L --fail --silent https://raw.githubusercontent.com/deepseek-ai/DeepSpec/main/scripts/data/README.md | sed -n '1,220p'"


# DeepSpec

DeepSpec is a full-stack codebase for training and evaluating draft models for speculative decoding. It contains data preparation utilities, draft model implementations, training code, and evaluation scripts.

## Environment

Install the Python dependencies:

```bash
python -m pip install -r requirements.txt
```

Data preparation additionally requires an inference engine to serve the target model when regenerating answers; see [scripts/data/README.md](./scripts/data/README.md) for details.

## Workflow

Run the stages in order — each stage's output feeds the next:

1. **Data Preparation** — download prompts, regenerate target answers, and build the target cache.
2. **Training** — train a draft model against the cached target outputs.
3. **Evaluation** — measure speculative-decoding acceptance on benchmark tasks.

## Data Preparation

See [scripts/data/README.md](./scripts/data/README.md) for the step-by-step data pipeline:

1. download and split training data,
2. regenerate answers,
3. prepare the target cache (storage warning: this can be very large — roughly 38 TB for the default `Qwen/Qwen3-4B` setting).

## Training

```bash
bash scripts/train/train.sh
```

`train.sh` launches `train.py`, which spawns one worker per visible GPU. Select the algorithm and target model by pointing `config_path` at one of the configs under [config/](./config/) (e.g. `config/dspark/dspark_qwen3_4b.py`); see the script header for the full list of configs, how to override `config_path` / `target_cache_dir`, and how to use `--opts` to override individual config fields. Checkpoints are written to `~/checkpoints/<project_name>/<exp_name>/step_*`.

Hardware: the default configs and scripts assume a single node with 8 GPUs. For fewer GPUs, reduce `CUDA_VISIBLE_DEVICES`.

## Evaluation

```bash
bash scripts/eval/eval.sh
```

`eval.sh` runs `eval.py` against a trained draft checkpoint over the speculative-decoding benchmarks in [eval_datasets/](./eval_datasets/) (gsm8k, math500, aime25, humaneval, mbpp, livecodebench, mt-bench, alpaca, arena-hard-v2). Set:

- `target_name_or_path` — the target model the draft was trained against (e.g. `Qwen/Qwen3-4B`),
- `draft_name_or_path` — the draft checkpoint, e.g. `~/checkpoints/deepspec/dspark_block8_qwen3_4b/step_latest`, or one of the Hugging Face repo IDs listed in [Released Checkpoints](#released-checkpoints).

### Released Checkpoints

The checkpoints below are the ones used for Table 1 in the [paper](./DSpark_paper.pdf). Each checkpoint was trained on [open-perfectblend](https://huggingface.co/datasets/mlabonne/open-perfectblend) data generated by its corresponding target model in non-thinking mode, and is the direct output of the corresponding training configuration under [config/](./config/).

| Algorithm | `Qwen/Qwen3-4B` | `Qwen/Qwen3-8B` | `Qwen/Qwen3-14B` | `google/gemma-4-12B-it` |
| --- | --- | --- | --- | --- |
| Eagle3 | [deepseek-ai/eagle3_qwen3_4b_ttt7](https://huggingface.co/deepseek-ai/eagle3_qwen3_4b_ttt7) | [deepseek-ai/eagle3_qwen3_8b_ttt7](https://huggingface.co/deepseek-ai/eagle3_qwen3_8b_ttt7) | [deepseek-ai/eagle3_qwen3_14b_ttt7](https://huggingface.co/deepseek-ai/eagle3_qwen3_14b_ttt7) | [deepseek-ai/eagle3_gemma4_12b_ttt7](https://huggingface.co/deepseek-ai/eagle3_gemma4_12b_ttt7) |
| DFlash | [deepseek-ai/dflash_qwen3_4b_block7](https://huggingface.co/deepseek-ai/dflash_qwen3_4b_block7) | [deepseek-ai/dflash_qwen3_8b_block7](https://huggingface.co/deepseek-ai/dflash_qwen3_8b_block7) | [deepseek-ai/dflash_qwen3_14b_block7](https://huggingface.co/deepseek-ai/dflash_qwen3_14b_block7) | [deepseek-ai/dflash_gemma4_12b_block7](https://huggingface.co/deepseek-ai/dflash_gemma4_12b_block7) |
| DSpark | [deepseek-ai/dspark_qwen3_4b_block7](https://huggingface.co/deepseek-ai/dspark_qwen3_4b_block7) | [deepseek-ai/dspark_qwen3_8b_block7](https://huggingface.co/deepseek-ai/dspark_qwen3_8b_block7) | [deepseek-ai/dspark_qwen3_14b_block7](https://huggingface.co/deepseek-ai/dspark_qwen3_14b_block7) | [deepseek-ai/dspark_gemma4_12b_block7](https://huggingface.co/deepseek-ai/dspark_gemma4_12b_block7) |

> [!IMPORTANT]
> If you cite these results in a new paper, align your setup with the training settings in this repository; otherwise, the comparison is not meaningful. For domain-specific use, fine-tune the draft model again for better results, especially if the target model is expected to run in thinking mode.

## Supported Algorithms

Currently, DeepSpec includes three draft models: [DSpark](./DSpark_paper.pdf), [DFlash](https://arxiv.org/abs/2602.06036) and [Eagle3](https://arxiv.org/abs/2503.01840).

## License

DeepSpec is released under the [MIT License](./LICENSE). It includes code adapted
from third-party projects under their own licenses; see [NOTICE](./NOTICE) for the
full attribution.

## Acknowledgements

DeepSpec builds on the ideas and code of several excellent open-source projects:

- [SpecForge](https://github.com/sgl-project/SpecForge) (Apache-2.0) — the overall training framework and Eagle3 implementation; portions of the Eagle3 modeling, loss, optimizer, attention, and evaluation code are adapted from it. Adapted files carry an in-file attribution comment, and the full notice is recorded in [NOTICE](./NOTICE).
- [DFlash](https://github.com/z-lab/dflash) (MIT) — the DFlash draft-model design and training recipe.
- [Qwen3](https://github.com/QwenLM/Qwen3) and [Gemma](https://github.com/google-deepmind/gemma) — the target model families supported in this repo.

We thank the authors and maintainers of these projects. Contributions of new algorithms are welcome.


# Data Preparation

This directory contains an example data preparation pipeline using `Qwen/Qwen3-4B` as the target model.

DeepSpec trains draft models against a target model. The data pipeline does three things:

1. download and split prompt data,
2. regenerate assistant answers with the target model,
3. precompute the target cache used by training.

The example below targets `Qwen/Qwen3-4B`, but the same pipeline applies to other models (e.g. Gemma). To switch targets, change the model name (`--model` / `model_path`) and adjust the sampling parameters (`--temperature`, `--top-p`, `--top-k` and `--min-p`) to match the recommended generation settings for that model. Output paths in the examples reference `qwen3_4b`; rename them as needed.

The wrapper script [prepare_data.sh](./prepare_data.sh) records the default settings. The individual Python scripts are also documented below for users who want to run each stage manually.

## Outputs

Default outputs:

```text
train_datasets/perfectblend_train.jsonl
train_datasets/qwen3_4b/perfectblend_train_regen.jsonl
~/.cache/deepspec/qwen3_4b_target_cache
```

The example scripts assume a single machine with eight visible GPUs by default. For fewer GPUs, edit `num_workers` and `CUDA_VISIBLE_DEVICES` in the shell scripts.

## Step 1: Download And Split Data

The source dataset is `mlabonne/open-perfectblend`. The train split is written as JSONL, and the held-out user turns are written under `eval_datasets/`.

```bash
python scripts/data/download_and_split.py \
    --dataset-name mlabonne/open-perfectblend \
    --test-size 0.05 \
    --train-output-path train_datasets/perfectblend_train.jsonl \
    --test-output-dir eval_datasets \
    --skip-existing
```

This produces:

```text
train_datasets/perfectblend_train.jsonl
eval_datasets/perfectblend.jsonl
```

## Step 2: Regenerate Answers With Qwen3-4B

This step serves the target model and regenerates assistant answers against it. Any OpenAI-compatible inference engine works (SGLang, vLLM, TGI, etc.) — the example below uses [SGLang](https://github.com/sgl-project/sglang), but you can swap in whatever engine you prefer as long as it exposes an OpenAI-compatible `/v1` endpoint. SGLang is not in `requirements.txt`; install it separately, e.g. `pip install "sglang[all]"`.

Start local sglang servers in one terminal:

```bash
bash scripts/data/launch_sglang_server.sh
```

By default this starts eight `Qwen/Qwen3-4B` workers on ports `30000` to `30007` and writes logs to:

```text
logs/sglang_qwen3_4b/
```

In another terminal, regenerate the assistant answers:

```bash
python scripts/data/generate_train_data.py \
    --model Qwen/Qwen3-4B \
    --server-address \
        127.0.0.1:30000 \
        127.0.0.1:30001 \
        127.0.0.1:30002 \
        127.0.0.1:30003 \
        127.0.0.1:30004 \
        127.0.0.1:30005 \
        127.0.0.1:30006 \
        127.0.0.1:30007 \
    --concurrency 32 \
    --temperature 0.7 \
    --top-p 0.8 \
    --top-k 20 \
    --min-p 0 \
    --max-tokens 4096 \
    --disable-thinking \
    --resume \
    --input-file-path train_datasets/perfectblend_train.jsonl \
    --output-file-path train_datasets/qwen3_4b/perfectblend_train_regen.jsonl
```

This produces:

```text
train_datasets/qwen3_4b/perfectblend_train_regen.jsonl
```

If any samples fail, the script writes them to:

```text
train_datasets/qwen3_4b/perfectblend_train_regen_error.jsonl
```

Stop the sglang servers before the next step if they are using the same GPUs.

## Step 3: Prepare Target Cache

The training loop reads a precomputed target cache instead of repeatedly running the target model. Prepare it with:

```bash
export CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES:-0,1,2,3,4,5,6,7}
export MASTER_ADDR=${MASTER_ADDR:-127.0.0.1}
export MASTER_PORT=${MASTER_PORT:-29500}
export RANK=${RANK:-0}
export WORLD_SIZE=${WORLD_SIZE:-1}

python scripts/data/prepare_target_cache.py \
    --config config/dspark/dspark_qwen3_4b.py \
    --train-data-path train_datasets/qwen3_4b/perfectblend_train_regen.jsonl \
    --output-dir ${HOME}/.cache/deepspec/qwen3_4b_target_cache \
    --local-batch-size 16
```

> **Storage warning:** The target cache stores per-token hidden states for the
> full training set and can be very large. With the default `Qwen/Qwen3-4B`
> setting it takes roughly **38 TB** of disk. Make sure the `--output-dir`
> filesystem has enough free space (scaling with dataset size, sequence length,
> and target hidden dimension) before running this step. If storage is limited,
> use a smaller training set and/or reduce `model.target_layer_ids` in the config
> (fewer captured layers means proportionally less cache).

This produces the cache consumed by [scripts/train/train.sh](../train/train.sh):

```text
~/.cache/deepspec/qwen3_4b_target_cache
```

## Wrapper Script

The wrapper script combines the default public commands:

```bash
bash scripts/data/prepare_data.sh
```

Use the manual commands above if you want to stop and restart services between stages, change sampling parameters, use fewer GPUs, or inspect intermediate outputs.


Question: arxiv 2606.19348 DSpark

Question: arxiv 2606.19348 DSpark

DSpark Adaptive speculative decoding arxiv 2606.19348


`pdftotext` is not installed in this container, so I’m switching to a lightweight Python PDF extractor for the DSpark paper. I’m keeping the deployment app itself dependency-light; this is just for research.


/bin/zsh -lc "python3 - <<'PY'
try:
    import pypdf
    print('pypdf available')
except Exception as e:
    print('missing', type(e).__name__, e)
PY"


/bin/zsh -lc "python3 -m pip install --quiet pypdf && python3 - <<'PY'
from pypdf import PdfReader
reader = PdfReader('/tmp/DSpark_paper.pdf')
for i, page in enumerate(reader.pages[:10]):
    text = page.extract_text() or ''
    print(f'\\n--- PAGE {i+1} ---\\n')
    print(text[:4500])
PY"


error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.


/bin/zsh -lc "python3 -m venv /tmp/deepspec-pdf-venv && /tmp/deepspec-pdf-venv/bin/pip install --quiet pypdf && /tmp/deepspec-pdf-venv/bin/python - <<'PY'
from pypdf import PdfReader
reader = PdfReader('/tmp/DSpark_paper.pdf')
print('pages', len(reader.pages))
for i, page in enumerate(reader.pages):
    text = page.extract_text() or ''
    print(f'\\n--- PAGE {i+1} ---\\n')
    print(text[:5000])
PY"