Spaces:
Sleeping
feat(api): headless audio_to_prompt endpoint — audio in, SA3 prompt out
Browse files- new api_name="audio_to_prompt" route (hidden components + button, the
proven gradio_client pattern): file in -> deterministic SA3 prompt +
measured JSON out. No wallet, no LLM, no pollen — structural prompt
from outputs.sa3_variation_prompt
- fast=True skips demucs + basic-pitch (~2s vs ~30s); reuses analyze()'s
run_stems/run_midi flags
- same ingest as the UI upload: video containers + aac-family are
ffmpeg-extracted via _prepare_source_audio
- gated by Space secret AUDIO_BRIEF_API_TOKEN (constant-time compare);
open when unset
- returns {prompt, match_style_prompt, measured{...}, fast_mode,
stages_ok, errors}
- docs/API.md with gradio_client usage + gating setup
Verified locally end-to-end via gradio_client.handle_file: fast + full
(4-stem demucs) paths, token accept/reject, video extract, missing-file
error. Endpoint appears in view_api.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- app.py +126 -0
- docs/API.md +76 -0
|
@@ -1966,6 +1966,101 @@ def send_tile_to_analysis(tile_id: str | None, session_id: str | None = ""):
|
|
| 1966 |
)
|
| 1967 |
|
| 1968 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1969 |
def build_ui() -> gr.Blocks:
|
| 1970 |
# Theme + CSS must live on the Blocks instance (not on .launch()) so
|
| 1971 |
# HF Spaces — which auto-launches `demo` without our launch args —
|
|
@@ -4798,6 +4893,37 @@ def build_ui() -> gr.Blocks:
|
|
| 4798 |
js=NOTEPAD_BOOTSTRAP_JS,
|
| 4799 |
)
|
| 4800 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4801 |
return demo
|
| 4802 |
|
| 4803 |
|
|
|
|
| 1966 |
)
|
| 1967 |
|
| 1968 |
|
| 1969 |
+
# ── Headless API: audio → derived SA3 prompt ─────────────────────────────
|
| 1970 |
+
# Exposed via a hidden button with api_name="audio_to_prompt" (registered
|
| 1971 |
+
# in build_ui). Deliberately wallet-free: the returned prompt is the
|
| 1972 |
+
# DETERMINISTIC structural prompt (outputs.sa3_variation_prompt) built
|
| 1973 |
+
# from measurements alone — no Pollinations LLM call, no pollen cost, so
|
| 1974 |
+
# the endpoint works for any caller. LLM-polished prose stays a UI-only
|
| 1975 |
+
# feature (needs the per-session wallet).
|
| 1976 |
+
#
|
| 1977 |
+
# Gate: if the Space secret AUDIO_BRIEF_API_TOKEN is set, callers must
|
| 1978 |
+
# pass a matching `token`. Unset → open (fine for local/dev). This keeps
|
| 1979 |
+
# casual abuse of the CPU-heavy demucs stage out on the public Space.
|
| 1980 |
+
def _api_token_ok(token: str) -> bool:
|
| 1981 |
+
import os
|
| 1982 |
+
required = os.environ.get("AUDIO_BRIEF_API_TOKEN", "").strip()
|
| 1983 |
+
if not required:
|
| 1984 |
+
return True # no gate configured — open endpoint
|
| 1985 |
+
# Constant-time compare so the token can't be guessed byte-by-byte.
|
| 1986 |
+
import hmac
|
| 1987 |
+
return hmac.compare_digest(str(token or "").strip(), required)
|
| 1988 |
+
|
| 1989 |
+
|
| 1990 |
+
def audio_to_prompt_api(audio_path: str | None,
|
| 1991 |
+
token: str = "",
|
| 1992 |
+
fast: bool = False,
|
| 1993 |
+
bpm_prior: str = "default") -> dict:
|
| 1994 |
+
"""Headless endpoint: an audio (or video) file in → a ready-to-use SA3
|
| 1995 |
+
prompt + the measured analysis JSON out. No wallet, no LLM.
|
| 1996 |
+
|
| 1997 |
+
Params (positional order for gradio_client.predict):
|
| 1998 |
+
audio_path : uploaded file path (wav/mp3/flac/ogg + mp4/mov/m4a/… —
|
| 1999 |
+
video/aac are ffmpeg-extracted, same as the UI upload).
|
| 2000 |
+
token : shared secret; required only when AUDIO_BRIEF_API_TOKEN
|
| 2001 |
+
is set on the Space.
|
| 2002 |
+
fast : skip demucs stems + basic-pitch bass-MIDI (~2 s instead
|
| 2003 |
+
of ~30 s). Prompt omits stem/bassline detail.
|
| 2004 |
+
bpm_prior : genre slug or numeric string to seed the beat tracker
|
| 2005 |
+
(see BPM_PRIORS) — "default" is fine for most.
|
| 2006 |
+
|
| 2007 |
+
Returns a JSON-able dict:
|
| 2008 |
+
{ prompt, match_style_prompt, measured{…}, fast_mode, stages_ok, errors }
|
| 2009 |
+
"""
|
| 2010 |
+
if not _api_token_ok(token):
|
| 2011 |
+
raise gr.Error("invalid or missing API token")
|
| 2012 |
+
if not audio_path:
|
| 2013 |
+
raise gr.Error("no audio file provided")
|
| 2014 |
+
|
| 2015 |
+
# Same ingest path as the UI: video containers + aac-family → WAV.
|
| 2016 |
+
src = _prepare_source_audio(audio_path)
|
| 2017 |
+
|
| 2018 |
+
a = analyze(
|
| 2019 |
+
src,
|
| 2020 |
+
bpm_prior=(bpm_prior or "default"),
|
| 2021 |
+
run_stems=not fast,
|
| 2022 |
+
run_midi=not fast,
|
| 2023 |
+
run_tags=False,
|
| 2024 |
+
run_embedding=False,
|
| 2025 |
+
)
|
| 2026 |
+
|
| 2027 |
+
# Decode is the only stage whose failure makes the output meaningless
|
| 2028 |
+
# (no BPM/key/anything). Surface that as an error rather than a prompt
|
| 2029 |
+
# full of '?'. Individual heavy-stage failures (demucs/basic-pitch)
|
| 2030 |
+
# are non-fatal — the structural prompt still stands on librosa data.
|
| 2031 |
+
if a.bpm is None and a.errors:
|
| 2032 |
+
raise gr.Error(f"analysis failed: {a.errors}")
|
| 2033 |
+
|
| 2034 |
+
return {
|
| 2035 |
+
"prompt": outputs.sa3_variation_prompt(a),
|
| 2036 |
+
"match_style_prompt": outputs.sa3_match_style_prompt(a),
|
| 2037 |
+
"measured": {
|
| 2038 |
+
"bpm": a.bpm,
|
| 2039 |
+
"key": a.key,
|
| 2040 |
+
"key_mode": a.key_mode,
|
| 2041 |
+
"key_confidence": a.key_correlation,
|
| 2042 |
+
"duration_s": a.duration_s,
|
| 2043 |
+
"lufs_i": a.lufs_i,
|
| 2044 |
+
"lufs_lra": a.lufs_lra,
|
| 2045 |
+
"true_peak_db": a.true_peak_db,
|
| 2046 |
+
"sections": a.sections,
|
| 2047 |
+
"stems": sorted(a.stems) if a.stems else [],
|
| 2048 |
+
"stem_stats": a.stem_stats or None,
|
| 2049 |
+
"voiceover_present": a.voiceover_present,
|
| 2050 |
+
"bass_midi": _bass_midi_summary(a),
|
| 2051 |
+
},
|
| 2052 |
+
"fast_mode": bool(fast),
|
| 2053 |
+
"stages_ok": [k for k, v in {
|
| 2054 |
+
"bpm_key": a.bpm is not None,
|
| 2055 |
+
"sections": bool(a.sections),
|
| 2056 |
+
"loudness": a.lufs_i is not None,
|
| 2057 |
+
"stems": bool(a.stems),
|
| 2058 |
+
"bass_midi": _bass_midi_summary(a) is not None,
|
| 2059 |
+
}.items() if v],
|
| 2060 |
+
"errors": a.errors or [],
|
| 2061 |
+
}
|
| 2062 |
+
|
| 2063 |
+
|
| 2064 |
def build_ui() -> gr.Blocks:
|
| 2065 |
# Theme + CSS must live on the Blocks instance (not on .launch()) so
|
| 2066 |
# HF Spaces — which auto-launches `demo` without our launch args —
|
|
|
|
| 4893 |
js=NOTEPAD_BOOTSTRAP_JS,
|
| 4894 |
)
|
| 4895 |
|
| 4896 |
+
# ── Headless API surface ─────────────────────────────────────────
|
| 4897 |
+
# Hidden components + a hidden button carry the api_name route so
|
| 4898 |
+
# gradio_client (and plain HTTP POST to /gradio_api/call/…) can
|
| 4899 |
+
# reach audio_to_prompt_api. Visibility is UI-only — the endpoint
|
| 4900 |
+
# is registered server-side regardless. This is the same pattern
|
| 4901 |
+
# the app's other api_name routes use, and it plays nicely with
|
| 4902 |
+
# gradio_client.handle_file for the upload.
|
| 4903 |
+
with gr.Group(visible=False):
|
| 4904 |
+
_api_audio_in = gr.File(
|
| 4905 |
+
type="filepath",
|
| 4906 |
+
file_types=[
|
| 4907 |
+
"audio",
|
| 4908 |
+
".mp3", ".wav", ".flac", ".ogg", ".aiff", ".aif",
|
| 4909 |
+
".m4a", ".aac", ".opus", ".wma",
|
| 4910 |
+
".mp4", ".mov", ".m4v", ".webm", ".mkv", ".avi",
|
| 4911 |
+
],
|
| 4912 |
+
label="api audio in",
|
| 4913 |
+
)
|
| 4914 |
+
_api_token_in = gr.Textbox(value="", label="api token")
|
| 4915 |
+
_api_fast_in = gr.Checkbox(value=False, label="fast (skip demucs)")
|
| 4916 |
+
_api_bpm_in = gr.Textbox(value="default", label="bpm prior")
|
| 4917 |
+
_api_json_out = gr.JSON(label="api result")
|
| 4918 |
+
_api_btn = gr.Button("audio_to_prompt")
|
| 4919 |
+
_api_btn.click(
|
| 4920 |
+
fn=audio_to_prompt_api,
|
| 4921 |
+
inputs=[_api_audio_in, _api_token_in, _api_fast_in, _api_bpm_in],
|
| 4922 |
+
outputs=[_api_json_out],
|
| 4923 |
+
api_name="audio_to_prompt",
|
| 4924 |
+
show_progress="hidden",
|
| 4925 |
+
)
|
| 4926 |
+
|
| 4927 |
return demo
|
| 4928 |
|
| 4929 |
|
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# audio-brief API — audio in → SA3 prompt out
|
| 2 |
+
|
| 3 |
+
The Space exposes a headless endpoint that runs the analysis pipeline and
|
| 4 |
+
returns a ready-to-use SA3 prompt plus the measured JSON. **No wallet, no
|
| 5 |
+
LLM** — the prompt is the deterministic structural prompt built from
|
| 6 |
+
measurements alone.
|
| 7 |
+
|
| 8 |
+
- **Endpoint:** `/audio_to_prompt`
|
| 9 |
+
- **Base URL:** `https://kalamishere-audio-brief.hf.space/`
|
| 10 |
+
- **Auth:** optional shared token (see *Gating* below).
|
| 11 |
+
|
| 12 |
+
## Parameters (positional)
|
| 13 |
+
|
| 14 |
+
| # | Name | Type | Default | Notes |
|
| 15 |
+
|---|---|---|---|---|
|
| 16 |
+
| 1 | `audio` | file | — | wav/mp3/flac/ogg + mp4/mov/m4a/webm/… (video & aac-family are ffmpeg-extracted) |
|
| 17 |
+
| 2 | `token` | str | `""` | required only when the Space secret `AUDIO_BRIEF_API_TOKEN` is set |
|
| 18 |
+
| 3 | `fast` | bool | `false` | skip demucs stems + bass-MIDI → ~2 s instead of ~30 s; prompt omits stem/bassline detail |
|
| 19 |
+
| 4 | `bpm_prior` | str | `"default"` | genre slug or numeric string to seed the beat tracker |
|
| 20 |
+
|
| 21 |
+
## Returns
|
| 22 |
+
|
| 23 |
+
```json
|
| 24 |
+
{
|
| 25 |
+
"prompt": "# SA3 variation prompt\nBPM: 128 (anchor)\nKey: A minor\n…",
|
| 26 |
+
"match_style_prompt": "…",
|
| 27 |
+
"measured": {
|
| 28 |
+
"bpm": 128.0, "key": "A", "key_mode": "minor", "key_confidence": 0.82,
|
| 29 |
+
"duration_s": 30.0, "lufs_i": -9.1, "lufs_lra": 4.2, "true_peak_db": -0.3,
|
| 30 |
+
"sections": [...], "stems": ["bass","drums","other","vocals"],
|
| 31 |
+
"stem_stats": {...}, "voiceover_present": false, "bass_midi": {...}
|
| 32 |
+
},
|
| 33 |
+
"fast_mode": false,
|
| 34 |
+
"stages_ok": ["bpm_key","sections","loudness","stems","bass_midi"],
|
| 35 |
+
"errors": []
|
| 36 |
+
}
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
## Python (recommended)
|
| 40 |
+
|
| 41 |
+
```python
|
| 42 |
+
from gradio_client import Client, handle_file
|
| 43 |
+
|
| 44 |
+
c = Client("kalamishere/audio-brief") # or the full .hf.space URL
|
| 45 |
+
r = c.predict(
|
| 46 |
+
handle_file("track.wav"),
|
| 47 |
+
"YOUR_TOKEN", # "" if the Space isn't gated
|
| 48 |
+
True, # fast=True
|
| 49 |
+
"default",
|
| 50 |
+
api_name="/audio_to_prompt",
|
| 51 |
+
)
|
| 52 |
+
print(r["prompt"])
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
## Gating (protect the CPU-heavy endpoint)
|
| 56 |
+
|
| 57 |
+
The public Space runs demucs on every full call, so leave it gated. Set a
|
| 58 |
+
Space secret in **Settings → Variables and secrets**:
|
| 59 |
+
|
| 60 |
+
```
|
| 61 |
+
AUDIO_BRIEF_API_TOKEN = <a long random string>
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
When set, calls must pass a matching `token` (constant-time compared) or
|
| 65 |
+
get `invalid or missing API token`. When unset, the endpoint is open —
|
| 66 |
+
fine for local runs.
|
| 67 |
+
|
| 68 |
+
## Notes / limits
|
| 69 |
+
|
| 70 |
+
- **Speed:** full analysis is demucs-bound (~30 s on HF free CPU). Use
|
| 71 |
+
`fast=true` for interactive/high-volume use, or upgrade the Space tier.
|
| 72 |
+
- **Concurrency:** the queue caps concurrency at 4 (see `demo.queue`).
|
| 73 |
+
Heavy parallel traffic on the free tier will swap — see the low-RAM
|
| 74 |
+
ceiling note in the pipeline docs.
|
| 75 |
+
- **YouTube isn't an API input** — pass a file. (The UI's YouTube box
|
| 76 |
+
yt-dlp's to a file first; that flow is UI-only.)
|