kalamishere Claude Fable 5 commited on
Commit
c8b024e
·
1 Parent(s): 109ee6a

feat(api): headless audio_to_prompt endpoint — audio in, SA3 prompt out

Browse files

- new api_name="audio_to_prompt" route (hidden components + button, the
proven gradio_client pattern): file in -> deterministic SA3 prompt +
measured JSON out. No wallet, no LLM, no pollen — structural prompt
from outputs.sa3_variation_prompt
- fast=True skips demucs + basic-pitch (~2s vs ~30s); reuses analyze()'s
run_stems/run_midi flags
- same ingest as the UI upload: video containers + aac-family are
ffmpeg-extracted via _prepare_source_audio
- gated by Space secret AUDIO_BRIEF_API_TOKEN (constant-time compare);
open when unset
- returns {prompt, match_style_prompt, measured{...}, fast_mode,
stages_ok, errors}
- docs/API.md with gradio_client usage + gating setup

Verified locally end-to-end via gradio_client.handle_file: fast + full
(4-stem demucs) paths, token accept/reject, video extract, missing-file
error. Endpoint appears in view_api.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Files changed (2) hide show
  1. app.py +126 -0
  2. docs/API.md +76 -0
app.py CHANGED
@@ -1966,6 +1966,101 @@ def send_tile_to_analysis(tile_id: str | None, session_id: str | None = ""):
1966
  )
1967
 
1968
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1969
  def build_ui() -> gr.Blocks:
1970
  # Theme + CSS must live on the Blocks instance (not on .launch()) so
1971
  # HF Spaces — which auto-launches `demo` without our launch args —
@@ -4798,6 +4893,37 @@ def build_ui() -> gr.Blocks:
4798
  js=NOTEPAD_BOOTSTRAP_JS,
4799
  )
4800
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4801
  return demo
4802
 
4803
 
 
1966
  )
1967
 
1968
 
1969
+ # ── Headless API: audio → derived SA3 prompt ─────────────────────────────
1970
+ # Exposed via a hidden button with api_name="audio_to_prompt" (registered
1971
+ # in build_ui). Deliberately wallet-free: the returned prompt is the
1972
+ # DETERMINISTIC structural prompt (outputs.sa3_variation_prompt) built
1973
+ # from measurements alone — no Pollinations LLM call, no pollen cost, so
1974
+ # the endpoint works for any caller. LLM-polished prose stays a UI-only
1975
+ # feature (needs the per-session wallet).
1976
+ #
1977
+ # Gate: if the Space secret AUDIO_BRIEF_API_TOKEN is set, callers must
1978
+ # pass a matching `token`. Unset → open (fine for local/dev). This keeps
1979
+ # casual abuse of the CPU-heavy demucs stage out on the public Space.
1980
+ def _api_token_ok(token: str) -> bool:
1981
+ import os
1982
+ required = os.environ.get("AUDIO_BRIEF_API_TOKEN", "").strip()
1983
+ if not required:
1984
+ return True # no gate configured — open endpoint
1985
+ # Constant-time compare so the token can't be guessed byte-by-byte.
1986
+ import hmac
1987
+ return hmac.compare_digest(str(token or "").strip(), required)
1988
+
1989
+
1990
+ def audio_to_prompt_api(audio_path: str | None,
1991
+ token: str = "",
1992
+ fast: bool = False,
1993
+ bpm_prior: str = "default") -> dict:
1994
+ """Headless endpoint: an audio (or video) file in → a ready-to-use SA3
1995
+ prompt + the measured analysis JSON out. No wallet, no LLM.
1996
+
1997
+ Params (positional order for gradio_client.predict):
1998
+ audio_path : uploaded file path (wav/mp3/flac/ogg + mp4/mov/m4a/… —
1999
+ video/aac are ffmpeg-extracted, same as the UI upload).
2000
+ token : shared secret; required only when AUDIO_BRIEF_API_TOKEN
2001
+ is set on the Space.
2002
+ fast : skip demucs stems + basic-pitch bass-MIDI (~2 s instead
2003
+ of ~30 s). Prompt omits stem/bassline detail.
2004
+ bpm_prior : genre slug or numeric string to seed the beat tracker
2005
+ (see BPM_PRIORS) — "default" is fine for most.
2006
+
2007
+ Returns a JSON-able dict:
2008
+ { prompt, match_style_prompt, measured{…}, fast_mode, stages_ok, errors }
2009
+ """
2010
+ if not _api_token_ok(token):
2011
+ raise gr.Error("invalid or missing API token")
2012
+ if not audio_path:
2013
+ raise gr.Error("no audio file provided")
2014
+
2015
+ # Same ingest path as the UI: video containers + aac-family → WAV.
2016
+ src = _prepare_source_audio(audio_path)
2017
+
2018
+ a = analyze(
2019
+ src,
2020
+ bpm_prior=(bpm_prior or "default"),
2021
+ run_stems=not fast,
2022
+ run_midi=not fast,
2023
+ run_tags=False,
2024
+ run_embedding=False,
2025
+ )
2026
+
2027
+ # Decode is the only stage whose failure makes the output meaningless
2028
+ # (no BPM/key/anything). Surface that as an error rather than a prompt
2029
+ # full of '?'. Individual heavy-stage failures (demucs/basic-pitch)
2030
+ # are non-fatal — the structural prompt still stands on librosa data.
2031
+ if a.bpm is None and a.errors:
2032
+ raise gr.Error(f"analysis failed: {a.errors}")
2033
+
2034
+ return {
2035
+ "prompt": outputs.sa3_variation_prompt(a),
2036
+ "match_style_prompt": outputs.sa3_match_style_prompt(a),
2037
+ "measured": {
2038
+ "bpm": a.bpm,
2039
+ "key": a.key,
2040
+ "key_mode": a.key_mode,
2041
+ "key_confidence": a.key_correlation,
2042
+ "duration_s": a.duration_s,
2043
+ "lufs_i": a.lufs_i,
2044
+ "lufs_lra": a.lufs_lra,
2045
+ "true_peak_db": a.true_peak_db,
2046
+ "sections": a.sections,
2047
+ "stems": sorted(a.stems) if a.stems else [],
2048
+ "stem_stats": a.stem_stats or None,
2049
+ "voiceover_present": a.voiceover_present,
2050
+ "bass_midi": _bass_midi_summary(a),
2051
+ },
2052
+ "fast_mode": bool(fast),
2053
+ "stages_ok": [k for k, v in {
2054
+ "bpm_key": a.bpm is not None,
2055
+ "sections": bool(a.sections),
2056
+ "loudness": a.lufs_i is not None,
2057
+ "stems": bool(a.stems),
2058
+ "bass_midi": _bass_midi_summary(a) is not None,
2059
+ }.items() if v],
2060
+ "errors": a.errors or [],
2061
+ }
2062
+
2063
+
2064
  def build_ui() -> gr.Blocks:
2065
  # Theme + CSS must live on the Blocks instance (not on .launch()) so
2066
  # HF Spaces — which auto-launches `demo` without our launch args —
 
4893
  js=NOTEPAD_BOOTSTRAP_JS,
4894
  )
4895
 
4896
+ # ── Headless API surface ─────────────────────────────────────────
4897
+ # Hidden components + a hidden button carry the api_name route so
4898
+ # gradio_client (and plain HTTP POST to /gradio_api/call/…) can
4899
+ # reach audio_to_prompt_api. Visibility is UI-only — the endpoint
4900
+ # is registered server-side regardless. This is the same pattern
4901
+ # the app's other api_name routes use, and it plays nicely with
4902
+ # gradio_client.handle_file for the upload.
4903
+ with gr.Group(visible=False):
4904
+ _api_audio_in = gr.File(
4905
+ type="filepath",
4906
+ file_types=[
4907
+ "audio",
4908
+ ".mp3", ".wav", ".flac", ".ogg", ".aiff", ".aif",
4909
+ ".m4a", ".aac", ".opus", ".wma",
4910
+ ".mp4", ".mov", ".m4v", ".webm", ".mkv", ".avi",
4911
+ ],
4912
+ label="api audio in",
4913
+ )
4914
+ _api_token_in = gr.Textbox(value="", label="api token")
4915
+ _api_fast_in = gr.Checkbox(value=False, label="fast (skip demucs)")
4916
+ _api_bpm_in = gr.Textbox(value="default", label="bpm prior")
4917
+ _api_json_out = gr.JSON(label="api result")
4918
+ _api_btn = gr.Button("audio_to_prompt")
4919
+ _api_btn.click(
4920
+ fn=audio_to_prompt_api,
4921
+ inputs=[_api_audio_in, _api_token_in, _api_fast_in, _api_bpm_in],
4922
+ outputs=[_api_json_out],
4923
+ api_name="audio_to_prompt",
4924
+ show_progress="hidden",
4925
+ )
4926
+
4927
  return demo
4928
 
4929
 
docs/API.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # audio-brief API — audio in → SA3 prompt out
2
+
3
+ The Space exposes a headless endpoint that runs the analysis pipeline and
4
+ returns a ready-to-use SA3 prompt plus the measured JSON. **No wallet, no
5
+ LLM** — the prompt is the deterministic structural prompt built from
6
+ measurements alone.
7
+
8
+ - **Endpoint:** `/audio_to_prompt`
9
+ - **Base URL:** `https://kalamishere-audio-brief.hf.space/`
10
+ - **Auth:** optional shared token (see *Gating* below).
11
+
12
+ ## Parameters (positional)
13
+
14
+ | # | Name | Type | Default | Notes |
15
+ |---|---|---|---|---|
16
+ | 1 | `audio` | file | — | wav/mp3/flac/ogg + mp4/mov/m4a/webm/… (video & aac-family are ffmpeg-extracted) |
17
+ | 2 | `token` | str | `""` | required only when the Space secret `AUDIO_BRIEF_API_TOKEN` is set |
18
+ | 3 | `fast` | bool | `false` | skip demucs stems + bass-MIDI → ~2 s instead of ~30 s; prompt omits stem/bassline detail |
19
+ | 4 | `bpm_prior` | str | `"default"` | genre slug or numeric string to seed the beat tracker |
20
+
21
+ ## Returns
22
+
23
+ ```json
24
+ {
25
+ "prompt": "# SA3 variation prompt\nBPM: 128 (anchor)\nKey: A minor\n…",
26
+ "match_style_prompt": "…",
27
+ "measured": {
28
+ "bpm": 128.0, "key": "A", "key_mode": "minor", "key_confidence": 0.82,
29
+ "duration_s": 30.0, "lufs_i": -9.1, "lufs_lra": 4.2, "true_peak_db": -0.3,
30
+ "sections": [...], "stems": ["bass","drums","other","vocals"],
31
+ "stem_stats": {...}, "voiceover_present": false, "bass_midi": {...}
32
+ },
33
+ "fast_mode": false,
34
+ "stages_ok": ["bpm_key","sections","loudness","stems","bass_midi"],
35
+ "errors": []
36
+ }
37
+ ```
38
+
39
+ ## Python (recommended)
40
+
41
+ ```python
42
+ from gradio_client import Client, handle_file
43
+
44
+ c = Client("kalamishere/audio-brief") # or the full .hf.space URL
45
+ r = c.predict(
46
+ handle_file("track.wav"),
47
+ "YOUR_TOKEN", # "" if the Space isn't gated
48
+ True, # fast=True
49
+ "default",
50
+ api_name="/audio_to_prompt",
51
+ )
52
+ print(r["prompt"])
53
+ ```
54
+
55
+ ## Gating (protect the CPU-heavy endpoint)
56
+
57
+ The public Space runs demucs on every full call, so leave it gated. Set a
58
+ Space secret in **Settings → Variables and secrets**:
59
+
60
+ ```
61
+ AUDIO_BRIEF_API_TOKEN = <a long random string>
62
+ ```
63
+
64
+ When set, calls must pass a matching `token` (constant-time compared) or
65
+ get `invalid or missing API token`. When unset, the endpoint is open —
66
+ fine for local runs.
67
+
68
+ ## Notes / limits
69
+
70
+ - **Speed:** full analysis is demucs-bound (~30 s on HF free CPU). Use
71
+ `fast=true` for interactive/high-volume use, or upgrade the Space tier.
72
+ - **Concurrency:** the queue caps concurrency at 4 (see `demo.queue`).
73
+ Heavy parallel traffic on the free tier will swap — see the low-RAM
74
+ ceiling note in the pipeline docs.
75
+ - **YouTube isn't an API input** — pass a file. (The UI's YouTube box
76
+ yt-dlp's to a file first; that flow is UI-only.)