Spaces:
Configuration error
Configuration error
Add self-describing evidence exports
Browse files
README.md
CHANGED
|
@@ -104,7 +104,7 @@ Candidate comparisons to run:
|
|
| 104 |
|
| 105 |
For model comparisons, choose the candidate model before **Load models**, run **Run benchmark suite**, add at least three real microphone rows with **Run 3 real-mic series**, then click **Unload models** and choose the next candidate. The benchmark table is preserved across unloads, so **Copy JSON** can export the summary and all rows after the comparison set. For TTS voice/step comparisons, change the voice or step slider after loading and rerun **Run TTS benchmark** plus the text/loopback benchmarks; those values are captured per row.
|
| 106 |
|
| 107 |
-
Use **Download JSON** after benchmark or real-mic runs on the hosted Space to save a timestamped evidence file with the same payload as **Copy JSON** and `window.browserSpeakBench.exportResults()`. The export includes `generatedAt`, `hostMetadata` with the hosted Space commit when the static host exposes it, the current `runtime`, an `evidence` summary for real-mic/hardware-WebGPU/hosted-build readiness, aggregate `summary`, and raw `results`. The **Run evidence capture** button starts a timestamped `browser-speak-evidence-*.json` download when the capture sequence finishes; if the browser blocks the automatic download, **Download JSON** saves the same payload manually. Benchmark rows are also autosaved in browser storage for the current host and Space commit, so a reload during manual evidence capture can restore rows from the same deployed build; **Clear** removes both the visible rows and the saved copy.
|
| 108 |
|
| 109 |
Use `node tools/audit-browser-evidence.mjs path/to/browser-speak-benchmarks.json` to audit a JSON file downloaded from the hosted browser UI. This checks that the export came from the static Space revision, reports the page-generated `evidence` summary, verifies that at least three real-mic rows meet the WER/identity/latency gates from raw rows, and verifies that at least one completed row reports hardware WebGPU rather than a software adapter. Browser-downloaded JSON cannot prove no-server network behavior or Chrome launch flags by itself, so pair it with `node tools/run-hosted-smoke.mjs` for network evidence or `node tools/run-real-mic-series.mjs` for harness-collected real-mic provenance.
|
| 110 |
|
|
|
|
| 104 |
|
| 105 |
For model comparisons, choose the candidate model before **Load models**, run **Run benchmark suite**, add at least three real microphone rows with **Run 3 real-mic series**, then click **Unload models** and choose the next candidate. The benchmark table is preserved across unloads, so **Copy JSON** can export the summary and all rows after the comparison set. For TTS voice/step comparisons, change the voice or step slider after loading and rerun **Run TTS benchmark** plus the text/loopback benchmarks; those values are captured per row.
|
| 106 |
|
| 107 |
+
Use **Download JSON** after benchmark or real-mic runs on the hosted Space to save a timestamped evidence file with the same payload as **Copy JSON** and `window.browserSpeakBench.exportResults()`. The export includes `schemaVersion`, a unique `exportId`, `generatedAt`, `hostMetadata` with the hosted Space commit when the static host exposes it, the current `runtime`, an `evidence` summary for real-mic/hardware-WebGPU/hosted-build readiness, an embedded `evidenceGuide` with the audit command and row requirements, aggregate `summary`, and raw `results`. The **Run evidence capture** button starts a timestamped `browser-speak-evidence-*.json` download when the capture sequence finishes; if the browser blocks the automatic download, **Download JSON** saves the same payload manually. Benchmark rows are also autosaved in browser storage for the current host and Space commit, so a reload during manual evidence capture can restore rows from the same deployed build; **Clear** removes both the visible rows and the saved copy.
|
| 108 |
|
| 109 |
Use `node tools/audit-browser-evidence.mjs path/to/browser-speak-benchmarks.json` to audit a JSON file downloaded from the hosted browser UI. This checks that the export came from the static Space revision, reports the page-generated `evidence` summary, verifies that at least three real-mic rows meet the WER/identity/latency gates from raw rows, and verifies that at least one completed row reports hardware WebGPU rather than a software adapter. Browser-downloaded JSON cannot prove no-server network behavior or Chrome launch flags by itself, so pair it with `node tools/run-hosted-smoke.mjs` for network evidence or `node tools/run-real-mic-series.mjs` for harness-collected real-mic provenance.
|
| 110 |
|
app.js
CHANGED
|
@@ -133,6 +133,7 @@ const DEFAULT_LOOPBACK_PREROLL_MS = 180;
|
|
| 133 |
const DEFAULT_LOOPBACK_PROMPT = "Identify this browser demo.";
|
| 134 |
const REAL_MIC_TARGET_RUNS = 3;
|
| 135 |
const REAL_MIC_MAX_WER = 0.25;
|
|
|
|
| 136 |
const BENCHMARK_STORAGE_KEY = "browser-speak:benchmark-results:v1";
|
| 137 |
const BENCHMARK_STORAGE_VERSION = 1;
|
| 138 |
const BENCHMARK_STORAGE_MAX_ROWS = 80;
|
|
@@ -1869,16 +1870,86 @@ function exportBenchmarkSummary() {
|
|
| 1869 |
}
|
| 1870 |
|
| 1871 |
function benchmarkExportPayload() {
|
|
|
|
|
|
|
| 1872 |
return {
|
| 1873 |
-
|
|
|
|
|
|
|
| 1874 |
hostMetadata: { ...state.hostMetadata },
|
| 1875 |
runtime: runtimeEnvironment(),
|
| 1876 |
-
evidence
|
|
|
|
| 1877 |
summary: exportBenchmarkSummary(),
|
| 1878 |
results: [...state.benchmarkResults],
|
| 1879 |
};
|
| 1880 |
}
|
| 1881 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1882 |
function downloadBenchmarkJson({
|
| 1883 |
prefix = "browser-speak-benchmarks",
|
| 1884 |
message = "Benchmark JSON download started.",
|
|
|
|
| 133 |
const DEFAULT_LOOPBACK_PROMPT = "Identify this browser demo.";
|
| 134 |
const REAL_MIC_TARGET_RUNS = 3;
|
| 135 |
const REAL_MIC_MAX_WER = 0.25;
|
| 136 |
+
const BENCHMARK_EXPORT_SCHEMA_VERSION = "browser-speak-benchmarks/v2";
|
| 137 |
const BENCHMARK_STORAGE_KEY = "browser-speak:benchmark-results:v1";
|
| 138 |
const BENCHMARK_STORAGE_VERSION = 1;
|
| 139 |
const BENCHMARK_STORAGE_MAX_ROWS = 80;
|
|
|
|
| 1870 |
}
|
| 1871 |
|
| 1872 |
function benchmarkExportPayload() {
|
| 1873 |
+
const generatedAt = new Date().toISOString();
|
| 1874 |
+
const evidence = evidenceSummaryForRows(state.benchmarkResults);
|
| 1875 |
return {
|
| 1876 |
+
schemaVersion: BENCHMARK_EXPORT_SCHEMA_VERSION,
|
| 1877 |
+
exportId: createExportId(),
|
| 1878 |
+
generatedAt,
|
| 1879 |
hostMetadata: { ...state.hostMetadata },
|
| 1880 |
runtime: runtimeEnvironment(),
|
| 1881 |
+
evidence,
|
| 1882 |
+
evidenceGuide: browserEvidenceGuide(evidence),
|
| 1883 |
summary: exportBenchmarkSummary(),
|
| 1884 |
results: [...state.benchmarkResults],
|
| 1885 |
};
|
| 1886 |
}
|
| 1887 |
|
| 1888 |
+
function createExportId() {
|
| 1889 |
+
const random = globalThis.crypto?.randomUUID?.();
|
| 1890 |
+
return random ? `browser-speak-${random}` : `browser-speak-${Date.now().toString(36)}`;
|
| 1891 |
+
}
|
| 1892 |
+
|
| 1893 |
+
function browserEvidenceGuide(evidence) {
|
| 1894 |
+
return {
|
| 1895 |
+
auditCommand: "node tools/audit-browser-evidence.mjs <downloaded-json>",
|
| 1896 |
+
pairedNetworkCheck: "node tools/run-hosted-smoke.mjs",
|
| 1897 |
+
requirements: {
|
| 1898 |
+
hostedSpaceMetadata: {
|
| 1899 |
+
hostSuffix: ".hf.space",
|
| 1900 |
+
commitHeader: "x-repo-commit",
|
| 1901 |
+
},
|
| 1902 |
+
realMic: {
|
| 1903 |
+
prompt: APP_IDENTITY_PROMPT,
|
| 1904 |
+
requiredRows: REAL_MIC_TARGET_RUNS,
|
| 1905 |
+
maxMedianWer: REAL_MIC_MAX_WER,
|
| 1906 |
+
requiredSignals: [
|
| 1907 |
+
"completed mic rows",
|
| 1908 |
+
"mic input stats",
|
| 1909 |
+
"identity answer pass",
|
| 1910 |
+
"finite speech-end-to-first-audio latency",
|
| 1911 |
+
],
|
| 1912 |
+
},
|
| 1913 |
+
hardwareWebgpu: {
|
| 1914 |
+
requiredRows: 1,
|
| 1915 |
+
requiredSignals: [
|
| 1916 |
+
"row stack device is webgpu",
|
| 1917 |
+
"browser reports WebGPU available",
|
| 1918 |
+
"adapter is not classified as software",
|
| 1919 |
+
],
|
| 1920 |
+
},
|
| 1921 |
+
},
|
| 1922 |
+
current: {
|
| 1923 |
+
realMicPassed: evidence.realMic.passed,
|
| 1924 |
+
realMicRows: evidence.realMic.rows,
|
| 1925 |
+
hardwareWebgpuPassed: evidence.hardwareWebgpu.passed,
|
| 1926 |
+
hardwareWebgpuRows: evidence.hardwareWebgpu.hardwareRows,
|
| 1927 |
+
hostedPassed: evidence.hosted.passed,
|
| 1928 |
+
},
|
| 1929 |
+
nextActions: browserEvidenceNextActions(evidence),
|
| 1930 |
+
limitations: [
|
| 1931 |
+
"Downloaded browser JSON proves page-reported rows and adapter metadata.",
|
| 1932 |
+
"Pair with hosted smoke for no-server network evidence.",
|
| 1933 |
+
"Use the real-mic harness for Chrome launch provenance when fake-capture flags must be ruled out.",
|
| 1934 |
+
],
|
| 1935 |
+
};
|
| 1936 |
+
}
|
| 1937 |
+
|
| 1938 |
+
function browserEvidenceNextActions(evidence) {
|
| 1939 |
+
const actions = [];
|
| 1940 |
+
if (!evidence.hosted.passed) {
|
| 1941 |
+
actions.push("Run and download evidence from the hosted static HF Space.");
|
| 1942 |
+
}
|
| 1943 |
+
if (!evidence.realMic.passed) {
|
| 1944 |
+
actions.push(`Run evidence capture or Run 3 real-mic series, saying "${APP_IDENTITY_PROMPT}" each time.`);
|
| 1945 |
+
}
|
| 1946 |
+
if (!evidence.hardwareWebgpu.passed) {
|
| 1947 |
+
actions.push("Run evidence capture in a browser exposing a hardware WebGPU adapter.");
|
| 1948 |
+
}
|
| 1949 |
+
if (actions.length === 0) actions.push("Audit this JSON with tools/audit-browser-evidence.mjs.");
|
| 1950 |
+
return actions;
|
| 1951 |
+
}
|
| 1952 |
+
|
| 1953 |
function downloadBenchmarkJson({
|
| 1954 |
prefix = "browser-speak-benchmarks",
|
| 1955 |
message = "Benchmark JSON download started.",
|