Claude commited on
Add rows layout toggle and C Protocol candidate data model
Browse filesCircular/Rows layout toggle in the pipeline workspace (layered
top-down DAG as an alternative to the radial view). Added the C
Protocol (Code, Claim, Consensus, Cost, Connectivity, each 0-1, Code
gates the tier below 0.3) as the candidate-repo scoring model,
seeded with real GitHub data for the Avatar node's six candidates.
First worked-example report (Avatar) written to docs/reports/.
- docs/reports/avatar.md +121 -0
- src/app/pipeline/Workspace.tsx +9 -3
- src/components/PipelineGraph.tsx +6 -3
- src/lib/candidates.ts +248 -0
- src/lib/graph-layout.ts +68 -4
docs/reports/avatar.md
ADDED
|
@@ -0,0 +1,121 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Node Report: Avatar (Talking-Head Generation)
|
| 2 |
+
|
| 3 |
+
**Pipeline:** Beryl — Live Conversational Avatar
|
| 4 |
+
**Category:** avatar (face/body synthesis — lip sync, expression, head motion)
|
| 5 |
+
**Report generated:** 2026-07-13
|
| 6 |
+
**Auditor:** Samantha (LAYER pipeline-audit rubric, 5 failure modes)
|
| 7 |
+
**Status:** Tier B — see §5
|
| 8 |
+
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
## 1. Abstract
|
| 12 |
+
|
| 13 |
+
The Avatar node converts a driving audio signal and a static reference image into
|
| 14 |
+
a photorealistic video stream of a talking human. It is the node where two of the
|
| 15 |
+
rubric's five failure modes — the Uncanny Valley and the End-of-Speech Pop —
|
| 16 |
+
originate, and consequently the node that decides whether the finished pipeline
|
| 17 |
+
reads as "alive" or "off." This report surveys six candidate implementations
|
| 18 |
+
under the **C Protocol** — Code, Claim, Consensus, Cost, Connectivity, each
|
| 19 |
+
scored 0–1 and rendered live as stars as each is verified — and separately
|
| 20 |
+
audits the currently-selected implementation, LivePortrait via
|
| 21 |
+
FasterLivePortrait, against the five documented failure modes.
|
| 22 |
+
|
| 23 |
+
## 2. Background
|
| 24 |
+
|
| 25 |
+
Talking-head generation (THG) research since 2023 has split into two lineages:
|
| 26 |
+
|
| 27 |
+
- **Warping/GAN-based** (LivePortrait and its derivatives): a reference-image
|
| 28 |
+
keypoint-warping approach. Cheap, real-time capable, moderate photorealism
|
| 29 |
+
ceiling.
|
| 30 |
+
- **Diffusion-based** (EchoMimic, Hallo, JoyVASA): higher photorealism ceiling,
|
| 31 |
+
substantially higher per-frame compute cost, generally not real-time without
|
| 32 |
+
further distillation work.
|
| 33 |
+
|
| 34 |
+
Beryl's live-conversation requirement (sub-frame-budget latency, streaming
|
| 35 |
+
output) makes the warping/GAN lineage the default starting point, with
|
| 36 |
+
diffusion-based approaches held as a quality upgrade path once (and only once)
|
| 37 |
+
their latency is independently measured against the real-time budget.
|
| 38 |
+
|
| 39 |
+
## 3. Mechanism
|
| 40 |
+
|
| 41 |
+
A THG stage generally decomposes into three sub-stages, mirrored in this
|
| 42 |
+
pipeline's own node children:
|
| 43 |
+
|
| 44 |
+
1. **Motion extraction** — derive keypoint or motion-coefficient deltas from the
|
| 45 |
+
driving audio (via phoneme/viseme timing) and, for identity-lock, from
|
| 46 |
+
periodic Vision re-checks of the reference frame.
|
| 47 |
+
2. **Warping** — apply the extracted motion to the reference image's feature
|
| 48 |
+
representation. This is the most compute-expensive sub-stage and the first
|
| 49 |
+
place to look when frame rate drops.
|
| 50 |
+
3. **Generation** — render the final RGB frame from the warped feature
|
| 51 |
+
representation. Smaller generators trade fidelity for speed.
|
| 52 |
+
|
| 53 |
+
## 4. Candidate Implementations — the C Protocol
|
| 54 |
+
|
| 55 |
+
Five axes, each scored 0–1: **Code** (does it run, is it official — a gate, not
|
| 56 |
+
just an average input: below 0.3 caps the tier at C regardless of the other
|
| 57 |
+
four), **Claim** (research credibility), **Consensus** (community stars/traction),
|
| 58 |
+
**Cost** (compute/financial efficiency), **Connectivity** (fit into a real-time
|
| 59 |
+
pipeline).
|
| 60 |
+
|
| 61 |
+
| Candidate | Stars | Paper | Code | Claim | Consensus | Cost | Connectivity | Tier |
|
| 62 |
+
|---|---:|---|---:|---:|---:|---:|---:|:-:|
|
| 63 |
+
| **LiveTalking** (lipku) | 8,378 | — (engineering) | 0.95 | 0.55 | 0.95 | 0.85 | 0.95 | A |
|
| 64 |
+
| **FasterLivePortrait** (warmshao) | 1,153 | — (engineering) | 0.90 | 0.60 | 0.70 | 0.90 | 0.85 | A |
|
| 65 |
+
| **EchoMimicV2** (antgroup) | 4,614 | CVPR 2025 | 0.85 | 0.90 | 0.85 | 0.55 | 0.65 | A |
|
| 66 |
+
| **Hallo2** (fudan-generative-vision) | 3,718 | ICLR 2025 | 0.85 | 0.95 | 0.90 | 0.35 | 0.55 | B |
|
| 67 |
+
| **JoyVASA** (jdh-algo) | 874 | — | 0.75 | 0.65 | 0.70 | 0.45 | 0.50 | B |
|
| 68 |
+
| **VASA-1** (no official release) | 105† | Microsoft Research | 0.05 | 0.80 | 0.20 | 0.00 | 0.00 | C |
|
| 69 |
+
|
| 70 |
+
† Largest of three competing *unofficial* reproduction attempts; none endorsed
|
| 71 |
+
by the original authors. See §5, Failure 5.
|
| 72 |
+
|
| 73 |
+
**Reading this table**: look at VASA-1's row first — 0.80 on Claim, 0.05 on Code.
|
| 74 |
+
That mismatch *is* the finding: a paper can look excellent and still be a Tier C
|
| 75 |
+
pipeline dependency because nothing exists to run. Hallo2 shows the softer
|
| 76 |
+
version of the same lesson — the strongest Claim in the set (ICLR 2025) paired
|
| 77 |
+
with the weakest Cost, because it was built for offline quality, not live
|
| 78 |
+
latency. A high Claim score is a reason to *read* the paper, not a reason to
|
| 79 |
+
skip measuring the code.
|
| 80 |
+
|
| 81 |
+
## 5. Failure-Mode Audit (Current Selection: LivePortrait / FasterLivePortrait)
|
| 82 |
+
|
| 83 |
+
| # | Failure | Verdict | Finding |
|
| 84 |
+
|---|---|:-:|---|
|
| 85 |
+
| 1 | Uncanny Valley | **AT-RISK** | Architecture-inherent to audio-driven THG generally, not disproved by choice of model. Requires a real measurement — frame-to-frame head-pose variance and upper-face action-unit activation range — not an assumption of solved-ness. |
|
| 86 |
+
| 2 | End-of-Speech Pop | **AT-RISK** | Known integration failure class for exactly this pattern: compositing a generated mouth region back onto a static source image. Must be verified experimentally at the silence transition. |
|
| 87 |
+
| 3 | Cascaded Latency | **PASS** (node-local) | ~33ms/frame (~30fps) in isolation is within budget. This node is the *last hop* of the pipeline-level cascade, not an offender on its own — see the pipeline-level audit for the FAIL verdict on the architecture as a whole. |
|
| 88 |
+
| 4 | VRAM Death | **PASS** | 8GB fp16, comfortably fits alongside the rest of the stack on a single 24GB GPU. |
|
| 89 |
+
| 5 | VASA-1 Problem | **PASS** | LivePortrait and FasterLivePortrait are both public, open-source, and independently reproducible — unlike Maxine ARA (Enterprise-gated, can't even run it to check) and unlike VASA-1 itself (no official repo at all). |
|
| 90 |
+
|
| 91 |
+
**Net tier: B.** Not because anything measured has failed, but because two of
|
| 92 |
+
the five failure modes are honestly unresolved rather than falsely marked safe.
|
| 93 |
+
A Tier A verdict here would be the rubber-stamping Samantha's rubric exists to
|
| 94 |
+
prevent.
|
| 95 |
+
|
| 96 |
+
## 6. Open Questions (feed into knowledge graph)
|
| 97 |
+
|
| 98 |
+
1. Run the frame-to-frame head-pose variance measurement (Failure 1) against
|
| 99 |
+
FasterLivePortrait's actual output — currently asserted as a risk, not yet
|
| 100 |
+
measured.
|
| 101 |
+
2. Run the silence-transition color/saturation delta measurement (Failure 2)
|
| 102 |
+
against the TTS→Avatar handoff specifically.
|
| 103 |
+
3. Independently benchmark EchoMimicV2 and Hallo2 latency/VRAM against the
|
| 104 |
+
real-time budget rather than accepting the "likely too slow" inference in
|
| 105 |
+
§4 — the claim strength of both papers means this is worth confirming, not
|
| 106 |
+
dismissing.
|
| 107 |
+
4. Study LiveTalking's architecture directly — it is the closest existing
|
| 108 |
+
real-world implementation of what Beryl is attempting, and may shortcut
|
| 109 |
+
integration work at the Avatar↔TTS boundary specifically.
|
| 110 |
+
|
| 111 |
+
## 7. References
|
| 112 |
+
|
| 113 |
+
- LiveTalking — https://github.com/lipku/LiveTalking
|
| 114 |
+
- FasterLivePortrait — https://github.com/warmshao/FasterLivePortrait
|
| 115 |
+
- EchoMimicV2 (CVPR 2025) — https://github.com/antgroup/echomimic_v2
|
| 116 |
+
- Hallo2 (ICLR 2025) — https://github.com/fudan-generative-vision/hallo2
|
| 117 |
+
- JoyVASA — https://github.com/jdh-algo/JoyVASA
|
| 118 |
+
- VASA-1 (unofficial reproduction, no official release) — https://github.com/vasavatar/VASA-1
|
| 119 |
+
|
| 120 |
+
*Star/fork counts pulled live from the GitHub API on 2026-07-13. Re-verify
|
| 121 |
+
before treating as current.*
|
src/app/pipeline/Workspace.tsx
CHANGED
|
@@ -9,6 +9,7 @@ import NodeDetails from "@/components/NodeDetails";
|
|
| 9 |
import ExpertChat from "@/components/ExpertChat";
|
| 10 |
import { berylPipeline } from "@/lib/beryl-pipeline";
|
| 11 |
import { FAILURE_MODES, type ImplementationMode } from "@/lib/types";
|
|
|
|
| 12 |
|
| 13 |
export default function Workspace() {
|
| 14 |
const searchParams = useSearchParams();
|
|
@@ -21,6 +22,7 @@ export default function Workspace() {
|
|
| 21 |
const [showMetrics, setShowMetrics] = useState(true);
|
| 22 |
const [tab, setTab] = useState<"details" | "expert">("details");
|
| 23 |
const [auditOpen, setAuditOpen] = useState(false);
|
|
|
|
| 24 |
|
| 25 |
const failingAudit = pipeline.pipelineAudit?.some((a) => a.verdict === "fail");
|
| 26 |
|
|
@@ -71,9 +73,12 @@ export default function Workspace() {
|
|
| 71 |
>
|
| 72 |
<SlidersHorizontal size={12} /> Metrics
|
| 73 |
</button>
|
| 74 |
-
<
|
| 75 |
-
|
| 76 |
-
|
|
|
|
|
|
|
|
|
|
| 77 |
</div>
|
| 78 |
</header>
|
| 79 |
|
|
@@ -124,6 +129,7 @@ export default function Workspace() {
|
|
| 124 |
selectedId={selectedId}
|
| 125 |
activeModes={activeModes}
|
| 126 |
showMetrics={showMetrics}
|
|
|
|
| 127 |
onSelect={selectNode}
|
| 128 |
onToggleExpand={toggleExpand}
|
| 129 |
onSetMode={setMode}
|
|
|
|
| 9 |
import ExpertChat from "@/components/ExpertChat";
|
| 10 |
import { berylPipeline } from "@/lib/beryl-pipeline";
|
| 11 |
import { FAILURE_MODES, type ImplementationMode } from "@/lib/types";
|
| 12 |
+
import type { LayoutMode } from "@/lib/graph-layout";
|
| 13 |
|
| 14 |
export default function Workspace() {
|
| 15 |
const searchParams = useSearchParams();
|
|
|
|
| 22 |
const [showMetrics, setShowMetrics] = useState(true);
|
| 23 |
const [tab, setTab] = useState<"details" | "expert">("details");
|
| 24 |
const [auditOpen, setAuditOpen] = useState(false);
|
| 25 |
+
const [layoutMode, setLayoutMode] = useState<LayoutMode>("circular");
|
| 26 |
|
| 27 |
const failingAudit = pipeline.pipelineAudit?.some((a) => a.verdict === "fail");
|
| 28 |
|
|
|
|
| 73 |
>
|
| 74 |
<SlidersHorizontal size={12} /> Metrics
|
| 75 |
</button>
|
| 76 |
+
<button
|
| 77 |
+
onClick={() => setLayoutMode((m) => (m === "circular" ? "rows" : "circular"))}
|
| 78 |
+
className="flex items-center gap-1.5 text-xs px-3 py-1.5 rounded-full border border-[var(--panel-border)] text-[var(--muted)] hover:text-white transition"
|
| 79 |
+
>
|
| 80 |
+
<LayoutGrid size={12} /> {layoutMode === "circular" ? "Circular" : "Rows"}
|
| 81 |
+
</button>
|
| 82 |
</div>
|
| 83 |
</header>
|
| 84 |
|
|
|
|
| 129 |
selectedId={selectedId}
|
| 130 |
activeModes={activeModes}
|
| 131 |
showMetrics={showMetrics}
|
| 132 |
+
layoutMode={layoutMode}
|
| 133 |
onSelect={selectNode}
|
| 134 |
onToggleExpand={toggleExpand}
|
| 135 |
onSetMode={setMode}
|
src/components/PipelineGraph.tsx
CHANGED
|
@@ -11,7 +11,7 @@ import {
|
|
| 11 |
} from "@xyflow/react";
|
| 12 |
import "@xyflow/react/dist/style.css";
|
| 13 |
import type { ImplementationMode, Pipeline } from "@/lib/types";
|
| 14 |
-
import { computeLayout } from "@/lib/graph-layout";
|
| 15 |
import StageNode from "./nodes/StageNode";
|
| 16 |
|
| 17 |
const nodeTypes = { stage: StageNode };
|
|
@@ -22,6 +22,7 @@ export default function PipelineGraph({
|
|
| 22 |
selectedId,
|
| 23 |
activeModes,
|
| 24 |
showMetrics,
|
|
|
|
| 25 |
onSelect,
|
| 26 |
onToggleExpand,
|
| 27 |
onSetMode,
|
|
@@ -31,12 +32,13 @@ export default function PipelineGraph({
|
|
| 31 |
selectedId: string | null;
|
| 32 |
activeModes: Record<string, ImplementationMode>;
|
| 33 |
showMetrics: boolean;
|
|
|
|
| 34 |
onSelect: (id: string) => void;
|
| 35 |
onToggleExpand: (id: string) => void;
|
| 36 |
onSetMode: (id: string, mode: ImplementationMode) => void;
|
| 37 |
}) {
|
| 38 |
const { nodes, edges } = useMemo(() => {
|
| 39 |
-
const { positions, flatChildren } = computeLayout(pipeline, expandedIds);
|
| 40 |
const nodes: Node[] = [];
|
| 41 |
|
| 42 |
pipeline.nodes.forEach((n) => {
|
|
@@ -115,10 +117,11 @@ export default function PipelineGraph({
|
|
| 115 |
});
|
| 116 |
|
| 117 |
return { nodes, edges };
|
| 118 |
-
}, [pipeline, expandedIds, selectedId, activeModes, showMetrics, onSelect, onToggleExpand, onSetMode]);
|
| 119 |
|
| 120 |
return (
|
| 121 |
<ReactFlow
|
|
|
|
| 122 |
nodes={nodes}
|
| 123 |
edges={edges}
|
| 124 |
nodeTypes={nodeTypes}
|
|
|
|
| 11 |
} from "@xyflow/react";
|
| 12 |
import "@xyflow/react/dist/style.css";
|
| 13 |
import type { ImplementationMode, Pipeline } from "@/lib/types";
|
| 14 |
+
import { computeLayout, type LayoutMode } from "@/lib/graph-layout";
|
| 15 |
import StageNode from "./nodes/StageNode";
|
| 16 |
|
| 17 |
const nodeTypes = { stage: StageNode };
|
|
|
|
| 22 |
selectedId,
|
| 23 |
activeModes,
|
| 24 |
showMetrics,
|
| 25 |
+
layoutMode,
|
| 26 |
onSelect,
|
| 27 |
onToggleExpand,
|
| 28 |
onSetMode,
|
|
|
|
| 32 |
selectedId: string | null;
|
| 33 |
activeModes: Record<string, ImplementationMode>;
|
| 34 |
showMetrics: boolean;
|
| 35 |
+
layoutMode: LayoutMode;
|
| 36 |
onSelect: (id: string) => void;
|
| 37 |
onToggleExpand: (id: string) => void;
|
| 38 |
onSetMode: (id: string, mode: ImplementationMode) => void;
|
| 39 |
}) {
|
| 40 |
const { nodes, edges } = useMemo(() => {
|
| 41 |
+
const { positions, flatChildren } = computeLayout(pipeline, expandedIds, layoutMode);
|
| 42 |
const nodes: Node[] = [];
|
| 43 |
|
| 44 |
pipeline.nodes.forEach((n) => {
|
|
|
|
| 117 |
});
|
| 118 |
|
| 119 |
return { nodes, edges };
|
| 120 |
+
}, [pipeline, expandedIds, selectedId, activeModes, showMetrics, layoutMode, onSelect, onToggleExpand, onSetMode]);
|
| 121 |
|
| 122 |
return (
|
| 123 |
<ReactFlow
|
| 124 |
+
key={layoutMode}
|
| 125 |
nodes={nodes}
|
| 126 |
edges={edges}
|
| 127 |
nodeTypes={nodeTypes}
|
src/lib/candidates.ts
ADDED
|
@@ -0,0 +1,248 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
// The C Protocol — five evaluation axes, each scored 0-1, rendered as stars.
|
| 2 |
+
// Code is a gate, not just an average input: a candidate with near-zero Code
|
| 3 |
+
// cannot score above Tier C regardless of how strong its other four axes look.
|
| 4 |
+
// That rule exists specifically to encode Failure 5 (the VASA-1 Problem) as
|
| 5 |
+
// arithmetic, not just prose: a brilliant claim with no repo doesn't count.
|
| 6 |
+
|
| 7 |
+
export type CScore = {
|
| 8 |
+
value: number; // 0-1
|
| 9 |
+
note: string;
|
| 10 |
+
};
|
| 11 |
+
|
| 12 |
+
export const C_PROTOCOL_DIMENSIONS = ["code", "claim", "consensus", "cost", "connectivity"] as const;
|
| 13 |
+
export type CDimension = (typeof C_PROTOCOL_DIMENSIONS)[number];
|
| 14 |
+
|
| 15 |
+
// Fixed color order — never reassigned per candidate, per dimension identity.
|
| 16 |
+
export const C_PROTOCOL_COLOR: Record<CDimension, string> = {
|
| 17 |
+
code: "#3987e5", // blue
|
| 18 |
+
claim: "#199e70", // aqua
|
| 19 |
+
consensus: "#c98500", // yellow
|
| 20 |
+
cost: "#008300", // green
|
| 21 |
+
connectivity: "#9085e9", // violet
|
| 22 |
+
};
|
| 23 |
+
|
| 24 |
+
export const C_PROTOCOL_LABEL: Record<CDimension, string> = {
|
| 25 |
+
code: "Code",
|
| 26 |
+
claim: "Claim",
|
| 27 |
+
consensus: "Consensus",
|
| 28 |
+
cost: "Cost",
|
| 29 |
+
connectivity: "Connectivity",
|
| 30 |
+
};
|
| 31 |
+
|
| 32 |
+
export type Candidate = {
|
| 33 |
+
id: string;
|
| 34 |
+
name: string;
|
| 35 |
+
org: string;
|
| 36 |
+
githubUrl: string;
|
| 37 |
+
category: "avatar" | "asr" | "llm" | "tts" | "keyframer" | "orchestration";
|
| 38 |
+
paperVenue?: string;
|
| 39 |
+
stars?: number;
|
| 40 |
+
forks?: number;
|
| 41 |
+
verifiedAt: string;
|
| 42 |
+
hasOfficialRepo: boolean;
|
| 43 |
+
scores: Record<CDimension, CScore>;
|
| 44 |
+
verdict: string;
|
| 45 |
+
};
|
| 46 |
+
|
| 47 |
+
function tierFromScores(scores: Record<CDimension, CScore>): "A" | "B" | "C" {
|
| 48 |
+
// Code gate: no reproducible code caps the tier at C, full stop.
|
| 49 |
+
if (scores.code.value < 0.3) return "C";
|
| 50 |
+
const avg =
|
| 51 |
+
(scores.code.value +
|
| 52 |
+
scores.claim.value +
|
| 53 |
+
scores.consensus.value +
|
| 54 |
+
scores.cost.value +
|
| 55 |
+
scores.connectivity.value) /
|
| 56 |
+
5;
|
| 57 |
+
if (avg >= 0.75) return "A";
|
| 58 |
+
if (avg >= 0.5) return "B";
|
| 59 |
+
return "C";
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
// Star/fork counts pulled live from the GitHub API. Re-verify before treating
|
| 63 |
+
// as current — this is exactly the discipline Failure 5 demands.
|
| 64 |
+
const RAW: Array<Omit<Candidate, "verdict"> & { verdictDraft: string }> = [
|
| 65 |
+
{
|
| 66 |
+
id: "live-talking",
|
| 67 |
+
name: "LiveTalking",
|
| 68 |
+
org: "lipku",
|
| 69 |
+
githubUrl: "https://github.com/lipku/LiveTalking",
|
| 70 |
+
category: "avatar",
|
| 71 |
+
stars: 8378,
|
| 72 |
+
forks: 1350,
|
| 73 |
+
verifiedAt: "2026-07-13",
|
| 74 |
+
hasOfficialRepo: true,
|
| 75 |
+
scores: {
|
| 76 |
+
code: { value: 0.95, note: "Real, official, actively maintained (commits within the last day). Runs as-is." },
|
| 77 |
+
claim: { value: 0.55, note: "No peer-reviewed paper attached — this is an engineering integration project, proven in production rather than in a journal." },
|
| 78 |
+
consensus: { value: 0.95, note: "8,378 stars, 1,350 forks — one of the most active repos in this comparison." },
|
| 79 |
+
cost: { value: 0.85, note: "Built on lightweight methods (wav2lip/MuseTalk/ER-NeRF class) explicitly chosen for low VRAM and real-time throughput." },
|
| 80 |
+
connectivity: { value: 0.95, note: "Built explicitly as an end-to-end real-time streaming pipeline (lip-sync + TTS + LLM chat) — closest real-world reference to Beryl's own architecture." },
|
| 81 |
+
},
|
| 82 |
+
verdictDraft: "The single closest real-world reference for what Beryl is trying to build. Study its architecture before writing new THG code.",
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
id: "faster-live-portrait",
|
| 86 |
+
name: "FasterLivePortrait",
|
| 87 |
+
org: "warmshao",
|
| 88 |
+
githubUrl: "https://github.com/warmshao/FasterLivePortrait",
|
| 89 |
+
category: "avatar",
|
| 90 |
+
stars: 1153,
|
| 91 |
+
forks: 130,
|
| 92 |
+
verifiedAt: "2026-07-13",
|
| 93 |
+
hasOfficialRepo: true,
|
| 94 |
+
scores: {
|
| 95 |
+
code: { value: 0.9, note: "Real, official, updated within the last day. ONNX/TensorRT paths both run." },
|
| 96 |
+
claim: { value: 0.6, note: "Builds on LivePortrait's peer-reviewed base; this repo itself is an engineering optimization, not new research." },
|
| 97 |
+
consensus: { value: 0.7, note: "1,153 stars — smaller than headline projects, but real and current." },
|
| 98 |
+
cost: { value: 0.9, note: "Purpose-built for speed — this is the actual fix for LivePortrait's latency, not a theoretical one." },
|
| 99 |
+
connectivity: { value: 0.85, note: "Drop-in acceleration of LivePortrait with explicit real-time support." },
|
| 100 |
+
},
|
| 101 |
+
verdictDraft: "Best-in-class for the Avatar node specifically on the cost/latency axis Beryl needs.",
|
| 102 |
+
},
|
| 103 |
+
{
|
| 104 |
+
id: "echomimic-v2",
|
| 105 |
+
name: "EchoMimicV2",
|
| 106 |
+
org: "antgroup",
|
| 107 |
+
githubUrl: "https://github.com/antgroup/echomimic_v2",
|
| 108 |
+
category: "avatar",
|
| 109 |
+
paperVenue: "CVPR 2025",
|
| 110 |
+
stars: 4614,
|
| 111 |
+
forks: 540,
|
| 112 |
+
verifiedAt: "2026-07-13",
|
| 113 |
+
hasOfficialRepo: true,
|
| 114 |
+
scores: {
|
| 115 |
+
code: { value: 0.85, note: "Real, official, actively maintained. Runs as published." },
|
| 116 |
+
claim: { value: 0.9, note: "CVPR 2025 peer-reviewed — a top-tier venue, real published results." },
|
| 117 |
+
consensus: { value: 0.85, note: "4,614 stars, 540 forks, real ComfyUI ecosystem." },
|
| 118 |
+
cost: { value: 0.55, note: "Diffusion-based — heavier per-frame cost than GAN/warping derivatives. Needs a measured latency/VRAM check before assuming real-time fit." },
|
| 119 |
+
connectivity: { value: 0.65, note: "Semi-body diffusion output is richer but not architected for a tight real-time loop the way LiveTalking is." },
|
| 120 |
+
},
|
| 121 |
+
verdictDraft: "Credible and well-cited, but cost profile needs verification against Beryl's real-time budget before adoption.",
|
| 122 |
+
},
|
| 123 |
+
{
|
| 124 |
+
id: "hallo2",
|
| 125 |
+
name: "Hallo2",
|
| 126 |
+
org: "fudan-generative-vision",
|
| 127 |
+
githubUrl: "https://github.com/fudan-generative-vision/hallo2",
|
| 128 |
+
category: "avatar",
|
| 129 |
+
paperVenue: "ICLR 2025",
|
| 130 |
+
stars: 3718,
|
| 131 |
+
forks: 536,
|
| 132 |
+
verifiedAt: "2026-07-13",
|
| 133 |
+
hasOfficialRepo: true,
|
| 134 |
+
scores: {
|
| 135 |
+
code: { value: 0.85, note: "Real, official, actively maintained. Runs as published." },
|
| 136 |
+
claim: { value: 0.95, note: "ICLR 2025 — one of the top peer-reviewed venues in ML. The strongest research claim in this entire set." },
|
| 137 |
+
consensus: { value: 0.9, note: "3,718 stars for Hallo2 alone; sibling Hallo has 8,655 — a very strong ecosystem." },
|
| 138 |
+
cost: { value: 0.35, note: "High-resolution diffusion pipeline built for quality over speed — likely a poor fit for Beryl's live-conversation latency budget as-is." },
|
| 139 |
+
connectivity: { value: 0.55, note: "Optimized for long-duration, high-resolution offline generation, not a tight conversational real-time loop." },
|
| 140 |
+
},
|
| 141 |
+
verdictDraft: "The best research claim in this set does not automatically mean the best fit. Don't let Failure 5's inverse — great paper, wrong use case — trip you.",
|
| 142 |
+
},
|
| 143 |
+
{
|
| 144 |
+
id: "joyvasa",
|
| 145 |
+
name: "JoyVASA",
|
| 146 |
+
org: "jdh-algo",
|
| 147 |
+
githubUrl: "https://github.com/jdh-algo/JoyVASA",
|
| 148 |
+
category: "avatar",
|
| 149 |
+
stars: 874,
|
| 150 |
+
forks: 87,
|
| 151 |
+
verifiedAt: "2026-07-13",
|
| 152 |
+
hasOfficialRepo: true,
|
| 153 |
+
scores: {
|
| 154 |
+
code: { value: 0.75, note: "Real, official, actively maintained, runs as published." },
|
| 155 |
+
claim: { value: 0.65, note: "Real code, no major top-tier venue publication found in this pass — credible but not independently peer-reviewed here." },
|
| 156 |
+
consensus: { value: 0.7, note: "874 stars, actively updated." },
|
| 157 |
+
cost: { value: 0.45, note: "Diffusion-based — likely needs further optimization before real-time use, similar to EchoMimicV2's tradeoff." },
|
| 158 |
+
connectivity: { value: 0.5, note: "Generalizes to animal animation too — flexible, but not built for a real-time streaming loop." },
|
| 159 |
+
},
|
| 160 |
+
verdictDraft: "Interesting for its generalization (portrait + animal), but not a clear win over LiveTalking/FasterLivePortrait for Beryl's specific use case.",
|
| 161 |
+
},
|
| 162 |
+
{
|
| 163 |
+
id: "vasa-1",
|
| 164 |
+
name: "VASA-1",
|
| 165 |
+
org: "Microsoft Research (no official release)",
|
| 166 |
+
githubUrl: "https://github.com/vasavatar/VASA-1",
|
| 167 |
+
category: "avatar",
|
| 168 |
+
paperVenue: "Microsoft Research",
|
| 169 |
+
stars: 105,
|
| 170 |
+
forks: 14,
|
| 171 |
+
verifiedAt: "2026-07-13",
|
| 172 |
+
hasOfficialRepo: false,
|
| 173 |
+
scores: {
|
| 174 |
+
code: { value: 0.05, note: "No official repo exists. Every GitHub result for 'VASA-1' is an unofficial, unendorsed reproduction attempt (105, 318, and 22 stars respectively) — none complete, none authoritative." },
|
| 175 |
+
claim: { value: 0.8, note: "The paper's presented results are genuinely impressive on their own terms — this high score is the trap: a strong claim paired with near-zero Code is exactly the pattern Failure 5 exists to catch." },
|
| 176 |
+
consensus: { value: 0.2, note: "The largest unofficial attempt (johndpope/VASA-1-hack, 318 stars) is explicitly labeled 'wip — running some training with overfitting.' Fragmented, not consolidated." },
|
| 177 |
+
cost: { value: 0, note: "Cannot be measured — there is no runnable reference implementation." },
|
| 178 |
+
connectivity: { value: 0, note: "No official code exists to connect to anything." },
|
| 179 |
+
},
|
| 180 |
+
verdictDraft: "This is Failure 5's namesake for a reason. Brilliant claim, zero official reproduction. Archive it as a research reference only — never as a pipeline dependency.",
|
| 181 |
+
},
|
| 182 |
+
{
|
| 183 |
+
id: "glm-4-voice",
|
| 184 |
+
name: "GLM-4-Voice",
|
| 185 |
+
org: "zai-org",
|
| 186 |
+
githubUrl: "https://github.com/zai-org/GLM-4-Voice",
|
| 187 |
+
category: "llm",
|
| 188 |
+
stars: 3203,
|
| 189 |
+
forks: 281,
|
| 190 |
+
verifiedAt: "2026-07-13",
|
| 191 |
+
hasOfficialRepo: true,
|
| 192 |
+
scores: {
|
| 193 |
+
code: { value: 0.85, note: "Real, official, updated within the last few days." },
|
| 194 |
+
claim: { value: 0.7, note: "From a reputable lab (Zhipu/Z.ai), no single top-tier venue confirmed in this pass but a credible, active org." },
|
| 195 |
+
consensus: { value: 0.8, note: "3,203 stars, active." },
|
| 196 |
+
cost: { value: 0.65, note: "One end-to-end model likely avoids the redundant encode/decode hops of three separate models — needs a direct VRAM/latency measurement against the current stack." },
|
| 197 |
+
connectivity: { value: 0.7, note: "Architecturally different, not a drop-in node swap: an end-to-end speech-to-speech model that COLLAPSES ASR->LLM->TTS into one call — the single most important find in this comparison for the Cascaded Latency problem." },
|
| 198 |
+
},
|
| 199 |
+
verdictDraft: "Worth a serious architecture-level evaluation: does replacing three pipeline nodes with one end-to-end model actually solve Failure 3, or just relocate it?",
|
| 200 |
+
},
|
| 201 |
+
{
|
| 202 |
+
id: "wan2-1",
|
| 203 |
+
name: "Wan2.1",
|
| 204 |
+
org: "Wan-Video",
|
| 205 |
+
githubUrl: "https://github.com/Wan-Video/Wan2.1",
|
| 206 |
+
category: "keyframer",
|
| 207 |
+
stars: 16543,
|
| 208 |
+
forks: 2957,
|
| 209 |
+
verifiedAt: "2026-07-13",
|
| 210 |
+
hasOfficialRepo: true,
|
| 211 |
+
scores: {
|
| 212 |
+
code: { value: 0.95, note: "Real, official, updated within hours." },
|
| 213 |
+
claim: { value: 0.85, note: "Real, open, from Alibaba, extensively used and cited." },
|
| 214 |
+
consensus: { value: 0.98, note: "16,543 stars, 2,957 forks — the most active repo in this entire comparison." },
|
| 215 |
+
cost: { value: 0.7, note: "Ships multiple model sizes (1.3B/14B) — an actual quantization/scale path, not a 32GB+ dead end." },
|
| 216 |
+
connectivity: { value: 0.7, note: "Text/image-to-video generation with a real ComfyUI ecosystem — fits the offline/async Keyframer role well." },
|
| 217 |
+
},
|
| 218 |
+
verdictDraft: "The clear flagship choice for the Keyframer node. No competing claim in this set comes close on consensus or activity.",
|
| 219 |
+
},
|
| 220 |
+
{
|
| 221 |
+
id: "triton",
|
| 222 |
+
name: "Triton Inference Server",
|
| 223 |
+
org: "triton-inference-server",
|
| 224 |
+
githubUrl: "https://github.com/triton-inference-server/server",
|
| 225 |
+
category: "orchestration",
|
| 226 |
+
stars: 10823,
|
| 227 |
+
forks: 1807,
|
| 228 |
+
verifiedAt: "2026-07-13",
|
| 229 |
+
hasOfficialRepo: true,
|
| 230 |
+
scores: {
|
| 231 |
+
code: { value: 0.95, note: "Real, official, NVIDIA-maintained, daily commit activity." },
|
| 232 |
+
claim: { value: 0.9, note: "NVIDIA-maintained since 2018, extremely mature, no reproducibility question here." },
|
| 233 |
+
consensus: { value: 0.95, note: "10,823 stars, 1,807 forks, daily commit activity." },
|
| 234 |
+
cost: { value: 0.9, note: "The serving layer itself has minimal overhead; it reduces aggregate cost by enabling shared-GPU concurrency." },
|
| 235 |
+
connectivity: { value: 0.95, note: "Purpose-built to orchestrate multi-model serving with dynamic batching — exactly the Orchestration node's job." },
|
| 236 |
+
},
|
| 237 |
+
verdictDraft: "No real alternative needed here — this is the settled choice.",
|
| 238 |
+
},
|
| 239 |
+
];
|
| 240 |
+
|
| 241 |
+
export const CANDIDATES: Candidate[] = RAW.map(({ verdictDraft, ...c }) => ({
|
| 242 |
+
...c,
|
| 243 |
+
verdict: verdictDraft,
|
| 244 |
+
}));
|
| 245 |
+
|
| 246 |
+
export function candidateTier(c: Candidate): "A" | "B" | "C" {
|
| 247 |
+
return tierFromScores(c.scores);
|
| 248 |
+
}
|
src/lib/graph-layout.ts
CHANGED
|
@@ -1,5 +1,7 @@
|
|
| 1 |
import type { Pipeline, PipelineNode } from "./types";
|
| 2 |
|
|
|
|
|
|
|
| 3 |
export type LayoutResult = {
|
| 4 |
positions: Record<string, { x: number; y: number }>;
|
| 5 |
flatChildren: Record<string, PipelineNode>;
|
|
@@ -8,10 +10,7 @@ export type LayoutResult = {
|
|
| 8 |
const RING_1_RADIUS = 460;
|
| 9 |
const RING_2_RADIUS = 230;
|
| 10 |
|
| 11 |
-
|
| 12 |
-
pipeline: Pipeline,
|
| 13 |
-
expandedIds: Set<string>,
|
| 14 |
-
): LayoutResult {
|
| 15 |
const top = pipeline.nodes.filter((n) => n.category !== "orchestration");
|
| 16 |
const hub = pipeline.nodes.find((n) => n.category === "orchestration");
|
| 17 |
const count = top.length;
|
|
@@ -44,3 +43,68 @@ export function computeLayout(
|
|
| 44 |
|
| 45 |
return { positions, flatChildren };
|
| 46 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
import type { Pipeline, PipelineNode } from "./types";
|
| 2 |
|
| 3 |
+
export type LayoutMode = "circular" | "rows";
|
| 4 |
+
|
| 5 |
export type LayoutResult = {
|
| 6 |
positions: Record<string, { x: number; y: number }>;
|
| 7 |
flatChildren: Record<string, PipelineNode>;
|
|
|
|
| 10 |
const RING_1_RADIUS = 460;
|
| 11 |
const RING_2_RADIUS = 230;
|
| 12 |
|
| 13 |
+
function computeCircularLayout(pipeline: Pipeline, expandedIds: Set<string>): LayoutResult {
|
|
|
|
|
|
|
|
|
|
| 14 |
const top = pipeline.nodes.filter((n) => n.category !== "orchestration");
|
| 15 |
const hub = pipeline.nodes.find((n) => n.category === "orchestration");
|
| 16 |
const count = top.length;
|
|
|
|
| 43 |
|
| 44 |
return { positions, flatChildren };
|
| 45 |
}
|
| 46 |
+
|
| 47 |
+
// Rows: nodes grouped into horizontal bands by pipeline order (top-down DAG),
|
| 48 |
+
// matching a layered "structural" view rather than a circular loop.
|
| 49 |
+
const ROW_ORDER: Record<string, number> = {
|
| 50 |
+
vision: 0,
|
| 51 |
+
asr: 0,
|
| 52 |
+
llm: 1,
|
| 53 |
+
tts: 2,
|
| 54 |
+
keyframer: 3,
|
| 55 |
+
avatar: 3,
|
| 56 |
+
video: 4,
|
| 57 |
+
orchestration: 5,
|
| 58 |
+
};
|
| 59 |
+
|
| 60 |
+
const ROW_HEIGHT = 190;
|
| 61 |
+
const COLUMN_WIDTH = 260;
|
| 62 |
+
const CHILD_ROW_HEIGHT = 110;
|
| 63 |
+
|
| 64 |
+
function computeRowsLayout(pipeline: Pipeline, expandedIds: Set<string>): LayoutResult {
|
| 65 |
+
const positions: Record<string, { x: number; y: number }> = {};
|
| 66 |
+
const flatChildren: Record<string, PipelineNode> = {};
|
| 67 |
+
|
| 68 |
+
const rows = new Map<number, PipelineNode[]>();
|
| 69 |
+
pipeline.nodes.forEach((n) => {
|
| 70 |
+
const row = ROW_ORDER[n.category] ?? 0;
|
| 71 |
+
if (!rows.has(row)) rows.set(row, []);
|
| 72 |
+
rows.get(row)!.push(n);
|
| 73 |
+
});
|
| 74 |
+
|
| 75 |
+
const sortedRowIndices = Array.from(rows.keys()).sort((a, b) => a - b);
|
| 76 |
+
|
| 77 |
+
sortedRowIndices.forEach((rowIndex) => {
|
| 78 |
+
const nodesInRow = rows.get(rowIndex)!;
|
| 79 |
+
const rowWidth = (nodesInRow.length - 1) * COLUMN_WIDTH;
|
| 80 |
+
nodesInRow.forEach((n, i) => {
|
| 81 |
+
const x = i * COLUMN_WIDTH - rowWidth / 2;
|
| 82 |
+
const y = rowIndex * ROW_HEIGHT;
|
| 83 |
+
positions[n.id] = { x, y };
|
| 84 |
+
|
| 85 |
+
if (expandedIds.has(n.id) && n.children?.length) {
|
| 86 |
+
const childCount = n.children.length;
|
| 87 |
+
const childRowWidth = (childCount - 1) * (COLUMN_WIDTH * 0.7);
|
| 88 |
+
n.children.forEach((child, ci) => {
|
| 89 |
+
flatChildren[child.id] = child;
|
| 90 |
+
positions[child.id] = {
|
| 91 |
+
x: x + ci * (COLUMN_WIDTH * 0.7) - childRowWidth / 2,
|
| 92 |
+
y: y + CHILD_ROW_HEIGHT,
|
| 93 |
+
};
|
| 94 |
+
});
|
| 95 |
+
}
|
| 96 |
+
});
|
| 97 |
+
});
|
| 98 |
+
|
| 99 |
+
return { positions, flatChildren };
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
export function computeLayout(
|
| 103 |
+
pipeline: Pipeline,
|
| 104 |
+
expandedIds: Set<string>,
|
| 105 |
+
mode: LayoutMode = "circular",
|
| 106 |
+
): LayoutResult {
|
| 107 |
+
return mode === "rows"
|
| 108 |
+
? computeRowsLayout(pipeline, expandedIds)
|
| 109 |
+
: computeCircularLayout(pipeline, expandedIds);
|
| 110 |
+
}
|