Meridian / docs /studio_walkthrough.md
yycc's picture
Meridian
9f57754
|
Raw
History Blame Contribute Delete
17.1 kB

Meridian Studio — walkthrough film

← Research article · Studio guide

Status: a 38-second preview-only overview is available; a matching generated take is still pending. Watch the concise Studio overview · Editing recipe.

The concise edit uses restrained English titles and enlarged details of the actual UI: one source → camera position and aim → held source time → geometric reference. It omits repetitive authoring and backend waits, disclosed on screen, without accelerating the remaining actions. The complete supplied input and 243-frame geometric reference are retained; the UI capture is resampled from 25 to 24 fps. It is silent, with no final generated video implied. The original recording remains unchanged:

Watch the revised, uncut recording · 10-second geometric reference · Before / after camera comparison · Session evidence.

NBA3 replaces the flower scene as the current walkthrough candidate: approach → airborne hold with camera travel → resumed dunk and landing. The earlier flower pilot is retained, not overwritten.

Revision 02 replaces the large orbit/approach with small athlete-framed lateral travel. The athlete and hoop are substantially more legible in the held reference, and the source map continues through the dunk and landing. Review included contact sheets covering all 243 reference frames and larger comparisons at output frames 60, 134 and 179. The backend's average coverage rose from 61.2% to 83.9%; that is a geometric diagnostic, not a generated-quality score.

The reference still has conspicuous disocclusion holes and tearing around the athlete's outline. This is a more useful authoring demonstration, not an approved cinematic result. Native-speed motion review and a matching generated take are still needed before publishing the research-page film. The rejected first camera pilot is retained for comparison, not silently replaced.

The September 13 recordings ran through Chromium against the existing Studio at 127.0.0.1:8412 on GPU 0, with permission to execute outside the restricted sandbox. It records actual seven-key authoring and geometric previews; no final generation was submitted. The running main-repository Studio has the same authoring controls as the release, with minor comment/warning-text differences; the served HTML is retained. Revision 02 completed 81 POST requests with no recorded API/browser errors and no /render request. The preview workflow passed its live checks; the script's --render branch remains untested. This is raw workflow evidence, not yet an approved research-page film.

The older files named browser_studio.png are screenshots of a separate source/control gallery, not this camera-authoring application. Do not substitute them for a Studio demonstration.

Record with Playwright

Recording script — run it in a terminal where Chromium can launch and http://127.0.0.1:8412 is reachable. That address means the machine running the script; use an existing private tunnel or --url if the Studio runs elsewhere. Do not expose the unauthenticated service publicly. The script connects to an existing service; it does not launch, restart or cancel it.

Arrange a free service/GPU slot first. Even without --render, uploading triggers reconstruction, and editing triggers point-cloud, thumbnail and full-path warp work. The script waits between edits; it is not a CPU-only recording tool and cannot determine whether other users need that GPU.

From the release root, make a preview-only pilot:

PY=/home/chenyun/miniforge3/envs/wan_new/bin/python
"$PY" docs/record_studio_walkthrough.py \
  --source videos-all/nba3_teacher30/nba3_full_event.mp4 \
  --camera-style nba-glide \
  --url http://127.0.0.1:8412 \
  --out videos-all/studio_walkthrough/nba3_preview_02

This NBA3 plate contains the complete event in 124 frames at 24 fps, already prepared from the supplied clip. The hold uses frame 60, during the airborne ball sweep before the dunk; the action then resumes through the landing. For another scene, choose a clean clip with at least 124 normalized frames before its first detected cut. It stops if that prepared span is shorter; it does not silently adapt the timing sketch. Retain source permission/attribution when substituting footage.

The pilot uses actual UI controls to:

  1. Upload and reconstruct, select 243 output frames, then start from the source-camera path.
  2. Add and retime five intermediate keys using the source-time table below.
  3. Look through keys, make small sideways Shift-drags, and adjust the aim to retain the athlete and hoop.
  4. Return to the overview, scrub the hold, and play the real grey-hole and magenta references.

nba-glide is specific to the NBA3 full-event plate. It reads the reconstructed torso point near normalized source-image coordinate (0.367, 0.435) at frame 60, then uses genuine pointer gestures to place it near (0.40, 0.435) during the hold. This preserves space for the ball and hoop rather than aiming every key at the scene centre. It does not inject camera state or replace UI responses. Read-only projection calculations guide the automated gestures; this is not an automatic subject-tracking feature.

Nominal sideways offsets reach 0.024 scene-centre-depth units, with no forward push. Exact positions and aims are retained in path_preview.json; these depth-normalized units are not metres. The older nominal 30° orbit workflow remains available as --camera-style orbit-pilot (the script's default for compatibility), not as the recommended NBA3 path. Neither workflow is a reproduction of an approved CLI take, a large-angle benchmark or a guarantee of generated quality. Inspect both downloaded reference videos continuously before spending time on final generation. Passing the Studio's clearance gate is not a visual-quality verdict.

To capture the same scripted workflow including a new generated take, use a new directory and add --render:

"$PY" docs/record_studio_walkthrough.py \
  --source videos-all/nba3_teacher30/nba3_full_event.mp4 \
  --camera-style nba-glide \
  --out videos-all/studio_walkthrough/nba3_render_01 \
  --render

This is a new session, not a resume of the preview. The script retains and checks the accepted job's payload against the path displayed in that recording. It never bypasses a disabled Render button. Add --headed to watch in Chromium on a machine with a display; let the automation finish without editing the same page. Default headless mode records the same viewport without needing a desktop. --timeout sets each backend wait in seconds; the default is 1800. A timeout or closed browser does not cancel an already submitted job. Check its recorded job ID before retrying.

If Playwright or Chromium is missing, install them in the recording environment first:

"$PY" -m pip install playwright
"$PY" -m playwright install chromium

What gets saved

  • studio_walkthrough_raw.webm: the actual 1920 × 1080 browser viewport, including real waits. It records neither browser chrome nor audio; do not rely on it to include the OS mouse cursor. No fake cursor, replacement UI, simulated responses or accelerated preview are injected.
  • input.* and, when present, input_provenance.json: a retained upload and its adjacent source record.
  • prepared.json, path_preview.json, warp_preview.json: the normalized span, geometry, exact edited keys, source-frame map and preview diagnostics. Preview-only runs do not claim a render payload.
  • preview_truth.mp4, preview_holes.mp4, numbered screenshots: reference footage and review stills.
  • With nba-glide, subject_anchor.json: the selected reconstructed torso point and its source projection.
  • session.json, studio_served.html: source/download hashes, served UI, request payloads and client-observed wall-clock milestones. These timestamps are relative to script startup, not exact WebM edit points or an interactive-latency benchmark.
  • With --render: render_request.json, job.json, and the matching source.mp4, render.mp4, out.mp4, grid.mp4. source.mp4 follows the authored source-time map; it is not the untouched input.

The API does not expose checkpoint identities or the launch recipe. Retain the service launch command, checkpoint/adapter identifiers and server log separately. Also preserve the normalized upload from <service --work>/clips/<clip>/clip.mp4 if an exact input archive is needed; <clip> is recorded in prepared.json. The script does not inspect the server filesystem or guess its configuration.

For an MP4 viewing copy, without cutting waits or changing playback speed:

ffmpeg -n -i videos-all/studio_walkthrough/nba3_render_01/studio_walkthrough_raw.webm \
  -c:v libx264 -crf 18 -pix_fmt yuv420p -movflags +faststart \
  videos-all/studio_walkthrough/nba3_render_01/studio_walkthrough_review.mp4

Keep the raw recording. A 45-second research-page film is a separate editorial pass, following the outline below, with omitted waits disclosed. Use the downloaded full out.mp4 for the cinematic reveal, not a screen-recorded crop of its small comparison pane. Do not publish a Studio-film link until the actual recording and generated motion have been reviewed.

The story

Design the observation. See the reference. Generate the shot.

One beautiful source, one deliberate camera path, one uninterrupted generated result. Show that the released Studio is an authoring tool—not only a gallery or a menu of orbit presets. The point is the relationship between an edit and its visible consequence, rather than a tour of every control.

Place the film in the research article's Studio section, after the method and speed discussion. Keep the cinematic hero separate: the hero shows the result; this film explains how to author it.

Capture outline · approximately 45 seconds

These are editorial allocations, not measured service timings. Extend the capture if an operation needs longer; do not speed up pointer movement or pretend the model generated instantaneously.

Passage Actual screen action Minimal caption
Establish · 0–4 s Show the chosen source, then the actual Studio with a prepared source span. The source must remain identifiable. One source. A new observation.
Author · 4–16 s Show the multi-key path, look through a key, Shift-drag sideways and adjust its aim. Keep the athlete and hoop legible; do not exaggerate the small travel. Place the camera. Shape its path.
Shape time · 16–23 s Show two keys sharing a source frame at different output frames. Scrub across the hold and the subsequent advancing segment. Hold the moment. Keep the camera moving.
Inspect · 23–29 s Let the genuine full-path warp finish. Play the grey-hole reference; briefly switch to the magenta diagnostic. Keep one actual edit-to-preview response at native speed. Preview the geometry before generating.
Generate · 29–32 s Click Render this take and show the real progress screen. If waiting is cut, say so and report the retained run's elapsed time. Generation wait omitted: [measured duration].
Reveal · 32–42.125 s Play the matching 243-frame output intact at 24 fps, large and uncluttered. Generated view.
Close · about 3 s End on the Studio's source / reference / output comparison or a quiet wordmark. Meridian Studio · included in the code release.

The final take must be generated from the exact Studio keys shown. An existing CLI flower or motorcycle output is useful for choosing a scene, but is not evidence of an unexecuted Studio path.

Source and camera design

NBA3 is the current walkthrough source. Its wide view makes the approach, airborne ball sweep, dunk and landing legible as one event. The retained 124-frame plate covers the full supplied clip; its frame 60 maps to original frame 73 / PTS 2.435767 s. See the exact input preparation. These are source-file timestamps, not a claim about physical capture speed. The footage is user-supplied; public redistribution rights and endorsement have not been established.

The Spring flower scene remains an alternate, with Blender Foundation attribution and CC BY 4.0 notice in its input preparation.

Use a source export appropriate for the Studio's maximum 124-frame prepared window. Do not claim the Studio reproduced a 243-source-frame CLI reconstruction; selecting a 243-frame output does not enlarge its prepared source window. Begin with a modest, well-framed path. Only increase travel after inspecting the projection—an impressive trajectory that loses the athlete is a worse demo.

For a 124-frame prepared span starting at start, this timing sketch fits a 243-frame output:

Output index t Source index Camera intention, to tune in the actual scene
0 start + 0 Establish the source-side composition.
40 start + 40 Follow the source framing with a small lateral offset.
60 start + 60 Begin the time hold with breathing room around the subject.
105 start + 60 Glide sideways while retaining the athlete.
145 start + 60 Travel sideways, keeping the aim on the subject.
179 start + 60 Ease back toward the source-side camera before action resumes.
242 start + 123 Let the action advance again.

This uses all 124 prepared source frames and adds 119 held output frames. It preserves the prepared input's pace outside the hold; any slow motion already in that input remains baked in. The exact camera keys are saved with each recording. This table specifies intent, not a promise of seamless generated motion.

What “real-time preview” may honestly mean

  • Browser 3D view: a loaded point cloud and camera handles redraw during interaction. Capture this normally; do not attach an FPS or latency claim without measuring it.
  • Full-path geometric reference: generated by the backend after a committed valid edit. This requires GPU work and video encoding; it can wait behind other service work.
  • Final generated video: a separate render job. Never label its replay as a live model response.

The point-cloud overview is not the full conditioning tensor. The grey-hole video is a compressed preview of that reference; the magenta view is diagnostic and is not given to the model.

Capture and acceptance checklist

  • Record the actual released service/index.html, not a UI mock or the videos-all gallery.
  • Arrange a dedicated service/GPU recording slot separately. Do not interrupt existing jobs to make this recording. No service was started or render queued as part of this documentation update.
  • Capture at native 1920 × 1080 or another readable desktop size. Keep pointer motion deliberate; show the key table when explaining source time. Avoid cinematic overlays on the actual output.
  • Retain the source export, normalized frame window, exact /render path payload, seed, model recipe, output files and status timings alongside the raw capture. The UI has no path-export button, so retain the request through browser network tools or the client used for the session.
  • Disclose omitted reconstruction/generation waits. Keep one representative edit-to-warp update unaccelerated, including its real waiting time. Cold reconstruction is not interactive preview.
  • Check subject visibility, transitions into/out of the hold, thin geometry and newly exposed backgrounds continuously. Sampled frames and successful playback alone are insufficient.
  • Confirm the generated take matches the recorded path and source map. Record any crops or omitted output frames; the first choice is to keep the complete generated take intact.
  • Add source credits and transformation notices. Do not imply that a view inferred from a film is documentary footage, or that a nominal orbit angle was measured in the generated output.

Once recorded and reviewed, add a real poster and MP4 link to the research article. Until then, link this plan explicitly as a plan; no broken “Watch Studio” button or fabricated placeholder video.