joyai-echo / examples.py
multimodalart's picture
multimodalart HF Staff
Port the authors' multishot_inference_dmd pipeline to ZeroGPU xlarge (fp8-cast DiT, module-scope load)
4d13bfa verified
Raw
History Blame Contribute Delete
9.88 kB
"""Example shot prompts, copied verbatim from the authors' own
``prompts/test_*.json`` in https://github.com/jd-opensource/JoyAI-Echo.
Only the first two shots of each story are used, to keep a demo run short.
"""
VLOG_1 = (
"ID_A is a young woman in her twenties with shoulder-length dark brown hair, a slim "
"build, and a soft natural look. ID_A wears a loose light beige knit top and relaxed dark "
"trousers, creating a quiet home-lifestyle appearance. ID_A’s voice is a clear young "
"female voice with a soft mid-high register, gentle breathiness, and intimate vlog-style "
"pacing. The mouth movement is clearly visible in the frame and remains naturally "
"synchronized with the spoken line, especially on slower reflective phrases and lightly "
"smiled vowel shapes. At normal speed, ID_A steps into the center of the frame, settles "
"the posture, and begins speaking with a calm and thoughtful expression. The lip movement "
"aligns closely with the audio throughout the full sentence, especially on the slower "
"emotional beats. In a soft young female voice with reflective warmth, ID_A says, \"I did "
"not plan to record tonight, but something about this room felt different when I walked "
"back in.\" The shot uses realistic indoor vlog-style imagery with soft practical light, "
"neutral tones, and a warm personal atmosphere. The tone feels calm, introspective, and "
"naturally domestic rather than performative. A stable medium shot frames ID_A from the "
"waist up, keeping the face, upper-body posture, and visible lip movement clearly "
"readable while preserving some of the room behind. The background includes a white "
"curtain, soft string lights, part of a small table, and the warm interior of a well-kept "
"room. The setting feels personal, lived-in, and quiet. Very soft indoor room tone, light "
"fabric movement, and subtle foot placement are audible. No prominent background music. "
"The audio focus remains on speech and subtle room ambience."
)
VLOG_2 = (
"ID_A is a young woman in her twenties with shoulder-length dark brown hair, soft natural "
"makeup, and a gentle but observant expression. ID_A wears a loose light beige knit top "
"and dark relaxed trousers. ID_A’s voice is a youthful female voice with clear diction, a "
"soft intimate tone, and slow personal-story pacing. The mouth movement remains visible "
"and synchronized as ID_A turns slightly toward the window side of the room while "
"continuing to talk. At normal speed, ID_A turns slightly to the side and glances toward "
"the curtain as if the room itself has triggered the reflection. The spoken line remains "
"clearly mouth-synced, including subtle lip compression at clause endings. In a soft, "
"thoughtful female voice, ID_A says, \"Maybe it is because I have been too busy to really "
"look at any of this for a while.\" The visual style remains realistic and soft, with "
"warm-neutral indoor tones and a personal diary-video mood. The emotional atmosphere is "
"quiet and self-aware. A slight pan follows ID_A as the subject turns toward the curtain, "
"keeping facial expression and mouth articulation visible while revealing more of the "
"side wall. The curtain, a faint edge of the window area, and small warm decorative "
"lights remain visible. The room continues to feel intimate and uncluttered. Light fabric "
"movement, a soft shift of feet, and low indoor ambience are audible. No background "
"music. The room’s quiet and the speaker’s voice dominate."
)
WAREHOUSE_1 = (
"ID_A is a warehouse worker seated on a low metal bench, is a man in his thirties with "
"short black hair, tan skin, a sturdy medium build, and a practical tired face. He wears "
"a maroon T-shirt, grey work shorts, and is pulling on tall white rubber boots for "
"wet-floor industrial work. ID_A’s voice is a low, steady male voice with a calm "
"working-class plainness, slightly rough from long hours and warehouse dust, with natural "
"conversational pacing. His mouth is visible and synchronized with the line. At normal "
"speed, ID_A braces one hand on the bench and pulls the rubber boot firmly over his heel, "
"glancing down to check the fit before looking ahead. His lip movement aligns with the "
"line. ID_A says, \"If the floor is wet again tonight, I want to know before it gets "
"halfway across the building.\" Naturalistic industrial drama with realistic warehouse "
"lighting, practical textures, and documentary-like physical detail. A medium-close "
"static shot focuses on ID_A from the knees up, keeping the boot-pulling action and his "
"face in view. A worn warehouse prep area with concrete floor, metal lockers, stacked "
"plastic crates, and a hanging fluorescent light casting a cold industrial glow. Rubber "
"boot squeak, fabric friction, distant forklift reverse beep, fluorescent hum, and "
"far-off metal clatter are audible. No prominent background music, only a faint low "
"industrial tension pad underneath the natural ambience."
)
WAREHOUSE_2 = (
"ID_A is a warehouse worker adjusting the second white rubber boot, is a man in his "
"thirties with short black hair, tan skin, and a practical, focused expression. He wears "
"a maroon T-shirt and grey shorts, with both tall white rubber boots nearly secured. "
"ID_A’s voice is a low male voice with dry humor, steady breath, and an unhurried "
"practical rhythm. His mouth is visible and synchronized with speech. At normal speed, "
"ID_A presses the heel down, smooths the top of the boot with one hand, and gives it a "
"final tug. His mouth movement stays aligned with the line. ID_A says, \"Nobody notices "
"proper boots until the place starts flooding and everyone suddenly wants a pair.\" "
"Realistic industrial workplace footage with tactile detail and grounded working-life "
"atmosphere. A lower medium shot centered on the bench, boots, and torso, keeping facial "
"reaction visible. Metal bench, industrial lockers, scuffed concrete, and stacked storage "
"bins remain visible behind him. Rubber flex, bench creak, distant pallet drag, "
"fluorescent buzz, and soft ventilation rumble are audible. No distinct score, just low "
"atmospheric tension under the room tone."
)
RALLY_1 = (
"ID_A is an American male racing driver in his thirties with a lean athletic build, sharp "
"cheekbones, short black hair, light stubble, and a weathered face marked by experience "
"and pressure. ID_A wears a partially unzipped professional rally racing suit over a dark "
"fireproof base layer, with a serious focused expression before entering the car. ID_A’s "
"voice is a low, steady male baritone with a rough edge, controlled breathing, and "
"restrained emotional force. The face is shown clearly in close framing, and the mouth "
"movement is highly visible and tightly synchronized with the spoken line. At normal "
"speed, ID_A looks past the camera toward the road, then lowers his gaze for a brief "
"controlled breath before speaking. Lip, jaw, and breath timing remain closely aligned "
"with the line. In a low, steady male voice, ID_A says, \"I did not come back for a clean "
"drive. I came back because this mountain still owes me an answer.\" Realistic motorsport "
"film language with cool daylight, hard facial detail, and restrained cinematic tension. "
"The tone is serious, professional, and emotionally loaded rather than flashy. Stable "
"cinematic close-up centered on ID_A’s face and upper shoulders for strong identity "
"anchoring and clear lip-sync. The background shows a mountain rally start zone with "
"safety barriers, blurred crew members, a dust-coated race car, open air, and a winding "
"road disappearing into pale rocky terrain. Engine idling in the background, faint wind "
"over the mountain road, distant metallic clicks from the cooling car body, and soft "
"fabric movement from the racing suit are audible. A very low cinematic tension pulse may "
"be present, but it stays far beneath the spoken line and mechanical ambience."
)
RALLY_2 = (
"ID_A is an American male racing driver in his thirties with short black hair, light "
"stubble, and an intensely focused expression. ID_A wears a full rally suit with sponsor "
"patches and tightens his gloves with practiced speed. ID_A’s voice is a firm low male "
"voice with clipped diction and restrained intensity. His face remains visible enough for "
"strong identity continuity, and the mouth movement stays naturally synchronized with the "
"spoken line. At normal speed, ID_A tightens both gloves, glances once toward the "
"driver-side door, and speaks without wasting motion. The mouth movement stays clearly "
"synchronized. In a low controlled male voice, ID_A says, \"If I leave anything unfinished "
"in the first section, the road will take it from me later.\" Realistic and tactile "
"rally-film detail, emphasizing gloves, suit fabric, and body tension over broad "
"spectacle. The mood is concentrated and severe. Medium-close shot following glove "
"tightening and upper-body preparation while preserving facial readability and lip-sync. "
"The race car fills part of the background, along with mountain haze, service tents, "
"blurred crew, and safety markers at the start line. Glove strap tightening, zipper "
"movement, engine idle, wind across metal surfaces, and a faint crunch of gravel "
"underfoot are audible. A restrained low-frequency cinematic pulse remains secondary to "
"the live motorsport soundscape."
)