Prompt samples for p-video-2

app.py

@@ -2513,6 +2513,10 @@ video_combined_dir = _resolve_data_path(
     data_dir / "video_editing_combined",
     space_root.parent / "video_editing_combined",
 )
+text_to_video_combined_dir = _resolve_data_path(
+    data_dir / "video_generation_combined",
+    space_root.parent / "video_generation_combined",
+)
 qwen_path = _resolve_data_path(
     data_dir / "qwen_image_bench_model_price_and_median_generation_time_10_august.csv",
     space_root.parent / "qwen_image_bench_model_price_and_median_generation_time_10_august.csv",
@@ -2599,6 +2603,7 @@ text_to_video_display_columns = [
 oneig_samples = load_sample_comparison_data(oneig_combined_dir)
 qwen_samples = load_sample_comparison_data(qwen_combined_dir)
 video_samples = load_sample_comparison_data(video_combined_dir)
+text_to_video_samples = load_sample_comparison_data(text_to_video_combined_dir)
 
 metrics = [
     {"id": "datapoint_elo", "column": "Datapoint Elo"},
@@ -2676,7 +2681,7 @@ datasets = [
             "second of video is Fal wall time, except Pruna models which use "
             "model execution time."
         ),
-        "samples": None,
+        "samples": text_to_video_samples,
     },
     {
         "id": "video_editing",

data/video_generation_combined/generations.jsonl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8395b068f818174bbbbd0af450c6cedcc76142352f35ccf565f55c4837cb43b4
+size 1135969

data/video_generation_combined/prompts.jsonl

@@ -0,0 +1,90 @@
+{"prompt_id": "v-bench2__all__p0000", "text": "Garden, zoom in.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0001", "text": "Garden, zoom out.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0002", "text": "Garden, tilt up.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0003", "text": "The camera starts at the top of the forest, where thick morning mist drifts between the treetops, with sunlight filtering through the leaves and casting dappled spots of light, the air fresh and moist. As the camera slowly moves downward, dewdrops glisten on the leaves, a gentle breeze rustling the leaves, making a soft, soothing sound. The camera moves along a woodland path, occasionally capturing a squirrel leaping between trees, its agile figure darting across the branches. The shot continues, eventually arriving at a tranquil lake. The surface of the water is like a mirror, reflecting the towering trees and distant mountains. In the distance, a heron takes off from the lakeshore, breaking the stillness of the water, ripples forming as the bird skims the surface. The camera follows its flight and gradually pulls back, showing the entire serene beauty of the lake.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0004", "text": "The scene opens in the vast desert, with the camera angled low over the sand, fine grains drifting in the gentle breeze, the undulating dunes in the distance bathed in a faint orange glow. As the camera moves forward, the outlines of the dunes become clearer, and the early morning sky begins to glow with warm hues, transitioning the desert from deep brown to golden yellow. The scene cuts to the top of a dune, where the camera follows a caravan of camels making their way across the endless desert, their bells jingling softly in the wind. The first rays of the sun break through the gaps between the dunes, casting golden light on the sand, as the desert comes to life, glowing in the morning's embrace. Finally, the camera pulls back, revealing the vast expanse of the desert merging with the horizon, the sun climbing higher, illuminating the entire landscape in a mystical, warm glow.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0005", "text": "The camera begins in a vast grassland, where the lush green grass sways gently in the breeze, the air fresh, and the soft rustling of the leaves fills the space. As the camera moves forward, the grass turns from a vibrant green to golden hues, sunlight pouring over the land, the textures of the grass becoming more pronounced in the play of light and shadow. In the distance, a herd of cattle grazes peacefully. Their figures become clearer as the camera moves closer, occasionally lifting their heads and gazing into the distance. The scene shifts to a lakeshore, where the lake reflects the sky, grasslands, and distant mountains, the water clear and calm, with gentle ripples created by the breeze. Finally, the camera pulls back, following the herd as they move off into the distance, the vastness of the grassland merging with the expansive sky, creating a feeling of peaceful openness.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0006", "text": "The camera starts at a tranquil coastline, where the waves gently crash against the rocks, the salty sea breeze fills the air, and the atmosphere feels fresh and alive. As the camera moves forward, the surface of the water glistens with golden light from the setting sun, the sound of the waves becoming more distinct as the tide retreats, revealing a moist sandy shore. The scene shifts to the beach, where a few seagulls peck at the sand, occasionally flying up and breaking the stillness of the sky. The sun slowly sinks below the horizon, painting the sky in vibrant shades of orange and red, as the golden light on the water fades, and the waves begin to intensify. Finally, the camera pulls back, and the entire coastline fades into twilight, with the sea and sky blending together in a serene, solitary embrace.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0007", "text": "The race began, and the first runner quickly took off, leading the other teams. Everyone was focused on his performance. During the baton handoff of the first runner, Team A made a slight mistake, and the baton nearly fell. Despite this, the runner quickly passed it to the second runner. The second runner showed great determination and chased hard, finally overtaking Team B on the bend. However, due to the earlier mistake, Team A's lead was not significant. During the third runner's handoff, the runner from Team A nervously accelerated and managed to maintain the lead, but Team C quickly caught up thanks to their excellent pace control. During the fourth runner's handoff, the final sprint became crucial. The Team A runner started to accelerate in the second-to-last turn and, with a burst of strength, successfully widened the gap, steadily running toward the finish line and winning the race.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0008", "text": "The match began, and Team A quickly organized an attack, breaking through Team B's defense with fast passes. The forward accurately kicked the ball into the goal, taking a 1-0 lead. Team B did not panic; they used a long pass to quickly counterattack. The forward calmly shot past the goalkeeper, and the ball went straight into the net, making it 1-1. At this point, the game entered a stalemate. Both teams' defenses were solid, and the match became a fierce battle. In the second half, Team A had a corner kick. The high ball delivered by the player was headed in by the center-back who jumped high, putting Team A in the lead at 2-1. In the final moments of the game, Team B got a penalty kick. The forward took the shot without hesitation, sending the ball into the net to make it 2-2. Finally, in the last moments of extra time, Team A used a quick counterattack and a long-range shot from outside the box flew straight into the corner, securing the win with a 3-2 score.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0009", "text": "The race began, and all the runners sprinted quickly. The runner from Team A quickly took the lead. After a period of steady running, the runners from Team A and Team B gradually created a gap, almost running shoulder to shoulder. However, at the 25 km mark, the runner from Team A suddenly began to feel unwell, slowing down noticeably. The runner from Team B seized the opportunity and caught up, eventually overtaking Team A to take the lead at the 30 km mark. As the race entered the second half, the Team A runner gave it their all, regained their rhythm, and caught up with Team B at the 35 km mark. The two were neck and neck in the final sprint, and with only 200 meters remaining, the Team A runner pushed through with determination and accelerated to cross the finish line first by a narrow margin, winning the tough race.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0010", "text": "The race began, and the runners quickly started. The Team A runner took the lead initially due to a powerful start. However, the Team B runner did not rush to chase but instead steadily adjusted their pace, ensuring they had more energy for the latter part of the race. By the third lap, the Team A runner began to tire, and the Team B runner gradually reduced the gap, eventually overtaking Team A in the fourth lap. The Team C runner, meanwhile, began to accelerate, and in the last two laps, with strong willpower, they overtook Team B and moved into the lead. In the final sprint, the Team A runner gritted their teeth and tried to close the gap, but with only 50 meters left, the Team C runner exploded with speed, crossing the finish line with a clear lead and winning the race.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0011", "text": "The match began, and Team A quickly found their rhythm. A series of fast attacks put pressure on Team B’s defense. Team A's forward made two three-pointers, quickly increasing the lead. After the first quarter, Team B adjusted their tactics, strengthened their defense, and gradually started to counterattack, narrowing the gap. In the third quarter, Team A’s key player was injured and forced to leave the game. Team B took advantage of this by speeding up their attacks, overtaking the score. In the fourth quarter, Team A's bench players caught up with a series of fast breaks, and in the final moments, a player from Team A calmly made a game-winning three-pointer from beyond the arc, securing a narrow 1-point victory.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0012", "text": "The match began, and Team A quickly gained the upper hand with several precise kills, taking a 11-6 lead. The Team B player stayed calm and gradually adjusted, finding ways to respond. In the second set, Team B improved their serving quality, scoring several powerful smashes to win a set, leveling the score at 1-1. In the deciding set, the Team A player showed signs of fatigue, but with precise net control and quick reflexes, they pulled ahead. In the crucial final point, a lightning-fast smash left Team B with no chance to return, and Team A won the match 21-19.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0013", "text": "A lion with the wings of an eagle, soaring through the sky with majestic ease.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0014", "text": "A giraffe with the scales of a fish, able to glide smoothly through the water.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0015", "text": "A wolf with the body of a horse, galloping across a vast plains with wild abandon.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0016", "text": "A bear with the antlers of a deer, roaming the forest with a regal presence.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0017", "text": "A cheetah with the shell of a tortoise, moving quickly but with a protective outer layer.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0018", "text": "A wooden toy is placed gently on the surface of a small bowl of water.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0019", "text": "A river changes from blue to brown.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0020", "text": "The leaves gradually change from red to green.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0021", "text": "A river changes from brown to blue.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0022", "text": "A car changes from white to red.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0023", "text": "A car changes from red to white.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0024", "text": "A dog is on the left of a table, then the dog runs to the front of the table.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0025", "text": "A dog is on the left of a sofa, then the dog runs to the front of the sofa.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0026", "text": "A dog is on the right of a table, then the dog runs to the left of the table.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0027", "text": "A dog is on the right of a sofa, then the dog runs to the front of the sofa.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0028", "text": "A dog is on the right of a rock, then the dog runs to the left of the rock.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0029", "text": "A dog is behind a chair, then the dog runs to the right of the chair.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0030", "text": "A man is doing yoga.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0031", "text": "A woman is doing yoga.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0032", "text": "people are doing yoga.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0033", "text": "A man is running.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0034", "text": "A woman is running.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0035", "text": "people are running.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0036", "text": "A man is walking.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0037", "text": "A woman is walking.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0038", "text": "people are walking.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0039", "text": "A man is dancing.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0040", "text": "A woman is dancing.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0041", "text": "A man is playing basketball.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0042", "text": "A woman is playing basketball.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0043", "text": "One person hands a cup of water to another.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0044", "text": "One person passes a ball to another.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0045", "text": "Two people shake hands.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0046", "text": "One person ties the shoelaces of another person.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0047", "text": "One person opens the door for another person.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0048", "text": "Two people exchange a book.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0049", "text": "One person puts a coat on another person.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0050", "text": "One person places a chair for another to sit in.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0051", "text": "An orange dog is running.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0052", "text": "Two orange dogs are running.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0053", "text": "A brown dog is on the left of an apple, then the dog moves to the right of the apple.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0054", "text": "An orange cat is running.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0055", "text": "Equal amounts of yellow and blue paint are rapidly combined, with the mixture being vigorously stirred until fully blended.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0056", "text": "Equal amounts of black and white paint are rapidly combined, with the mixture being vigorously stirred until fully blended.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0057", "text": "Equal amounts of white and black paint are rapidly combined, with the mixture being vigorously stirred until fully blended.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0058", "text": "Equal amounts of red and purple paint are rapidly combined, with the mixture being vigorously stirred until fully blended.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0059", "text": "A bowl of soup is tilted in the space station, with the liquid slowly spreading in all directions.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0060", "text": "A jar of peanut butter is opened in the space station, with the viscous liquid slowly dispersing.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0061", "text": "A container of cream is opened in the space station, with the liquid slowly dispersing into the air.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0062", "text": "A cup of tea is carefully tilted in the space station, and the liquid floats in various directions.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0063", "text": "A bottle of ketchup is gently squeezed in the space station, with the thick liquid spreading into the environment.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0064", "text": "A bottle of water is opened in the space station, and the water starts to float out in irregular shapes.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0065", "text": "A person is sitting on the couch, then suddenly they get up and start sweeping the floor.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0066", "text": "A dog is playing with a ball, then it suddenly starts lying down on the carpet.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0067", "text": "A person is cooking dinner, then they suddenly start organizing the pantry.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0068", "text": "A cat is watching birds through the window, then it suddenly starts grooming itself.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0069", "text": "A person is typing on a keyboard, then they suddenly get up and start making the bed.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0070", "text": "A horse is trotting in the field, then it suddenly starts drinking from a stream.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0071", "text": "A person is drinking a glass of water, then they suddenly start cleaning the windows.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0072", "text": "A dog is sitting in the yard, then it suddenly starts running in circles.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0073", "text": "A person is slurping noodles from a steaming bowl.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0074", "text": "A person is eating hamburger.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0075", "text": "A person is eating ice cream.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0076", "text": "A person is drinking coffee from a cup.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0077", "text": "A person is spreading butter on toast.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0078", "text": "A person is eating spaghetti with a fork.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0079", "text": "A person is biting into an apple.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0080", "text": "A person is peeling a banana.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0081", "text": "The camera orbits around. Castle, the camera circles around.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0082", "text": "The camera orbits around. Volcano, the camera circles around.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0083", "text": "The camera orbits around. Statue, the camera circles around.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0084", "text": "The camera orbits around. Clock Tower, the camera circles around.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0085", "text": "The camera orbits around. Playground, the camera circles around.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0086", "text": "A timelapse captures the transformation of water in an untextured bottle as the temperature significantly drops below 0°C.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0087", "text": "A timelapse captures the transformation of a river as the temperature significantly drops below 0°C.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0088", "text": "A timelapse captures the transformation of juice in an untextured bottle as the temperature significantly drops below 0°C.", "dataset": "v-bench2", "category": "all"}
+{"prompt_id": "v-bench2__all__p0089", "text": "A timelapse captures the transformation of milk in an untextured bottle as the temperature significantly drops below 0°C.", "dataset": "v-bench2", "category": "all"}

model_display.py

@@ -103,6 +103,7 @@ MODEL_DISPLAY_NAMES = {
     "ltx_2_5_pro": "LTX 2.5 Pro",
     "minimax_h3": "MiniMax H3",
     "minimax_h3_max": "MiniMax H3 Max",
+    "minimax_h3_max_turbo": "MiniMax H3 Max Turbo",
     "minimax_h3_max_turbo__prompt_expansion_mode_balanced": "MiniMax H3 Max Turbo",
     "seedance_2_5_turbo": "Seedance 2.5 Turbo",
     "veo_3_1_lite": "Veo 3.1 Lite",

ui.py

@@ -88,9 +88,9 @@ and price**. Each view is a **dataset** scored with a **metric**, written as
    and lower price (or time). Only datasets with price or generation time
    can open this tab (not Arena AI).
 4. **Samples**: the same prompts, side by side. Only for datasets we
-   generated (Qwen Image Dataset, OneIG Alignment Dataset, and the
-   Pruna Internal Video-Edit Benchmark). Video-edit samples show the source
-   clip first, then each model's edit.
+   generated (VBench-2.0 Dataset, Qwen Image Dataset, OneIG Alignment
+   Dataset, and the Pruna Internal Video-Edit Benchmark). Video-edit
+   samples show the source clip first, then each model's edit.
 
 ## How a score is made
 
@@ -113,8 +113,8 @@ prompt suites, so samples are not shown.
 VBench-2.0 prompts, comparing P-Video-2 variants with Fal-hosted models.
 Quality is Datapoint Elo and Rapidata Elo from pairwise preference. Price
 is USD per second of output video. Time per second of video is Fal wall
-time, except Pruna models which use model execution time. Samples are not
-shown for this dataset.
+time, except Pruna models which use model execution time. Samples are
+available.
 
 ### Pruna Internal Video-Edit Benchmark
 Pruna's internal video-to-video editing benchmark, collected by our