Spaces:
Runtime error
A newer version of the Gradio SDK is available:
6.4.0
Evaluation Results
Summary Statistics
- Total Test Cases: 20
- Average Score: 90.8%
- Score Range: 70.0% - 100.0%
- Average Generation Time: 271.7s (~4.5 minutes)
- Time Range: 259.0s - 289.7s
Score Distribution
| Category | Count | Percentage |
|---|---|---|
| 🏆 Excellent (≥90%) | 13 | 65.0% |
| ✓ Good (80-89%) | 3 | 15.0% |
| ⚠️ Acceptable (70-79%) | 4 | 20.0% |
| ✗ Poor (<70%) | 0 | 0.0% |
Component-wise Average Scores (Before Weighting)
| Component | Average Score | Weight |
|---|---|---|
| Room Counts | 75.0% | 30% |
| Room Presence | 97.4% | 20% |
| Adjacency | 97.5% | 30% |
| Constraints | 97.7% | 20% |
Complete Test Results
| Test ID | Score | Gen Time | Room Counts | Room Presence | Adjacency | Constraints |
|---|---|---|---|---|---|---|
| 🏆 two_bedroom_apt_01 | 100.0% | 262.2s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 open_plan_loft_01 | 100.0% | 274.7s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 family_home_01 | 100.0% | 276.7s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 master_suite_home_01 | 100.0% | 265.3s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 four_bedroom_house_01 | 100.0% | 270.8s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 guest_suite_home_01 | 100.0% | 282.2s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 dual_master_suite_01 | 100.0% | 269.5s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 balcony_apartment_01 | 100.0% | 285.2s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 multigenerational_home_01 | 100.0% | 289.7s | 100.0% | 100.0% | 100.0% | 100.0% |
| 🏆 single_floor_accessible_01 | 95.0% | 262.5s | 83.3% | 100.0% | 100.0% | 100.0% |
| 🏆 three_bed_family_townhouse_01 | 95.0% | 268.3s | 83.3% | 100.0% | 100.0% | 100.0% |
| 🏆 entertainment_home_01 | 90.8% | 274.9s | 83.3% | 100.0% | 100.0% | 79.2% |
| 🏆 basic_studio_01 | 90.0% | 259.0s | 66.7% | 100.0% | 100.0% | 100.0% |
| ✓ luxury_penthouse_01 | 87.5% | 270.8s | 83.3% | 87.5% | 100.0% | 75.0% |
| ✓ one_bedroom_apt_01 | 85.0% | 270.0s | 50.0% | 100.0% | 100.0% | 100.0% |
| ✓ separated_zones_01 | 85.0% | 285.7s | 100.0% | 100.0% | 50.0% | 100.0% |
| ⚠️ home_office_layout_01 | 73.5% | 267.8s | 25.0% | 80.0% | 100.0% | 100.0% |
| ⚠️ work_from_home_layout_01 | 73.5% | 263.0s | 25.0% | 80.0% | 100.0% | 100.0% |
| ⚠️ compact_efficiency_01 | 70.0% | 269.7s | 0.0% | 100.0% | 100.0% | 100.0% |
| ⚠️ student_apartment_01 | 70.0% | 266.0s | 0.0% | 100.0% | 100.0% | 100.0% |
Detailed Results
🏆 two_bedroom_apt_01 - 100.0%
Description: Standard 2BR family apartment
Prompt: Create a spacious two-bedroom apartment (100m² total) with living room (35m²), kitchen (18m²), dining room (15m²), two bedrooms (25m² and 20m²), and two bathrooms. Kitchen should connect to dining room.
Generation Time: 262.2s (~4.4 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 open_plan_loft_01 - 100.0%
Description: Modern loft with open concept
Prompt: Design a modern open-plan loft (85m² total) with spacious combined living/dining area (40m²), open kitchen (18m²), one bedroom (22m²) with ensuite bathroom (5m²). Contemporary style.
Generation Time: 274.7s (~4.6 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 family_home_01 - 100.0%
Description: Family home with multiple bedrooms
Prompt: Create a large family home (140m² total) with living room (40m²), dining room (20m²), kitchen (20m²), three bedrooms (30m², 25m², 20m²), and two bathrooms (8m² each). Kitchen adjacent to dining room.
Generation Time: 276.7s (~4.6 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 master_suite_home_01 - 100.0%
Description: Home with master suite
Prompt: Design a comfortable home (110m² total) with living room (35m²), kitchen (18m²), spacious master bedroom (35m²) with ensuite bathroom (8m²), and one additional bedroom (20m²). Master bedroom must be adjacent to its bathroom.
Generation Time: 265.3s (~4.4 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 four_bedroom_house_01 - 100.0%
Description: Large family house
Prompt: Create a large family house (180m² total) with spacious living room (45m²), dining room (25m²), kitchen (22m²), four bedrooms (35m², 30m², 25m², 20m²), and three bathrooms (10m², 8m², 6m²). Living room and dining room should be adjacent. Kitchen adjacent to dining room. Traditional family layout.
Generation Time: 270.8s (~4.5 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 guest_suite_home_01 - 100.0%
Description: Home with separated guest suite
Prompt: Create a luxury home (150m² total) with main living areas (living room 40m², kitchen 20m², dining room 20m²), spacious master bedroom (35m²) with bathroom (10m²), and a separate guest suite with bedroom (25m²) and its own bathroom (8m²). Master and guest bedrooms each adjacent to their respective bathrooms.
Generation Time: 282.2s (~4.7 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 dual_master_suite_01 - 100.0%
Description: Dual master suite configuration
Prompt: Design a luxury apartment (160m² total) with two master bedrooms (40m² and 35m²), each with its own ensuite bathroom (12m² and 10m²). Include spacious living room (45m²), elegant dining room (22m²), and gourmet kitchen (25m²). High-end finishes throughout.
Generation Time: 269.5s (~4.5 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 balcony_apartment_01 - 100.0%
Description: Apartment with outdoor space
Prompt: Design a modern apartment (80m² total) with living room (32m²), kitchen (14m²), bedroom (22m²), bathroom (6m²), and a balcony (6m²) accessible from living room. Indoor-outdoor living style.
Generation Time: 285.2s (~4.8 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 multigenerational_home_01 - 100.0%
Description: Tests complex multi-suite layouts
Prompt: Design a spacious multigenerational home (200m² total) with main living room (45m²), kitchen (22m²), dining room (20m²), master bedroom (35m²) with bathroom (10m²), two additional bedrooms (25m² each), shared bathroom (8m²), and a separate accessible suite for elderly parents with bedroom (30m²) and bathroom (10m²) adjacent to each other. Three-generation family layout.
Generation Time: 289.7s (~4.8 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 single_floor_accessible_01 - 95.0%
Description: Tests accessibility-friendly layouts
Prompt: Design an accessible barrier-free single-floor home (130m² total) with open living (38m²) and dining area (18m²), kitchen (20m²) adjacent to dining, spacious master bedroom (32m²) with accessible ensuite bathroom (10m²), one additional bedroom (25m²), and another accessible bathroom (8m²). Wide hallways and easy navigation for wheelchair access.
Generation Time: 262.5s (~4.4 minutes)
Component Scores:
- Room Counts: 83.3%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 three_bed_family_townhouse_01 - 95.0%
Description: Tests standard family townhouse layout
Prompt: Design a three-bedroom family townhouse (145m² total) with living room (40m²), separate dining room (18m²), kitchen (20m²) near dining room, three bedrooms (30m², 25m², 22m²), and two bathrooms (10m² and 7m²). Traditional townhouse layout for growing family.
Generation Time: 268.3s (~4.5 minutes)
Component Scores:
- Room Counts: 83.3%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
🏆 entertainment_home_01 - 90.8%
Description: Tests entertainment-focused layouts
Prompt: Create an entertainer's home (150m² total) with large open living (40m²) and dining area (22m²) open to each other, spacious gourmet kitchen (25m²) adjacent to dining, dedicated media/theater room (20m²), two bedrooms (25m² each), and two bathrooms (8m² each). Perfect for hosting guests.
Generation Time: 274.9s (~4.6 minutes)
Component Scores:
- Room Counts: 83.3%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 79.2%
🏆 basic_studio_01 - 90.0%
Description: Most basic layout - should score very high
Prompt: Create a compact studio apartment (45m² total) with cozy living area (20m²), small kitchen (10m²), and one bathroom (5m²). Minimalist layout.
Generation Time: 259.0s (~4.3 minutes)
Component Scores:
- Room Counts: 66.7%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
✓ luxury_penthouse_01 - 87.5%
Description: Tests luxury/high-end layouts with special rooms
Prompt: Create an ultra-luxury penthouse (220m² total) with expansive living room (60m²), formal dining room (28m²), gourmet kitchen (30m²) adjacent to dining, spacious master bedroom (45m²) with walk-in closet and luxurious ensuite bathroom (15m²), elegant study room (18m²), and comfortable guest bedroom (28m²) with guest bathroom (10m²). Premium materials and high ceilings throughout.
Generation Time: 270.8s (~4.5 minutes)
Component Scores:
- Room Counts: 83.3%
- Room Presence: 87.5%
- Adjacency: 100.0%
- Constraints: 75.0%
✓ one_bedroom_apt_01 - 85.0%
Description: Classic 1BR layout
Prompt: Design a comfortable one-bedroom apartment (70m² total) with living room (30m²), kitchen (15m²), bedroom (20m²), and bathroom (5m²). Open floor plan with living room adjacent to kitchen.
Generation Time: 270.0s (~4.5 minutes)
Component Scores:
- Room Counts: 50.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
✓ separated_zones_01 - 85.0%
Description: Functional separation of spaces
Prompt: Design a modern apartment (95m² total) with separate day and night zones: day zone has living room (35m²) and kitchen (15m²) adjacent to each other, night zone has two bedrooms (25m² and 20m²) and bathroom (8m²). Bedrooms should NOT be adjacent to kitchen. Contemporary functional layout.
Generation Time: 285.7s (~4.8 minutes)
Component Scores:
- Room Counts: 100.0%
- Room Presence: 100.0%
- Adjacency: 50.0%
- Constraints: 100.0%
⚠️ home_office_layout_01 - 73.5%
Description: Work-from-home layout
Prompt: Create a work-from-home apartment (90m² total) with living room (30m²), kitchen (15m²), bedroom (25m²), bathroom (6m²), and a dedicated home office room (14m²) for remote work. Modern professional layout.
Generation Time: 267.8s (~4.5 minutes)
Component Scores:
- Room Counts: 25.0%
- Room Presence: 80.0%
- Adjacency: 100.0%
- Constraints: 100.0%
⚠️ work_from_home_layout_01 - 73.5%
Description: Tests home office integration and zoning
Prompt: Create a professional work-from-home apartment (92m² total) with living room (30m²), kitchen (15m²), bedroom (24m²), bathroom (7m²), and dedicated home office (16m²) with good sound separation from living areas and bedroom. Quiet workspace prioritized.
Generation Time: 263.0s (~4.4 minutes)
Component Scores:
- Room Counts: 25.0%
- Room Presence: 80.0%
- Adjacency: 100.0%
- Constraints: 100.0%
⚠️ compact_efficiency_01 - 70.0%
Description: Tests compact/minimal space layouts
Prompt: Create a highly efficient compact apartment (48m² total) with small living room (18m²), compact galley kitchen (10m²) adjacent to living room, cozy bedroom (15m²), and efficient bathroom (5m²). Maximize every square meter.
Generation Time: 269.7s (~4.5 minutes)
Component Scores:
- Room Counts: 0.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%
⚠️ student_apartment_01 - 70.0%
Description: Tests minimal/basic student housing
Prompt: Design a budget-friendly student apartment (42m² total) with compact living area (15m² max), small kitchenette (10m² max), small bedroom (15m²), and basic bathroom (5m²). Affordable and functional layout for students.
Generation Time: 266.0s (~4.4 minutes)
Component Scores:
- Room Counts: 0.0%
- Room Presence: 100.0%
- Adjacency: 100.0%
- Constraints: 100.0%