deploy: 63a85616f5fc427cf1e1e7b425293131f2fce2b8

README.md

@@ -8,4 +8,169 @@ sdk_version: 4.36.1
 app_file: app.py
 pinned: false
 ---
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+
+# Layout Utility
+
+## Description
+
+The Layout Utility metric evaluates how efficiently a layout utilizes suitable space for element placement. It measures the utilization rate of regions that are appropriate for arranging elements, as determined by the negative (inverted) saliency map. This metric helps assess whether layouts make effective use of available non-salient (less visually important) space.
+
+## What It Measures
+
+This metric computes:
+
+- **Space utilization**: How much of the suitable (low-saliency) space is occupied by elements
+- **Placement efficiency**: Whether elements are placed in appropriate regions
+- **Canvas coverage**: How well the layout fills available design space
+
+Higher scores indicate better utilization of suitable space, though extremely high scores might indicate overcrowding.
+
+## Metric Details
+
+- Uses the **negative/inverted saliency map** (S') to identify suitable placement regions
+- Suitable space = regions with LOW visual importance (complement of salient areas)
+- Computes the ratio of element-covered suitable space to total suitable space
+- From PosterLayout (Hsu et al., CVPR 2023) for content-aware poster design
+- Higher utility means elements effectively use non-salient space
+
+## Usage
+
+### Installation
+
+```bash
+pip install evaluate
+```
+
+### Basic Example
+
+```python
+import evaluate
+import numpy as np
+
+# Load the metric with canvas dimensions
+metric = evaluate.load(
+    "creative-graphic-design/layout-utility",
+    canvas_width=360,
+    canvas_height=504
+)
+
+# Prepare data
+predictions = np.random.rand(1, 25, 4)  # normalized ltrb coordinates
+gold_labels = np.random.randint(0, 4, size=(1, 25))  # class labels
+# Paths to saliency map images (grayscale, 0-255)
+saliency_maps_1 = ["path/to/saliency_map_1.png"]
+saliency_maps_2 = ["path/to/saliency_map_2.png"]
+
+score = metric.compute(
+    predictions=predictions,
+    gold_labels=gold_labels,
+    saliency_maps_1=saliency_maps_1,
+    saliency_maps_2=saliency_maps_2
+)
+print(score)
+```
+
+### Batch Processing Example
+
+```python
+import evaluate
+
+# Load the metric
+metric = evaluate.load(
+    "creative-graphic-design/layout-utility",
+    canvas_width=360,
+    canvas_height=504
+)
+
+# Batch processing
+batch_size = 128
+predictions = np.random.rand(batch_size, 25, 4)
+gold_labels = np.random.randint(0, 4, size=(batch_size, 25))
+saliency_maps_1 = [f"path/to/saliency_{i}_1.png" for i in range(batch_size)]
+saliency_maps_2 = [f"path/to/saliency_{i}_2.png" for i in range(batch_size)]
+
+score = metric.compute(
+    predictions=predictions,
+    gold_labels=gold_labels,
+    saliency_maps_1=saliency_maps_1,
+    saliency_maps_2=saliency_maps_2
+)
+print(score)
+```
+
+## Parameters
+
+### Initialization Parameters
+
+- **canvas_width** (`int`, required): Width of the canvas in pixels
+- **canvas_height** (`int`, required): Height of the canvas in pixels
+
+### Computation Parameters
+
+- **predictions** (`list` of `lists` of `float`): Normalized bounding boxes in ltrb format (0.0 to 1.0)
+- **gold_labels** (`list` of `lists` of `int`): Class labels for each element (0 = padding)
+- **saliency_maps_1** (`list` of `str`): File paths to first set of saliency map images
+- **saliency_maps_2** (`list` of `str`): File paths to second set of saliency map images
+
+**Note**:
+
+- Saliency maps should be grayscale images (0-255) where brighter = more salient/important
+- The metric internally inverts these to find suitable (non-salient) placement regions
+- Very small elements (< 0.1% of canvas) are filtered out
+
+## Returns
+
+Returns a `float` value representing the utilization rate of suitable space (range: 0.0 to 1.0).
+
+## Interpretation
+
+- **Higher is generally better** (range: 0.0 to 1.0)
+- **Value ~1.0**: Very high utilization of suitable space (may indicate dense layout)
+- **Value 0.6-0.9**: Good space utilization
+- **Value 0.3-0.6**: Moderate utilization, room for more elements
+- **Value 0.0-0.3**: Low utilization, underused suitable space
+- **Value ~0.0**: Very sparse layout, most suitable space unused
+
+### Use Cases
+
+- **Content-aware layout generation**: Evaluate if layouts efficiently use available space
+- **Poster/flyer design**: Assess whether designs make good use of non-salient regions
+- **Space efficiency analysis**: Compare layout density across different designs
+- **Design quality**: Identify layouts that are too sparse or potentially too dense
+
+### Key Insights
+
+- **Balance is important**: Neither too sparse nor too dense is ideal
+- **Context-dependent**: Optimal utility varies by design type (minimalist vs. information-dense)
+- **Suitable space concept**: Based on inverted saliency - where elements CAN go, not where they SHOULD avoid
+- **Combine with occlusion**: High utility + low occlusion = good space utilization without blocking important content
+- **Not absolute quality**: High utility alone doesn't guarantee good design
+
+### Interpretation Considerations
+
+- Very high utility (>0.9) might indicate overcrowding
+- Very low utility (<0.2) might indicate wasteful use of space
+- Optimal range typically 0.4-0.8 depending on design intent
+
+## Citations
+
+```bibtex
+@inproceedings{hsu2023posterlayout,
+  title={Posterlayout: A new benchmark and approach for content-aware visual-textual presentation layout},
+  author={Hsu, Hsiao Yuan and He, Xiangteng and Peng, Yuxin and Kong, Hao and Zhang, Qing},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+  pages={6018--6026},
+  year={2023}
+}
+```
+
+## References
+
+- **Paper**: [PosterLayout (Hsu et al., CVPR 2023)](https://arxiv.org/abs/2303.15937)
+- **Reference Implementation**: [PosterLayout eval.py](https://github.com/PKU-ICST-MIPL/PosterLayout-CVPR2023/blob/main/eval.py#L144-L171)
+
+## Related Metrics
+
+- [Layout Occlusion](../layout_occlusion/): Evaluates coverage of salient (important) regions
+- [Layout Validity](../layout_validity/): Checks basic validity constraints
+- [Layout Unreadability](../layout_unreadability/): Evaluates text placement quality

layout-utility.py

@@ -5,6 +5,7 @@ import datasets as ds
 import evaluate
 import numpy as np
 import numpy.typing as npt
+from evaluate.utils.file_utils import add_start_docstrings
 from PIL import Image
 
 _DESCRIPTION = r"""\
@@ -26,6 +27,7 @@ _CITATION = """\
 """
 
 
+@add_start_docstrings(_DESCRIPTION, _KWARGS_DESCRIPTION)
 class LayoutUtility(evaluate.Metric):
     def __init__(
         self,
@@ -64,10 +66,10 @@ class LayoutUtility(evaluate.Metric):
             filepath = filepath[0]
 
         map_pil = Image.open(filepath)  # type: ignore
-        map_pil = map_pil.convert("L")
+        map_pil = map_pil.convert("L")  # type: ignore
 
         if map_pil.size != (self.canvas_width, self.canvas_height):
-            map_pil = map_pil.resize((self.canvas_width, self.canvas_height))
+            map_pil = map_pil.resize((self.canvas_width, self.canvas_height))  # type: ignore
 
         map_arr = np.array(map_pil)
         map_arr = map_arr / 255.0
@@ -111,7 +113,7 @@ class LayoutUtility(evaluate.Metric):
             predictions=predictions, gold_labels=gold_labels
         )
 
-        score = 0
+        score = []
 
         assert (
             len(predictions)
@@ -119,7 +121,6 @@ class LayoutUtility(evaluate.Metric):
             == len(saliency_maps_1)
             == len(saliency_maps_2)
         )
-        num_predictions = len(predictions)
         it = zip(predictions, gold_labels, saliency_maps_1, saliency_maps_2)
 
         for prediction, gold_label, smap_1, smap_2 in it:
@@ -145,5 +146,8 @@ class LayoutUtility(evaluate.Metric):
             total_utils = np.sum(c_smap * cal_mask)
 
             if total_not_sal and total_utils:
-                score += total_utils / total_not_sal
-        return score / num_predictions
+                # score += total_utils / total_not_sal
+                score.append(total_utils / total_not_sal)
+
+        # return score / num_predictions
+        return np.mean(score)

requirements.txt

@@ -1,89 +1,136 @@
-aiofiles==23.2.1 ; python_version >= "3.9" and python_version < "4.0"
-aiohttp==3.9.3 ; python_version >= "3.9" and python_version < "4.0"
-aiosignal==1.3.1 ; python_version >= "3.9" and python_version < "4.0"
-altair==5.2.0 ; python_version >= "3.9" and python_version < "4.0"
-annotated-types==0.6.0 ; python_version >= "3.9" and python_version < "4.0"
-anyio==4.2.0 ; python_version >= "3.9" and python_version < "4.0"
-arrow==1.3.0 ; python_version >= "3.9" and python_version < "4.0"
-async-timeout==4.0.3 ; python_version >= "3.9" and python_version < "3.11"
-attrs==23.2.0 ; python_version >= "3.9" and python_version < "4.0"
-binaryornot==0.4.4 ; python_version >= "3.9" and python_version < "4.0"
-certifi==2024.2.2 ; python_version >= "3.9" and python_version < "4.0"
-chardet==5.2.0 ; python_version >= "3.9" and python_version < "4.0"
-charset-normalizer==3.3.2 ; python_version >= "3.9" and python_version < "4.0"
-click==8.1.7 ; python_version >= "3.9" and python_version < "4.0"
-colorama==0.4.6 ; python_version >= "3.9" and python_version < "4.0"
-contourpy==1.2.0 ; python_version >= "3.9" and python_version < "4.0"
-cookiecutter==2.5.0 ; python_version >= "3.9" and python_version < "4.0"
-cycler==0.12.1 ; python_version >= "3.9" and python_version < "4.0"
-datasets==2.17.0 ; python_version >= "3.9" and python_version < "4.0"
-dill==0.3.8 ; python_version >= "3.9" and python_version < "4.0"
-evaluate[template]==0.4.1 ; python_version >= "3.9" and python_version < "4.0"
-exceptiongroup==1.2.0 ; python_version >= "3.9" and python_version < "3.11"
-fastapi==0.109.2 ; python_version >= "3.9" and python_version < "4.0"
-ffmpy==0.3.1 ; python_version >= "3.9" and python_version < "4.0"
-filelock==3.13.1 ; python_version >= "3.9" and python_version < "4.0"
-fonttools==4.48.1 ; python_version >= "3.9" and python_version < "4.0"
-frozenlist==1.4.1 ; python_version >= "3.9" and python_version < "4.0"
-fsspec==2023.10.0 ; python_version >= "3.9" and python_version < "4.0"
-fsspec[http]==2023.10.0 ; python_version >= "3.9" and python_version < "4.0"
-gradio-client==0.10.0 ; python_version >= "3.9" and python_version < "4.0"
-gradio==4.18.0 ; python_version >= "3.9" and python_version < "4.0"
-h11==0.14.0 ; python_version >= "3.9" and python_version < "4.0"
-httpcore==1.0.2 ; python_version >= "3.9" and python_version < "4.0"
-httpx==0.26.0 ; python_version >= "3.9" and python_version < "4.0"
-huggingface-hub==0.20.3 ; python_version >= "3.9" and python_version < "4.0"
-idna==3.6 ; python_version >= "3.9" and python_version < "4.0"
-importlib-resources==6.1.1 ; python_version >= "3.9" and python_version < "4.0"
-jinja2==3.1.3 ; python_version >= "3.9" and python_version < "4.0"
-jsonschema-specifications==2023.12.1 ; python_version >= "3.9" and python_version < "4.0"
-jsonschema==4.21.1 ; python_version >= "3.9" and python_version < "4.0"
-kiwisolver==1.4.5 ; python_version >= "3.9" and python_version < "4.0"
-markdown-it-py==3.0.0 ; python_version >= "3.9" and python_version < "4.0"
-markupsafe==2.1.5 ; python_version >= "3.9" and python_version < "4.0"
-matplotlib==3.8.2 ; python_version >= "3.9" and python_version < "4.0"
-mdurl==0.1.2 ; python_version >= "3.9" and python_version < "4.0"
-multidict==6.0.5 ; python_version >= "3.9" and python_version < "4.0"
-multiprocess==0.70.16 ; python_version >= "3.9" and python_version < "4.0"
-numpy==1.26.4 ; python_version >= "3.9" and python_version < "4.0"
-orjson==3.9.13 ; python_version >= "3.9" and python_version < "4.0"
-packaging==23.2 ; python_version >= "3.9" and python_version < "4.0"
-pandas==2.2.0 ; python_version >= "3.9" and python_version < "4.0"
-pillow==10.2.0 ; python_version >= "3.9" and python_version < "4.0"
-pyarrow-hotfix==0.6 ; python_version >= "3.9" and python_version < "4.0"
-pyarrow==15.0.0 ; python_version >= "3.9" and python_version < "4.0"
-pydantic-core==2.16.2 ; python_version >= "3.9" and python_version < "4.0"
-pydantic==2.6.1 ; python_version >= "3.9" and python_version < "4.0"
-pydub==0.25.1 ; python_version >= "3.9" and python_version < "4.0"
-pygments==2.17.2 ; python_version >= "3.9" and python_version < "4.0"
-pyparsing==3.1.1 ; python_version >= "3.9" and python_version < "4.0"
-python-dateutil==2.8.2 ; python_version >= "3.9" and python_version < "4.0"
-python-multipart==0.0.9 ; python_version >= "3.9" and python_version < "4.0"
-python-slugify==8.0.4 ; python_version >= "3.9" and python_version < "4.0"
-pytz==2024.1 ; python_version >= "3.9" and python_version < "4.0"
-pyyaml==6.0.1 ; python_version >= "3.9" and python_version < "4.0"
-referencing==0.33.0 ; python_version >= "3.9" and python_version < "4.0"
-requests==2.31.0 ; python_version >= "3.9" and python_version < "4.0"
-responses==0.18.0 ; python_version >= "3.9" and python_version < "4.0"
-rich==13.7.0 ; python_version >= "3.9" and python_version < "4.0"
-rpds-py==0.17.1 ; python_version >= "3.9" and python_version < "4.0"
-ruff==0.2.1 ; python_version >= "3.9" and python_version < "4.0"
-semantic-version==2.10.0 ; python_version >= "3.9" and python_version < "4.0"
-shellingham==1.5.4 ; python_version >= "3.9" and python_version < "4.0"
-six==1.16.0 ; python_version >= "3.9" and python_version < "4.0"
-sniffio==1.3.0 ; python_version >= "3.9" and python_version < "4.0"
-starlette==0.36.3 ; python_version >= "3.9" and python_version < "4.0"
-text-unidecode==1.3 ; python_version >= "3.9" and python_version < "4.0"
-tomlkit==0.12.0 ; python_version >= "3.9" and python_version < "4.0"
-toolz==0.12.1 ; python_version >= "3.9" and python_version < "4.0"
-tqdm==4.66.2 ; python_version >= "3.9" and python_version < "4.0"
-typer[all]==0.9.0 ; python_version >= "3.9" and python_version < "4.0"
-types-python-dateutil==2.8.19.20240106 ; python_version >= "3.9" and python_version < "4.0"
-typing-extensions==4.9.0 ; python_version >= "3.9" and python_version < "4.0"
-tzdata==2024.1 ; python_version >= "3.9" and python_version < "4.0"
-urllib3==2.2.0 ; python_version >= "3.9" and python_version < "4.0"
-uvicorn==0.27.1 ; python_version >= "3.9" and python_version < "4.0"
-websockets==11.0.3 ; python_version >= "3.9" and python_version < "4.0"
-xxhash==3.4.1 ; python_version >= "3.9" and python_version < "4.0"
-yarl==1.9.4 ; python_version >= "3.9" and python_version < "4.0"
-zipp==3.17.0 ; python_version >= "3.9" and python_version < "3.10"
+# This file was autogenerated by uv via the following command:
+#    uv export --package layout_utility --no-dev --no-hashes --format requirements-txt
+aiohappyeyeballs==2.6.1
+    # via aiohttp
+aiohttp==3.13.2
+    # via fsspec
+aiosignal==1.4.0
+    # via aiohttp
+anyio==4.12.0
+    # via httpx
+attrs==25.4.0
+    # via aiohttp
+certifi==2025.11.12
+    # via
+    #   httpcore
+    #   httpx
+    #   requests
+charset-normalizer==3.4.4
+    # via requests
+click==8.3.1
+    # via typer-slim
+colorama==0.4.6 ; sys_platform == 'win32'
+    # via
+    #   click
+    #   tqdm
+datasets==4.4.2
+    # via evaluate
+dill==0.4.0
+    # via
+    #   datasets
+    #   evaluate
+    #   multiprocess
+evaluate==0.4.6
+    # via layout-utility
+filelock==3.20.1
+    # via
+    #   datasets
+    #   huggingface-hub
+frozenlist==1.8.0
+    # via
+    #   aiohttp
+    #   aiosignal
+fsspec==2025.10.0
+    # via
+    #   datasets
+    #   evaluate
+    #   huggingface-hub
+h11==0.16.0
+    # via httpcore
+hf-xet==1.2.0 ; platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'
+    # via huggingface-hub
+httpcore==1.0.9
+    # via httpx
+httpx==0.28.1
+    # via
+    #   datasets
+    #   huggingface-hub
+huggingface-hub==1.2.3
+    # via
+    #   datasets
+    #   evaluate
+idna==3.11
+    # via
+    #   anyio
+    #   httpx
+    #   requests
+    #   yarl
+multidict==6.7.0
+    # via
+    #   aiohttp
+    #   yarl
+multiprocess==0.70.18
+    # via
+    #   datasets
+    #   evaluate
+numpy==2.2.6
+    # via
+    #   datasets
+    #   evaluate
+    #   pandas
+packaging==25.0
+    # via
+    #   datasets
+    #   evaluate
+    #   huggingface-hub
+pandas==2.3.3
+    # via
+    #   datasets
+    #   evaluate
+pillow==12.0.0
+    # via layout-utility
+propcache==0.4.1
+    # via
+    #   aiohttp
+    #   yarl
+pyarrow==22.0.0
+    # via datasets
+python-dateutil==2.9.0.post0
+    # via pandas
+pytz==2025.2
+    # via pandas
+pyyaml==6.0.3
+    # via
+    #   datasets
+    #   huggingface-hub
+requests==2.32.5
+    # via
+    #   datasets
+    #   evaluate
+shellingham==1.5.4
+    # via huggingface-hub
+six==1.17.0
+    # via python-dateutil
+tqdm==4.67.1
+    # via
+    #   datasets
+    #   evaluate
+    #   huggingface-hub
+typer-slim==0.21.0
+    # via huggingface-hub
+typing-extensions==4.15.0
+    # via
+    #   aiosignal
+    #   anyio
+    #   huggingface-hub
+    #   typer-slim
+tzdata==2025.3
+    # via pandas
+urllib3==2.6.2
+    # via requests
+xxhash==3.6.0
+    # via
+    #   datasets
+    #   evaluate
+yarl==1.22.0
+    # via aiohttp