deploy: 63a85616f5fc427cf1e1e7b425293131f2fce2b8

README.md

@@ -9,4 +9,146 @@ app_file: app.py
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Layout Overlay
+
+## Description
+
+The Layout Overlay metric measures the average IoU (Intersection over Union) of all pairs of layout elements, specifically excluding "underlay" or decoration elements. This metric is designed for poster and presentation layouts where underlay elements serve as backgrounds and should not be counted in overlap calculations.
+
+## What It Measures
+
+This metric computes:
+
+- **Non-underlay overlap**: IoU between all pairs of foreground elements (text, images, logos)
+- **Element collision**: How much non-decoration elements interfere with each other
+- **Foreground placement quality**: Whether foreground elements are properly spaced
+
+Underlay/decoration elements (like background shapes) are excluded from the calculation since they're intended to sit behind other elements.
+
+## Metric Details
+
+- Filters out decoration/underlay elements (typically class index 3 in PosterLayout)
+- Removes invalid elements (< 0.1% of canvas area)
+- Computes pairwise IoU for all remaining element pairs
+- Returns average IoU across all overlapping pairs
+- From PosterLayout (Hsu et al., CVPR 2023) for poster design evaluation
+
+## Usage
+
+### Installation
+
+```bash
+pip install evaluate
+```
+
+### Basic Example
+
+```python
+import evaluate
+import numpy as np
+
+# Load the metric with canvas dimensions
+metric = evaluate.load(
+    "creative-graphic-design/layout-overlay",
+    canvas_width=360,
+    canvas_height=504,
+    decoration_label_index=3  # underlay/decoration class
+)
+
+# Prepare data
+predictions = np.random.rand(1, 25, 4)  # normalized ltrb coordinates
+gold_labels = np.random.randint(0, 4, size=(1, 25))  # class labels
+score = metric.compute(predictions=predictions, gold_labels=gold_labels)
+print(score)
+```
+
+### Batch Processing Example
+
+```python
+import evaluate
+import numpy as np
+
+# Load the metric
+metric = evaluate.load(
+    "creative-graphic-design/layout-overlay",
+    canvas_width=360,
+    canvas_height=504,
+    decoration_label_index=3
+)
+
+# Batch processing
+batch_size = 128
+predictions = np.random.rand(batch_size, 25, 4)
+gold_labels = np.random.randint(0, 4, size=(batch_size, 25))
+score = metric.compute(predictions=predictions, gold_labels=gold_labels)
+print(score)
+```
+
+## Parameters
+
+### Initialization Parameters
+
+- **canvas_width** (`int`, required): Width of the canvas in pixels
+- **canvas_height** (`int`, required): Height of the canvas in pixels
+- **decoration_label_index** (`int`, optional, default=3): Class index for underlay/decoration elements to exclude
+
+### Computation Parameters
+
+- **predictions** (`list` of `lists` of `float`): Normalized bounding boxes in ltrb format (0.0 to 1.0)
+- **gold_labels** (`list` of `lists` of `int`): Class labels for each element (0 = padding)
+
+**Note**:
+
+- Elements with label == 0 are treated as padding
+- Elements with label == decoration_label_index are excluded (underlay)
+- Very small elements (< 0.1% of canvas) are filtered out
+
+## Returns
+
+Returns a `float` value representing the average IoU of overlapping element pairs (excluding underlay).
+
+## Interpretation
+
+- **Lower is better** (range: 0.0 to 1.0)
+- **Value of 0.0**: No overlap between foreground elements (ideal)
+- **Value of 0.1-0.3**: Minor overlap, possibly acceptable in dense layouts
+- **Value of 0.3-0.5**: Moderate overlap, may indicate placement issues
+- **Value > 0.5**: Significant overlap, likely problematic
+
+### Use Cases
+
+- **Poster/presentation layout evaluation**: Ensure foreground elements don't overlap excessively
+- **Content-aware design**: Evaluate layouts with distinct foreground and background layers
+- **Layered designs**: Assess foreground element placement independent of decoration layers
+- **Multi-layer layouts**: Focus on collision detection for primary content
+
+### Key Insights
+
+- **Underlay exclusion is important**: Decoration elements are meant to be behind others
+- **Context-specific**: Appropriate for designs with clear foreground/background separation
+- **Different from general overlap**: Focuses only on foreground element interactions
+- **Use with related metrics**: Combine with underlay effectiveness for full picture
+
+## Citations
+
+```bibtex
+@inproceedings{hsu2023posterlayout,
+  title={Posterlayout: A new benchmark and approach for content-aware visual-textual presentation layout},
+  author={Hsu, Hsiao Yuan and He, Xiangteng and Peng, Yuxin and Kong, Hao and Zhang, Qing},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+  pages={6018--6026},
+  year={2023}
+}
+```
+
+## References
+
+- **Paper**: [PosterLayout (Hsu et al., CVPR 2023)](https://arxiv.org/abs/2303.15937)
+- **Reference Implementation**: [PosterLayout eval.py](https://github.com/PKU-ICST-MIPL/PosterLayout-CVPR2023/blob/main/eval.py#L205-L222)
+
+## Related Metrics
+
+- [Layout Overlap](../layout_overlap/): General overlap metric for all elements
+- [Layout Underlay Effectiveness](../layout_underlay_effectiveness/): Evaluates underlay element placement
+- [Layout Average IoU](../layout_average_iou/): IoU-based overlap for all elements
+- [Layout Validity](../layout_validity/): Checks basic validity constraints

layout-overlay.py

@@ -4,13 +4,22 @@ import datasets as ds
 import evaluate
 import numpy as np
 import numpy.typing as npt
+from evaluate.utils.file_utils import add_start_docstrings
 
 _DESCRIPTION = r"""\
 Computes the average IoU of all pairs of elements except for underlay.
 """
 
 _KWARGS_DESCRIPTION = """\
-FIXME
+Args:
+    predictions (`list` of `lists` of `float`): A list of lists of floats representing normalized `ltrb`-format bounding boxes.
+    gold_labels (`list` of `lists` of `int`): A list of lists of integers representing class labels.
+
+Ruturns:
+    float: Average IoU except decoration (i.e., underlay) elements (used in PosterLayout).
+
+Examples::
+    FIXME
 """
 
 _CITATION = """\
@@ -24,16 +33,19 @@ _CITATION = """\
 """
 
 
+@add_start_docstrings(_DESCRIPTION, _KWARGS_DESCRIPTION)
 class LayoutOverlay(evaluate.Metric):
     def __init__(
         self,
         canvas_width: int,
         canvas_height: int,
+        decoration_label_index: int = 3,
         **kwargs,
     ) -> None:
         super().__init__(**kwargs)
         self.canvas_width = canvas_width
         self.canvas_height = canvas_height
+        self.decoration_label_index = decoration_label_index
 
     def _info(self) -> evaluate.EvaluationModuleInfo:
         return evaluate.MetricInfo(
@@ -114,8 +126,13 @@ class LayoutOverlay(evaluate.Metric):
 
         for gold_label, prediction in zip(gold_labels, predictions):
             ove = 0.0
-            mask = (gold_label > 0).reshape(-1) & (gold_label != 3).reshape(-1)
+
+            cond1 = (gold_label > 0).reshape(-1)
+            cond2 = (gold_label != self.decoration_label_index).reshape(-1)
+
+            mask = cond1 & cond2
             mask_box = prediction[mask]
+
             n = len(mask_box)
             for i in range(n):
                 bb1 = mask_box[i]

requirements.txt

@@ -1,89 +1,134 @@
-aiofiles==23.2.1 ; python_version >= "3.9" and python_version < "4.0"
-aiohttp==3.9.3 ; python_version >= "3.9" and python_version < "4.0"
-aiosignal==1.3.1 ; python_version >= "3.9" and python_version < "4.0"
-altair==5.2.0 ; python_version >= "3.9" and python_version < "4.0"
-annotated-types==0.6.0 ; python_version >= "3.9" and python_version < "4.0"
-anyio==4.2.0 ; python_version >= "3.9" and python_version < "4.0"
-arrow==1.3.0 ; python_version >= "3.9" and python_version < "4.0"
-async-timeout==4.0.3 ; python_version >= "3.9" and python_version < "3.11"
-attrs==23.2.0 ; python_version >= "3.9" and python_version < "4.0"
-binaryornot==0.4.4 ; python_version >= "3.9" and python_version < "4.0"
-certifi==2024.2.2 ; python_version >= "3.9" and python_version < "4.0"
-chardet==5.2.0 ; python_version >= "3.9" and python_version < "4.0"
-charset-normalizer==3.3.2 ; python_version >= "3.9" and python_version < "4.0"
-click==8.1.7 ; python_version >= "3.9" and python_version < "4.0"
-colorama==0.4.6 ; python_version >= "3.9" and python_version < "4.0"
-contourpy==1.2.0 ; python_version >= "3.9" and python_version < "4.0"
-cookiecutter==2.5.0 ; python_version >= "3.9" and python_version < "4.0"
-cycler==0.12.1 ; python_version >= "3.9" and python_version < "4.0"
-datasets==2.17.0 ; python_version >= "3.9" and python_version < "4.0"
-dill==0.3.8 ; python_version >= "3.9" and python_version < "4.0"
-evaluate[template]==0.4.1 ; python_version >= "3.9" and python_version < "4.0"
-exceptiongroup==1.2.0 ; python_version >= "3.9" and python_version < "3.11"
-fastapi==0.109.2 ; python_version >= "3.9" and python_version < "4.0"
-ffmpy==0.3.1 ; python_version >= "3.9" and python_version < "4.0"
-filelock==3.13.1 ; python_version >= "3.9" and python_version < "4.0"
-fonttools==4.48.1 ; python_version >= "3.9" and python_version < "4.0"
-frozenlist==1.4.1 ; python_version >= "3.9" and python_version < "4.0"
-fsspec==2023.10.0 ; python_version >= "3.9" and python_version < "4.0"
-fsspec[http]==2023.10.0 ; python_version >= "3.9" and python_version < "4.0"
-gradio-client==0.10.0 ; python_version >= "3.9" and python_version < "4.0"
-gradio==4.18.0 ; python_version >= "3.9" and python_version < "4.0"
-h11==0.14.0 ; python_version >= "3.9" and python_version < "4.0"
-httpcore==1.0.2 ; python_version >= "3.9" and python_version < "4.0"
-httpx==0.26.0 ; python_version >= "3.9" and python_version < "4.0"
-huggingface-hub==0.20.3 ; python_version >= "3.9" and python_version < "4.0"
-idna==3.6 ; python_version >= "3.9" and python_version < "4.0"
-importlib-resources==6.1.1 ; python_version >= "3.9" and python_version < "4.0"
-jinja2==3.1.3 ; python_version >= "3.9" and python_version < "4.0"
-jsonschema-specifications==2023.12.1 ; python_version >= "3.9" and python_version < "4.0"
-jsonschema==4.21.1 ; python_version >= "3.9" and python_version < "4.0"
-kiwisolver==1.4.5 ; python_version >= "3.9" and python_version < "4.0"
-markdown-it-py==3.0.0 ; python_version >= "3.9" and python_version < "4.0"
-markupsafe==2.1.5 ; python_version >= "3.9" and python_version < "4.0"
-matplotlib==3.8.2 ; python_version >= "3.9" and python_version < "4.0"
-mdurl==0.1.2 ; python_version >= "3.9" and python_version < "4.0"
-multidict==6.0.5 ; python_version >= "3.9" and python_version < "4.0"
-multiprocess==0.70.16 ; python_version >= "3.9" and python_version < "4.0"
-numpy==1.26.4 ; python_version >= "3.9" and python_version < "4.0"
-orjson==3.9.13 ; python_version >= "3.9" and python_version < "4.0"
-packaging==23.2 ; python_version >= "3.9" and python_version < "4.0"
-pandas==2.2.0 ; python_version >= "3.9" and python_version < "4.0"
-pillow==10.2.0 ; python_version >= "3.9" and python_version < "4.0"
-pyarrow-hotfix==0.6 ; python_version >= "3.9" and python_version < "4.0"
-pyarrow==15.0.0 ; python_version >= "3.9" and python_version < "4.0"
-pydantic-core==2.16.2 ; python_version >= "3.9" and python_version < "4.0"
-pydantic==2.6.1 ; python_version >= "3.9" and python_version < "4.0"
-pydub==0.25.1 ; python_version >= "3.9" and python_version < "4.0"
-pygments==2.17.2 ; python_version >= "3.9" and python_version < "4.0"
-pyparsing==3.1.1 ; python_version >= "3.9" and python_version < "4.0"
-python-dateutil==2.8.2 ; python_version >= "3.9" and python_version < "4.0"
-python-multipart==0.0.9 ; python_version >= "3.9" and python_version < "4.0"
-python-slugify==8.0.4 ; python_version >= "3.9" and python_version < "4.0"
-pytz==2024.1 ; python_version >= "3.9" and python_version < "4.0"
-pyyaml==6.0.1 ; python_version >= "3.9" and python_version < "4.0"
-referencing==0.33.0 ; python_version >= "3.9" and python_version < "4.0"
-requests==2.31.0 ; python_version >= "3.9" and python_version < "4.0"
-responses==0.18.0 ; python_version >= "3.9" and python_version < "4.0"
-rich==13.7.0 ; python_version >= "3.9" and python_version < "4.0"
-rpds-py==0.17.1 ; python_version >= "3.9" and python_version < "4.0"
-ruff==0.2.1 ; python_version >= "3.9" and python_version < "4.0"
-semantic-version==2.10.0 ; python_version >= "3.9" and python_version < "4.0"
-shellingham==1.5.4 ; python_version >= "3.9" and python_version < "4.0"
-six==1.16.0 ; python_version >= "3.9" and python_version < "4.0"
-sniffio==1.3.0 ; python_version >= "3.9" and python_version < "4.0"
-starlette==0.36.3 ; python_version >= "3.9" and python_version < "4.0"
-text-unidecode==1.3 ; python_version >= "3.9" and python_version < "4.0"
-tomlkit==0.12.0 ; python_version >= "3.9" and python_version < "4.0"
-toolz==0.12.1 ; python_version >= "3.9" and python_version < "4.0"
-tqdm==4.66.2 ; python_version >= "3.9" and python_version < "4.0"
-typer[all]==0.9.0 ; python_version >= "3.9" and python_version < "4.0"
-types-python-dateutil==2.8.19.20240106 ; python_version >= "3.9" and python_version < "4.0"
-typing-extensions==4.9.0 ; python_version >= "3.9" and python_version < "4.0"
-tzdata==2024.1 ; python_version >= "3.9" and python_version < "4.0"
-urllib3==2.2.0 ; python_version >= "3.9" and python_version < "4.0"
-uvicorn==0.27.1 ; python_version >= "3.9" and python_version < "4.0"
-websockets==11.0.3 ; python_version >= "3.9" and python_version < "4.0"
-xxhash==3.4.1 ; python_version >= "3.9" and python_version < "4.0"
-yarl==1.9.4 ; python_version >= "3.9" and python_version < "4.0"
-zipp==3.17.0 ; python_version >= "3.9" and python_version < "3.10"
+# This file was autogenerated by uv via the following command:
+#    uv export --package layout_overlay --no-dev --no-hashes --format requirements-txt
+aiohappyeyeballs==2.6.1
+    # via aiohttp
+aiohttp==3.13.2
+    # via fsspec
+aiosignal==1.4.0
+    # via aiohttp
+anyio==4.12.0
+    # via httpx
+attrs==25.4.0
+    # via aiohttp
+certifi==2025.11.12
+    # via
+    #   httpcore
+    #   httpx
+    #   requests
+charset-normalizer==3.4.4
+    # via requests
+click==8.3.1
+    # via typer-slim
+colorama==0.4.6 ; sys_platform == 'win32'
+    # via
+    #   click
+    #   tqdm
+datasets==4.4.2
+    # via evaluate
+dill==0.4.0
+    # via
+    #   datasets
+    #   evaluate
+    #   multiprocess
+evaluate==0.4.6
+    # via layout-overlay
+filelock==3.20.1
+    # via
+    #   datasets
+    #   huggingface-hub
+frozenlist==1.8.0
+    # via
+    #   aiohttp
+    #   aiosignal
+fsspec==2025.10.0
+    # via
+    #   datasets
+    #   evaluate
+    #   huggingface-hub
+h11==0.16.0
+    # via httpcore
+hf-xet==1.2.0 ; platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'
+    # via huggingface-hub
+httpcore==1.0.9
+    # via httpx
+httpx==0.28.1
+    # via
+    #   datasets
+    #   huggingface-hub
+huggingface-hub==1.2.3
+    # via
+    #   datasets
+    #   evaluate
+idna==3.11
+    # via
+    #   anyio
+    #   httpx
+    #   requests
+    #   yarl
+multidict==6.7.0
+    # via
+    #   aiohttp
+    #   yarl
+multiprocess==0.70.18
+    # via
+    #   datasets
+    #   evaluate
+numpy==2.2.6
+    # via
+    #   datasets
+    #   evaluate
+    #   pandas
+packaging==25.0
+    # via
+    #   datasets
+    #   evaluate
+    #   huggingface-hub
+pandas==2.3.3
+    # via
+    #   datasets
+    #   evaluate
+propcache==0.4.1
+    # via
+    #   aiohttp
+    #   yarl
+pyarrow==22.0.0
+    # via datasets
+python-dateutil==2.9.0.post0
+    # via pandas
+pytz==2025.2
+    # via pandas
+pyyaml==6.0.3
+    # via
+    #   datasets
+    #   huggingface-hub
+requests==2.32.5
+    # via
+    #   datasets
+    #   evaluate
+shellingham==1.5.4
+    # via huggingface-hub
+six==1.17.0
+    # via python-dateutil
+tqdm==4.67.1
+    # via
+    #   datasets
+    #   evaluate
+    #   huggingface-hub
+typer-slim==0.21.0
+    # via huggingface-hub
+typing-extensions==4.15.0
+    # via
+    #   aiosignal
+    #   anyio
+    #   huggingface-hub
+    #   typer-slim
+tzdata==2025.3
+    # via pandas
+urllib3==2.6.2
+    # via requests
+xxhash==3.6.0
+    # via
+    #   datasets
+    #   evaluate
+yarl==1.22.0
+    # via aiohttp