Upload folder using huggingface_hub

README.md

@@ -8,8 +8,6 @@ tags:
   - image-feature-extraction
   - embeddings
   - illustration
-  - art-style
-  - style-transfer
   - vision-transformer
   - dino
   - custom-architecture
@@ -23,13 +21,15 @@ model-index:
 
 # EgaraNet — Illustration Style Embedding Model
 
-EgaraNet is an embedding model that encodes the artistic style of illustrations into 1024-dimensional L2-normalized vectors. It was trained on approximately **1.2 million illustrations** from around **12,000 artists**, learning to produce embeddings where illustrations by the same artist are close together in the vector space. Produced under Article 30-4 of the Japanese Copyright Act.
+EgaraNet is an embedding model that encodes the artistic style of illustrations into 1024-dimensional L2-normalized vectors. It was trained on approximately 1.2 million illustrations from around 12,000 artists, learning to produce embeddings where illustrations by the same artist are close together in the vector space.
+
+> Produced under Article 30-4 of the Japanese Copyright Act.
 
 ## Model Description
 
 | | |
 |---|---|
-| **Architecture** | DINOv3 ViT-L/16 backbone + StyleNet (Transposed Transformer Block) head |
+| **Architecture** | DINOv3 ViT-L/16 backbone + StyleNet (Transposed Attention Transformer) head |
 | **Embedding Dim** | 1024 |
 | **Input** | RGB images, any resolution (multiples of 16 recommended) |
 | **Output** | L2-normalized style embedding vector |
@@ -38,8 +38,8 @@ EgaraNet is an embedding model that encodes the artistic style of illustrations
 ### Architecture
 
 - **Backbone**: DINOv3 ViT-L/16 (frozen during training) — extracts rich visual features
-- **StyleNet Head**: 3 × Transposed Transformer Blocks → Attention Pooling → Projection Head
-  - **Transposed Transformer Block (TTB)**: Computes cross-covariance attention in channel space (C×C), discarding spatial information to isolate artistic style. Uses RMSNorm and SwiGLU FFN.
+- **StyleNet Head**: 3 × Transposed Attention Transformer → Attention Pooling → Projection Head
+  - **Transposed Attention Transformer (TAT)**: Computes cross-covariance attention in channel space (C×C), discarding spatial information to isolate artistic style. Uses RMSNorm and SwiGLU FFN.
 
 ## Usage with Transformers
 
@@ -101,7 +101,7 @@ with torch.no_grad():
 ## Input Requirements
 
 - **Image format**: RGB images (PNG, JPEG, WebP, etc.)
-- **Resolution**: Dynamic — the model accepts images of **any resolution** where height and width are multiples of 16. The recommended preprocessing is `MaxResizeMod16(512)` which scales the long edge to 512px while preserving aspect ratio and snapping both dimensions to multiples of 16.
+- **Resolution**: Dynamic — the model accepts images of any resolution where height and width are multiples of 16. The recommended preprocessing is `MaxResizeMod16(512)` which scales the long edge to 512px while preserving aspect ratio and snapping both dimensions to multiples of 16.
 - **Batch size**: Dynamic. For batch inference, all images must have the same spatial dimensions. Process images with different aspect ratios individually or use padding.
 - **Normalization**: ImageNet statistics — mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
 
@@ -141,9 +141,11 @@ The model outputs an `EgaraNetOutput` dataclass with:
 
 ## References
 
-- DINOv3: [arXiv:2508.10104](https://arxiv.org/abs/2508.10104)
-- MANIQA: [arXiv:2204.08958](https://arxiv.org/abs/2204.08958)
-- Style Transfer (Gatys et al.): [CVPR 2016](https://openaccess.thecvf.com/content_cvpr_2016/html/Gatys_Image_Style_Transfer_CVPR_2016_paper.html)
+- **DINOv3**: "DINOv3" [arXiv:2508.10104](https://arxiv.org/abs/2508.10104)
+- **Style Transfer**: "Image Style Transfer Using Convolutional Neural Networks" [CVPR 2016](https://openaccess.thecvf.com/content_cvpr_2016/html/Gatys_Image_Style_Transfer_CVPR_2016_paper.html)
+- **Restormer**: "Restormer: Efficient Transformer for High-Resolution Image Restoration" [arXiv:2111.09881](https://arxiv.org/abs/2111.09881)
+- **MHTA**: "Multi-Head Transposed Attention Transformer for Sea Ice Segmentation in Sar Imagery" [IGARSS 2024](https://ieeexplore.ieee.org/document/10640437)
+- **MANIQA**: "MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment" [arXiv:2204.08958](https://arxiv.org/abs/2204.08958)
 
 ## Links
 

config.json

@@ -1,5 +1,5 @@
 {
-  "_comment": "EgaraNet: DINOv3 ViT-L backbone + StyleNet (Transposed Transformer Block) composite model config.",
+  "_comment": "EgaraNet: DINOv3 ViT-L backbone + StyleNet (Transposed Attention Transformer) composite model config.",
 
   "model_type": "egara_net",
   "architectures": ["EgaraNetModel"],
@@ -49,31 +49,31 @@
     "transformers_version": "4.56.0.dev0"
   },
 
-  "_section_stylenet": "--- StyleNet: Transposed Transformer Block (TTB) head ---",
+  "_section_stylenet": "--- StyleNet: Transposed Attention Transformer (TAT) head ---",
 
-  "ttb_input_dim": null,
-  "_ttb_input_dim_note": "null = auto-inferred from backbone_config.hidden_size at model init",
+  "tat_input_dim": null,
+  "_tat_input_dim_note": "null = auto-inferred from backbone_config.hidden_size at model init",
 
-  "ttb_hidden_dim": 1024,
-  "_ttb_hidden_dim_note": "Internal channel dimension of TTB layers. Matches backbone hidden_size for ViT-L.",
+  "tat_hidden_dim": 1024,
+  "_tat_hidden_dim_note": "Internal channel dimension of TAT layers. Matches backbone hidden_size for ViT-L.",
 
-  "ttb_output_dim": 1024,
-  "_ttb_output_dim_note": "Final L2-normalised style vector dimension.",
+  "tat_output_dim": 1024,
+  "_tat_output_dim_note": "Final L2-normalised style vector dimension.",
 
-  "ttb_num_layers": 3,
-  "_ttb_num_layers_note": "Number of stacked TransposedTransformerBlock layers.",
+  "tat_num_layers": 3,
+  "_tat_num_layers_note": "Number of stacked TransposedAttentionTransformer layers.",
 
-  "ttb_num_heads": 16,
-  "_ttb_num_heads_note": "Number of attention heads in TTB. Must divide ttb_hidden_dim evenly. (1024 / 16 = 64)",
+  "tat_num_heads": 16,
+  "_tat_num_heads_note": "Number of attention heads in TAT. Must divide tat_hidden_dim evenly. (1024 / 16 = 64)",
 
-  "_section_ttb_internals": "--- TTB internals (derived / documented for reference) ---",
-  "ttb_rms_norm_eps": 1e-05,
-  "ttb_swiglu_multiple": 64,
-  "_ttb_swiglu_note": "SwiGLU hidden = round_up(floor(hidden_dim * 8/3), multiple). E.g. 768->2048, 1024->2752.",
+  "_section_tat_internals": "--- TAT internals (derived / documented for reference) ---",
+  "tat_rms_norm_eps": 1e-05,
+  "tat_swiglu_multiple": 64,
+  "_tat_swiglu_note": "SwiGLU hidden = round_up(floor(hidden_dim * 8/3), multiple). E.g. 768->2048, 1024->2752.",
 
   "_section_attnpool": "--- Attention Pooling ---",
   "attn_pool_num_heads": 8,
-  "_attn_pool_note": "nn.MultiheadAttention heads used in AttentionPooling. Must divide ttb_hidden_dim.",
+  "_attn_pool_note": "nn.MultiheadAttention heads used in AttentionPooling. Must divide tat_hidden_dim.",
 
   "_section_head": "--- Projection Head (hidden -> output) ---",
   "head_act": "silu",

configuration_egara_net.py

@@ -10,8 +10,8 @@ Model structure
   │               config stored under `backbone_config`
   └style_net : StyleNet
       ├input_proj        : Identity or Linear (input_dim → hidden_dim)
-      ├ttb_layers [×N]   : TransposedTransformerBlock
-      │   ├RMSNorm + TTB attention (cross-covariance / transposed)
+      ├tat_layers [×N]   : TransposedAttentionTransformer
+      │   ├RMSNorm + TAT attention (cross-covariance / transposed)
       │   └RMSNorm + SwiGLU FFN
       ├attn_pool         : AttentionPooling (learned query → MHA)
       └head              : Linear → SiLU → Linear  (L2-normalised output)
@@ -22,10 +22,10 @@ Usage
 
     # Build from scratch
     cfg = EgaraNetConfig(
-        ttb_hidden_dim=1024,
-        ttb_output_dim=1024,
-        ttb_num_layers=3,
-        ttb_num_heads=16,
+        tat_hidden_dim=1024,
+        tat_output_dim=1024,
+        tat_num_layers=3,
+        tat_num_heads=16,
     )
     cfg.save_pretrained("./my_egara_net/")
 
@@ -75,35 +75,35 @@ class EgaraNetConfig(PretrainedConfig):
         ``"facebook/dinov3-vitl16-pretrain-lvd1689m"``.  Ignored when
         ``backbone_config`` is provided explicitly.
 
-    ttb_input_dim (int | None):
+    tat_input_dim (int | None):
         Input channel dimension fed into StyleNet.  ``None`` (default)
         means it is inferred automatically from
         ``backbone_config.hidden_size`` at model initialisation.
 
-    ttb_hidden_dim (int):
-        Internal channel width of every TTB layer and the AttentionPooling
+    tat_hidden_dim (int):
+        Internal channel width of every TAT layer and the AttentionPooling
         query.  Should match ``backbone_config.hidden_size`` (1024 for ViT-L).
 
-    ttb_output_dim (int):
+    tat_output_dim (int):
         Dimension of the final L2-normalised style vector.
 
-    ttb_num_layers (int):
-        Number of stacked ``TransposedTransformerBlock`` layers.
+    tat_num_layers (int):
+        Number of stacked ``TransposedAttentionTransformer`` layers.
 
-    ttb_num_heads (int):
-        Number of attention heads inside each TTB.
-        Must satisfy ``ttb_hidden_dim % ttb_num_heads == 0``.
+    tat_num_heads (int):
+        Number of attention heads inside each TAT.
+        Must satisfy ``tat_hidden_dim % tat_num_heads == 0``.
 
-    ttb_rms_norm_eps (float):
+    tat_rms_norm_eps (float):
         ε used in all ``RMSNorm`` layers.
 
-    ttb_swiglu_multiple (int):
+    tat_swiglu_multiple (int):
         SwiGLU hidden dim is rounded up to the nearest multiple of this
-        value.  ``hidden = round_up(floor(ttb_hidden_dim * 8/3), multiple)``.
+        value.  ``hidden = round_up(floor(tat_hidden_dim * 8/3), multiple)``.
 
     attn_pool_num_heads (int):
         Number of heads for ``nn.MultiheadAttention`` inside
-        ``AttentionPooling``.  Must divide ``ttb_hidden_dim``.
+        ``AttentionPooling``.  Must divide ``tat_hidden_dim``.
 
     head_act (str):
         Activation between the two linear layers in the projection head.
@@ -130,14 +130,14 @@ class EgaraNetConfig(PretrainedConfig):
         # Backbone
         backbone_config: dict | PretrainedConfig | None = None,
         backbone_model_id: str = "facebook/dinov3-vitl16-pretrain-lvd1689m",
-        # StyleNet / TTB
-        ttb_input_dim: int | None = None,
-        ttb_hidden_dim: int = 1024,
-        ttb_output_dim: int = 1024,
-        ttb_num_layers: int = 3,
-        ttb_num_heads: int = 16,
-        ttb_rms_norm_eps: float = 1e-5,
-        ttb_swiglu_multiple: int = 64,
+        # StyleNet / TAT
+        tat_input_dim: int | None = None,
+        tat_hidden_dim: int = 1024,
+        tat_output_dim: int = 1024,
+        tat_num_layers: int = 3,
+        tat_num_heads: int = 16,
+        tat_rms_norm_eps: float = 1e-5,
+        tat_swiglu_multiple: int = 64,
         # Attention Pooling
         attn_pool_num_heads: int = 8,
         # Projection Head
@@ -150,18 +150,18 @@ class EgaraNetConfig(PretrainedConfig):
         **kwargs: Any,
     ) -> None:
         # Validate constraints
-        if ttb_hidden_dim % ttb_num_heads != 0:
+        if tat_hidden_dim % tat_num_heads != 0:
             raise ValueError(
-                f"ttb_hidden_dim ({ttb_hidden_dim}) must be divisible by "
-                f"ttb_num_heads ({ttb_num_heads})."
+                f"tat_hidden_dim ({tat_hidden_dim}) must be divisible by "
+                f"tat_num_heads ({tat_num_heads})."
             )
-        if ttb_hidden_dim % attn_pool_num_heads != 0:
+        if tat_hidden_dim % attn_pool_num_heads != 0:
             raise ValueError(
-                f"ttb_hidden_dim ({ttb_hidden_dim}) must be divisible by "
+                f"tat_hidden_dim ({tat_hidden_dim}) must be divisible by "
                 f"attn_pool_num_heads ({attn_pool_num_heads})."
             )
-        if ttb_num_layers < 1:
-            raise ValueError(f"ttb_num_layers must be >= 1, got {ttb_num_layers}.")
+        if tat_num_layers < 1:
+            raise ValueError(f"tat_num_layers must be >= 1, got {tat_num_layers}.")
 
         # Backbone config
         if isinstance(backbone_config, dict):
@@ -171,13 +171,13 @@ class EgaraNetConfig(PretrainedConfig):
         self.backbone_model_id: str = backbone_model_id
 
         # StyleNet params
-        self.ttb_input_dim: int | None = ttb_input_dim
-        self.ttb_hidden_dim: int = ttb_hidden_dim
-        self.ttb_output_dim: int = ttb_output_dim
-        self.ttb_num_layers: int = ttb_num_layers
-        self.ttb_num_heads: int = ttb_num_heads
-        self.ttb_rms_norm_eps: float = ttb_rms_norm_eps
-        self.ttb_swiglu_multiple: int = ttb_swiglu_multiple
+        self.tat_input_dim: int | None = tat_input_dim
+        self.tat_hidden_dim: int = tat_hidden_dim
+        self.tat_output_dim: int = tat_output_dim
+        self.tat_num_layers: int = tat_num_layers
+        self.tat_num_heads: int = tat_num_heads
+        self.tat_rms_norm_eps: float = tat_rms_norm_eps
+        self.tat_swiglu_multiple: int = tat_swiglu_multiple
 
         # Attention Pooling
         self.attn_pool_num_heads: int = attn_pool_num_heads
@@ -205,20 +205,20 @@ class EgaraNetConfig(PretrainedConfig):
         )
 
     @property
-    def effective_ttb_input_dim(self) -> int:
+    def effective_tat_input_dim(self) -> int:
         """Actual input_dim used by StyleNet input_proj."""
-        return self.ttb_input_dim if self.ttb_input_dim is not None \
+        return self.tat_input_dim if self.tat_input_dim is not None \
                else self.backbone_hidden_size
 
     @property
-    def ttb_swiglu_hidden_dim(self) -> int:
+    def tat_swiglu_hidden_dim(self) -> int:
         """Derived SwiGLU intermediate dimension (not stored in JSON)."""
-        return _swiglu_hidden_dim(self.ttb_hidden_dim, self.ttb_swiglu_multiple)
+        return _swiglu_hidden_dim(self.tat_hidden_dim, self.tat_swiglu_multiple)
 
     @property
-    def ttb_head_dim(self) -> int:
-        """Per-head dimension inside TTB attention."""
-        return self.ttb_hidden_dim // self.ttb_num_heads
+    def tat_head_dim(self) -> int:
+        """Per-head dimension inside TAT attention."""
+        return self.tat_hidden_dim // self.tat_num_heads
 
     # Serialisation
 
@@ -245,14 +245,14 @@ class EgaraNetConfig(PretrainedConfig):
             f"EgaraNetConfig(\n"
             f"  backbone          : {backbone_name}"
             f" (hidden={getattr(self.backbone_config, 'hidden_size', '?')})\n"
-            f"  ttb_input_dim     : {self.effective_ttb_input_dim}  "
-            f"({'auto' if self.ttb_input_dim is None else 'explicit'})\n"
-            f"  ttb_hidden_dim    : {self.ttb_hidden_dim}\n"
-            f"  ttb_output_dim    : {self.ttb_output_dim}\n"
-            f"  ttb_num_layers    : {self.ttb_num_layers}\n"
-            f"  ttb_num_heads     : {self.ttb_num_heads}  "
-            f"(head_dim={self.ttb_head_dim})\n"
-            f"  ttb_swiglu_hidden : {self.ttb_swiglu_hidden_dim}\n"
+            f"  tat_input_dim     : {self.effective_tat_input_dim}  "
+            f"({'auto' if self.tat_input_dim is None else 'explicit'})\n"
+            f"  tat_hidden_dim    : {self.tat_hidden_dim}\n"
+            f"  tat_output_dim    : {self.tat_output_dim}\n"
+            f"  tat_num_layers    : {self.tat_num_layers}\n"
+            f"  tat_num_heads     : {self.tat_num_heads}  "
+            f"(head_dim={self.tat_head_dim})\n"
+            f"  tat_swiglu_hidden : {self.tat_swiglu_hidden_dim}\n"
             f"  attn_pool_heads   : {self.attn_pool_num_heads}\n"
             f"  head_act          : {self.head_act}\n"
             f"  image_size        : {self.image_size}  "
@@ -284,10 +284,10 @@ if __name__ == "__main__":
 
     cfg = EgaraNetConfig(
         backbone_config=backbone_cfg,
-        ttb_hidden_dim=backbone_cfg.get("hidden_size", 1024),
-        ttb_output_dim=backbone_cfg.get("hidden_size", 1024),
-        ttb_num_layers=3,
-        ttb_num_heads=16,
+        tat_hidden_dim=backbone_cfg.get("hidden_size", 1024),
+        tat_output_dim=backbone_cfg.get("hidden_size", 1024),
+        tat_num_layers=3,
+        tat_num_heads=16,
         attn_pool_num_heads=8,
     )
     print(cfg)
@@ -296,12 +296,12 @@ if __name__ == "__main__":
     tmp = pathlib.Path("/tmp/egara_net_test")
     cfg.save_pretrained(tmp)
     cfg2 = EgaraNetConfig.from_pretrained(tmp)
-    assert cfg.ttb_hidden_dim == cfg2.ttb_hidden_dim
-    assert cfg.ttb_output_dim == cfg2.ttb_output_dim
-    assert cfg.ttb_num_layers == cfg2.ttb_num_layers
+    assert cfg.tat_hidden_dim == cfg2.tat_hidden_dim
+    assert cfg.tat_output_dim == cfg2.tat_output_dim
+    assert cfg.tat_num_layers == cfg2.tat_num_layers
     print("Round-trip OK ✓")
 
     # Derived properties
-    print(f"SwiGLU hidden dim : {cfg.ttb_swiglu_hidden_dim}")
-    print(f"TTB head dim      : {cfg.ttb_head_dim}")
-    print(f"Effective input   : {cfg.effective_ttb_input_dim}")
+    print(f"SwiGLU hidden dim : {cfg.tat_swiglu_hidden_dim}")
+    print(f"TAT head dim      : {cfg.tat_head_dim}")
+    print(f"Effective input   : {cfg.effective_tat_input_dim}")

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bc07899ef37fc10821834d26888d52e1d6e5cc335f2e9aaf4a71c94e5f3c8ddb
+oid sha256:2f4e3c6f00d0ffe0e40c9481150465c297805cc482fc83881d0212cc1f8cb90f
 size 1389584856

modeling_egara_net.py

@@ -5,7 +5,7 @@ Hugging Face PreTrainedModel implementation for EgaraNet.
 
 EgaraNet is a composite model:
     backbone  : DINOv3 ViT  (frozen feature extractor)
-    style_net : StyleNet    (Transposed Transformer Block head)
+    style_net : StyleNet    (Transposed Attention Transformer head)
 
 Usage:
     from modeling_egara_net import EgaraNetModel
@@ -81,9 +81,9 @@ class SwiGLU(nn.Module):
         return self.w_down(F.silu(self.w_gate(x)) * self.w_up(x))
 
 
-class TransposedTransformerBlock(nn.Module):
+class TransposedAttentionTransformer(nn.Module):
     """
-    TTB — Transposed Transformer Block.
+    TAT — Transposed Attention Transformer.
     Cross-covariance attention in (HeadDim × HeadDim) space.
     """
 
@@ -151,13 +151,13 @@ class AttentionPooling(nn.Module):
 
 class StyleNet(nn.Module):
     """
-    StyleNet head: TTB layers → AttentionPooling → Projection → L2 norm.
+    StyleNet head: TAT layers → AttentionPooling → Projection → L2 norm.
     """
 
     def __init__(self, config: EgaraNetConfig):
         super().__init__()
-        input_dim = config.effective_ttb_input_dim
-        hidden_dim = config.ttb_hidden_dim
+        input_dim = config.effective_tat_input_dim
+        hidden_dim = config.tat_hidden_dim
 
         self.input_proj = (
             nn.Identity()
@@ -165,14 +165,14 @@ class StyleNet(nn.Module):
             else nn.Linear(input_dim, hidden_dim)
         )
 
-        self.ttb_layers = nn.ModuleList([
-            TransposedTransformerBlock(
+        self.tat_layers = nn.ModuleList([
+            TransposedAttentionTransformer(
                 dim=hidden_dim,
-                num_heads=config.ttb_num_heads,
-                eps=config.ttb_rms_norm_eps,
-                swiglu_multiple=config.ttb_swiglu_multiple,
+                num_heads=config.tat_num_heads,
+                eps=config.tat_rms_norm_eps,
+                swiglu_multiple=config.tat_swiglu_multiple,
             )
-            for _ in range(config.ttb_num_layers)
+            for _ in range(config.tat_num_layers)
         ])
 
         self.attn_pool = AttentionPooling(
@@ -182,12 +182,12 @@ class StyleNet(nn.Module):
         self.head = nn.Sequential(
             nn.Linear(hidden_dim, hidden_dim),
             nn.SiLU(),
-            nn.Linear(hidden_dim, config.ttb_output_dim),
+            nn.Linear(hidden_dim, config.tat_output_dim),
         )
 
     def forward(self, x: torch.Tensor) -> torch.Tensor:
         x = self.input_proj(x)
-        for layer in self.ttb_layers:
+        for layer in self.tat_layers:
             x = layer(x)
         x = self.attn_pool(x)
         x = self.head(x)

onnx/egara_net_fp16.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:022cec15e29198be338deeea41155f270cbd5c3191f48d6c1770227c23043e8b
-size 699500866
+oid sha256:5a030df2d519143d655bf5e78ead47ad31317a37e5822fe6538883c961b3299a
+size 699502106

onnx/egara_net_fp32.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:61cb32d90d105d8e8c4847e4b12fc9c92300d0960cf5f7398e653d9f10683b25
-size 1394094118
+oid sha256:46ebe59d3dd13f9a0d5e73a0b847e816e1b1c476fff1a04b1acb34892dc3012b
+size 1394095358

onnx/egara_net_int8.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0a8cb49f30537184284c2583b74257876319d6086a8c02ded1b3537b02044e15
-size 362130177
+oid sha256:cbe9745cf78a8ed2833dda5e02d2cd3932041a6033d413e1c5d2e5b18279c9c7
+size 362131297