Upload 5 files

README.md

@@ -0,0 +1,64 @@
+---
+license: bsd-3-clause
+tags:
+- pytorch
+- blip
+- matching
+---
+
+# Blip for Matching
+
+## Overview
+
+This is an experimental **Blip** codebase for **Matching**. It keeps the **giant** setup intentionally manageable so architecture changes can be inspected before a full training run.
+
+## Repository status
+
+- The Python file contains the model and runnable example or training entry point.
+- `config.json` records the generated architecture settings.
+- `training_args.json` records the default experiment recipe.
+- `model.safetensors` is a valid initialization checkpoint for smoke tests; it is **not** presented as a trained benchmark checkpoint.
+- No benchmark score is claimed in this repository.
+
+## Architecture
+
+| Item | Value |
+|---|---|
+| Architecture | Blip |
+| Scale | giant |
+| Attention | sparse |
+| Fusion | tucker |
+| Activation | mish |
+| Normalization | groupnorm |
+
+## Default experiment recipe
+
+The included configuration uses **sgd** with a **cosine** schedule. These are starting values in the script, not evidence of a completed run. For a meaningful evaluation, train all baselines with the same data exposure, tuning budget, and random seeds.
+
+## Quick check
+
+```bash
+python finetune.py --help
+```
+
+Inspect the script's `__main__` block for its generated smoke-test example. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.
+
+## Evaluation guidance
+
+A useful first evaluation would use **a paired validation set**, report the task metric across at least three seeds, and include a matched-capacity baseline. Keep training logs and environment versions with any published result.
+
+## Limitations
+
+The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.
+
+## Files
+
+- `finetune.py` — primary artifact
+- `README.md` — this documentation
+- `config.json` — architecture configuration
+- `training_args.json` — default experiment settings
+- `model.safetensors` — initialization checkpoint
+
+## License
+
+Released under **bsd-3-clause**. Review the source-data terms separately when this repository is used with external datasets.

config.json

@@ -0,0 +1,16 @@
+{
+  "architectures": [
+    "CustomResearchModel"
+  ],
+  "architecture": "blip",
+  "model_type": "blip",
+  "hidden_size": 384,
+  "num_hidden_layers": 6,
+  "num_attention_heads": 4,
+  "intermediate_size": 768,
+  "hidden_act": "mish",
+  "max_position_embeddings": 128,
+  "layer_norm_eps": 1e-12,
+  "checkpoint_status": "initialization-only",
+  "notes": "Untrained checkpoint for smoke tests; no benchmark claim."
+}

finetune.py

@@ -0,0 +1,53 @@
+import torch, torch.nn as nn, math
+
+class M(nn.Module):
+    def __init__(self, d=768, L=16, H=12, nc=10):
+        super().__init__()
+        self.pe = nn.Conv2d(3, d, 16, 16)
+        self.cls = nn.Parameter(torch.randn(1,1,d)*.02)
+        self.pos = nn.Parameter(torch.randn(1,197,d)*.02)
+        self.blks = nn.ModuleList([nn.TransformerEncoderLayer(d, H, d*4, .1, activation='gelu', batch_first=True, norm_first=True) for _ in range(L)])
+        self.ln = nn.LayerNorm(d)
+        self.te = nn.Embedding(30522, d, padding_idx=0)
+        self.tpos = nn.Parameter(torch.randn(1,128,d)*.02)
+        self.tblks = nn.ModuleList([nn.TransformerEncoderLayer(d, H, d*4, .1, activation='gelu', batch_first=True, norm_first=True) for _ in range(L)])
+        self.tln = nn.LayerNorm(d)
+        self.fuse = nn.ModuleList([nn.TransformerEncoderLayer(d, H, d*4, .1, activation='gelu', batch_first=True, norm_first=True) for _ in range(2)])
+        self.fln = nn.LayerNorm(d)
+        self.head = nn.Sequential(nn.Linear(d,d), nn.Mish(), nn.Dropout(.1), nn.Linear(d,nc))
+        self._init()
+
+    def _init(self):
+        for m in self.modules():
+            if isinstance(m, nn.Linear):
+                nn.init.trunc_normal_(m.weight, std=0.02)
+                if m.bias is not None: nn.init.zeros_(m.bias)
+
+    def enc_img(self, x):
+        x = self.pe(x).flatten(2).transpose(1,2)
+        x = torch.cat([self.cls.expand(x.size(0),-1,-1), x], 1)
+        x = x + self.pos
+        for b in self.blks: x = b(x)
+        return self.ln(x)
+
+    def enc_txt(self, ids, mask=None):
+        x = self.te(ids) + self.tpos[:, :ids.size(1)]
+        m = (mask == 0) if mask is not None else None
+        for b in self.tblks: x = b(x, src_key_padding_mask=m)
+        return self.tln(x)
+
+    def forward(self, img, ids, mask=None, lbl=None):
+        fi = self.enc_img(img)
+        ft = self.enc_txt(ids, mask)
+        x = ft
+        for f in self.fuse: x = f(x)
+        x = self.fln(x[:, 0])
+        logits = self.head(x)
+        loss = nn.functional.cross_entropy(logits, lbl) if lbl is not None else None
+        return {'logits': logits, 'loss': loss}
+
+if __name__ == '__main__':
+    m = M()
+    print(f'Params: {sum(p.numel() for p in m.parameters()):,}')
+    o = m(torch.randn(2,3,224,224), torch.randint(0,30522,(2,128)), torch.ones(2,128), torch.tensor([0,1]))
+    print(o['logits'].shape, o['loss'].item())

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:942106f79313f005044e24a9b3447078e1f0f9857fac7f54fd97107770626036
+size 198880

training_args.json

@@ -0,0 +1,9 @@
+{
+  "optimizer": "sgd",
+  "scheduler": "cosine",
+  "learning_rate": 0.0002,
+  "batch_size": 24,
+  "epochs": 10,
+  "seed": 123,
+  "status": "default recipe; not a completed run"
+}