Upload README.md with huggingface_hub

README.md

@@ -18,6 +18,7 @@ tags:
   - gujarati
   - odia
   - text-to-font
+  - retrieval
 library_name: peft
 base_model: openai/clip-vit-large-patch14
 pipeline_tag: feature-extraction
@@ -40,30 +41,73 @@ Indic font-intelligence family.
 This is, to our knowledge, the first open font↔language embedding
 model for Indic scripts (prior art: FontCLIP, Latin-centric).
 
+## 🖥️ Try it live
+
+**[→ Gradio demo: anilpai/lipika-demo](https://huggingface.co/spaces/anilpai/lipika-demo)**
+— the *Search by description* tab runs this model. Results render live
+in each font (hosted WOFF2), with attribute chips, similarity bars,
+match counts, a script filter, and a toggle for legacy 90s DTP fonts.
+The *Recognize* tab is the companion
+[image→font recognizer](https://huggingface.co/loopdesk-ai/lipika).
+
 ## What's in this repo
 
 | file | description |
 |---|---|
 | `adapter/` | LoRA adapter (r16, q/v proj, last 8 blocks of both towers) for `openai/clip-vit-large-patch14` |
-| `index.npz` | Precomputed embeddings for 595 font families × up to 3 faces (regular / heaviest / lightest) × per-script specimens (1250 entries) + family metadata |
-| `attributes.json` | 46 visual-attribute scores (0–100) per family (VLM-labeled, fused two-pass) |
+| `index.npz` | Precomputed embeddings for 595 font families × up to 3 faces (regular / heaviest / lightest) × per-script specimens (1250 entries), plus per-family metadata: attribute scores, scripts, `legacy` flag, legacy `encoding` |
+| `attributes.json` | 46 visual-attribute scores (0–100) per family (VLM-labeled, two-pass fused) |
 | `vocab.json` | The attribute vocabulary |
+| `specimens/` | 596 rendered specimen banners (1024×220 PNG), one per family — preview images used by the demo when no hosted WOFF2 exists |
+
+## Quickstart (lipika package — recommended)
+
+```bash
+pip install "lipika[search] @ git+https://github.com/Loopdesk-AI/lipika.git"
+```
+
+```python
+from pathlib import Path
+from huggingface_hub import snapshot_download
+from fontrecog.fontclip.search import FontClipSearcher
+
+fc = Path(snapshot_download("loopdesk-ai/lipika-fontclip",
+                            allow_patterns=["index.npz", "adapter/*"]))
+searcher = FontClipSearcher(fc / "index.npz", ckpt=str(fc / "adapter"))
+
+hits = searcher.search("a thin delicate Devanagari font",
+                       k=5, script="devanagari")
+# [{'family': 'Khula', 'score': 0.21, 'matched_script': 'devanagari',
+#   'matched_face': 'light', 'scripts': ['devanagari', 'latin']}, ...]
+
+stats = searcher.search_with_stats("a round playful Tamil font",
+                                   k=10, script="tamil")
+# {'results': [...], 'n_candidates': 31, 'n_shown': 10}
+
+# Legacy (non-Unicode 90s DTP) families — 326 of 595, e.g. Kruti Dev,
+# DevLys — are excluded by default; opt in explicitly:
+searcher.search("a retro 90s desktop-publishing Hindi font",
+                script="devanagari", include_legacy=True)
+```
 
-## Quickstart
+## Quickstart (raw transformers + peft)
 
 ```python
 import json, numpy as np, torch, torch.nn.functional as F
+from huggingface_hub import hf_hub_download
 from transformers import CLIPModel, CLIPTokenizerFast
 from peft import PeftModel
 
+repo = "loopdesk-ai/lipika-fontclip"
 model = CLIPModel.from_pretrained("openai/clip-vit-large-patch14")
-model = PeftModel.from_pretrained(model, "loopdesk-ai/lipika-fontclip",
+model = PeftModel.from_pretrained(model, repo,
                                   subfolder="adapter").merge_and_unload().eval()
 tok = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
 
-z = np.load("index.npz")  # hf_hub_download(..., "index.npz")
+z = np.load(hf_hub_download(repo, "index.npz"))
 E = torch.from_numpy(z["embeddings"])
-fams, scripts = [str(x) for x in z["families"]], [str(x) for x in z["scripts"]]
+fams = [str(x) for x in z["families"]]
+scripts = [str(x) for x in z["scripts"]]
 meta = json.loads(z["meta"].tobytes().decode())["families"]
 
 q = "a thin delicate Devanagari font"
@@ -75,14 +119,32 @@ best = {}
 for i in sims.argsort(descending=True).tolist():
     f = fams[i]
     if scripts[i] != "devanagari" or meta[f].get("legacy"):
-        continue  # legacy = non-Unicode 90s DTP faces, opt-in
+        continue
     best.setdefault(f, sims[i].item())
     if len(best) == 5:
         break
 print(list(best))
 ```
 
-Tested with `transformers==4.49.0`, `peft==0.14.0`.
+Tested with `transformers==4.49.0`, `peft>=0.14`. (`transformers>=5`
+changed the `get_text_features` return type — use `<5` or the lipika
+package.)
+
+## Index metadata
+
+`index.npz` → `meta` (JSON) → `families[family]`:
+
+| key | description |
+|---|---|
+| `attrs` | 46 attribute scores 0–100 (family-fused) |
+| `scripts` | scripts covered by the family |
+| `files` | font file per indexed face (`rep`/`heavy`/`light`) |
+| `legacy` | `true` for non-Unicode 90s DTP families |
+| `encoding` | legacy cmap encoding (e.g. `krutidev010`) or `null` — text must be converted before rendering in these fonts |
+
+Each family is indexed with up to 3 static faces so weight-specific
+queries ("very heavy…") can match the right face; `matched_face` in
+results tells you which one won.
 
 ## Training
 
@@ -94,20 +156,27 @@ Tested with `transformers==4.49.0`, `peft==0.14.0`.
 - **Recipe**: LoRA (r16, α16) on q/v projections of the last 8 blocks of
   both towers; symmetric InfoNCE with same-family negatives masked;
   hinged cross-script pair loss (same family, different script →
-  cos ≥ 0.9); hinged attribute-ranking loss against frozen text probes;
-  batch 256, 4k steps, best @2000.
+  cos ≥ 0.9); hinged attribute-ranking loss against frozen (detached)
+  text probes; batch 256, 4k steps on 1×H100, best checkpoint @2000.
+- Trainable probes for the ranking loss were falsified (the text tower
+  games the hinge and attribute directions collapse); detaching them is
+  what made the loss help.
 
 ## Evaluation
 
+Frozen 50-query benchmark + held-out families (30 families never seen
+in training), multi-face index:
+
 | metric | zero-shot CLIP | lipika-fontclip |
 |---|---|---|
-| Text→font P@5 (norm., 30 frozen queries) | 0.150 | **0.450** (3.0×) |
-| Attribute ranking (mean Spearman, held-out families) | −0.075 | **0.365** |
+| Text→font P@5 (norm., answerable queries) | 0.150 | **0.450** (3.0×) |
+| Attribute ranking (mean Spearman, held-out) | −0.075 | **0.365** |
 | Cross-script consistency (same family, cos) | 0.618 | **0.884** |
 | Bold-above-light ordering | 50.7% | **100%** |
 
-Zero-shot CLIP is near-chance on Indic typography; fine-tuning is what
-makes this usable.
+Zero-shot CLIP is near-chance on Indic typography (weight ordering is a
+coin flip; attribute correlations are negative) — fine-tuning is what
+makes this usable. 8/8 retrieval regression spot-checks pass.
 
 ## Limitations
 
@@ -116,11 +185,20 @@ makes this usable.
 - "Serif/high-contrast" concepts are weakly represented in the corpus
   and retrieval quality reflects that.
 - 326/595 families are legacy (non-Unicode) 90s DTP fonts (Kruti Dev,
-  DevLys, …); they flood stylistic queries unless filtered — the
-  `legacy` metadata flag is provided for this.
+  DevLys, …); they flood stylistic queries unless filtered — excluded
+  by default in the lipika searcher, `include_legacy=True` restores
+  them, and the `legacy`/`encoding` metadata supports custom handling.
 - English queries only.
 
+## Related
+
+- **Demo Space**: [anilpai/lipika-demo](https://huggingface.co/spaces/anilpai/lipika-demo)
+- **Recognizer model** (image→font): [loopdesk-ai/lipika](https://huggingface.co/loopdesk-ai/lipika)
+- **Code**: [Loopdesk-AI/lipika](https://github.com/Loopdesk-AI/lipika)
+  (`fontrecog.fontclip` — training, eval gates, index builder, searcher)
+
 ## License
 
-Apache-2.0 (adapter, index, labels). Fonts referenced are under their
-own licenses (mostly OFL); no font binaries are redistributed here.
+Apache-2.0 (adapter, index, labels, specimen images). Fonts referenced
+are under their own licenses (mostly OFL); no font binaries are
+redistributed here.