Add model card

README.md

@@ -0,0 +1,153 @@
+---
+language:
+- multilingual
+license: apache-2.0
+base_model: answerdotai/ModernBERT-large
+tags:
+- text-classification
+- multi-label-classification
+- topic-classification
+- modernbert
+- metacurate
+pipeline_tag: text-classification
+model-index:
+- name: topic-classifier-v13
+  results:
+  - task:
+      type: text-classification
+      name: Multi-label Topic Classification
+    metrics:
+    - type: f1
+      value: 0.7017
+      name: Tuned Macro F1 (F1-optimized thresholds)
+    - type: precision
+      value: 0.7578
+      name: Macro Precision (precision-biased thresholds)
+    - type: f1
+      value: 0.6409
+      name: Tuned Micro F1
+---
+
+# topic-classifier-v13
+
+Multi-label topic classifier for tech/AI web content, fine-tuned from
+[answerdotai/ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large).
+
+Developed by [Metacurate](https://metacurate.io) to classify ingested web documents
+into 516 granular tech/AI topic labels, supporting content discovery and filtering.
+
+## Model details
+
+| Property | Value |
+|---|---|
+| Base model | `answerdotai/ModernBERT-large` |
+| Task | Multi-label text classification |
+| Labels | 516 total / 478 active |
+| Max input length | 8,192 tokens |
+| Languages | Multilingual (trained on EN-translated text) |
+| Training epochs | 15 |
+| Learning rate | 2e-5 |
+| Batch size | 16 |
+| Warmup ratio | 0.1 |
+| Positive weight cap | 100.0 |
+
+## Performance
+
+Evaluated on a held-out 15% stratified validation split.
+
+| Threshold strategy | Macro F1 | Micro F1 | Macro Precision |
+|---|---|---|---|
+| Raw (0.5) | 0.6497 | 0.6130 | — |
+| F1-optimized per-label thresholds | **0.7017** | 0.6409 | — |
+| Precision-biased thresholds (F-beta=0.5, floor=0.5) | 0.6589 | 0.6287 | **0.7578** |
+
+The model ships with two threshold files:
+- `thresholds.json` — per-label thresholds that maximize F1
+- `thresholds_precision.json` — per-label thresholds tuned for F-beta (β=0.5, precision floor=0.5)
+
+For production use, `thresholds_precision.json` is recommended: it suppresses 38 low-precision labels
+entirely and raises thresholds on the remaining 478, trading a small F1 reduction for substantially
+higher precision (~75.8% macro precision).
+
+## Labels
+
+516 granular tech/AI topic labels derived from a data-driven taxonomy built over the Metacurate document corpus.
+478 labels are active in production (38 suppressed by precision floor).
+
+Example labels: `large language models`, `computer vision`, `reinforcement learning`,
+`cybersecurity`, `semiconductor industry`, `natural language processing`,
+`autonomous vehicles`, `quantum computing`, `blockchain technology`, `robotics`, ...
+
+Full label list: see `label_list.json` in this repository.
+
+## Usage
+
+### Direct inference with `transformers`
+
+```python
+import json
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+model_id = "metacurate/topic-classifier-v13"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForSequenceClassification.from_pretrained(model_id)
+model.eval()
+
+# Load labels and precision thresholds
+with open("label_list.json") as f:
+    labels = json.load(f)
+
+with open("thresholds_precision.json") as f:
+    thresh_data = json.load(f)
+thresholds = dict(zip(thresh_data["labels"], thresh_data["thresholds"]))
+
+text = "OpenAI released GPT-5 with improved reasoning and coding capabilities."
+
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=4096)
+with torch.no_grad():
+    logits = model(**inputs).logits
+probs = torch.sigmoid(logits).squeeze().tolist()
+
+active = [
+    (label, round(score, 4))
+    for label, score in zip(labels, probs)
+    if score >= thresholds.get(label, 1.0)
+]
+print(active)
+# e.g. [('large language models', 0.9123), ('AI startups', 0.8741), ...]
+```
+
+### Via the Metacurate inference service
+
+The model is served via a Modal FastAPI endpoint with per-label precision thresholds
+applied server-side. The service accepts a batch of texts and returns labels and scores
+per text.
+
+## Training data
+
+- ~13,000 real documents labeled by GPT-4.1-mini using the full taxonomy
+- ~2,700 supplementary synthetic records for low-support labels (generated with GPT-4.1-mini)
+- 15% stratified held-out validation split
+
+## Intended use
+
+Designed for classifying tech/AI web articles at ingestion time. Input is the full
+multilingual document text (title + body), translated to English where needed.
+Output is a set of topic labels for each document.
+
+**Not recommended for:**
+- General-purpose multi-label classification outside the tech/AI domain
+- Documents shorter than ~50 words (label coverage degrades)
+
+## Limitations
+
+- Taxonomy is tech/AI-centric; coverage of other domains is limited
+- 38 labels are suppressed in production due to insufficient training data precision
+- Performance varies by label; rare topics (< 50 training examples) have lower recall
+- Thresholds were tuned on a held-out split from the same distribution — out-of-distribution generalization is untested
+
+## Citation
+
+Developed at Metacurate for internal use. Not peer-reviewed.