Upload model card

README.md

@@ -1,10 +1,12 @@
 ---
 language: en
 license: mit
-library_name: model2vec
+library_name: transformers
 tags:
-  - model2vec
-  - static-embeddings
+  - transformers
+  - electra
+  - biomedical
+  - text-classification
   - topic-classification
   - openalex
   - scientific-papers
@@ -13,11 +15,18 @@ datasets:
   - jimnoneill/paper-to-field-training
 ---
 
-# Paper-to-Field Classifier
+# Paper-to-Field Classifier (v3)
 
-Lightweight CPU-based topic classifier for scientific paper abstracts using the
+Transformer-based topic classifier for scientific paper abstracts using the
 [OpenAlex taxonomy](https://docs.openalex.org/api-entities/topics) (4,516 topics → 245 subfields → 26 fields → 4 domains).
 
+## Performance
+
+| Metric | Accuracy |
+|--------|----------|
+| Field (26 classes) | **86.3%** |
+| Domain (4 classes) | **94.4%** |
+
 ## Usage
 
 ```python
@@ -35,26 +44,30 @@ print(result)
 # {
 #   'topic': {'id': 10209, 'name': 'Neural Machine Translation and Sequence Models', 'score': 0.87},
 #   'subfield': {'id': 1702, 'name': 'Artificial Intelligence'},
-#   'field': {'id': 17, 'name': 'Computer Science'},
+#   'field': {'id': 17, 'name': 'Computer Science', 'score': 0.95},
 #   'domain': {'id': 3, 'name': 'Physical Sciences'}
 # }
 ```
 
 ## Model Details
 
-- **Base model**: [minishlab/potion-base-32M](https://huggingface.co/minishlab/potion-base-32M) (Model2Vec)
-- **Fine-tuned on**: ~50K domain-balanced paper abstracts from OpenAlex
+- **Architecture**: BioM-ELECTRA-Large (~335M params) fine-tuned for 26-class field classification
+- **Fine-tuned on**: ~200K paper abstracts with DeepSeek-verified field labels (domain-balanced)
+- **Label quality**: Training labels verified by DeepSeek LLM, replacing noisy OpenAlex labels (~50% error rate)
 - **Taxonomy**: OpenAlex (4,516 topics, 245 subfields, 26 fields, 4 domains)
-- **Input**: Paper title + abstract (truncated to 500 chars)
-- **Inference**: CPU-only, ~3,000 papers/second
+- **Input**: Paper title + abstract (tokenizer truncates at 384 tokens)
+- **Field prediction**: Classification head (26 classes with sqrt-weighted cross-entropy for class imbalance)
+- **Topic resolution**: [CLS] embeddings + FAISS nearest-neighbor within predicted field
+- **GPU recommended** for inference (works on CPU but slower)
 
 ## Training
 
-Trained on OpenAlex bulk data with papers filtered for:
-- English language
-- Has abstract
-- Primary topic confidence score > 0.8
-- Domain-balanced sampling (~12.5K per domain)
+Trained on 200K domain-balanced paper abstracts from OpenAlex bulk data, re-annotated with
+DeepSeek LLM for high-quality field labels (confidence >= 0.7 filter applied).
+
+Hyperparameters: lr=1e-5, cosine schedule, batch=32 (grad accum 2 = effective 64), epochs=8,
+warmup=6%, label smoothing=0.1, fp16, early stopping (patience 5), sqrt inverse-frequency
+class weights.
 
 ## Install