Update README.md

README.md

@@ -1,3 +1,52 @@
----
-license: mit
----
+---
+license: mit
+---
+# PhenoGemini Qwen3-MoE Ranker Model Card
+
+## Overview
+
+This model ranks candidate disease-causing genes based on a patient’s phenotypic information. During inference, it does not generate a full free-text response. Instead, it performs gene-token subspace ranking on the logits at the final position and outputs a prioritized list of genes.
+
+## Prompt Wrapping
+
+Use `https://phenogemini.org` to generate a prompt that includes similar patients.
+
+## Model Loading
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "DISCOStudy/PhenoGemini"
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+
+# Load base model
+base = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype="auto",
+    device_map="auto",
+    trust_remote_code=True
+)
+
+```
+
+## Logit Masking (Gene-Token Subspace Ranking)
+
+This model does not rank over the full vocabulary. It first masks non-gene tokens, then ranks only within the gene-token subspace:
+
+```python
+with torch.no_grad():
+    prompt = "[PUT THE PROMPT GENERATED FROM https://phenogemini.org]"
+    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+    outputs = model(inputs["input_ids"])
+    # Take logits for the last token only
+    logits = outputs.logits[0][-1]
+    # Mask non-gene tokens
+    logits[:151669] = -torch.inf
+    # Rank gene tokens by logits
+    token_ids = torch.argsort(logits, descending=True)[:8332]
+    # Output gene tokens and logits; tokens include Entrez IDs
+    tokens = tokenizer.convert_ids_to_tokens(token_ids.tolist())
+    scores = logits[token_ids].tolist()
+```