Instructions to use Synthyra/DPLM2-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Synthyra/DPLM2-3B with Transformers:
# Load model directly from transformers import EsmForDPLM2 model = EsmForDPLM2.from_pretrained("Synthyra/DPLM2-3B", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -36,6 +36,24 @@ with torch.no_grad():
|
|
| 36 |
logits = mlm(**batch).logits
|
| 37 |
```
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
## DPLM2 modality types
|
| 40 |
DPLM2 infers `type_ids` automatically from `input_ids` and `attention_mask` when they are not provided.
|
| 41 |
|
|
|
|
| 36 |
logits = mlm(**batch).logits
|
| 37 |
```
|
| 38 |
|
| 39 |
+
## Experimental test-time training
|
| 40 |
+
|
| 41 |
+
TTT is disabled by default. Normal DPLM2 inference, embeddings, logits, and
|
| 42 |
+
`state_dict()` keys are unchanged unless you explicitly call `model.ttt(...)`.
|
| 43 |
+
The current implementation is experimental and trains only local LoRA adapters
|
| 44 |
+
on the PLM backbone with masked language modeling on the test protein. It can
|
| 45 |
+
help some difficult proteins, but it adds test-time compute and can degrade
|
| 46 |
+
already confident predictions.
|
| 47 |
+
|
| 48 |
+
```python
|
| 49 |
+
metrics = mlm.ttt(
|
| 50 |
+
seq="MSTNPKPQRKTKRNT",
|
| 51 |
+
ttt_config={"steps": 3, "ags": 1, "batch_size": 1},
|
| 52 |
+
)
|
| 53 |
+
mlm.ttt_reset()
|
| 54 |
+
print(metrics["losses"])
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
## DPLM2 modality types
|
| 58 |
DPLM2 infers `type_ids` automatically from `input_ids` and `attention_mask` when they are not provided.
|
| 59 |
|