Transformers
Safetensors
dplm2
custom_code
lhallee commited on
Commit
dad0049
·
verified ·
1 Parent(s): eb8ed46

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -36,6 +36,24 @@ with torch.no_grad():
36
  logits = mlm(**batch).logits
37
  ```
38
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  ## DPLM2 modality types
40
  DPLM2 infers `type_ids` automatically from `input_ids` and `attention_mask` when they are not provided.
41
 
 
36
  logits = mlm(**batch).logits
37
  ```
38
 
39
+ ## Experimental test-time training
40
+
41
+ TTT is disabled by default. Normal DPLM2 inference, embeddings, logits, and
42
+ `state_dict()` keys are unchanged unless you explicitly call `model.ttt(...)`.
43
+ The current implementation is experimental and trains only local LoRA adapters
44
+ on the PLM backbone with masked language modeling on the test protein. It can
45
+ help some difficult proteins, but it adds test-time compute and can degrade
46
+ already confident predictions.
47
+
48
+ ```python
49
+ metrics = mlm.ttt(
50
+ seq="MSTNPKPQRKTKRNT",
51
+ ttt_config={"steps": 3, "ags": 1, "batch_size": 1},
52
+ )
53
+ mlm.ttt_reset()
54
+ print(metrics["losses"])
55
+ ```
56
+
57
  ## DPLM2 modality types
58
  DPLM2 infers `type_ids` automatically from `input_ids` and `attention_mask` when they are not provided.
59