lhallee commited on
Commit
11518f1
·
verified ·
1 Parent(s): 8cb042d

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -50,6 +50,24 @@ with torch.no_grad():
50
  print(logits.shape) # (2, 11, 33)
51
  ```
52
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
  ### For working with attention maps
54
  ```python
55
  import torch
 
50
  print(logits.shape) # (2, 11, 33)
51
  ```
52
 
53
+ ### Experimental test-time training
54
+
55
+ TTT is disabled by default. Normal inference, embeddings, logits, and
56
+ `state_dict()` keys are unchanged unless you explicitly call `model.ttt(...)`.
57
+ The current implementation is experimental and trains only local LoRA adapters
58
+ on the ESM2 backbone using masked language modeling on the test protein. It can
59
+ help difficult proteins, but it adds test-time compute and can hurt already
60
+ confident predictions.
61
+
62
+ ```python
63
+ metrics = model.ttt(
64
+ seq="MSTNPKPQRKTKRNT",
65
+ ttt_config={"steps": 3, "ags": 1, "batch_size": 1},
66
+ )
67
+ model.ttt_reset()
68
+ print(metrics["losses"])
69
+ ```
70
+
71
  ### For working with attention maps
72
  ```python
73
  import torch