Upload README.md with huggingface_hub

README.md

@@ -31,44 +31,44 @@ config = AutoConfig.from_pretrained('Synthyra/ESMplusplus_large', trust_remote_c
 config.attn_backend = "flex"  # or "kernels_flash", "sdpa", "auto"
 model = AutoModelForMaskedLM.from_pretrained('Synthyra/ESMplusplus_large', config=config, trust_remote_code=True)
 ```
-
-`torch.compile(model)` is heavily recommended for sustained throughput, especially with Flex Attention.
-
-## Binder Design Regularizer
-
-The FastPLMs binder design tutorial uses the ESM++ model family as the
-masked-LM pseudoperplexity regularizer while FastPLMs ESMFold2 experimental
-models provide differentiable folding losses and final critics. The verified
-EGFR example defaults to `Synthyra/ESMplusplus_6B`; this 600M checkpoint exposes
-the same `AutoModelForMaskedLM` API and can be used as a lower-memory
-regularizer by editing `FastPLMsBinderDesign.lm_name` in
-`cookbook/tutorials/binder_design_fastplms.py`.
-
-Default verified run:
-
-```bash
-python cookbook/tutorials/binder_design_fastplms.py \
-  --backend local \
-  --target-name egfr \
-  --binder-sequence '################################################################################################################################' \
-  --not-antibody \
-  --steps 150 \
-  --batch-size 1 \
-  --seed 103 \
-  --output-dir binder_design_egfr_len128_seed103
-```
-
-The verified 6B-regularized result had hero mean iPTM `0.913870`, hero min iPTM
-`0.904600`, and all four ESMFold2 hero critics above `0.9`.
-
-See [`docs/binder_design.md`](https://github.com/Synthyra/FastPLMs/blob/main/docs/binder_design.md)
-for the complete workflow, output files, metrics, and Modal/local compute
-options.
-
-## Use with Hugging Face Transformers
-```python
-from transformers import AutoModelForMaskedLM
-model = AutoModelForMaskedLM.from_pretrained('Synthyra/ESMplusplus_large', trust_remote_code=True)
+
+`torch.compile(model)` is heavily recommended for sustained throughput, especially with Flex Attention.
+
+## Binder Design Regularizer
+
+The FastPLMs binder design tutorial uses the ESM++ model family as the
+masked-LM pseudoperplexity regularizer while FastPLMs ESMFold2 experimental
+models provide differentiable folding losses and final critics. The verified
+EGFR example defaults to `Synthyra/ESMplusplus_6B`; this 600M checkpoint exposes
+the same `AutoModelForMaskedLM` API and can be used as a lower-memory
+regularizer by editing `FastPLMsBinderDesign.lm_name` in
+`cookbook/tutorials/binder_design_fastplms.py`.
+
+Default verified run:
+
+```bash
+python cookbook/tutorials/binder_design_fastplms.py \
+  --backend local \
+  --target-name egfr \
+  --binder-sequence '################################################################################################################################' \
+  --not-antibody \
+  --steps 150 \
+  --batch-size 1 \
+  --seed 103 \
+  --output-dir binder_design_egfr_len128_seed103
+```
+
+The verified 6B-regularized result had hero mean iPTM `0.913870`, hero min iPTM
+`0.904600`, and all four ESMFold2 hero critics above `0.9`.
+
+See [`docs/binder_design.md`](https://github.com/Synthyra/FastPLMs/blob/main/docs/binder_design.md)
+for the complete workflow, output files, metrics, and Modal/local compute
+options.
+
+## Use with Hugging Face Transformers
+```python
+from transformers import AutoModelForMaskedLM
+model = AutoModelForMaskedLM.from_pretrained('Synthyra/ESMplusplus_large', trust_remote_code=True)
 tokenizer = model.tokenizer
 
 sequences = ['MPRTEIN', 'MSEQWENCE']
@@ -99,6 +99,24 @@ import torch
 model = AutoModelForMaskedLM.from_pretrained('Synthyra/ESMplusplus_large', trust_remote_code=True, dtype=torch.float16) # or torch.bfloat16
 ```
 
+## Experimental test-time training
+
+TTT is disabled by default. Normal ESM++ inference, embeddings, logits, and
+`state_dict()` keys are unchanged unless you explicitly call `model.ttt(...)`.
+The current implementation is experimental and trains only local LoRA adapters
+on the ESMC backbone with masked language modeling on the test protein. It can
+help some difficult proteins, but it adds test-time compute and can degrade
+already confident predictions.
+
+```python
+metrics = model.ttt(
+    seq="MSTNPKPQRKTKRNT",
+    ttt_config={"steps": 3, "ags": 1, "batch_size": 1},
+)
+model.ttt_reset()
+print(metrics["losses"])
+```
+
 ## Embed entire datasets with no new code
 To embed a list of protein sequences **fast**, just call embed_dataset. Sequences are sorted to reduce padding tokens, so the initial progress bar estimation is usually much longer than the actual time it will take.