Update README.md

README.md

@@ -1,3 +1,81 @@
 ---
 license: apache-2.0
+tags:
+- protein-language-model
+- bioinformatics
+- transformer
+- cnn
+- small-protein
+- smORF
+frameworks:
+- pytorch
 ---
+
+# TinyProteinTransformer (TPT)
+
+TinyProteinTransformer (TPT) is a lightweight CNN-Transformer hybrid encoder designed for microbial smORF-encoded small proteins. It was pretrained on the Global Microbial smORF Catalog (GMSC, >280M sequences) using masked language modeling combined with contrastive learning, producing compact yet expressive residue-level and sequence-level protein representations.
+
+This repository hosts the pretrained model weights for direct inference and downstream fine-tuning.
+
+For training scripts, ablation code, and the full experimental pipeline, see the GitHub repository: [F4NG66/TPT](https://github.com/F4NG66/TPT)
+
+## Files
+
+| File | Description |
+|---|---|
+| `TPT.pt` | Baseline pretrained TPT weights (CNN-Transformer hybrid encoder, hidden dim 640) |
+| `TPT_gated.py` | Gated variant of the TPT architecture|
+
+## Model Details
+
+- **Architecture:** CNN-Transformer hybrid encoder with multi-scale convolutional feature extraction and attention-based pooling
+- **Hidden dimension:** 640
+- **Transformer layers:** 20
+- **CNN kernel sizes:** 3, 5, 7, 9
+- **Max sequence length:** 128
+- **Pretraining objective:** Masked language modeling + contrastive learning (temperature τ = 0.1, loss weight λ = 0.05)
+- **Pretraining corpus:** GMSC10.90 (Global Microbial smORF Catalog)
+
+## Usage
+
+```python
+import torch
+from model import TinyProteinTransformer
+from utils import build_tokenizer
+
+tokenizer = build_tokenizer()
+model = TinyProteinTransformer(vocab_size=len(tokenizer))
+model.load_state_dict(torch.load("TPT.pt", map_location="cpu"))
+model.eval()
+
+def encode(seq, max_len=128):
+    ids = [tokenizer.get(aa, tokenizer["X"]) for aa in seq][:max_len]
+    ids += [tokenizer["PAD"]] * (max_len - len(ids))
+    return torch.tensor(ids).unsqueeze(0)
+
+with torch.no_grad():
+    x = encode("MKVLILACLVVVTITVS")
+    h = model.encode(x)                  # (1, L, 640) residue-level representations
+    embedding = model.attention_pool(h)  # (1, 640) sequence-level embedding
+```
+
+For classification tasks, attach a lightweight linear head on top of `embedding`.
+
+### Gated variant
+
+`TPT_gated.py` defines the gated architecture variant. Load its accompanying checkpoint the same way, using the model class defined in that script in place of `TinyProteinTransformer`.
+
+## Intended Use
+
+TPT embeddings can be used as frozen features for downstream classification tasks on small secreted/microbial proteins, including antimicrobial peptide (AMP) prediction, toxicity (TOX) prediction, bacteriocin (BCN) classification, and anti-CRISPR (Acr) protein classification.
+
+## Citation
+
+```bibtex
+@article{sheng2026tpt,
+  title={TPT: A Compact CNN-Transformer Encoder for Efficient Microbial Small Protein Modeling},
+  author={Sheng, Fang and Zhang, Junhe and Zhu, Chengkai},
+  journal={Frontiers in Microbiology},
+  year={2026}
+}
+```