Create README.md

README.md

@@ -0,0 +1,35 @@
+You can easily import our continually post-trained model with HuggingFace's `transformers`:
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+# Import our model. The package will take care of downloading the models automatically
+tokenizer = AutoTokenizer.from_pretrained("roberta-base")
+model = AutoModelForSequenceClassification.from_pretrained("UIC-Liu-Lab/CPT", trust_remote_code=True)
+
+# Tokenize input texts
+texts = [
+    "There's a kid on a skateboard.",
+    "A kid is skateboarding.",
+    "A kid is inside the house."
+]
+inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
+
+# Task id and smax
+t = torch.LongTensor([0]).to(model.device)	# using task 0's CL-plugin, choose from {0, 1, 2, 3}
+smax = 400
+
+# Get the model output!
+res = model(**inputs, return_dict=True, t=t, s=smax)
+```
+
+If you encounter any problem when directly loading the models by HuggingFace's API, you can also download the models manually from the [repo](https://huggingface.co/UIC-Liu-Lab/CPT/tree/main) and use `model = AutoModel.from_pretrained({PATH TO THE DOWNLOAD MODEL})`.
+
+Note: The post-trained weights you load contain un-trained classification heads. The post-training sequence is `Restaurant -> AI -> ACL -> AGNews`, you can use the downloaded weights to fine-tune the corresponding end-task. The results (MF1/Acc) will be consistent with follows.
+
+|                 | Restaurant    | AI            | ACL           | AGNews        | Avg.          |
+| --------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
+| UIC-Liu-Lab/CPT | 53.90 / 75.13 | 30.42 / 30.89 | 37.56 / 38.53 | 63.77 / 65.79 | 46.41 / 52.59 |
+
+##