metadata
license: mit
datasets:
- keiwoo/peptide
base_model:
- FacebookAI/roberta-base
pipeline_tag: feature-extraction
tags:
- biology
- medical
metrics:
- accuracy
This repository contains relevant information of our research article 'Supervised fine-tuning enhances unsupervised learning from 45 million amino acids in TCR and peptide sequences'.
Embedding examples
Transformers
You can use RoBERTpep with Transformers. To get started, install the necessary dependencies to setup your environment:
pip install -U transformers torch
Once setup you can proceed to run the model by running the snippet below:
from transformers import BertTokenizer, RobertaModel
import torch
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
tokenizer = BertTokenizer.from_pretrained('keiwoo/RoBERTpep', do_lower_case = False)
model = RobertaModel.from_pretrained('keiwoo/RoBERTpep').to(device)
outputs = model(**tokenizer(' '.join('KLGGALQAK'), return_tensors="pt").to(device))
print(outputs.last_hidden_state[0].shape)
# torch.Size([11, 1024]) [CLS+N+SEP, 1024]
Fine-tuning
Please refer to https://github.com/keiwoo/RoBERTcr