File size: 3,705 Bytes
2154dc3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 |
---
license: other
---
# How to use
```python
from transformers import AutoModel, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
"Taykhoom/Helix-mRNA-Wrapper",
trust_remote_code=True,
)
model = AutoModel.from_pretrained(
"Taykhoom/Helix-mRNA-Wrapper",
trust_remote_code=True,
).eval()
dna = "ACGUAGCAUCGGAUCUAUCUAUCGACACUUGGUUAUCGAUCUACGAGCAUCUCGUUAGC"
inputs = tokenizer(
dna,
return_tensors="pt",
truncation=True,
padding="longest",
max_length=tokenizer.model_max_length,
return_special_tokens_mask=True,
)
special_tokens_mask = inputs["special_tokens_mask"]
attention_mask = 1 - special_tokens_mask
embedding = model(
input_ids=inputs["input_ids"],
attention_mask=attention_mask,
).last_hidden_state # [1, sequence_length, 256]
```
# Performance Vs Original Helix-mRNA Models
Verify that the modified code produces the same embeddings as the original Helix-mRNA models.
Original Helix-mRNA code snippet:
```python
from helical.models.helix_mrna import HelixmRNA, HelixmRNAConfig
import torch
input_sequences = ["ACGUAGCAUCGGAUCUAUCUAUCGACACUUGGUUAUCGAUCUACGAGCAUCUCGUUAGC"]
helix_mrna_config = HelixmRNAConfig(batch_size=1)
helix_mrna = HelixmRNA(configurer=helix_mrna_config)
# prepare data for input to the model
processed_input_data = helix_mrna.process_data(input_sequences)
# generate the embeddings for the processed data
embedding = torch.Tensor(helix_mrna.get_embeddings(processed_input_data))
embedding_mean = torch.mean(embedding, dim=1) # [1, 256]
print(torch.mean(embedding_mean)) # Outputs tensor(-0.0033)
embedding_max = torch.max(embedding, dim=1)[0]
print(torch.mean(embedding_max)) # Outputs tensor(0.0989)
```
Modified code snippet using the wrapper:
```python
import torch
from transformers import AutoModel, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
"Taykhoom/Helix-mRNA-Wrapper",
trust_remote_code=True,
)
model = AutoModel.from_pretrained(
"Taykhoom/Helix-mRNA-Wrapper",
trust_remote_code=True,
).eval()
dna = "ACGUAGCAUCGGAUCUAUCUAUCGACACUUGGUUAUCGAUCUACGAGCAUCUCGUUAGC"
inputs = tokenizer(
dna,
return_tensors="pt",
truncation=True,
padding="longest",
max_length=tokenizer.model_max_length,
return_special_tokens_mask=True,
)
special_tokens_mask = inputs["special_tokens_mask"]
attention_mask = 1 - special_tokens_mask
embedding = model(
input_ids=inputs["input_ids"],
attention_mask=attention_mask,
).last_hidden_state # [1, sequence_length, 256]
embedding_mean = torch.mean(embedding, dim=1)
print(torch.mean(embedding_mean)) # Outputs tensor(-0.0033, grad_fn=<MeanBackward0>)
embedding_max = torch.max(embedding, dim=1)[0]
print(torch.mean(embedding_max)) # Outputs tensor(0.0989, grad_fn=<MeanBackward0>)
```
# License Notice
This repository contains modified versions of Helical code.
Modifications include:
- Removal of reliance on helical package
- Removal of some ease-of-use embedding generation code (to standardize usage) and other checks (see original repository for more details)
Not all of the original functionality may be preserved. These changes were made to better integrate with the mRNABench framework which focuses on embedding generation for mRNA sequences. Most of the required code was directly copied from the original Helical repository with minimal changes, so please refer to the original repository for full details on the implementation.
When using this repository, please adhere to the original license terms of the Helical code. This license can be found in this directory as `LICENSE`.
# Original Repository
The original Helical repository can be found at: https://github.com/helicalAI/helical
|