Bio Series
Collection
Embeddings and NLG related to biology / amino acid sequences • 10 items • Updated • 1
# Load model directly
from transformers import AutoModelForMaskedLM
model = AutoModelForMaskedLM.from_pretrained("monsoon-nlp/dna-blockdiff", trust_remote_code=True, dtype="auto")Quick Links
DNA and Block Diffusion
Untrained architecture test using the Block Diffusion architecture and AgroNT's six-nucleotide-length tokens.
Loading model
from transformers import AutoModelForMaskedLM
m = AutoModelForMaskedLM.from_pretrained(
"monsoon-nlp/dna-blockdiff",
trust_remote_code=True,
)
Generating text
cd bd3lms
python -u main.py \
loader.eval_batch_size=1 \
model=small \
algo=bd3lm \
algo.T=900 \
algo.backbone=hf_dit \
algo.sampler=analytic \
data=openwebtext-split \
model.length=2048 \
block_size=4 \
wandb=null \
mode=sample_eval \
eval.checkpoint_path="monsoon-nlp/dna-blockdiff" \
model.attn_backend=sdpa \
sampling.nucleus_p=0.9 \
sampling.kv_cache=true \
sampling.logdir=$PWD/sample_logs/samples_genlen_bd3lm_blocksize4 \
data.tokenizer_name_or_path="monsoon-nlp/dna-blockdiff"
- Downloads last month
- 6
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="monsoon-nlp/dna-blockdiff", trust_remote_code=True)