Hugging Face's logo

RyeAI
/
DA-BPE-CEREBRAS

Text Generation
Transformers
Safetensors
gpt2
Danish
BPE Tokenization
CerebrasGPT
text-generation-inference
Model card Files
xet
 Community
How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RyeAI/DA-BPE-CEREBRAS")
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RyeAI/DA-BPE-CEREBRAS")
model = AutoModelForCausalLM.from_pretrained("RyeAI/DA-BPE-CEREBRAS")
Quick Links

DA-BPE-CEREBRAS

This CerebrasGPT-111M model employs a standard Byte-Pair-Encoding (BPE) tokenizer for Danish text. It serves as a benchmark to compare against morphology-aware tokenizers.

Downloads last month
2
Safetensors
Model size
0.1B params
Tensor type
F16
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including RyeAI/DA-BPE-CEREBRAS