Feature Extraction
Transformers
PyTorch
Hungarian
byte representation
gradient boosting
hungarian
custom_code
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("SzegedAI/charmen-electra", trust_remote_code=True, device_map="auto")Quick Links
Charmen-Electra
A byte-based transformer model trained on Hungarian language. In order to use the model you will need a custom Tokenizer which is available at: https://github.com/szegedai/byte-offset-tokenizer.
Since we use a custom architecture with Gradient Boosting, Down- and Up-Sampling, you have to enable Trusted Remote Code like:
model = AutoModel.from_pretrained("SzegedAI/charmen-electra", trust_remote_code=True)
Acknowledgement
- Downloads last month
- 8
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="SzegedAI/charmen-electra", trust_remote_code=True)