metadata
language:
- tl
tags:
- tagalog
- text-generation
- custom-architecture
- pytorch
license: mit
Henyo-70M
Henyo is a custom Tagalog LLM trained on a subset of Wikipedia.
Model Architecture
- Parameter Count: 70M
- Architecture: Decoder-only Transformer (Custom)
- Features:
- SwiGLU Activation
- Grouped Query Attention (GQA)
- Rotary Positional Embeddings (RoPE)
- RMSNorm
Usage
This model uses a custom architecture code. You can load it using the AutoModel class with trust_remote_code=True (if code is uploaded) or by defining the class manually.
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("marcuscedricridia/Henyo-70M")
# Note: Since this is a custom model, you may need the inference script provided in the repo.