File size: 855 Bytes
199958e e8b4a80 199958e e8b4a80 199958e e8b4a80 199958e e8b4a80 199958e e8b4a80 199958e e8b4a80 199958e e8b4a80 199958e e8b4a80 199958e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
---
language:
- tl
tags:
- tagalog
- text-generation
- custom-architecture
- pytorch
license: mit
---
# Henyo-70M
**Henyo** is a custom Tagalog LLM trained on a subset of Wikipedia.
## Model Architecture
- **Parameter Count**: 70M
- **Architecture**: Decoder-only Transformer (Custom)
- **Features**:
- **SwiGLU** Activation
- **Grouped Query Attention (GQA)**
- **Rotary Positional Embeddings (RoPE)**
- **RMSNorm**
## Usage
This model uses a custom architecture code. You can load it using the `AutoModel` class with `trust_remote_code=True` (if code is uploaded) or by defining the class manually.
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("marcuscedricridia/Henyo-70M")
# Note: Since this is a custom model, you may need the inference script provided in the repo.
|