File size: 1,884 Bytes
0032d60 9c945c9 0032d60 9c945c9 0032d60 9c945c9 0032d60 9f62ab6 0032d60 9c945c9 0032d60 9c945c9 0032d60 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | ---
license: cc-by-nc-nd-4.0
language:
- en
- hi
- kn
- ta
- te
- ml
- mr
pipeline_tag: text-generation
tags:
- llama
- lora
- instruct
- ezaris
- multilingual
---
# Ezaris-Instruct
The **Ezaris** base model with the **v18 instruct LoRA** applied — the current working instruction-tuned checkpoint of the Ezaris program.
## Structure
```
base/ Ezaris base — 27.2B-token pretrain + 32B-token continued-pretraining (step 30,518, 2K ctx)
Llama-style decoder: 20 layers · 2048 hidden · 16 heads / 8 KV · vocab 131,072 (Asterizer 128K)
bf16 · tied embeddings · ~1.2B params
instruct_v18/ LoRA adapter (r=16, alpha=32, dropout=0.05) trained at step 4,000 on the base
```
## Load
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("ASTERIZER/Ezaris-Instruct/base", trust_remote_code=True, torch_dtype="auto")
tok = AutoTokenizer.from_pretrained("ASTERIZER/Ezaris-Instruct/base")
model = PeftModel.from_pretrained(base, "ASTERIZER/Ezaris-Instruct/instruct_v18")
model.eval()
prompt = "Explain artificial intelligence in simple terms."
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=128)
print(tok.decode(out[0], skip_special_tokens=True))
```
## Base model lineage
- **Pretrain**: 240 GB multilingual corpus (40% South-Indian, 26 scripts, ~40 languages) → `production_ready_pretrained_models/` (latest step 25,667)
- **CPT**: 32B tokens, 2K context, `cpt_32b_2k` → **step 30,518** (this `base/`)
- **Instruct**: SFT LoRA `v18` at step 4,000 (this `instruct_v18/`)
Full training sets, checkpoints, and fine-tuned versions: [Ezaris-Training-Sets](https://huggingface.co/datasets/ASTERIZER/Ezaris-Training-Sets).
## License
CC BY-NC-ND 4.0 — non-commercial, no derivatives, attribution required.
|