README.md

metadata

license: cc-by-nc-nd-4.0
language:
  - en
  - hi
  - kn
  - ta
  - te
  - ml
  - mr
pipeline_tag: text-generation
tags:
  - llama
  - lora
  - instruct
  - ezaris
  - multilingual

Ezaris-Instruct

The Ezaris base model with the v18 instruct LoRA applied — the current working instruction-tuned checkpoint of the Ezaris program.

Structure

base/         Ezaris base — 27.2B-token pretrain + 32B-token continued-pretraining (step 30,518, 2K ctx)
              Llama-style decoder: 20 layers · 2048 hidden · 16 heads / 8 KV · vocab 131,072 (Asterizer 128K)
              bf16 · tied embeddings · ~1.2B params
instruct_v18/ LoRA adapter (r=16, alpha=32, dropout=0.05) trained at step 4,000 on the base

Load

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("ASTERIZER/Ezaris-Instruct/base", trust_remote_code=True, torch_dtype="auto")
tok  = AutoTokenizer.from_pretrained("ASTERIZER/Ezaris-Instruct/base")
model = PeftModel.from_pretrained(base, "ASTERIZER/Ezaris-Instruct/instruct_v18")
model.eval()

prompt = "Explain artificial intelligence in simple terms."
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=128)
print(tok.decode(out[0], skip_special_tokens=True))

Base model lineage

• Pretrain: 240 GB multilingual corpus (40% South-Indian, 26 scripts, ~40 languages) → production_ready_pretrained_models/ (latest step 25,667)
• CPT: 32B tokens, 2K context, cpt_32b_2k → step 30,518 (this base/)
• Instruct: SFT LoRA v18 at step 4,000 (this instruct_v18/)

Full training sets, checkpoints, and fine-tuned versions: Ezaris-Training-Sets.

License

CC BY-NC-ND 4.0 — non-commercial, no derivatives, attribution required.