You need to agree to share your Hugging Face information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
To get access, please fill in the short form at https://saidzoda.com/access. We approve requests after we receive the form.
Log in or Sign Up to review the conditions and access this model content.
Gemma-4-E4B-it β Tajik adaptation (CPT β SFT β tool-calling)
How to get access: please fill in the short form at saidzoda.com/access. We approve requests after we receive the form.
Saidzoda Lab. One model line, trained 2026-09-22 β 2026-09-25 on 6ΓH100.
Base: google/gemma-4-E4B-it. Adapters are LoRA; only the final tool-calling stage is
published as a full merged model.
Contents
| path | what | size | format |
|---|---|---|---|
cpt-lora/ |
stage 1 β Tajik continual pretraining | 3.33 GB | LoRA adapter (r64, Ξ±64, rslora; embed_tokens+lm_head trained) |
sft-lora/ |
stage 2 β Tajik supervised fine-tune | 325 MB | LoRA adapter (r32) |
tool-full/ |
stage 3 β tool-calling, full merged model | 15 GB | safetensors |
Each LoRA applies to the output of the previous stage, not to the stock base:
base β cpt-lora β (merge) β sft-lora β (merge) β tool-full.
So sft-lora is not loadable on stock E4B β it expects the CPT-merged model.
Measured results
Frozen benches, method = log-likelihood, identical across every row so the deltas are comparable. Full sets (no subsampling).
| stage | belebele-tgk (900) | tajik-curated (657) |
|---|---|---|
stock gemma-4-E4B-it |
37.8% (340) | 28.3% (186) |
| + CPT | 63.8% (574) | 44.9% (295) |
| + SFT | 72.1% (649) | 53.4% (351) |
| + tool (this repo's full model) | 72.6% (653) | 53.7% (353) |
belebele 37.8 β 72.6 (+34.8 pp), curated 28.3 β 53.7 (+25.4 pp). CPT alone contributes +26.0 / +16.6 β the largest single step, because stock E4B is not competent in Tajik (37.8% against a 25% chance floor on a 4-choice benchmark).
Read before using the numbers
- These are log-likelihood scores. They are not comparable to chat/generative-mode scores, and LL generally understates an instruction-tuned model's practical performance.
- The CPT stage ran 1.5 of 2 planned epochs (stopped at step 26,500/35,332 by owner decision, mid-cosine at 16% of peak LR). It is not an annealed endpoint. Loss was still improving when stopped.
- The tool-calling stage adds +0.5/+0.3 on these MCQ benchmarks, which is inside noise β it targets tool-call formatting, not knowledge, so these benches are not the right measure of it.
- No GRPO stage. GRPO fails on E4B with
ValueError: Target module Gemma4ClippableLinear(...) is not supportedβ PEFT cannot wrap E4B's non-nn.Linearwrapper modules. Unresolved at publication.
Training
| stage | data | steps | seq | LR | schedule |
|---|---|---|---|---|---|
| CPT | Tajik text corpus | 26,500 / 35,332 | 2048 packed | 5e-5 | cosine |
| SFT | Tajik instruction set (1,112,888 rows) | 37,097 (2 ep) | 1024 | 1e-5 | cosine |
| tool | tool-call set (176,522 rows) | 2,705 (1 ep) | 1024 | 2e-5 | cosine |
Global batch 60 on all stages (6-GPU DDP). bf16, max_grad_norm 0.5.
Gated (manual) β access on request.