---
license: apache-2.0
language: [te]
library_name: transformers
tags: [udgamlabs, trained-from-scratch, base-model, telugu, checkpoints, gemma4]
---

![Vemana · Udgam Labs](./banner.png)

**UDGAM LABS** · Training checkpoints · Vemana, 278M parameters, Telugu

# Vemana-Checkpoints

Twelve checkpoints from the training run of [Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base), a 278M Telugu
language model trained from scratch on 12.0B token positions (one H100, about 16 hours, 26-27 October 2026, live at
[udgamlabs.com/live/vemana](https://udgamlabs.com/live/vemana)). Each folder is a complete Hugging Face model
(`Gemma4ForCausalLM`, float32) with the tokenizer, so any of them loads on its own:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
```

| Folder | Step | Token positions | Fresh-text bits per byte (416 pages, Sep-Oct 2026) |
|---|---|---|---|
| `m010-step-002289` | 2,289 | 1.2B | 0.420 |
| `m020-step-004578` | 4,578 | 2.4B | 0.414 |
| `stable-025-step-005722` | 5,722 | 3.0B | 0.405 |
| `gate-step-005722` | 5,722 | 3.0B | 0.383 (a copy cooled down from step 4,578, scored as a finished model) |
| `m030-step-006866` | 6,866 | 3.6B | 0.406 |
| `m040-step-009155` | 9,155 | 4.8B | 0.400 |
| `m050-step-011444` | 11,444 | 6.0B | 0.398 |
| `m060-step-013733` | 13,733 | 7.2B | 0.398 |
| `m070-step-016022` | 16,022 | 8.4B | 0.398 |
| `m080-step-018310` | 18,310 | 9.6B | 0.393 (start of the final cool-down) |
| `m090-step-020599` | 20,599 | 10.8B | 0.371 |
| `final-step-022888` | 22,888 | 12.0B | 0.361 (= Vemana-Base) |

The scores use the same script and pages as the [Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base) card
(lower is better). Most of the gain comes in the cool-down: the stable checkpoints level off near 0.398 between 40% and
70% of the run, and the last 20% brings the score to 0.361.

Each folder also holds `fresh.json` (the per-page scores behind the table) and `train_state.json` (the run's step,
schedule settings and the text it wrote at that point).

## Licence

Apache-2.0. Copyright 2026 Udgam Labs LLP. Training data credits are in the
[Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base) `NOTICE`.

---

**About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)