Laghu: a 152M chat model trained from scratch for $146. Everything is open.

It thinks before it answers (on/off), holds multi-turn conversations and chats, in 163 MB that runs in your browser.

UDGAM LABS · Research release · 22 October 2026

Today we are releasing Laghu, a small English chat model that we trained from scratch, together with everything we used to make it. This article is written for two kinds of readers. If you build with models, you will find commands, settings and file paths. If you do not, every idea is explained in plain words before its numbers. Laghu (लघु) is Sanskrit for "small, light".

Try it in your browser · Space · Model · Code · Technical report · Collection

TL;DR

• What it is. Laghu is a 152M-parameter English chat model. We trained it from scratch: a random starting point, our own 32,768-token tokenizer and our own data recipe. It is a research release.
• What it does. It thinks before it answers when you switch thinking on, holds multi-turn conversations and chats. It takes up to 16K tokens of input. The recommended file is 163 MB and runs in a browser tab, and Laghu also runs in Ollama, LM Studio, llama.cpp, MLX and transformers.
• What it cost. The main pretraining run cost $146.26: 8× A100 GPUs for 13.2 hours. The whole project cost about $194.
• How good it is. The chat model is level with SmolLM2-135M-Instruct on judged chat quality (51.0% win rate) and on multi-turn follow-ups (41.7% vs 37.5%, no significant difference), and ahead of it on maths with thinking (7.9% vs 1.8% on GSM8K). It is level with our own 384M model Gargi on chat quality and multi-turn at 40% of its size, and trails it on maths. The base model, Laghu-Base (152M), predicts new text better than SmolLM2-135M (135M) and Falcon-H1-Tiny-90M-Base (91M): 0.919 bits per byte against 0.941 and 0.979.
• Where it is behind. Falcon-H1-Tiny-90M-Instruct, trained with about 12× our compute and a preference-tuning stage, is ahead on all three of our chat categories. Laghu's maths (GSM8K 7.9%) and instruction following (IFEval 26.1) are weak.
• What is open. The weights of all three models, every checkpoint that survives (with optimiser states), the training code, the full recipe, data rebuild scripts, the chat-tuning data, every evaluation output item by item, what failed, and the cost.
• How we judged it. We wrote down the tests and the bar before measuring. Laghu did not clear the bar (0 of 3 categories). We publish it anyway, with everything.

Try it in a minute

About thinking. Ollama, LM Studio, llama.cpp and the MLX server turn thinking on by default; the commands above turn it off. Thinking helps only on short arithmetic (GSM8K 4.8% → 8.0%) and makes instruction following and multi-turn conversation worse, so leave it off for normal chat.

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("udgamlabs/Laghu")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu")

messages = [{"role": "user", "content": "A notebook costs 3 dollars. How much do 7 notebooks cost?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=True,
                                 return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7, top_p=0.95, top_k=64)
text = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
answer = text.rpartition("<channel|>")[2]          # the answer follows the thinking block, if there is one
print(answer.replace("<turn|>", "").strip())

This asks a short arithmetic question with thinking on, the one case where thinking helps. For normal chat, use enable_thinking=False (the default). Decode with skip_special_tokens=False and keep the text after the last <channel|>, as above; otherwise the thinking text stays in front of the answer. You need a transformers release that supports the Gemma 4 architecture; the model card names the version.

Everything we are releasing

Laghu went public today as seven model repositories, two datasets, a Space, a GitHub repository and a collection, plus a technical report. Here is what each one is and who it is for.

What is not here. The pretraining checkpoints from steps 2,000 to 74,000 and the earlier stretch checkpoints were lost on 21 October 2026 (see "What went wrong"). The 52.6B-token pretraining pool is published as a rebuild script with its source list, versions and manifest, not as files: its shards keep only the text, and would lose the per-document credit that some source licences require. The long-input data is a rebuild script too.

How we made Laghu

The whole process on one page

Making a chat model has three main stages. First the model reads a lot of text and learns to predict what comes next (pretraining). Then it learns to handle long inputs (the long-context stretch). Then it is shown example conversations, so that it answers questions instead of just continuing text (chat tuning). Each box on the right is a file you can download.

14 public text sources ─► tokenizer ─► 52.6B-token pool (decontaminated)
                                              │
                                              ▼
                     Pretraining: 8× A100, 13.2 h, $146 ─────────────► Laghu-Base
                                              │
                                              ▼
                     Long-context stretch: 1 GPU, ~4 h ──────────────► Laghu-Base-32K
                                              │
                                              ▼
                     Chat tuning: 41,467 conversations, 2 passes ────► Laghu
                                              │
                                              ▼
                     Conversion for local apps ──────────────────────► Laghu-GGUF, Laghu-MLX, Laghu-MLX-8bit

Words we use

1. The tokenizer

In plain words. A model cannot read letters directly. A tokenizer cuts text into tokens and gives each token a number. We trained our own tokenizer with 32,768 tokens. Our 384M model Gargi uses the same one.

The details. BPE with byte fallback, so any text can be encoded. Vocabulary 32,768, shared with Gargi. The input and output embeddings are tied: one matrix turns token numbers into vectors and turns vectors back into token scores. The tokenizer and the chat template used in training are in tokenizer/ on GitHub; the training script is udgam/tokenizer_train.py.

2. The model's shape

In plain words. Laghu uses the same layout as Google's Gemma 4 text models, scaled down to 152M parameters. We chose a standard layout so that tools that already run Gemma 4 run Laghu with no custom code: transformers, llama.cpp, Ollama, LM Studio, MLX, and llama.cpp compiled for the browser.

Inside, "attention" is how the model looks back at earlier text. Looking back over everything costs memory that grows with the length of the input. So in three of every four layers Laghu looks back only over the most recent 512 tokens (a "sliding window"), and every fourth layer looks at the whole input. This keeps memory small. In llama.cpp the 21 sliding-window layers need a fixed 10.5 MiB at any input length, and the whole server needs about 154 MiB at a 4K context and 264 MiB at 16K.

The details.

3. The pretraining data

In plain words. We gathered 52.6 billion tokens from 14 public sources whose licences allow training: educational web pages, rewritten educational text, maths, textbook-style synthetic text, Wikipedia, permissively licensed code, and NVIDIA's chat and science sets. We then removed every document that overlapped the tests we report, to keep those tests clean. The text came in two mixes: a broad "stable" mix for most of training, and a "cool-down" mix for the end with less web text and more maths, step-by-step solutions and chat.

The details.

• Pool v3: 52.6B tokens. Phase A (stable) 42.17B, phase B (cool-down) 10.33B, validation 0.10B. Built by udgam/build_mix.py --mix v3 on a 32-vCPU cloud machine in about 3.7 hours.
• Decontamination: every document was checked for shared 13-word runs (13-grams) against 10 evaluation sets (BFCL, IFEval, MMLU, HellaSwag, ARC, MATH-500, PIQA, OpenBookQA, GSM8K and CommonsenseQA). 21,427 documents were dropped (0.027%).
• Licence policy: the builder keeps an allow-list of generating models. It leaves out text written by models whose licences reach models trained on their outputs, and any non-commercial or unclear source.
• Shares of the pool (the builder's own token estimates per source; they do not add up exactly to the 52.6B tokenized total, and the report gives the tokenized counts):

4. Token bags

In plain words. Normally a model reads one token at a time and guesses the next one. For the first quarter of training we did something cheaper. Each input position held a "bag" of 4 tokens: the model saw the average of the 4 and had to predict all 4 tokens of the next bag. That way it skimmed through about 4 times as much text per step in that phase. For the remaining three quarters it trained the normal way, one token at a time.

The details. Token Superposition Training (TST) with bags of 4 tokens per position for the first 25% of steps. Each position's input is the mean embedding of its 4 tokens, and the loss is the mean cross-entropy over the 4 tokens of the next bag (nanochat/nanochat/gemma4.py, forward(..., tst_s=...)). The run had 40.0B token positions but read about 70B tokens in all; the stable mix was read about 1.47 times. Bits per byte measured during the bag phase are not comparable with next-token bits per byte, so the pretraining chart below starts after it.

5. The pretraining run

In plain words. We rented 8 A100 GPUs for 13.2 hours, at a cost of $146.26. In each step the model read 524,288 token positions, made its guesses, and nudged its 152 million numbers to guess better. It did that 76,294 times. Three choices shaped the run:

• The learning rate is how big each nudge is. We held it steady, then lowered it over the last 65% of training to 5% of its peak, so that the model settles.
• The cool-down mix. From 80% of the way through, the model read the cool-down mix.
• Weight averaging. Over the last 10% we kept a running average of the weights, called an EMA. Averaging smooths out step-to-step noise. Laghu-Base is that average.

The run had no loss spikes (sudden jumps in error that can derail training).

The details.

The settings are in udgam3/runs/main/main.conf, run by ops/pod_train.sh.

Held-out bits per byte during pretraining, from the training run's own scorer, after the token-bag phase (lower is better). The red line marks the switch to the cool-down mix. This scorer is not the one we use for the comparisons under Results, so read this chart for the trend, not the level.

6. The long-context stretch

In plain words. Pretraining used inputs of 2,048 tokens, and the base model loses track beyond that: hide a 5-digit code in a 4K-token text and it fails to find it. So we trained for about 4 more hours on inputs of 32,768 tokens: long documents (books, papers, government text and code) mixed with more ordinary text. Then we tested it by hiding the code in texts of different lengths. It finds the code 96-100% of the time up to 16K tokens, and never at 24K or 32K. So we say Laghu handles 16K tokens, not 32K.

The details.

• From the EMA weights at step 76,294: 1,900 steps × 524,288 tokens = 996M tokens at a 32,768-token context, on one RTX PRO 6000 Blackwell GPU (Colab G4), about 4 hours and about 40 Colab compute units.
• Hidden-code test (one 5-digit code, 100 trials per length): 100 / 100 / 99 / 96% at 4K / 8K / 12K / 16K tokens; 0% at 24K and 32K. The 2K base model fails from 4K.
• Cost on short text, measured before and after: the commonsense average on validation splits went from 50.81 to 50.05 (−0.76; our limit was 1.0, so it passed). Bits per byte at 2K: web +0.40%, chat-formatted task text +0.82%, new text −0.04%, reasoning text +1.10%. The reasoning figure missed our 1% limit narrowly; we accepted it.
• Scripts: udgam2/ctx32k/build_long32k.py and udgam3/runs/ctx32k/build_ctx32k_data.py (data), udgam3/runs/ctx32k/colab_ctx32k_pipeline.sh (training).

7. Chat tuning

In plain words. A base model only continues text; it does not follow requests. To make it answer, we showed it 41,467 example conversations, and it learned to write the assistant's side. About a third of them have several turns, so it also learned to follow up on what was said earlier. The examples cover everyday questions, follow-up questions, maths, instructions with checkable rules, Python tool use, long documents, safety, and who it is ("I'm Laghu, a small language model made by Udgam Labs."). Its training data includes logic puzzles. We showed the whole set twice.

The details.

• Size: 41,467 conversations, 34.5% multi-turn. 28.2M tokens per pass, of which 9.57M are trained on (the assistant's replies; prompts and tool results are not). 2 passes, so about 19.1M trained-on tokens. Sequences up to 16K tokens.
• Sources: answers by DeepSeek-V4.1-Flash to real-user-style prompts (from WildChat, Chatbot Arena and OpenAssistant), which cost $9.49 of API use; multi-turn conversations by gpt-oss-120b; verified rows from our data factory, among them 1,023 logic puzzles generated by our own Python code; and identity rows. Every row in laghu-sft names its source, generator and licence. A Python tool mode is in the data; we make no quality claim for it.
• A fix before tuning: the 12 chat-format tokens never appeared in plain-text pretraining. With tied embeddings, they ended up with identical embeddings (cosine similarity above 0.999). We re-drew them before chat tuning.
• How many passes: we trained 1, 2 and 3 passes. Held-out loss preferred 1 pass (1.724 against 1.784 for 2). A rule we wrote before generating anything compared actual replies instead: a judge preferred the 2-pass model in 60.0% of 200 separate, leak-free conversations (95% CI 54.9-65.0), and its accuracy on GSM8K training problems with thinking was 7.0% against 4.0%. Laghu is the 2-pass model.
• Scripts: udgam3/sft/gen/ (generation, checks and screening), udgam3/sft/render_v1.py, udgam3/sft/colab_sft_pipeline.sh; the pick rule is udgam3/sft/devpick/DEV_RULE.md.

8. The thinking switch

In plain words. With thinking on, Laghu first writes out its reasoning and then gives the answer. Many maths examples in the chat data come in two versions, with and without the reasoning, so the model learned both habits, and a switch picks one. At this size, thinking helps only on short arithmetic, and the reasoning itself is often wrong even when the final answer is right. That is why we recommend leaving it off for normal chat.

The details. Laghu uses the Gemma 4 chat template. With enable_thinking=True it writes a thought block between <|channel>thought and <channel|>, then the answer, then <turn|>. The switches: transformers enable_thinking=False (the default), Ollama --think=false, llama-server --reasoning off, MLX server --chat-template-args '{"enable_thinking":false}', LM Studio API "reasoning_effort": "none".

9. Reinforcement learning: tried, not used

In plain words. Reinforcement learning lets a model try many answers to a problem and rewards the right ones. We tried it once on maths, combined with learning from Gargi's answers (distillation). It made maths no better and made looping worse, so it failed the test we had set for it in advance. The released Laghu does not use it.

The details. Run R1: GRPO plus on-policy distillation from Gargi, 40 steps, on Colab (4.6 compute units). Accuracy on GSM8K training problems went from 8.6% to 8.2% against a bar of 11.6%, and the loop rate on MATH rose from 10.5% to 19.5%. Runs R2 and R3 were never started. This is our third reinforcement-learning failure at 384M parameters or fewer (Gargi had two). The run is in rl-r1-failed/ in Laghu-Checkpoints; the scripts and the gate are in udgam3/rl/ (GATES.md).

10. Formats, speed and memory

In plain words. To run Laghu on your own computer, we converted it to GGUF (for llama.cpp, Ollama and LM Studio) and to MLX (for Apple silicon Macs). We also made smaller files by storing each number in 8 bits (quantisation), and checked that they behave almost exactly like the full model. 4- and 5-bit versions lost clearly, so we did not publish them. In the browser, the same 163 MB file runs through llama.cpp compiled to WebAssembly.

The details.

Mean KL measures how far a file's next-token probabilities drift from the full-precision file (0 means identical). Top-1 agreement is how often both pick the same most likely next token. Q8_0's perplexity is 52.11 against 52.24 for F16.

• Speed on a Mac mini (Apple M6, 24 GB): about 330 tokens per second generating and about 21,300 tokens per second reading the prompt in llama.cpp (Q8_0, Metal); about 456 tokens per second generating in MLX (8-bit).
• Memory in llama-server (Q8_0, one slot): 154 MiB at a full 4K context, 264 MiB at a full 16K context. llama-server caps the context at 16,384 tokens.
• Chat format: checked in llama.cpp (--jinja), Ollama and MLX with thinking off and on, multi-turn conversations and tool calls.
• Ollama: ollama run hf.co/udgamlabs/Laghu-GGUF:Q8_0 uses a template and parameters that Hugging Face builds from the repository's template and params files. We checked them byte for byte, and in a local Ollama store the prompts matched the transformers template's length in 12 of 12 cases. The thinking shows as a separate block. Tool calls need the Modelfile route: download Laghu-Q8_0.gguf and Modelfile, then ollama create laghu -f Modelfile (tested with Ollama 0.35.1).
• LM Studio (0.4.26): loads Q8_0, chats, keeps context across turns and returns the thinking apart from the answer. Through the API, "reasoning_effort": "none" turns thinking off; chat_template_kwargs is ignored. We did not check the app's own thinking toggle.
• Browser: wllama 3.6.1 (llama.cpp compiled to WebAssembly). The page's chat format matches the transformers template on 8 of 8 test conversations, by text and by token count. With the COOP/COEP headers it runs multi-threaded. We do not claim a browser speed; the page shows you your own.

Results, honestly

How we measured

In plain words. We measured every model ourselves, with the same scripts, the same test items and the same settings, on the same kind of GPU, on 19 October 2026. Before measuring Laghu's chat model, we wrote down the three categories, the three rivals, the statistics, and the bar for a best-in-class claim: ahead of every rival in all three categories. Because we made nine comparisons, we applied a Holm correction, a standard adjustment that stops one lucky result from counting as a win.

The details. Chat models: vLLM, bf16, each model's own chat template, the same items for every model, on CUDA (Colab G4). Base models: lm-evaluation-harness 0.4.13, bf16, zero-shot (GSM8K 5-shot), <bos> added. Held-out text: one script (udgam3/evals/own/bpb_hf.py) and the same documents for every model. The pre-registration and its amendment log are in udgam3/PREREGISTRATION.md.

The base model on new text

In plain words. Give a model text it has never seen, and ask how surprised it is by each next piece. "Bits per byte" measures that surprise: roughly, how many yes/no questions the model needs to pin down each byte of the text. Lower is better. Our test set is 1,695 documents written in September and October 2026: Wikipedia pages created between 1 September and 15 October, and arXiv abstracts submitted between 1 and 15 October. It is new text, which makes it the cleanest comparison we have.

Laghu-Base read about 70 billion tokens in pretraining (about 71 billion across all of Laghu's training stages), about 28× fewer than the 2 trillion that SmolLM2-135M was pretrained on, and it still predicts this new text better: 0.919 against 0.941 bits per byte (difference 0.022, 95% CI 0.020-0.024). It is also ahead of Falcon-H1-Tiny-90M-Base (difference 0.060, CI 0.057-0.062), and 0.005 behind our own Gargi-Base, which is 2.5× larger (CI 0.004-0.006). The intervals come from a paired bootstrap: we resampled the 1,695 documents 4,000 times and recomputed the gap each time. Token counts use each model's own tokenizer, so the ratios are approximate.

Read this next to the limits. This is a result on one held-out set, not a general efficiency claim. Falcon-H1-Tiny-90M-Instruct, trained with about 12× our compute and a preference-tuning stage, is ahead of Laghu on all three chat categories below, and Laghu's maths (GSM8K 7.9%) and instruction following (IFEval 26.1) are weak. Laghu-Base also trails SmolLM2-135M on the five-task multiple-choice average (49.9 against 51.8). Other small models have also matched SmolLM2-135M on far fewer tokens; GPT-X3-150M, for example, advertises 27× fewer.

For developers: all three held-out sets.

Laghu-Base is ahead of SmolLM2-135M on new text and reasoning text (reasoning: by 0.059, CI 0.056-0.062), level on web text (−0.0001, CI −0.0035 to +0.0037), ahead of Falcon-H1-Tiny-90M-Base and Udgam-384M-Base on all three, and slightly behind Gargi-Base. The web and reasoning sets are held-out slices of our own data lineage, which favours our models; the new-text set is the clean comparison. The bootstrap script is udgam3/evals/own/bpb_compare.py.

Multiple-choice benchmarks

In plain words. These are standard quizzes: pick the right ending to a sentence, the right answer to a science question, the sensible way to do an everyday task. At this size, scores sit close to guessing on some of them: MMLU (four answer choices) at 25.3 is about what guessing gives.

On the 5-task average, Laghu-Base is 5th of the 19 models up to 200M parameters that we measured on 19 October 2026 (18 rivals plus Laghu). GPT-X3-150M has the highest average in that set. Newer models, such as Boris-2-Preview-2 (21 October), are not in it. We pre-registered no base-model claim and publish every result.

The chat model against size-class rivals

In plain words. We fixed three categories in advance:

• Maths with thinking: GSM8K, 1,319 grade-school word problems. Strict scoring: a reply cut off by the length limit counts as wrong.
• Multi-turn: a second question that depends on the first answer. We count how often the follow-up is right, over 144 checkable items.
• Chat quality: a judge model (DeepSeek-V4.1-Flash) compares Laghu's reply with the rival's reply to the same prompt, 324 held-out real-user-style prompts, each pair shown in both orders, with an adjustment for reply length. A 50% win rate means level.

After the Holm correction over nine comparisons:

• Laghu is ahead of SmolLM2-135M-Instruct and MobileLLM-R1.5 on maths with thinking.
• Laghu is ahead of MobileLLM-R1.5 on multi-turn.
• Laghu is level with SmolLM2-135M-Instruct on multi-turn and chat quality, and level with MobileLLM-R1.5 on chat quality. Against MobileLLM-R1.5, Laghu's raw win rate is 91.0%, but that number is confounded by reply length; the length-controlled 57.3% has a wide interval.
• Falcon-H1-Tiny-90M-Instruct is ahead of Laghu in all three categories. Falcon is smaller (91M), but used about 12× more training compute (≈ 4.4 × 10²⁰ against 3.6 × 10¹⁹ FLOPs), 800B tokens and a preference-tuning (DPO) stage. The Falcon figures come from TII's published description; the FLOPs are our estimate.
• Categories won against all three rivals: none.

MobileLLM-R1.5-140M results are reported for noncommercial research under the FAIR Noncommercial Research License; we acknowledge Meta's materials.

Level with our own 384M Gargi

Laghu matches our 384M Gargi on chat quality and multi-turn at 40% of its size, and trails it on maths. "Matches" here means no significant difference:

• chat quality: Laghu's win rate against Gargi is 45.9% (p = 0.06);
• multi-turn: 41.7% against Gargi's 49.3% (p = 0.12), and against 40.3% for our Udgam-384M-Chat (p = 0.89);
• maths with thinking: 7.9% against Gargi's 35.6%.

Other measurements (reported, not claimed)

Small differences are noise. At this size, bf16 greedy decoding is not batch-invariant: for Laghu, 32 of 40 check items gave identical first 512 tokens when run in a batch and one at a time.

Where it falls short

1. Maths. 7.9% on GSM8K (strict, with thinking). The arithmetic inside its thinking is often wrong. Falcon-H1-Tiny-90M-Instruct scores 23.3%; Gargi 35.6%.
2. Instruction following. IFEval prompt-level strict 26.1, against 60.8 for Falcon-H1-Tiny-90M-Instruct. Thinking on makes it worse (17.7).
3. It trails Falcon-H1-Tiny-90M-Instruct on all three pre-registered chat categories. Falcon is smaller (91M) but used about 12× more training compute (800B tokens and a DPO stage).
4. Loops. With greedy decoding, 8.3% of replies loop to the length cap with thinking off and 12.0% with thinking on. The shipped sampling settings bring this to 1.2-1.9%.
5. Long context. Recall works up to 16K tokens and fails beyond (0% at 24K and 32K).
6. Knowledge. A 152M model knows little and states wrong facts confidently. Open questions often get incoherent answers. Do not rely on it for facts, advice or decisions that matter.
7. Thinking mode helps maths slightly (GSM8K 4.8% → 8.0%) but hurts instruction following and multi-turn (41.7% → 32.6%).

It was trained on English only, and its safety training is light: it can produce wrong, biased or harmful text. The base models have no safety tuning.

What went wrong

We publish the failures with the model. Each one is in the technical report with its evidence.

1. The release gate failed. Laghu won 0 of 3 pre-registered categories against all three rivals. It is released only because we decided to publish it as a fully open research release (see the next section).
2. Reinforcement learning failed. Run R1 (GRPO plus distillation from Gargi) moved GSM8K-train accuracy from 8.6% to 8.2% against an 11.6% bar, and the MATH loop rate rose from 10.5% to 19.5%. R2 and R3 were never run. It is our third reinforcement-learning failure at 384M parameters or fewer.
3. A leaky validation split. We split the chat data by row id, which put related rows on both sides: thinking on/off twins, several questions about one document, and multi-turn rows from one seed. About 37% of eligible validation conversations had a near-copy in training. So the pick between 1, 2 and 3 passes used 200 separate, leak-free conversations.
4. Held-out loss picked the wrong length. Loss preferred 1 pass (1.724 against 1.784). A rule based on actual replies, written in advance, picked 2 passes (judge win rate 60.0% [54.9, 65.0]; GSM8K-train 7.0% against 4.0%).
5. The 32K stretch overshot a limit. Bits per byte on reasoning text rose 1.10% against the 1% limit we had set. We accepted it.
6. Chat tokens collapsed before tuning. With tied embeddings and plain-text pretraining, the 12 never-seen chat tokens ended with identical embeddings (cosine above 0.999). We fixed this by re-drawing them before chat tuning.
7. We lost checkpoints. The pretraining checkpoints saved every 2,000 steps (steps 2,000-74,000) and about 6 earlier 32K checkpoints were deleted on 21 October 2026 by a storage clean-up that squashed file history. That was our error. Only what survives is published, so the release cannot show how Laghu learned over the whole run.
8. The main run failed at its first start. It died at step 0, because a validation file had 1 row group for 8 GPUs (about $1.9 lost). Later, a monitor crash on a network stall hung the experiment-tracking sync for 5.5 hours; we backfilled it from the run log.
9. The pre-registration was finished late. The rival measurements were not done before the main run, so part 2 of the pre-registration was written in two stages after training, but before each result it covers was measured. The amendment log notes this.
10. The run came in below projection. Before the run we projected 0.900-0.912 bits per byte on new text after the 32K stretch [estimate]. We measured 0.921.

How we pre-registered, and why we released anyway

In plain words. Before measuring, we wrote down what we would test, against whom, how we would count, and what result would allow a best-in-class claim. Writing this first means the result cannot be tuned after the fact. The result came in below the bar. We chose to publish it as measured, with everything, and to make no best-in-class claim.

The timeline.

• 17 October 2026: part 1 of the pre-registration, written before any training.
• 19 October 2026, morning: part 2a, the base-model claims (none; every result reported), written before Laghu's base suite ran.
• 19 October 2026, evening: part 2b, the three chat categories, Holm correction over 9 comparisons, a loop limit of 10% or less and a bar against our own models, written before any Laghu chat result. Five minutes later, still before measuring, we changed the judge to DeepSeek-V4.1-Flash, because our account could not reach the pre-registered gpt-oss-120b judge.
• 19 October 2026, later that evening: the gate failed, 0 of 3. Finding out cost 1.4 Colab compute units and $0.45.
• 21 October 2026: we decided to release Laghu as a fully open research release with no best-in-class claim. No result or claim changed.

What it cost

Money that produced no kept result (failed starts, stopped runs) was about $5.05, or 2.6% of the total [estimate].

How to reproduce it

Every stage ran from the files in github.com/udgamlabs/laghu. The scripts are published exactly as they ran. Private Hugging Face repositories are renamed to YOUR_HF_NAMESPACE/... placeholders (docs/NAME_MAP.md), and absolute paths became $LAGHU_REPO, the root of the repository.

Check our numbers without retraining. laghu-evals has every model's output and score on every item. udgam3/evals/own/bpb_compare.py re-runs the bootstrap on the per-document bits. Some benchmarks ship as ids and scores only; the dataset's join_items.py puts their text back from pinned public revisions.

Start from a checkpoint. The files in Laghu-Checkpoints are nanochat torch.save files. Load them with the training code in nanochat/, or convert one to the transformers format with python -m scripts.export_hf --checkpoint-dir <folder> --weights final --out <dir>.

Licences

• Weights and our code: Apache-2.0, "Copyright 2026 Udgam Labs LLP". Files that come from karpathy/nanochat keep their MIT notice (see NOTICE). The chat template is Google's Gemma 4 template (Apache-2.0).
• Data we made or curated: CC-BY-4.0. Third-party rows keep their own licence, recorded per row in laghu-sft.
• Evaluation outputs: CC-BY-4.0 for what we produced; other models' outputs carry the notices on the laghu-evals card. MobileLLM results are reported for noncommercial research under the FAIR Noncommercial Research License, as aggregate scores only.
• Every source, its licence and the credit it asks for: docs/DATA_LICENSES.md.

Credits

Laghu learned from FineWeb-Edu, FineMath, FinePhrase and FinePDFs-Edu (Hugging Face), Ultra-FineWeb (OpenBMB), DCLM-Edu (DataComp-LM), SYNTH (Pleias), Nemotron datasets (NVIDIA), github-code-clean, Wikipedia, the Common Pile, WildChat (AI2), Arena prompts (LMArena), oasst2 (OpenAssistant) and GSM8K (OpenAI). Answers were written by DeepSeek-V4.1-Flash, gpt-oss-120b and the models in our data factory, and each row names its generator. The training code builds on karpathy/nanochat, and the architecture and chat format follow Google's Gemma 4. Thank you to everyone who made these open.

Citation

@misc{udgamlabs2026laghu,
  title  = {Laghu: a fully open 152M chat model trained from scratch},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Laghu}
}

---

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com

Laghu by Udgam Labs

Udgam Labs · AI research from first principles