Upload README.md

README.md

@@ -1,5 +1,5 @@
 ---
-license: apache-2.0
+license: gpl-3.0
 datasets:
 - HuggingFaceFW/fineweb-edu
 - mattwesney/General_Inquiry_Thinking-Chain-Of-Thought
@@ -11,152 +11,163 @@ language:
 - en
 tags:
 - small
-- haiku
-new_version: CompactAI-O/TMLM-Haiku-2.3
+- glint
+new_version: CompactAI-O/Glint-0.4
 ---
-# TMLM-Haiku-2
+
+# Glint-0.3
 
 > A 1M-parameter language model that speaks English, technically.
 
-    WARNING: This model was trained on a shoestring budget and a prayer. 
-    It does not answer questions correctly. It does not follow instructions well. 
-    It does, however, occasionally produce output that sounds profound until you read it twice.
+```
+WARNING: This model was trained on a shoestring budget and a prayer.
+It does not answer questions correctly. It does not follow instructions well.
+It does, however, occasionally produce output that sounds profound
+until you read it twice.
+```
 
 ---
 
 ## Overview
 
-TMLM-Haiku-2 is a tiny autoregressive language model with approximately one million parameters. That is not a typo. In an era where models are measured in billions, we went the other direction, mostly because we could.
+Glint-0.3 is a tiny autoregressive language model with about one million parameters. That is not a typo. While everyone else is measuring models in billions, we went the other direction. Mostly because we could.
 
-It was trained on English text at a ratio of 100 tokens per parameter. The math is simple: 1M parameters × 100 tokens = \~100M total tokens. We split this budget deliberately: roughly two-thirds (\~66.7M tokens) went into general pretraining, after which the resulting checkpoint was fine-tuned on instruction data using the remaining one-third (~33.3M tokens).
+It was trained on 100 tokens per parameter. The math: 1M params x 100 tokens = ~100M total. Two thirds went to pretraining (~66.7M tokens), then we used the rest to fine-tune on instruction data.
 
-This approach lets us squeeze more signal out of every parameter. It does not make the model smart. It just makes it slightly less confused than it would have been otherwise.
+This does not make the model smart. It just makes it slightly less confused than it would have been otherwise.
 
-Curiously, the pretrained checkpoint tends to produce more fluent output than the instruction-tuned version. It turns out that teaching a model to follow directions might actually make it worse at saying things. Who knew.
+Funny thing: the pretrained checkpoint sounds more fluent than the instruction-tuned one. Turns out teaching a model to follow directions can make it worse at saying things. Who knew.
 
 ---
 
-## Efficiency Claim (The One We Are Allowed to Make)
+## The one claim we are allowed to make
 
-At 100 tokens per parameter, TMLM-Haiku-2 is **one of the most efficient models out there** at this scale. If you were to compare it against another 1M parameter model trained on fewer tokens, these weights would likely outperform them.
+At 100 tokens per parameter, Glint-0.3 is one of the most efficient models at this scale. Against another 1M model trained on fewer tokens, these weights would likely win.
 
-We are not claiming state of the art. We are claiming that, for a model this small, we got more out of the data than most would. That is a modest victory, but we will take it.
+We are not saying state of the art. We are saying we got more out of the data than most would. Modest victory. We will take it.
 
 ---
 
-## Sample Outputs
+## Sample outputs
 
-### Instruction-Tuned
+### Instruction-tuned
 
-    User: Is food safe to eat?
-    Model: As an Age, in relation (a rush-rict), a decisios is the **exactly* estimate** 
-           (each labeling at * *Crise*).
+```
+User: Is food safe to eat?
+Model: As an Age, in relation (a rush-rict), a decisios is the **exactly* estimate**
+       (each labeling at * *Crise*).
 
-           When you cannot find all faces (doinins one way tea t become flawed) are no 
-           longer rartints like awarded, n but wheret dogs control crashed ttega y
+       When you cannot find all faces (doinins one way tea t become flawed) are no
+       longer rartints like awarded, n but wheret dogs control crashed ttega y
+```
 
 ### Pretrained
 
-    User: A cat, commonly known as the
-    Model: primary current burden of transportation wasnt impossible. A recence includes 
-           a personal crisis for participated populatably, to make sure technology 
-           systems change from an illustrious phase orbit oncold, rock areas, along 
-           with localizators outside again
+```
+User: A cat, commonly known as the
+Model: primary current burden of transportation wasnt impossible. A recence includes
+       a personal crisis for participated populatably, to make sure technology
+       systems change from an illustrious phase orbit oncold, rock areas, along
+       with localizators outside again
+```
 
-We recommend treating these outputs as experimental prose. If you find meaning in them, that is probably more about you than the model.
+If you find meaning in these, that says more about you than the model.
 
 ---
 
-## Technical Specifications
+## Specs
 
-| Attribute | Value |
-|-----------|-------|
+| Thing | Value |
+|-------|-------|
 | Parameters | ~1,000,000 |
 | Language | English |
 | Tokenization | Word-level |
 | Architecture | Lightweight Transformer |
 | Total Tokens | ~100M (100 tokens/param) |
-| Pretraining Tokens | ~66.7M (2/3 of budget) |
-| Instruction Tokens | ~33.3M (1/3 of budget) |
-| Target Throughput | ~1M tokens/sec |
-| License | MIT |
-| Repository | https://huggingface.co/CompactAI/TMLM-Haiku-2 |
+| Pretraining Tokens | ~66.7M |
+| Instruction Tokens | ~33.3M |
+| Throughput | ~1M tokens/sec |
 
 ---
 
-## Getting Started
+## Usage
 
-    from transformers import AutoModelForCausalLM, AutoTokenizer
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
-    model_name = "CompactAI/TMLM-Haiku-2"
-    tokenizer = AutoTokenizer.from_pretrained(model_name)
-    model = AutoModelForCausalLM.from_pretrained(model_name)
+model_name = "CompactAI-O/Glint-0.3"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
 
-    prompt = "A cat, commonly known as the"
-    inputs = tokenizer(prompt, return_tensors="pt")
-    outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.9)
+prompt = "A cat, commonly known as the"
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.9)
 
-    print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
 
-Pro tip: Adjust temperature between 0.8 and 1.2 for optimal levels of confusion.
+Try temperature between 0.8 and 1.2 for peak confusion.
 
 ---
 
-## Reasonable Use Cases
+## What is this actually for?
 
-- Generating creative writing prompts that nobody asked for
+- Generating writing prompts nobody asked for
 - Studying how small models fail in charming ways
 - Populating game worlds with NPCs that speak in riddles
-- Teaching students why bigger is not always better (Pun intended)
-- Amusing yourself during long training runs
-
----
+- Teaching that bigger is not always better
+- Entertaining yourself during long training runs
 
-## Unreasonable Use Cases
+## What is it not for
 
-- Anything requiring factual accuracy
-- Customer support automation
+- Facts. Any facts.
+- Customer support
 - Medical, legal, or financial advice (oh hell no)
 - Replacing a search engine
-- Expecting the model to know what it is talking about
+- Expecting it to know what it is talking about
 
 ---
 
-## Philosophy
+## Why does this exist?
 
-TMLM-Haiku-2 exists because we wondered what would happen if we trained a very small model on a very large dataset and then asked it to talk. The answer, as you have seen, is complicated.
+We wondered what would happen if you trained a very small model on a very large dataset and then asked it to talk. The answer, as you can see, is complicated.
 
-The training strategy was simple: allocate two-thirds of the token budget to broad pretraining, then use the remainder to nudge the model toward instruction following. This does not produce a capable assistant. It does produce a model that learned as much as it could, given the constraints.
+We put two thirds of the token budget into pretraining and used the rest to nudge it toward instruction following. This does not produce a capable assistant. It produces a model that learned as much as it could, given the constraints.
 
-This project is part of CompactAI, an ongoing effort to explore language modeling at the edge of feasibility. We believe that interesting things can happen when you remove the safety net of scale. Sometimes those things are useful. Sometimes they are just funny.
+This is part of CompactAI, an ongoing exploration of language modeling at the edge of feasibility. Interesting things happen when you remove the safety net of scale. Sometimes those things are useful. Sometimes they are just funny.
 
 ---
 
 ## Contributing
 
 We welcome:
-- Bug reports, especially those accompanied by entertaining failure cases
-- Prompts that coax unexpectedly poetic output from the model
-- Research collaborations focused on ultra-small model dynamics
-- Feedback on how to make a 1M parameter model slightly less confused
-Note:
-If its a bug we will include fixes in later stages of TMLM-Haiku & other varients if present
-Please do not submit pull requests that add more parameters. That defeats the purpose. Please.
+- Bug reports, especially if the failure case is entertaining
+- Prompts that coax unexpectedly poetic output from this thing
+- Research collaborations on ultra-small model dynamics
+- Ideas for making a 1M parameter model slightly less confused
+
+Please do not submit PRs that add more parameters. That defeats the purpose.
 
 ---
 
 ## Citation
 
-    @misc{tmlm-haiku-2-2026,
-      title={TMLM-Haiku-2: A 1M-Parameter English Language Model for Experimental Use},
-      author={CompactAI},
-      year={2026},
-      howpublished={\url{https://huggingface.co/CompactAI/TMLM-Haiku-2}},
-      note={Trained with hope. Deploy with caution.}
-    }
+```
+@misc{glint03,
+  title={Glint-0.3: A 1M-Parameter English Language Model for Experimental Use},
+  author={CompactAI},
+  year={2026},
+  howpublished={\url{https://huggingface.co/CompactAI-O/Glint-0.3}},
+  note={Trained with hope. Deploy with caution.}
+}
+```
 
 ---
 
 > The model generates text. Whether that text means anything is a question for philosophers.
 
-Train small. Expect less. Laugh anyway.
+Train small. Expect less. Laugh anyway.
+
+---
+
+*Built by [CompactAI](https://huggingface.co/CompactAI-O).*