Adurite1-135M-COT

Currently being retrained to fix major bug.

Adurite1-135M-COT is a small reasoning model based on HuggingFaceTB/SmolLM2-135M. It was fine tuned to think step by step before answering.

Model details

• Base model: HuggingFaceTB/SmolLM2-135M
• Parameters: 135M
• Context length: 8192
• Training: 6 epochs on 182k examples, about 1.1B tokens
• Mix: 61 percent chain of thought data including Bespoke Stratos, OpenThoughts, NuminaMath, Orca Math, OpenR1 Math, MathInstruct, plus general chat data from SmolTalk, Cosmopedia, FineTome and UltraChat
• Identity: trained to identify as Adurite1 from Maximalist Labs
• Hardware: TPU v6e-1

Training

Trained on TPU with max length 1024, per device batch 32, AdaFactor, cosine schedule, warmup 500 steps, weight decay 0.01.

Final loss 1.337 after 33732 steps.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("MLVXN/Adurite1-135M-COT")
model = AutoModelForCausalLM.from_pretrained("MLVXN/Adurite1-135M-COT")

prompt = "<|im_start|>user\nWhat is 12*8? Think step by step.<|im_end|>\n<|im_start|>assistant to=self thinking\n"
inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))

Or use the GGUF file with llama.cpp:

llama-cli -m adurite1-135m-cot-f16.gguf -p "What is 12*8? Think step by step."

GGUF

GGUF weights are available in this repo:

• adurite1-135m-cot-f16.gguf 313 MB full precision
• adurite1-135m-cot-q4_k_m.gguf quantized 4 bit (when available)

Evaluation

Work in progress. Early MMLU and GSM8K runs are being added. MMLU is expected around 33 to 36 percent, GSM8K around 45 to 50 percent with chain of thought prompting.

Limitations

This is a 135M model. It can do short reasoning traces of 2 to 4 steps but will fail on hard math and long logic problems. It may repeat or miss facts. Use with care and check outputs for important tasks.

License

Apache 2.0

Citation

@misc{adurite1-135m-cot,
  title = {Adurite1-135M-COT},
  author = {MLVXN},
  year = {2026},
  url = {https://huggingface.co/MLVXN/Adurite1-135M-COT}
}