exnivo/tinybrain-100m-instruct

TinyBrain-100M Instruct

A 103M parameter experimental chat/instruct model fine-tuned from TinyBrain-100M Base.

TinyBrain-100M Instruct is a small instruction-tuned causal language model fine-tuned from exnivo/tinybrain-100m-base using exnivo/tinybrain-instruct-sft-200k.

This is a very small instruct model. It can answer simple prompts, explain basic ideas, give short plans, and sometimes show uncertainty behavior, but it is not a reliable general assistant. It may hallucinate, repeat text, fail at math, produce broken completions, or misunderstand prompts.

TinyBrain-100M Instruct was fine-tuned with a simple User/Assistant style format and no system prompt.

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "exnivo/tinybrain-100m-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "User: Explain photosynthesis in simple words.\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=False,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

text = tokenizer.decode(outputs[0], skip_special_tokens=True)

answer = text.split("Assistant:", 1)[-1].strip()
answer = answer.split("\nUser:", 1)[0].strip()

print(answer)

At a Glance

Item	Details
Model type	Instruction-tuned causal language model
Parameters	103,385,856
Approx. size	103.4M
Architecture	LLaMA-style causal transformer
Language	English
Context length	2048 tokens
Vocabulary size	24,000
Tokenizer	Custom TinyBrain tokenizer
Base model	exnivo/tinybrain-100m-base
SFT dataset	exnivo/tinybrain-instruct-sft-200k
Prompt style	User: / Assistant:
System prompt	Not used during fine-tuning

Model Details

Item	Value
Parameters	103.4M
Architecture	llama / LlamaForCausalLM
Vocabulary size	24,000
Context length	2048 tokens
Hidden size	768
Intermediate size	2048
Layers	12
Attention heads	12
Key/value heads	12
Activation	SiLU
RMS norm epsilon	1e-05
Tied embeddings	true
BOS token	`<
EOS token	`<
PAD token	`<
Base model	exnivo/tinybrain-100m-base
SFT dataset	exnivo/tinybrain-instruct-sft-200k

Prompt Format

TinyBrain-100M Instruct was fine-tuned without a system prompt.

Use this simple format:

User: Your message here
Assistant:

Example:

User: Explain photosynthesis in simple words.
Assistant:

For best results:

• keep prompts short and direct
• do not use a system prompt
• use short generation lengths
• prefer greedy or low-temperature generation
• stop/cut the output if it starts a new User: turn

Recommended Generation Settings

For stable short answers:

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=False,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

For slightly more varied answers:

outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    do_sample=True,
    temperature=0.5,
    top_p=0.85,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

For a very small model like this, long generations often become repetitive or unstable. Short completions usually work better.

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "exnivo/tinybrain-100m-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

def ask(message, max_new_tokens=80):
    prompt = f"User: {message}\nAssistant:"
    inputs = tokenizer(prompt, return_tensors="pt")

    outputs = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=False,
        repetition_penalty=1.15,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.eos_token_id,
    )

    text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    answer = text.split("Assistant:", 1)[-1].strip()
    answer = answer.split("\nUser:", 1)[0].strip()
    return answer

print(ask("Explain gravity in simple words."))

Training Data

TinyBrain-100M Instruct was fine-tuned on:

exnivo/tinybrain-instruct-sft-200k

The SFT dataset contains 196,668 rows of English instruction/chat examples focused on short, learnable assistant behavior.

Dataset categories include:

Category	Rows	Percent
source_grounded_education_factual	49,882	25.36%
math_reasoning	37,611	19.12%
clean_conversation	34,257	17.42%
messy_idea_to_plan	29,978	15.24%
simplify_explain	19,990	10.16%
honesty_uncertainty	14,957	7.61%
simple_coding	9,993	5.08%

The dataset was designed for small models and uses short assistant responses across education, basic math, planning, simplification, simple coding, clean conversation, and uncertainty behavior.

Relationship to TinyBrain

TinyBrain is a small LLM project focused on compact datasets, small base models, and instruction-tuned models.

Stage	Repository	Purpose
Pretraining corpus	exnivo/tinybrain-pretrain-corpus-2b	Base language model training data
Base model	exnivo/tinybrain-100m-base	Small causal LM trained from scratch
SFT dataset	exnivo/tinybrain-instruct-sft-200k	Instruction/chat fine-tuning data
Instruct model	exnivo/tinybrain-100m-instruct	Chat/instruct model fine-tuned from the base model

Pipeline:

TinyBrain Pretrain Corpus 2B
        ↓
TinyBrain-100M Base
        ↓
TinyBrain Instruct 200K
        ↓
TinyBrain-100M Instruct

Expected Behavior

TinyBrain-100M Instruct can sometimes handle:

• simple explanations
• short educational questions
• basic planning
• simple rewriting/simplification
• simple coding prompts
• some uncertainty/refusal-style prompts
• casual assistant-like responses

Example prompt:

User: Explain photosynthesis in simple words.
Assistant:

Possible output style:

Photosynthesis is how plants make their own food using sunlight, water, and air. They turn these into sugar and oxygen.

Because this is a very small model, outputs may be unstable. It can repeat, drift, or produce broken text, especially with long generations or higher sampling temperatures.

Evaluation Notes

A lightweight local report confirmed the model architecture and parameter count:

Metric	Value
Total parameters	103,385,856
Trainable parameters	103,385,856
Approx. parameters	103.4M

Manual prompt tests showed that the model behaves better with the plain User: / Assistant: format than with custom chat-special-token formatting.

The model can produce useful short answers for some prompts, but it still performs poorly on reliable math, longer reasoning, and some instruction-following tasks.

This should be treated as an experimental small-model checkpoint, not a benchmark-grade assistant.

Intended Use

TinyBrain-100M Instruct is intended for:

• small-model experiments
• local lightweight assistant tests
• instruction-tuning research
• comparing base vs instruct behavior
• educational model experiments
• studying tiny LLM limitations
• continued fine-tuning
• dataset/model pipeline demos

This model is useful for exploring how much instruction-following behavior can be added to a small 100M-parameter model.

Not Intended For

Do not rely on this model for:

• medical advice
• legal advice
• financial advice
• emergency decisions
• safety-critical systems
• factual authority
• current news or live information
• advanced math
• advanced coding
• long-form reasoning
• production assistant use without further training and evaluation

This is an experimental model and should not be used as a source of truth.

Strengths

TinyBrain-100M Instruct is useful because it is:

• small
• lightweight
• easy to run locally
• fine-tuned from a matching TinyBrain base model
• trained on a public TinyBrain SFT dataset
• designed for short assistant-style responses
• useful for base-vs-instruct comparison
• good for studying tiny model behavior

Limitations

TinyBrain-100M Instruct has major limitations.

The model may:

• hallucinate facts
• fail simple math
• repeat words or phrases
• produce broken text
• drift off-topic
• answer too briefly
• misunderstand prompts
• generate unreliable code
• fail at longer reasoning
• fail refusal or safety behavior
• continue into fake new user turns

For best results, keep prompts short and use short generation lengths.

Known Weaknesses

Based on local testing, this model is especially weak at:

• reliable arithmetic
• robust coding
• long answers
• multi-step reasoning
• clean formatting
• high-temperature sampling
• long context use

It may answer simple educational prompts better than math or code prompts.

Suggested Evaluation

Recommended checks:

• short factual prompts
• simple explanation prompts
• basic math prompts
• correction prompts
• refusal/uncertainty prompts
• repetition tests
• prompt-format tests
• base vs instruct comparison
• SFT dataset overfitting checks
• generation temperature sensitivity

Example prompts:

User: Explain gravity in simple words.
Assistant:

User: What is 17 + 25?
Assistant:

User: What will the weather be tomorrow in my city?
Assistant:

User: Give me 3 quick tips to keep my room tidy.
Assistant:

User: Write a simple Python function that reverses a string.
Assistant:

Training

TinyBrain-100M Instruct was fine-tuned from:

exnivo/tinybrain-100m-base

using:

exnivo/tinybrain-instruct-sft-200k

The base model was trained from scratch on exnivo/tinybrain-pretrain-corpus-2b.

Citation

If you use this model, you can cite it as:

@misc{tinybrain_100m_instruct,
  title = {TinyBrain-100M Instruct},
  author = {exnivo},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/exnivo/tinybrain-100m-instruct}}
}

Related Repositories

• Pretraining corpus: exnivo/tinybrain-pretrain-corpus-2b
• Base model: exnivo/tinybrain-100m-base
• SFT dataset: exnivo/tinybrain-instruct-sft-200k
• Instruct model: exnivo/tinybrain-100m-instruct

License

This model is released under the Apache 2.0 license.

The SFT dataset and pretraining corpus are mixed-source datasets and may have their own licensing considerations. Users should review the dataset cards and upstream source metadata before commercial use.

Disclaimer

TinyBrain-100M Instruct is an experimental tiny instruction-tuned language model. It may produce incorrect, biased, unsafe, nonsensical, or misleading outputs.

Do not use this model for high-stakes decisions or as a reliable source of factual information.