MinimaLabs/min-spark-1.1

min-spark 1.1

min-spark 1.1 is a 5.76M-parameter language model with native effort levels. You can have it answer quickly or let it spend more compute on the same prompt, whichever you prefer at generation time.

What's new in 1.1

This release is min-spark 1.0 trained for longer. The architecture and vocabulary are untouched, and the parameter count is still 5.76M. What changed is the amount of training. We continued pre-training on another 6.8B tokens, bringing the total up to 16.81B.

That extra training made most of the benchmarks better. ARC-Easy dipped a little, which we consider a fair trade for the gains everywhere else:

Benchmark	min-spark 1.0	min-spark 1.1
BLiMP	69.19%	71.78%
ARC-Easy	37.08%	36.45%
ARC-Challenge	23.21%	23.04%
HellaSwag	27.92%	28.24%
PIQA	54.35%	57.40%
WikiText-2 byte-ppl	2.7747	2.7248

Effort levels

If you have used large reasoning models before, effort levels there usually mean a thinking-token budget or a modified system prompt. min-spark takes a different route: each level selects how many passes the model makes through its shared core, so the compute difference happens inside the network itself. The interface is the same one introduced in min-spark 1.0, and nothing about it changed in this release.

Effort	Character	Recommended use
low	Fastest	High-throughput completion
medium	Balanced	General generation
high	Most compute	Highest available quality

In practice, grammar quality benefits most from raising effort. BLiMP jumps noticeably between low and medium, while the other tasks stay roughly flat across settings.

Effort	BLiMP	ARC-Easy	ARC-Challenge	HellaSwag	PIQA	WikiText-2 byte-ppl
min-spark 1.1 low	69.40%	34.76%	23.04%	28.24%	57.40%	2.8313
min-spark 1.1 medium	71.78%	36.45%	22.70%	27.79%	56.69%	2.7249
min-spark 1.1 high	71.78%	36.45%	22.78%	27.75%	56.64%	2.7248

Evaluation

All scores below are zero-shot with lm-eval 0.4.12. BLiMP reports accuracy, ARC-Easy, ARC-Challenge, HellaSwag and PIQA report length-normalized accuracy, and WikiText-2 is byte-level perplexity where lower is better.

Benchmarks

Here is how the best-effort scores compare against current sub-10M entries on the Open SLM leaderboard. Peer numbers come from their leaderboard submissions.

Model	Params	BLiMP	ARC-Easy	ARC-Challenge	HellaSwag	PIQA
min-spark 1.1	5.76M	71.78%	36.45%	23.04%	28.24%	57.40%
cma-8M	7.8M		35.35%	23.29%	28.19%	58.22%
BananaMind-2-Nano	9.97M		36.20%	23.38%	27.50%	55.98%
Qana-mini-5M	4.94M		34.97%	23.21%	27.60%	57.18%
michel-nano-v2	8M	72.52%	35.90%	21.84%	27.40%	56.75%

The BananaMind Base Bench tells a similar story. At the best effort setting, version 1.0 reached an Elo of 920 and this release reaches 935:

Model	Elo	Accuracy
min-spark 1.0	920	39.71%
min-spark 1.1	935	41.14%

Splitting those results by effort shows an interesting difference between the two releases. Version 1.0 scored 905 at low effort and then saturated at 920 once you gave it more compute. Version 1.1 keeps improving all the way up: 910 at low, 932 at medium, and 935 at high. In other words, the extra training made higher effort settings actually worth using.

Usage

min-spark 1.1 works with Transformers through remote code loading:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "MinimaLabs/min-spark-1.1",
    trust_remote_code=True,
).to("cuda")
tokenizer = AutoTokenizer.from_pretrained(
    "MinimaLabs/min-spark-1.1",
    trust_remote_code=True,
)

prompt = "The meaning of life is"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(
    **inputs,
    effort="high",
    max_new_tokens=64,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The Transformers pipeline API works too:

from transformers import pipeline

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
result = pipe(prompt, effort="high", max_new_tokens=64)
print(result[0]["generated_text"])

If you would rather skip Transformers entirely, the repo ships a small standalone script:

python generate.py -p "The meaning of life is" -e high

A few practical notes. Generation processes one sequence at a time, though right-padded batches work fine for evaluation. There is no KV cache yet, so each generated token recomputes the whole prompt, which slows down long generations somewhat. The context window is 512 tokens.

Architecture

Field	Value
Parameters	5,758,572
Architecture	Meiosis (tied-embedding looped decoder)
Vocabulary	4,096-token byte-level BPE
Embedding width	288
Heads	6 query · 2 KV (GQA)
FFN hidden size	768
LoRA rank	16
Blocks	1 prelude · 3 shared body blocks · 1 coda · final RMSNorm
Context window	512 tokens
Effort (loop count)	low = 2 · medium = 3 · high = 4

Training

Field	Value
Training tokens	16.81B total (10.01B base + 6.8B continued pre-training)
Precision	fp16 autocast with gradient scaling
Context length	512 tokens
Global batch size	32
Optimizer	Muon (matrices) + NAdamW (auxiliary parameters), weight decay 0.01
Learning-rate schedule	Warmup → stable → cooldown
Attention masking	Intra-document
Checkpoint	Final checkpoint of the continued training run

Reproducing the evaluation

python run_lmeval.py \
  --effort medium \
  --tasks blimp,arc_easy,arc_challenge,hellaswag,piqa,wikitext

Add --limit N if you want a quicker run while poking at things.

Limitations

This is a small base model. It has not been instruction-tuned, so do not expect conversational behavior, alignment, or safety filtering. Its factual recall, reasoning, and long-form coherence are limited by its size, as you would expect from 5.76M parameters.

The context window is fixed at 512 tokens and generation accepts one sequence at a time with no KV cache. All published results were collected within that window, and we cannot promise anything about behavior beyond it.

License

Apache-2.0. See LICENSE.