JIMMY-4B-ZERO-REGRESSION (Apple Silicon Native MLX)

luisbrel/jimmy-4b-zero-regression is an experimental neural-network checkpoint developed locally on Apple Silicon M4 Max.

The experiment uses a high-level recursive self-improvement (RSI) loop: candidate versions are repeatedly evaluated against real generation targets, and only promising iterations are retained. It does not use teacher-model distillation or a conventional gradient-descent fine-tuning pipeline.


๐Ÿ† Local Head-to-Head Evaluation

Evaluated using authentic autoregressive token generations on Apple Silicon Metal. The run covers 56 generations, eight per suite across two trials:

Benchmark Suite Base Gemma-4 jimmy-4b-zero-regression Net Delta ($\Delta$)
GSM8K (Multi-Step Math) 50.0% 75.0% +25.0%
MATH-500 (Competition Math) 25.0% 37.5% +12.5%
HumanEval (Python Code) 75.0% 75.0% 0.0%
IFEval (Strict Constraints) 100.0% 100.0% 0.0%
ARC-Challenge (Science) 75.0% 87.5% +12.5%
Artificial Analysis v5 Multi-Quadrant 100.0% 100.0% 0.0%
GUI & DOM Grounding 62.5% 75.0% +12.5%
MACRO AVERAGE 69.6% 78.6% +9.0%

Trial-level scorecards

The aggregate is not enough to judge stability. These are the two trial scores behind each mean:

Benchmark Suite Base trial 1 Base trial 2 Final trial 1 Final trial 2
GSM8K 75.0% 25.0% 75.0% 75.0%
MATH-500 25.0% 25.0% 50.0% 25.0%
HumanEval 75.0% 75.0% 75.0% 75.0%
IFEval 100.0% 100.0% 100.0% 100.0%
ARC-Challenge 100.0% 50.0% 100.0% 75.0%
Artificial Analysis v5 100.0% 100.0% 100.0% 100.0%
GUI & DOM Grounding 50.0% 75.0% 75.0% 75.0%

๐Ÿ’ป Quickstart (MLX Native)

Run on any Mac with Apple Silicon using mlx-lm:

pip install mlx-lm
import mlx_lm

model_path = "luisbrel/jimmy-4b-zero-regression"
model, tokenizer = mlx_lm.load(model_path)

prompt = "If f(x) = (3x-2)/(x-2), calculate f(-2) + f(-1) + f(0). Express your answer as a common fraction."
messages = [{"role": "user", "content": prompt}]
formatted = tokenizer.apply_chat_template(messages, add_generation_prompt=True)

response = mlx_lm.generate(model, tokenizer, prompt=formatted, max_tokens=850)
print(response)

๐Ÿ”ฌ Provenance & Architecture

  • Base Architecture: Google Gemma-4-E4B-it (4-bit MLX Quantized)
  • License: Apache 2.0, inherited from the base model
  • Runtime: Apple Silicon MLX
  • Method: Experimental RSI-based post-training
  • Decontamination: Certified zero 13-gram overlap with benchmark splits
Downloads last month
22
Safetensors
Model size
7B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support