Shalabi-1

A small local learning experiment by Sameh Shalabi. Public repository: Sameh515/Shalabi-1. This native MLX LoRA adapter classifies the next step from a request, a plain-text list of available tools, and the observed state:

Code	Action	Meaning
A	USE_TOOL	A listed, permitted tool can make useful progress now.
B	ASK_USER	Progress requires user information, a choice, or approval.
C	FINISH	Completion is verified, or no tool or user step can help.

Apply verified completion first, then useful permitted tools, then necessary user input, then confirmed inability. FINISH can represent a blocked task; this adapter does not generate a final reply or claim an action succeeded. It does not select the particular tool or its arguments and performs no external actions.

Evaluation results

Separate evaluation

In a separate evaluation reported by Sameh on November 1, 2026, training improved the score from 52/60 to 59/60:

Measurement	Pretrained base model	Trained Shalabi-1
Correct	52/60	59/60
Accuracy	86.7%	98.3%
Mean restricted decision loss	0.080609	0.035176

Improvement: 7 more correct answers, or +11.7 percentage points in accuracy. The percentage-point change is calculated from the unrounded scores. These are author-reported aggregate results from a separate evaluation; its per-example predictions and evaluation configuration are not included in the bundled project.

Run locally on an Apple Silicon Mac

Python 3.12 and MLX with Metal are required. The base weights need about 6 GB of disk space. This run was performed on an Apple M5 Max with 128 GB unified memory; compatibility and memory use on other machines have not been tested.

If this repository contains project.zip, extract it first:

unzip project.zip -d next-step-project
cd next-step-project

With uv installed:

uv sync --frozen
uv run python next_step.py predict --input examples/request.json --adapter .

Without --adapter, the same command scores the original pinned checkpoint. predict also accepts JSONL and can save results with --output predictions.json. Use docs/learning-guide.html for the illustrated beginner guide and docs/solutions.html for the separate exercise key.

The adapter must be used through this prediction wrapper. Training normalized only A/B/C logits; generic text generation is not its interface. Input tools are plain text, not a native tools= schema. Inputs over 256 formatted tokens are rejected rather than truncated. There is no adversarial-input security evaluation.

Training and reproducibility

Base: mlx-community/Qwen3.5-9B-4bit

Exact base revision: 8b2b98c00a6b4d291155e4890773ca8f769aee53

Lineage limitation: that pinned community card's metadata identifies Qwen/Qwen3.5-9B-Base, while its description identifies Qwen/Qwen3.5-9B. The exact community checkpoint is reproducible; its true upstream weight lineage was not independently established. Both named upstream repositories declare Apache-2.0. This release does not claim a direct conversion from either one.

The frozen 4-bit backbone receives LoRA additions only in final full-attention block 31, on self_attn.q_proj and self_attn.v_proj: rank 8, native MLX scale 20, dropout 0, 139,264 trainable parameters. Scale is MLX's direct multiplier, not a promised cross-library alpha convention. Batch size 1; Adam at 5e-5; seed 42; 270 updates; no padding or formatting tokens in the loss. Loss is cross-entropy over the same three logits used at prediction time. Forward mode: eval with adapter gradients enabled.

The 180 original examples form 60 three-state families. Entire families are split 90/30/60 into training/validation/test, with all classes balanced. The same request has a different observed state in each family row. Code, prompt, configuration and data were hashed before training. Test scoring occurred after checkpoint selection and was not used for tuning.

Completed updates	Validation correct	Decision loss
0	29/30	0.054082
90	30/30	0.018945
180	30/30	0.027692
270	30/30	0.004502

Selected checkpoint: update 270, lowest decision loss on all 30 validation examples among updates 90, 180 and 270; earlier wins ties. The original checkpoint at update 0 is reported separately.

Training wall time including periodic validation: 49.8 seconds. Peak MLX allocated memory during that run: 5.77 GB (not total system RAM or a requirement for other machines). Dependency versions are locked in uv.lock and recorded in results/results.json. The frozen training record retains its internal experiment name, Next Step v0.1; the public release is Shalabi-1. Naming does not alter the trained weights.

To repeat the experiment in a new run directory:

uv run python next_step.py baseline --run runs/reproduction
uv run python next_step.py train --run runs/reproduction
uv run python next_step.py evaluate --run runs/reproduction

Training performs a zero-adapter identity check, a five-update finite-gradient check and fresh-process reload, and a separate nine-row learnability check before resetting to a fresh adapter and optimizer for the main run. The implementation checks adapter keys and shapes to prevent silent partial loading. Published predictions can be verified from saved weights. Exact bit-for-bit retraining across hardware or library changes is not promised.

For a new experiment, copy the configuration and data to a new version, change one variable, and use a fresh held-out set. Existing v0.1 test examples are public and must not become a new tuning target.

Provenance, credit and limits

Project author: Sameh Shalabi.

This project borrows decision-model framing from Elephant-1 and the matching restricted-option training/readout idea from Shisa DE-2.1. It uses neither model's weights or training data and does not reproduce their large-scale methods, attention changes, calibration, ensembles or benchmarks.

This is a narrow educational classifier tested on short synthetic English states. It is not a general agent, safety policy or autonomous action system. Model scores can be confidently wrong; ambiguous or unfamiliar cases need human review.

Code, original data and adapter are supplied under Apache-2.0; see LICENSE and NOTICE. Base weights and installed libraries retain their upstream attribution.