LFM2.5-1.2B-Instruct-WSP-JSONFix

LiquidAI/LFM2.5-1.2B-Instruct (revision b9d816ab510f9a28dca37cb182e9da8b0226e13c) with a training-free, sparse, reversible weight edit that stops the model from wrapping strict-JSON answers in markdown code fences. No gradient descent, fine-tuning, LoRA or dataset updates were used: 16 columns of the FFN down-projection (w2) in layers 12 to 15 were scaled by exactly 0.5, changing 32,768 stored BF16 values (0.0028% of the model).

This is a behavior repair, not a new capability.

Held-out test results (evaluated once, after selection)

Base This model
Strict-JSON exact match (greedy + 3 sampled seeds, 256 units) 167/256 196/256 (29 fixed, 0 broken)
Strict-JSON exact match, greedy (64 prompts) 42/64 49/64
Replies wrapped in code fences 41/256 0/256
Top-level JSON arrays exact 76/128 97/128 (0 broken)
JSON objects exact 91/128 99/128
Prompts that request a code block (fence kept) 48/48 48/48
Short exact answers / general sanity 57/64, 64/64 57/64, 64/64
Termination tasks 36/64 35/64
WikiText-2 perplexity 130.547 -0.073% relative

Matched controls (same size, random columns or random directions) stayed at base level (166 to 167/256). Benchmark: regenerated, template-based, 64 strict-JSON test prompts (32 objects, 32 arrays), small sample; see wsp/WSP_1_2B_Experiment_Report.md for methods, all 12 candidates, gates and limitations. Most fences in the base model came from weakly worded instructions such as "Answer in JSON."

Edit

tensor columns (scaled x0.5)
model.layers.12.feed_forward.w2.weight 4731, 7264, 7919
model.layers.13.feed_forward.w2.weight 488, 5186, 6715
model.layers.14.feed_forward.w2.weight 3659
model.layers.15.feed_forward.w2.weight 99, 532, 660, 926, 2017, 2439, 3263, 3682, 5191

wsp/ contains the sparse patch (chosen_patch.safetensors, coordinates plus original and edited values), its manifest, and apply_patch.py / rebuild_patch.py to apply, revert or regenerate it bitwise from the base model.

Usage

Same as the base model (chat template, tokenizer and generation settings unchanged):

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("yasserrmd/LFM2.5-1.2B-Instruct-WSP-JSONFix")
model = AutoModelForCausalLM.from_pretrained("yasserrmd/LFM2.5-1.2B-Instruct-WSP-JSONFix", dtype="bfloat16")

License and modification notice

Derivative of LiquidAI/LFM2.5-1.2B-Instruct by Liquid AI, distributed under the LFM Open License v1.0 (LICENSE, unchanged). Modified files: model.safetensors (the 32,768 values listed above). All other model files are copied from the base revision.

Downloads last month
-
Safetensors
Model size
1B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for yasserrmd/LFM2.5-1.2B-Instruct-WSP-JSONFix

Finetuned
(143)
this model
Quantizations
1 model