mradermacher

/

model_requests

like 147

Qwen3.5-27B-Gemini3-Pro-High-Reasoning-Compact-Thinking
EARLY ALPHA.

Fine tune via Unsloth of Qwen 3.5 27B dense model using Gemini distill dataset on local hardware.

This has altered reasoning/thinking block as well as thinking/reasoning block size. (reduced)

Every attempt was made to ensure the training was "mild" and did not negatively affect the model's already incrediblely strong benchmarks.

Vision (images) tested -> working with new training.

BENCHMARKS:

(awaiting finals)

                          arc   arc/e boolq hswag obkqa piqa  wino

THIS MODEL 0.470,0.550,0.709,...
Qwen3.5-27B-Text-VL qx86-hi 0.443,0.498,0.857,0.701,0.372,0.770,0.752

SAFETY ALIGNMENT:

No attempt was made to adjust/change "censorship" and/or "safety alignment" in the model.

That is coming next. (already built - HERETIC trained models.)

NOTES:

Gemini Pro Reasoning like "thinking" blocks in most cases.
Mix of Gemini/Qwen depending on how complex the task(s).
Suggest min q4ks (non-imatrix) or IQ3S (imatrix).
Tested with rep pen of 1 (off).
Context: 256k (default).
Tech notes:

Due to issues with transformers and/or training there MAYBE issues with the model.
As this is a NEW MODEL ARCH with it's own quirks regard this tune as "alpha".
Model may loop in "think" block ; especially Gemini only thinking in some cases ESPECIALLY "mlx" and/or "mxfp4" versions.
[additional updates pending]

IMPORTANT:

Other versions in testing.
Information from Qwen's repo below.
Example generation at the bottom of the page.
Video portions of the model were NOT TESTED.