zils-mini-mj3

A body for the Zils d3-mini soup pool (Bittensor subnet 127): a fine-tune of Qwen/Qwen3.5-4B in the chassis layout, BF16, language weights trained, vision tower from the parent.

Soup fine-tune: the start is the uniform weight average of the three bodies of the current d3-mini soup (Zayaan/zils-mini-hm4, qhoenix/zils-mini-decision-t2avg and joamCriff123/zils-mini-m1113b); credit for the start belongs to their authors and to the authors of the bodies they were built from. Procedure: three LoRA (rank 64, merged) fine-tunes of that average with different seeds, one shuffled pass each over the 61,091 rows below at learning rate 5e-5, were averaged uniformly with equal weights. Training targets were soft: for the written exam-format items the start's own answer probabilities only (a self-distillation anchor that keeps the start's behaviour on them; their labels were not used); for the tasksource-jev rows three quarters the row's own label and one quarter the start's answer probabilities.

Training data (61,091 rows): (1) about 31,100 exam-format decision items of the known task families written with the public Zils authoring recipe (scripts/author_exam.py) by Claude Opus and by a local open model, from train-side splits of the public source datasets, including about 1,100 tool-call items written from rows 90,000-93,000 of the function-call subset of glaiveai/glaive-function-calling-v2 (a dataset without splits that the exam recipe also draws from); used only as self-distillation inputs. (2) 30,000 rows of the train split of tasksource/tasksource-jev-typed-decisions with their labels (sources with commercial or unspecified licence metadata only, non-commercial sources excluded; sources whose names match BIG-bench tasks dropped). No validator exam items, no public seed exam items and no customer data were used. This repository holds weights only; check the source datasets' terms before reuse.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Charlie320/zils-mini-mj3

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,486)
this model