zils-lite-jq6

A body for the Zils d3-lite soup pool (Bittensor subnet 127): a fine-tune of Qwen/Qwen3.5-0.8B in the chassis layout, BF16, language weights trained, vision tower from the parent.

Soup fine-tune: the start is the uniform weight average of the two bodies of the 14 November 2026 d3-lite soup (VadymPikulyk/zils-lite-dh2 and qhoenix/zils-lite-decision-avg-abc), both Apache-2.0; credit for the start belongs to their authors and to the authors of the bodies they were built from. Procedure: six LoRA (rank 32, merged) fine-tunes of that average with different seeds, one pass each at learning rate 5e-5 (three over 30,000 rows and three over 45,913 rows of the data below), were averaged uniformly with equal weights. Training targets were soft and label-free: the answer probabilities of the current d3-mini soup average (teacher distillation from the larger pool's soup: the uniform weight average of Zayaan/zils-mini-hm4, qhoenix/zils-mini-decision-t2avg and joamCriff123/zils-mini-m1113b).

Training data (30,000 or 45,913 rows per seed, one shuffled pass): rows of the train split of tasksource/tasksource-jev-typed-decisions (about 630 source tasks; sources with commercial or unspecified licence metadata only, non-commercial sources excluded; sources whose names match BIG-bench tasks dropped; no Glaive rows), used only as unlabeled distillation inputs (their labels were not used). No exam-format written items, no validator exam items, no public seed exam items and no customer data were used. This repository holds weights only; check the source datasets' terms before reuse.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
0.9B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Charlie320/zils-lite-jq6

Finetuned
(924)
this model