Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
danielhanchen 
posted an update 5 days ago

noice.
will there be training for Qwen 4 27B (possibly with ngrams)?

·

Yes of course. Qwen3.8 is already supported in Unsloth

Any news about release Ornith 1.5 on UD3?

Use my dataset. It’s the same that JEV-27B-VL was trained on. https://huggingface.co/datasets/SargeDev/jev-distill-corpus-v3

The 20.7% to 74.3% is the plain mean of the three columns in your guide. Weighted by the 3,000-row test set the guide describes, it is 28.3% to 73.7%.

From the guide's table, Qwen3.5-0.8B:
typed-decisions 36 -> 73, BANKING77 7 -> 74, CLINC150 19 -> 76.
Mean of the three: 20.67 -> 74.33.
Weighted 2000 / 500 / 500: 28.33 -> 73.67.

Two things fall out of that.

The 2B flips. Same arithmetic gives 11.7 -> 66.0 unweighted and 22.3 -> 72.0 weighted. On the 3,000 rows, 2B lands below 0.8B (72.0 vs 73.7). The Test accuracy column says the reverse, 81 vs 78. Neither 78 nor 81 matches the weighted mix, so that column looks like a different split.

The starting point is mostly format. 2B at 1% on BANKING77 and CLINC150 is chance (1.3% and 0.7%). That reads as "can't name a valid intent", not "doesn't know the domain". So the gain is two things at once: learning the output space, and learning the task.

Also, banking77 and clinc150 are in the 12 training sources, so those two columns are in-domain held-out rather than transfer.

Which split is the 78% / 81% Test accuracy column?