Luth-Instruct-GGUF

Luth-1.7B-Instruct is a French fine-tuned variant of the Qwen3-1.7B model, enhanced using the Luth-SFT dataset to significantly improve its capabilities in French instruction following, mathematics, and general knowledge while maintaining and even boosting its English performance. It was trained by full fine-tuning with Axolotl and later merged with the base Qwen3-1.7B, thus preserving its English competencies alongside marked improvements in French benchmarks. The model demonstrates strong performance on selected French and English benchmarks, including ifeval, gpqa-diamond, mmlu, math-500, arc-chall, and hellaswag, showing notable gains over comparable models in both languages. It is designed for tasks requiring bilingual proficiency with pronounced strength in French and is supported by available evaluation, training, and data scripts on GitHub. The model is suitable for instruction-following applications in contexts demanding enhanced French language understanding without compromising English language capabilities. It is openly accessible under an appropriate license for research and usage.

Model Name	Model Size	Download Link
Luth-1.7B-Instruct-GGUF	1.7B	Hugging Face
Luth-0.6B-Instruct-GGUF	0.6B	Hugging Face

Model Files

Luth-1.7B-Instruct

File Name	Quant Type	File Size
Luth-1.7B-Instruct.BF16.gguf	BF16	3.45 GB
Luth-1.7B-Instruct.F16.gguf	F16	3.45 GB
Luth-1.7B-Instruct.F32.gguf	F32	6.89 GB
Luth-1.7B-Instruct.Q2_K.gguf	Q2_K	778 MB
Luth-1.7B-Instruct.Q3_K_L.gguf	Q3_K_L	1 GB
Luth-1.7B-Instruct.Q3_K_M.gguf	Q3_K_M	940 MB
Luth-1.7B-Instruct.Q3_K_S.gguf	Q3_K_S	867 MB
Luth-1.7B-Instruct.Q4_0.gguf	Q4_0	1.05 GB
Luth-1.7B-Instruct.Q4_1.gguf	Q4_1	1.14 GB
Luth-1.7B-Instruct.Q4_K.gguf	Q4_K	1.11 GB
Luth-1.7B-Instruct.Q4_K_M.gguf	Q4_K_M	1.11 GB
Luth-1.7B-Instruct.Q4_K_S.gguf	Q4_K_S	1.06 GB
Luth-1.7B-Instruct.Q5_0.gguf	Q5_0	1.23 GB
Luth-1.7B-Instruct.Q5_1.gguf	Q5_1	1.32 GB
Luth-1.7B-Instruct.Q5_K.gguf	Q5_K	1.26 GB
Luth-1.7B-Instruct.Q5_K_M.gguf	Q5_K_M	1.26 GB
Luth-1.7B-Instruct.Q5_K_S.gguf	Q5_K_S	1.23 GB
Luth-1.7B-Instruct.Q6_K.gguf	Q6_K	1.42 GB
Luth-1.7B-Instruct.Q8_0.gguf	Q8_0	1.83 GB

Luth-0.6B-Instruct

File Name	Quant Type	File Size
Luth-0.6B-Instruct.BF16.gguf	BF16	1.2 GB
Luth-0.6B-Instruct.F16.gguf	F16	1.2 GB
Luth-0.6B-Instruct.F32.gguf	F32	2.39 GB
Luth-0.6B-Instruct.Q2_K.gguf	Q2_K	296 MB
Luth-0.6B-Instruct.Q3_K_L.gguf	Q3_K_L	368 MB
Luth-0.6B-Instruct.Q3_K_M.gguf	Q3_K_M	347 MB
Luth-0.6B-Instruct.Q3_K_S.gguf	Q3_K_S	323 MB
Luth-0.6B-Instruct.Q4_0.gguf	Q4_0	382 MB
Luth-0.6B-Instruct.Q4_1.gguf	Q4_1	409 MB
Luth-0.6B-Instruct.Q4_K.gguf	Q4_K	397 MB
Luth-0.6B-Instruct.Q4_K_M.gguf	Q4_K_M	397 MB
Luth-0.6B-Instruct.Q4_K_S.gguf	Q4_K_S	383 MB
Luth-0.6B-Instruct.Q5_0.gguf	Q5_0	437 MB
Luth-0.6B-Instruct.Q5_1.gguf	Q5_1	464 MB
Luth-0.6B-Instruct.Q5_K.gguf	Q5_K	444 MB
Luth-0.6B-Instruct.Q5_K_M.gguf	Q5_K_M	444 MB
Luth-0.6B-Instruct.Q5_K_S.gguf	Q5_K_S	437 MB
Luth-0.6B-Instruct.Q6_K.gguf	Q6_K	495 MB
Luth-0.6B-Instruct.Q8_0.gguf	Q8_0	639 MB

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):