ArnaudClaudeML

/

blaniza-assistant

like 0

Dear Hengyu,

Thank you for your kind words and for reaching out — I'm glad the model has been useful for your project.

I need to correct an error in the model card: blaniza-assistant is not based on LenguajeNaturalAI/leniachat-qwen2-1.5B-v0. That appears to be a mistake in the README I wrote at the time. I apologize for the confusion.

The actual base model is NousResearch/Meta-Llama-3.1-8B-Instruct (Llama 3.1 8B Instruct). To answer your specific questions:

1. Direct fine-tuning: Yes, it is a direct fine-tune of the base model with no intermediate checkpoints or models involved. The process was a single QLoRA training run (4-bit quantization + LoRA adapters with rank 16).
2. Inheritance: The LoRA adapters were merged directly into the base NousResearch/Meta-Llama-3.1-8B-Instruct weights using merge_and_unload(). There was no merging, distillation, or mixing with any other released models.

Since your project seems oriented toward the Qwen ecosystem, you might be interested in another model I have available on the same account: ASISTENTE-SFTT-Qwen2p5-7B-LoRA-merged. It was fine-tuned from Qwen/Qwen2.5-7B-Instruct using the same dataset and a similar QLoRA
approach, with the following configuration:

• LoRA rank: 32, alpha: 64, dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Training: 2 epochs, learning rate 2e-4, batch size 4 with gradient accumulation steps of 4 (effective batch size 16), using SFTTrainer from TRL
• Quantization: 4-bit NF4 with double quantization and bfloat16 compute dtype
• Final model: LoRA adapters merged into the full base weights via merge_and_unload()

This model is a direct fine-tune of Qwen/Qwen2.5-7B-Instruct with no intermediate models involved, and may be a better fit if you are building within the Qwen2.5 ecosystem.

Best regards,
Arnaud