Kunpeng-4x7B-mistral

Architecture: Mixture of Experts (MoE)

A Moe Model of "Mistral-7B-Instruct-v0.2", "Mistral-7B-v0.1", "Starling-LM-7B-alpha", and "Mistral-7B-Instruct-v0.1" then fine-tuned with "WizardLM_evol_instruct_70k" for q_proj, v_proj, and gate.