Mistral
Collection
(Mistral) Models merged & fine tuned by WithIN Us Ai • 1 item • Updated
NeuralPipe-7B-slerp
NeuralHermes 2.5 - Mistral 7B
NeuralHermes is based on the teknium/OpenHermes-2.5-Mistral-7B model that has been further fine-tuned with Direct Preference Optimization (DPO) using the mlabonne/chatml_dpo_pairs dataset. It surpasses the original model on most benchmarks (see results).
It is directly inspired by the RLHF process described by Intel/neural-chat-7b-v3-1's authors to improve performance. I used the same dataset and reformatted it to apply the ChatML template.
The code to train this model is available on Google Colab and GitHub. It required an A100 GPU for about an hour.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support