File size: 1,658 Bytes
a19eebf 21e49e8 a19eebf 7bea740 6e47b7a 7bea740 a19eebf 21e49e8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
base_model: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
datasets:
- mlfoundations-dev/oh-dcft-v3.1-claude-3-5-sonnet-20241022
- Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
- nothingiisreal/Claude-3-Opus-Instruct-15K
language:
- en
license: apache-2.0
pipeline_tag: text-generation
tags:
- mistral
- claude
quantized_by: ayan4m1
inference: true
fine-tuning: true
library_name: transformers
---
## Claudette-7B - A Mistral fine-tuning with Claude data
Using unsloth for fine-tuning:
```
==((====))== Unsloth 2025.2.4: Fast Llama patching. Transformers: 4.48.2.
\\ /| GPU: NVIDIA A100-SXM4-40GB. Max memory: 39.557 GB. Platform: Linux.
O^O/ \_/ \ Torch: 2.5.1+cu124. CUDA: 8.0. CUDA Toolkit: 12.4. Triton: 3.1.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.29. FA2 = False]
"-____-" Free Apache license: http://github.com/unslothai/unsloth
```
Original model: https://huggingface.co/unsloth/mistral-7b-instruct-v0.3-bnb-4bit
Applied Claude-sourced datasets containing ~200k question/answer pairs for fine-tuning.
## Training loss
## Prompt format
```
<s>[INST]{prompt}[/INST]
```
## Comparison
In my non-exhaustive testing, this model performs as well or better than Llama3.1-8B-Sonnet in half the execution time.
## Release History
* v0.1 - [2025-02-12] Initial release, trained to 1024 steps
## Credits
Thanks to Mistral AI, [mlfoundations-dev](https://huggingface.co/mlfoundations-dev), [Gryphe](https://huggingface.co/Gryphe), and [nothingisreal](https://huggingface.co/nothingiisreal) for providing the data used to create this fine-tuning. |