|
|
--- |
|
|
base_model: |
|
|
- N-Bot-Int/MistThena7B |
|
|
tags: |
|
|
- text-generation-inference |
|
|
- transformers |
|
|
- mistral |
|
|
- rp |
|
|
- gguf |
|
|
language: |
|
|
- en |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- N-Bot-Int/Iris-Uncensored-R1 |
|
|
- N-Bot-Int/Moshpit-Combined-R2-Uncensored |
|
|
- N-Bot-Int/Mushed-Dataset-Uncensored |
|
|
- N-Bot-Int/Muncher-R1-Uncensored |
|
|
- unalignment/toxic-dpo-v0.1 |
|
|
library_name: transformers |
|
|
new_version: N-Bot-Int/MistThena7BV2-GGUF |
|
|
--- |
|
|
# Support Us Through |
|
|
- [](https://ko-fi.com/J3J61D8NHV) |
|
|
- [https://ko-fi.com/nexusnetworkint](Official Ko-FI link!) |
|
|
|
|
|
 |
|
|
# GGUF Version |
|
|
**GGUF** with Quants! Allowing you to run models using KoboldCPP and other AI Environments! |
|
|
|
|
|
|
|
|
# Quantizations: |
|
|
| Quant Type | Benefits | Cons | |
|
|
|---------------|---------------------------------------------------|---------------------------------------------------| |
|
|
| **Q4_K_M** | ✅ Smallest size (fastest inference) | ❌ Lowest accuracy compared to other quants | |
|
|
| | ✅ Requires the least VRAM/RAM | ❌ May struggle with complex reasoning | |
|
|
| | ✅ Ideal for edge devices & low-resource setups | ❌ Can produce slightly degraded text quality | |
|
|
| **Q5_K_M** | ✅ Better accuracy than Q4, while still compact | ❌ Slightly larger model size than Q4 | |
|
|
| | ✅ Good balance between speed and precision | ❌ Needs a bit more VRAM than Q4 | |
|
|
| | ✅ Works well on mid-range GPUs | ❌ Still not as accurate as higher-bit models | |
|
|
| **Q8_0** | ✅ Highest accuracy (closest to full model) | ❌ Requires significantly more VRAM/RAM | |
|
|
| | ✅ Best for complex reasoning & detailed outputs | ❌ Slower inference compared to Q4 & Q5 | |
|
|
| | ✅ Suitable for high-end GPUs & serious workloads | ❌ Larger file size (takes more storage) | |
|
|
|
|
|
# Model Details: |
|
|
Read the Model details on huggingface |
|
|
[Model Detail Here!](https://huggingface.co/N-Bot-Int/MistThena7B) |
