Hugging Face's logo

ghostai1
/
GHOSTAI-Spooky

Text Generation
GGUF
English
llama.cpp
mistral
instruct
lora
horror
roleplay
conversational
Model card Files
xet
 Community
ghostai1 commited on
Commit
2e39b36
·
verified ·
1 Parent(s): 1acbf39

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -26
README.md CHANGED
@@ -2,63 +2,74 @@
2
  license: mit
3
  ---
4
 
5
- <!--
6
- GHOSTAI • HORROR GGUF RELEASE README
7
- Drop this into README.md at the root of your Hugging Face repo.
8
- -->
9
 
10
  <p align="center">
11
- <img src="https://capsule-render.vercel.app/api?type=waving&color=0:0b0b0f,50:2b0a2a,100:0b0b0f&height=160&section=header&text=GHOSTAI%20%E2%80%94%20HORROR%20GGUF&fontSize=44&fontColor=EAEAEA&animation=twinkling" />
12
  </p>
13
 
14
  <p align="center">
15
  <img alt="GGUF" src="https://img.shields.io/badge/GGUF-llama.cpp-8A2BE2?style=for-the-badge">
16
- <img alt="Base" src="https://img.shields.io/badge/Base-Mistral%207B%20Instruct%20v0.3-5B2C83?style=for-the-badge">
17
- <img alt="Quant" src="https://img.shields.io/badge/Quant-Q4__K__M%20%7C%20F16-3A0CA3?style=for-the-badge">
18
  <img alt="Theme" src="https://img.shields.io/badge/Theme-Horror-8B0000?style=for-the-badge">
 
19
  </p>
20
 
21
  <p align="center">
22
- <b>GHOSTAI</b> is a horror-flavored GGUF release (llama.cpp-ready) built from a LoRA fine-tune on <code>mistralai/Mistral-7B-Instruct-v0.3</code>.
23
- <br/>
24
- Pick your haunt: <b>F16</b> for max fidelity or <b>Q4_K_M</b> for the best everyday balance.
25
  </p>
26
 
27
  ---
28
 
29
  ## 🩸 What’s inside
30
 
31
- This repo contains **GGUF** files for fast local inference using **llama.cpp**-compatible runtimes.
32
 
33
- ### 🎃 Spooky file set
34
 
35
- | Codename | File | Format | Use case |
36
- |---|---|---:|---|
37
- | **GHOSTAI_FOGF16** | `model.f16.gguf` | f16 | Maximum quality (largest) |
38
- | **GHOSTAI_CRYPT_Q4KM** | `model.Q4_K_M.gguf` | Q4_K_M | Best default (quality/size) |
39
- | **GHOSTAI_WHISPER_IQ1S** | `model.IQ1_S.gguf` | IQ1_S | Tiny build (quality drop) |
40
- | **GHOSTAI_RAGDOLL_Q2K** | `model.Q2_K.gguf` | Q2_K | Fallback if IQ1_S unsupported |
 
 
 
 
 
 
41
 
42
- > Not all files may exist in every release—this table lists the intended set. Use the “Files” panel to confirm what’s included.
 
 
 
43
 
44
  ---
45
 
46
- ## 🧬 Base model
47
 
48
- - **Base**: `mistralai/Mistral-7B-Instruct-v0.3`
49
- - **Release type**: GGUF export (llama.cpp ecosystem)
50
- - **Training method**: LoRA fine-tune → merged → GGUF → quantized
 
 
 
 
 
51
 
52
  ---
53
 
54
  ## ⚰️ Quickstart (llama.cpp)
55
 
56
- ### 1) Run on GPU (CUDA build)
57
 
58
  ```bash
59
  ./llama-cli \
60
- -m model.Q4_K_M.gguf \
61
  -ngl 99 \
62
  -c 4096 \
63
- -p "You are GHOSTAI. Speak like a calm narrator in a horror novel. Keep it concise."
 
64
 
 
2
  license: mit
3
  ---
4
 
5
+ <!-- GHOSTAI • HORROR GGUF (7B) — README -->
 
 
 
6
 
7
  <p align="center">
8
+ <img src="https://capsule-render.vercel.app/api?type=waving&color=0:0b0b0f,50:2b0a2a,100:0b0b0f&height=160&section=header&text=GHOSTAI%20%E2%80%94%20HORROR%20GGUF%20(7B)&fontSize=42&fontColor=EAEAEA&animation=twinkling" />
9
  </p>
10
 
11
  <p align="center">
12
  <img alt="GGUF" src="https://img.shields.io/badge/GGUF-llama.cpp-8A2BE2?style=for-the-badge">
13
+ <img alt="7B" src="https://img.shields.io/badge/Size-7B-5B2C83?style=for-the-badge">
 
14
  <img alt="Theme" src="https://img.shields.io/badge/Theme-Horror-8B0000?style=for-the-badge">
15
+ <img alt="Quant" src="https://img.shields.io/badge/Quants-Q8__0%20%7C%20Q6__K%20%7C%20Q5__K__M%20%7C%20Q4__K__M-3A0CA3?style=for-the-badge">
16
  </p>
17
 
18
  <p align="center">
19
+ <b>GHOSTAI</b> is a <b>horror-themed</b> <b>7B</b> GGUF release for the <b>llama.cpp</b> ecosystem.<br/>
20
+ This repo contains <b>quantized GGUFs only</b> (no FP16).
 
21
  </p>
22
 
23
  ---
24
 
25
  ## 🩸 What’s inside
26
 
27
+ Quantized GGUF files (7B) ready for llama.cpp-compatible runtimes.
28
 
29
+ ### 🎃 Files in this release
30
 
31
+ | File | Quant | Approx size | Rough RAM needed (4k ctx) |
32
+ |---|---:|---:|---:|
33
+ | `ghostai-horror-7b.Q8_0.gguf` | Q8_0 | ~7.2 GB | ~10–11 GB |
34
+ | `ghostai-horror-7b.Q6_K.gguf` | Q6_K | ~5.5 GB | ~8–9 GB |
35
+ | `ghostai-horror-7b.Q5_K_M.gguf` | Q5_K_M | ~4.8 GB | ~7–8 GB |
36
+ | `ghostai-horror-7b.Q5_K_S.gguf` | Q5_K_S | ~4.7 GB | ~7–8 GB |
37
+ | `ghostai-horror-7b.Q4_K_M.gguf` | Q4_K_M | ~4.1 GB | ~6–7 GB |
38
+ | `ghostai-horror-7b.Q4_K_S.gguf` | Q4_K_S | ~3.9 GB | ~6–7 GB |
39
+ | `ghostai-horror-7b.Q3_K_M.gguf` | Q3_K_M | ~3.3 GB | ~5–6 GB |
40
+ | `ghostai-horror-7b.Q3_K_S.gguf` | Q3_K_S | ~3.0 GB | ~5–6 GB |
41
+ | `ghostai-horror-7b.Q2_K.gguf` | Q2_K | ~2.5 GB | ~4–5 GB |
42
+ | `ghostai-horror-7b.TQ1_0.gguf` | TQ1_0 | ~1.6 GB | ~3–4 GB |
43
 
44
+ **RAM notes (rough):**
45
+ - “Rough RAM needed” assumes **~4k context** and typical llama.cpp overhead.
46
+ - If you run **8k context**, add roughly **+1–2 GB**.
47
+ - GPU offload doesn’t remove the need for RAM; it shifts some weight/KV usage to VRAM depending on settings.
48
 
49
  ---
50
 
51
+ ## 🧟 Which quant should I use?
52
 
53
+ - **Best default:** `Q4_K_M`
54
+ - **Higher quality:** `Q5_K_M` or `Q6_K`
55
+ - **If you have plenty of RAM:** `Q8_0`
56
+ - **Low RAM:** `Q3_K_S` / `Q2_K`
57
+ - **Tiny / experimental:** `TQ1_0` (expect quality loss)
58
+
59
+ These formats are **not “CPU vs GPU.”**
60
+ You can run any quant on CPU-only or with GPU offload.
61
 
62
  ---
63
 
64
  ## ⚰️ Quickstart (llama.cpp)
65
 
66
+ ### GPU offload (CUDA build)
67
 
68
  ```bash
69
  ./llama-cli \
70
+ -m ghostai-horror-7b.Q4_K_M.gguf \
71
  -ngl 99 \
72
  -c 4096 \
73
+ -p "You are GHOSTAI. Speak like a calm horror narrator. Keep it tight and vivid."
74
+
75