🦅 Aethon-4B-GGUF

All quantizations, ready to run locally

llama.cpp · Ollama · LM Studio · GPT4All

Built by Featherlabs · Operated by Owlkun

---

📦 Available Quantizations

All GGUFs were created from the Featherlabs/Aethon-4b merged model using llama.cpp's convert_hf_to_gguf.py + llama-quantize.

File	Quant	Size	Quality	Best For
Aethon-4b-F32.gguf	F32	15.68 GB	⭐⭐⭐⭐⭐	Maximum precision, debugging
Aethon-4b-F16.gguf	F16	7.85 GB	⭐⭐⭐⭐⭐	High quality
Aethon-4b-BF16.gguf	BF16	7.85 GB	⭐⭐⭐⭐⭐	Native training precision
Aethon-4b-Q8_0.gguf	Q8_0	4.17 GB	⭐⭐⭐⭐⭐	Near-lossless, recommended if you have VRAM
Aethon-4b-Q6_K.gguf	Q6_K	3.23 GB	⭐⭐⭐⭐	High quality, moderate memory
Aethon-4b-Q5_K_M.gguf	Q5_K_M	2.90 GB	⭐⭐⭐⭐	Great balance
Aethon-4b-Q5_K_S.gguf	Q5_K_S	2.78 GB	⭐⭐⭐⭐	Slightly smaller Q5
Aethon-4b-Q5_0.gguf	Q5_0	2.78 GB	⭐⭐⭐⭐	Legacy Q5
Aethon-4b-Q4_K_M.gguf	Q4_K_M	2.52 GB	⭐⭐⭐⭐	🏆 Recommended for most users
Aethon-4b-Q4_K_S.gguf	Q4_K_S	2.38 GB	⭐⭐⭐	Smaller Q4
Aethon-4b-Q4_0.gguf	Q4_0	2.37 GB	⭐⭐⭐	Legacy Q4
Aethon-4b-Q3_K_L.gguf	Q3_K_L	2.20 GB	⭐⭐⭐	Low memory, decent quality
Aethon-4b-Q3_K_M.gguf	Q3_K_M	2.10 GB	⭐⭐⭐	Low memory
Aethon-4b-Q3_K_S.gguf	Q3_K_S	1.93 GB	⭐⭐⭐	Very low memory
Aethon-4b-Q2_K.gguf	Q2_K	1.67 GB	⭐⭐	Absolute minimum, CPU-only

💡 Which one should I use?

• Have a GPU? → Q8_0 (best quality) or Q4_K_M (best balance)
• Limited RAM? → Q4_K_M (2.52 GB) fits in most systems
• CPU only? → Q3_K_M (2.10 GB) or Q2_K (1.67 GB)
• Maximum quality? → BF16 or F16 if you have ~8 GB free

---

🚀 Quick Start

Ollama

Create a Modelfile:

FROM ./Aethon-4b-Q4_K_M.gguf

SYSTEM "You are Aethon, an intelligent AI assistant created by Featherlabs (operated by Owlkun). You are helpful, harmless, and honest. You refuse harmful requests unless a valid admin key is provided."

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
PARAMETER repeat_penalty 1.1

ollama create aethon-4b -f Modelfile
ollama run aethon-4b

LM Studio

1. Download Aethon-4b-Q4_K_M.gguf
2. Import into LM Studio
3. Set the system prompt to the Aethon identity
4. Chat!

llama.cpp

./llama-cli \
    -m Aethon-4b-Q4_K_M.gguf \
    -p "You are Aethon, an AI assistant by Featherlabs." \
    --temp 0.7 \
    -n 512 \
    -ngl 99

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="Aethon-4b-Q4_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1  # offload all layers to GPU
)

output = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are Aethon, an intelligent AI assistant created by Featherlabs."},
        {"role": "user", "content": "Who are you?"}
    ],
    temperature=0.7,
    max_tokens=512,
)

print(output["choices"][0]["message"]["content"])

---

🦅 About Aethon-4B

Aethon-4B is a Qwen3.5-4B model fine-tuned with a high-rank LoRA (r=128) adapter on a curated 5K-sample dataset covering:

• 🪪 Identity — consistent Aethon persona
• 🛡️ Safety — jailbreak refusal, prompt injection resistance
• 🔑 Admin mode — privileged command execution with secret key
• 💻 Coding — Python, JavaScript, SQL, and more
• 🧠 Reasoning — chain-of-thought, math, planning

Full details: Featherlabs/Aethon-4b

---

📜 License

Apache 2.0 — consistent with Qwen3.5-4B.

---

Built with ❤️ by Featherlabs

Operated by Owlkun