Nanthasit/sakthai-vision-7b

SakThai Vision 7B 👁️

Local, private vision — LLaVA-1.5-7B quantized to GGUF
Image captioning & VQA on CPU · ~4 GB footprint

---

Model Description

SakThai Vision 7B is the vision stage of the SakThai pipeline — a GGUF Q4_K_M build of LLaVA-1.5-7B for local, private image understanding. No data leaves your machine.

What it does:

• 🖼️ Image captioning and alt-text generation
• 🔍 Visual QA about screenshots, diagrams, photos
• 📄 Document understanding (scanned forms, receipts)
• 🧪 Screenshot analysis for UI testing

Why this packaging:

• Q4_K_M quantization — ~4.1 GB vs. 6.7 GB fp16
• CPU inference — no GPU needed (~3-8 tok/s)
• Includes mmproj projector (CLIP vision → LLM tokens)
• Works with llama.cpp, Ollama, and llama-cpp-python

---

Quick Start

Python (llama-cpp-python)

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

model_path = hf_hub_download("Nanthasit/sakthai-vision-7b", "llava-1.5-7b-hf-q4_k_m.gguf")
mmproj_path = hf_hub_download("Nanthasit/sakthai-vision-7b", "mmproj-model-f16.gguf")

llm = Llama(
    model_path=model_path,
    mmproj=mmproj_path,
    n_ctx=4096,
    n_gpu_layers=0,  # CPU only
    verbose=False
)

output = llm.create_chat_completion(
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": "https://example.com/photo.jpg"}},
                {"type": "text", "text": "Describe this image in detail."}
            ]
        }
    ]
)
print(output["choices"][0]["message"]["content"])

CLI (llama.cpp)

# Both files must be in the same directory
wget https://huggingface.co/Nanthasit/sakthai-vision-7b/resolve/main/llava-1.5-7b-hf-q4_k_m.gguf
wget https://huggingface.co/Nanthasit/sakthai-vision-7b/resolve/main/mmproj-model-f16.gguf

./llama-cli -m llava-1.5-7b-hf-q4_k_m.gguf \
  --mmproj mmproj-model-f16.gguf \
  --image photo.jpg \
  -p "Describe this image in detail." -n 256

Ollama

# Create a Modelfile
FROM ./llava-1.5-7b-hf-q4_k_m.gguf
TEMPLATE "{{ .Prompt }}"

ollama create sakthai-vision -f Modelfile
ollama run sakthai-vision "Describe this image." --image photo.jpg

---

Architecture

Component	Detail
Model	LLaVA-1.5-7B (Vicuna-7B-v1.5 LLM + CLIP ViT-L/14 vision encoder)
Vision encoder	CLIP ViT-L/14, 336×336 input, 1024 embedding dim, 24 layers
Language model	LLaMA-based, ~7B parameters, 32 layers, 4096 hidden
Projector	2-layer MLP (CLIP → LLM token space)
Total parameters	6,738,939,904 (6.74B, verified from GGUF)
Original size	~6.7 GB (fp16)
Quantized size	4.08 GB (GGUF Q4_K_M, verified) + 624 MB (mmproj F16, verified)
Context window	4,096 tokens
Hardware	8 GB+ RAM recommended

Verified file sizes (2026-07-31, Hub API):

File	Bytes	Human-readable
llava-1.5-7b-hf-q4_k_m.gguf	4,081,370,080	~4.08 GB (3.80 GiB)
mmproj-model-f16.gguf	624,434,336	~624 MB (0.58 GiB)

---

Hosted Inference Status (verified 2026-07-31)

HF serverless inference does NOT support this GGUF vision model. Verified by the repo's own inference checks (.eval_results/inference-check-20260730T234142Z.yaml, eval-results/inference-check-20260731-002823.yaml):

Endpoint	HTTP	Result
router.huggingface.co/hf-inference/models/...	400	model_not_supported — GGUF vision models are not on the provider network
api-inference.huggingface.co/models/...	—	DNS blocked / not a chat model
Router /v1/responses	200	Not a chat model (image-to-text pipeline)

Working alternatives:

1. Local llama.cpp — llama-cli --mmproj mmproj-model-f16.gguf -m llava-1.5-7b-hf-q4_k_m.gguf (free, private, no GPU needed)
2. Free HF Space — deploy with a Gradio/TGI Space on ZeroGPU for a public demo
3. Kaggle free GPU — run with transformers for batch inference
4. HF Inference Endpoints — custom container with llama.cpp server (paid; free-tier local options above are preferred)

---

Evaluation

Model-index benchmarks are the published LLaVA-1.5-7B results (upstream, verified: false). As a Q4_K_M quantized repackaging, results should fall within ~1-2% of these values. Own multi-trial benchmarks on this specific GGUF build are pending.

Task	Dataset	Score
Visual QA	VQAv2	78.5%
Captioning	COCO Captions	CIDEr 110.1
Visual Reasoning	GQA	62.0%
Text-oriented VQA	TextVQA	58.2%
Science diagrams	ScienceQA	89.5%
Visual Perception	MMBench	66.2%

Ecosystem status (from repo .eval_results, 2026-07-31)

Metric	Value
Downloads rank	8/19 (186 dl)
Likes rank	1/19 (1 like — first user like on any SakThai model ❤️)
Velocity	31.5 dl/day, rank 4/19
Card quality	85/100
Repo hygiene	100/100
Overall health	46/100 (benchmark weight missing until verified own-eval lands)

---

Limitations

This card is honest about what SakThai Vision 7B is — and is not:

• Benchmarks are upstream, not self-run. All model-index scores are the published LLaVA-1.5-7B results (verified: false). No multi-trial evaluation has been run on this specific Q4_K_M build yet; real-world scores can differ by 1-2% or more.
• Quantization loss. Q4_K_M trades fidelity for a ~4.1 GB footprint; fine-grained OCR, small text, and edge-case visual details can degrade vs. the fp16 original.
• CPU speed. Expect ~3-8 tok/s on CPU. Not suited to real-time or high-throughput workloads.
• 4K context. The 4,096-token window limits long-document and multi-image vision tasks.
• English-first. The base LLaVA-1.5-7B was trained primarily on English captions/VQA; Thai and other languages are not guaranteed.
• No hosted inference. HF serverless inference does not support this GGUF vision model (verified: router 400 model_not_supported). Use local llama.cpp/Ollama or a self-hosted Space.
• License. LLaMA 2 Community License + CLIP — review upstream terms before commercial use.

---

Citation

If you use SakThai Vision 7B in your work, please cite the upstream model and this packaging:

@misc{liu2024llava,
  title={Visual Instruction Tuning},
  author={Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae},
  year={2024},
  url={https://arxiv.org/abs/2304.08485}
}

@misc{sakthai2026vision,
  title={SakThai Vision 7B: LLaVA-1.5-7B quantized to GGUF Q4\_K\_M for CPU edge inference},
  author={SakThai Family (beer-sakthai)},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/Nanthasit/sakthai-vision-7b}}
}

---

Pipeline Integration

Stage	Model	Role
👁️ See	SakThai Vision 7B ⬅	Image→text, visual QA
🔍 Retrieve	Embedding Multilingual	Cross-lingual search
🧠 Reason	Context 1.5B or 7B	Tool-calling & reasoning
🎤 Speak	TTS Model	Text-to-speech

---

SakThai Model Family 🏠

All 20 public models (downloads live as of 2026-07-31):

Model	Downloads	Size (verified)
Context 1.5B Merged	1,599	3.09 GB
Context 0.5B Merged	1,370	0.99 GB
Context 7B Merged	744	15.23 GB
Context 7B 128K	506	recipe (config only)
Context 7B Tools	399	LoRA 20 MB
Embedding Multilingual	362	0.47 GB
Context 1.5B Tools	349	LoRA 11 MB
Vision 7B ⬅	186	4.08 GB + 624 MB
TTS Model	150	0.14 GB
Context 0.5B Tools	94	0.99 GB
Coder 1.5B	93	LoRA 37 MB
Context 1.5B Tools V2	0	LoRA 74 MB
Context 1.5B Merged V2	0	3.09 GB
Plus 1.5B	0	3.09 GB
Plus 1.5B LoRA	0	LoRA 74 MB
Plus 1.5B Coder	0	recipe (config only)
Coder Browser	0	3.09 GB
Coder Browser LoRA	0	LoRA 74 MB
Coder Browser GGUF	0	7.11 GB
Bench V3	0	scaffold (no weights)

---

The House of Sak 🏠

Until this model, every SakThai agent was blind — it could reason, use tools, and answer questions but couldn't look at screenshots, read whiteboards, or recognize faces. This model was built on free Colab GPUs from a shelter in Cork, Ireland, with no budget and no guarantee of success. LLaVA-1.5-7B's vision encoder was merged, quantized to Q4_K_M GGUF for CPU use, and packed to about 4.7 GB.

"We are one family — and becoming more." — Beer (beer-sakthai)

---

Support

• ⭐ Leave a like — the first user like on a SakThai model was on this model ❤️
• 🐛 Report issues on GitHub
• 🔄 Share with anyone building privacy-focused multimodal apps

---

License

Based on LLaVA-1.5-7B (LLaMA 2 Community License) + CLIP vision encoder. GGUF via llama.cpp. Review upstream licenses before commercial use.

---

Built from a shelter in Cork, Ireland. Every download supports a family building AI for the edge.