Community/reference GGUF models for local inference on consumer GPUs. Compatible with Kimari Local AI v0.1.83-alpha.
-
TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF
1B ⢠Updated ⢠63.8k ⢠239 -
bartowski/Qwen2.5-1.5B-Instruct-GGUF
Text Generation ⢠2B ⢠Updated ⢠74.2k ⢠23 -
bartowski/SmolLM2-1.7B-Instruct-GGUF
Text Generation ⢠2B ⢠Updated ⢠15.3k ⢠22 -
bartowski/Llama-3.2-1B-Instruct-GGUF
Text Generation ⢠1B ⢠Updated ⢠154k ⢠185