Hugging Face's logo

intercroc
/
gemma_3_finetune

GGUF
gemma3
llama.cpp
unsloth
vision-language-model
conversational
Model card Files
xet
 Community
How to use from the
Use from the
llama-cpp-python library
# Gated model: Login with a HF token with gated access permission
hf auth login
# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="intercroc/gemma_3_finetune",
	filename="",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

gemma_3_finetune : GGUF

This model was finetuned and converted to GGUF format using Unsloth.

Example usage:

  • For text only LLMs: ./llama.cpp/llama-cli -hf intercroc/gemma_3_finetune --jinja
  • For multimodal models: ./llama.cpp/llama-mtmd-cli -hf intercroc/gemma_3_finetune --jinja

Available Model files:

  • gemma-3-12b-it.Q4_K_M.gguf
  • gemma-3-12b-it.BF16-mmproj.gguf

⚠️ Ollama Note for Vision Models

Important: Ollama currently does not support separate mmproj files for vision models.

To create an Ollama model from this vision model:

  1. Place the Modelfile in the same directory as the finetuned bf16 merged model
  2. Run: ollama create model_name -f ./Modelfile (Replace model_name with your desired name)

This will create a unified bf16 model that Ollama can use.

Note

The model's BOS token behavior was adjusted for GGUF compatibility. This was trained 2x faster with Unsloth

Downloads last month
-
GGUF
Model size
12B params
Architecture
gemma3
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support