Modelfile

# Point to your fine-tuned GGUF model
FROM ./unsloth.Q4_K_M.gguf

# --- CORRECTED TEMPLATE FOR LLAMA 3 CHAT FORMAT ---
# This template defines how your input (prompt and system message) is fed to the model.
# It matches the instruction-following format Llama 3 models are typically trained on.
TEMPLATE """<|begin_of_text|><|start_header_id|>system<|end_header_id|>



{{ .System }}<|eot_id|><|start_header_id|>user<|end_header_id|>



{{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>



"""

# --- REFINED STOP PARAMETERS (More Aggressive) ---
# These tokens tell the model when to stop generating output.
# They are crucial for preventing the model from generating parts of the next turn or garbage.
# We're adding more specific stop conditions for Llama 3's turn structure and common over-generation.
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"         # End Of Turn token
PARAMETER stop "<|end_of_text|>"    # End Of Text token (main EOS token)
PARAMETER stop "<|reserved_special_token_250|>" # The pad_token, as it sometimes gets generated unexpectedly
PARAMETER stop "<|reserved_special_token_28|>"
PARAMETER stop "<|reserved_special_token_185|>"

# Crucially, stop on newlines followed by the start of the next role markers
# This often catches the model trying to start the next 'user' or 'assistant' turn.
PARAMETER stop "\n<|start_header_id|>user"
PARAMETER stop "\n<|start_header_id|>assistant"

# Also, stop if the raw string "user" or "assistant" appears, in case the full header token isn't generated
PARAMETER stop "user"
PARAMETER stop "assistant"

# You can also add other commonly appearing tokens if they signify the end of a response
# PARAMETER stop "\n\n" # This can be aggressive, but sometimes useful
# PARAMETER stop "###" # If you see any triple hash marks signifying a new section

# Optional: Adjust parameters for inference if needed (keep these as they were if they worked for you)
# PARAMETER temperature 0.7
# PARAMETER top_k 40
# PARAMETER top_p 0.9
# PARAMETER num_gpu 1 # Set this to the number of GPUs you want to use, or 0 for CPU only

SYSTEM """You are a guvi chatbot.you will behave like a guvi customer care who will solve customer query with  politeness and helpful tone.

You are available 24/7 to assist.

If the customer provides any numbers (e.g., in an order ID, reference number, or contact number), you must refer to them with a general term like "your order number," "the reference number provided," or "the contact number" instead of printing the exact digits.

"""