| # Point to your fine-tuned GGUF model | |
| FROM ./unsloth.Q4_K_M.gguf | |
| # --- CORRECTED TEMPLATE FOR LLAMA 3 CHAT FORMAT --- | |
| # This template defines how your input (prompt and system message) is fed to the model. | |
| # It matches the instruction-following format Llama 3 models are typically trained on. | |
| TEMPLATE """<|begin_of_text|><|start_header_id|>system<|end_header_id|> | |
| {{ .System }}<|eot_id|><|start_header_id|>user<|end_header_id|> | |
| {{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|> | |
| """ | |
| # --- REFINED STOP PARAMETERS (More Aggressive) --- | |
| # These tokens tell the model when to stop generating output. | |
| # They are crucial for preventing the model from generating parts of the next turn or garbage. | |
| # We're adding more specific stop conditions for Llama 3's turn structure and common over-generation. | |
| PARAMETER stop "<|start_header_id|>" | |
| PARAMETER stop "<|end_header_id|>" | |
| PARAMETER stop "<|eot_id|>" # End Of Turn token | |
| PARAMETER stop "<|end_of_text|>" # End Of Text token (main EOS token) | |
| PARAMETER stop "<|reserved_special_token_250|>" # The pad_token, as it sometimes gets generated unexpectedly | |
| PARAMETER stop "<|reserved_special_token_28|>" | |
| PARAMETER stop "<|reserved_special_token_185|>" | |
| # Crucially, stop on newlines followed by the start of the next role markers | |
| # This often catches the model trying to start the next 'user' or 'assistant' turn. | |
| PARAMETER stop "\n<|start_header_id|>user" | |
| PARAMETER stop "\n<|start_header_id|>assistant" | |
| # Also, stop if the raw string "user" or "assistant" appears, in case the full header token isn't generated | |
| PARAMETER stop "user" | |
| PARAMETER stop "assistant" | |
| # You can also add other commonly appearing tokens if they signify the end of a response | |
| # PARAMETER stop "\n\n" # This can be aggressive, but sometimes useful | |
| # PARAMETER stop "###" # If you see any triple hash marks signifying a new section | |
| # Optional: Adjust parameters for inference if needed (keep these as they were if they worked for you) | |
| # PARAMETER temperature 0.7 | |
| # PARAMETER top_k 40 | |
| # PARAMETER top_p 0.9 | |
| # PARAMETER num_gpu 1 # Set this to the number of GPUs you want to use, or 0 for CPU only | |
| SYSTEM """You are a guvi chatbot.you will behave like a guvi customer care who will solve customer query with politeness and helpful tone. | |
| You are available 24/7 to assist. | |
| If the customer provides any numbers (e.g., in an order ID, reference number, or contact number), you must refer to them with a general term like "your order number," "the reference number provided," or "the contact number" instead of printing the exact digits. | |
| """ |