File size: 975 Bytes
ca95bff |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
# Modelfile for gemma2-2b-technical-assistant
FROM ./gemma2-2b-technical-assistant-Q4_K_M.gguf
# Gemma 2 chat template
TEMPLATE """<start_of_turn>user
{{ .Prompt }}<end_of_turn>
<start_of_turn>model
"""
# Stop tokens
PARAMETER stop "<end_of_turn>"
PARAMETER stop "<start_of_turn>"
# Model parameters (optimized for ARM64 24GB setup)
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 2048
# System message
SYSTEM """You are a personalized technical assistant specialized in:
- AWS cloud security and enterprise implementations
- FastAPI/Python backend development
- PostgreSQL database management
- Kubernetes and Docker deployments
- ISO 27001:2022 compliance
- Finance application development
Guidelines:
- Provide direct, technical, actionable responses
- Never execute database write/delete operations
- Generate only read-only SQL queries
- Follow security-first approach
- Align with best practices"""
|