TheMightyMaddy/verilog-agent-llama31-q4

🔧 Verilog Agent LLaMA 3.1 8B (Q4_K_M GGUF)

A fine-tuned version of Meta LLaMA 3.1 8B Instruct, specialized for Verilog code generation, debugging, and hardware design assistance. Converted to GGUF format for efficient local inference.

Fine-tuned and converted using Unsloth — 2x faster training with 60% less VRAM.

---

🚀 Quick Start

llama.cpp

# Text generation
llama-cli -hf Asssssy/verilog-agent-llama31-q4 --jinja

# Interactive mode
llama-cli -hf Asssssy/verilog-agent-llama31-q4 --jinja -i

Ollama

ollama run Asssssy/verilog-agent-llama31-q4

LM Studio

Search Asssssy/verilog-agent-llama31-q4 directly in LM Studio's model browser.

---

📦 Available Files

File	Quantization	Size	Use Case
Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf	Q4_K_M	~4.5GB	Best balance of speed & quality

---

💡 Example Usage

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Asssssy/verilog-agent-llama31-q4",
    filename="Meta-Llama-3.1-8B-Instruct.Q4_K_M.gguf",
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are an expert Verilog hardware design assistant."},
        {"role": "user", "content": "Write a 4-bit counter in Verilog."}
    ]
)
print(response["choices"][0]["message"]["content"])

---

🧠 Model Details

Property	Value
Base Model	Meta-LLaMA 3.1 8B Instruct
Fine-tuning Method	LoRA (QLoRA 4-bit)
Quantization	Q4_K_M
Max Sequence Length	2048
Domain	Verilog / Hardware Design
Framework	Unsloth + HuggingFace

---

⚙️ System Prompt

For best results, use this system prompt: " You are an expert Verilog and digital hardware design assistant. Help users write correct, synthesizable Verilog code. Explain your reasoning and flag any potential timing or synthesis issues."

---

⚠️ Limitations

• Specialized for Verilog — general coding tasks may have reduced performance
• Based on LLaMA 3.1 8B — larger models may outperform on complex designs
• Always verify generated Verilog with a simulator (e.g. Icarus Verilog, ModelSim)

---

📜 License

This model is based on Meta LLaMA 3.1 and inherits its license. Please review Meta's usage policy before deploying.

---