Solana-CodeLlama-7B-v1 (Anchor Specialized)

Overview

Solana-CodeLlama-7B-v1 is a domain-specialized language model fine-tuned for writing production-ready Solana Smart Contracts using the Anchor Framework.

While general coding models (like GPT-4 or standard CodeLlama) often hallucinate outdated syntax or struggle with Rust's strict ownership rules, this model was trained on a high-purity synthetic dataset of 10,000 algorithmic examples, focusing specifically on:

• Anchor Macros: Correct usage of #[derive(Accounts)], #[program], #[account].
• Security Constraints: Proper PDA seed validation and constraint checks (e.g., #[account(mut, seeds = [...], bump)]).
• Rust & SPL Tokens: Accurate CPI calls to the SPL Token program.

Performance & Benchmarks

The model was evaluated against the base CodeLlama-7B-Instruct model on a specific "Solana Hold-Out Set".

Metric	Base Model (Zero-Shot)	Solana-CodeLlama-7B-v1
Accuracy (Validation)	~35% (Hallucinates Python/Solidtiy)	97.26%
Accounts Struct	❌ FAIL	✅ PASS
Context Validation	❌ FAIL	✅ PASS
PDA Initialization	❌ FAIL	✅ PASS
SPL Token Transfer	❌ FAIL	✅ PASS

> "The model didn't just learn; it absorbed the syntax structure instantly, dropping loss to 0.02 in < 2 epochs."

Dataset

• Source: 100% Synthetic (Algorithmic Generation).
• Size: 10,000 Verified Examples.
• Methodology: We utilized a "Textbook Quality" approach, generating examples with perfect compile-ready logic rather than scraping noisy GitHub repositories.

Usage

1. Using Unsloth (Fastest)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "your-username/Solana-CodeLlama-7B-v1",
    max_seq_length = 2048,
    dtype = None,
    load_in_4bit = True,
)

prompt = """Write a Solana Anchor program to initialize a user vault."""
# ... Apply chat template ...

2. Using GGUF (Ollama / LM Studio)

This model is available in GGUF format for local deployment on consumer hardware (MacBook M1/M2/M3, NVIDIA RTX 3060/4090/5090).

• Solana-CodeLlama-7B-v1.Q4_K_M.gguf (Recommended for 8GB+ RAM)
• Solana-CodeLlama-7B-v1.Q8_0.gguf (High Precision)

Training Details

• Hardware: NVIDIA RTX 5090 (32GB VRAM).
• Framework: Unsloth (Open Source).
• Precision: Mixed Precision (BF16).
• LoRA Rank: 16.
• Batch Size: 8 (Effective).

License

Based on CodeLlama (Llama 2 Community License).

---

Fine-tuned with ❤️ using Unsloth.