# chipforge-llm

LoRA adapters for Verilog HDL generation, fine-tuned on Nemotron-30B.

## phase_1/v0.1/

| Field | Value |
|-------|-------|
| Base model | NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 |
| Training data | 36,321 Verilog modules (filtered ≤10k tokens) |
| Method | LoRA CPT (Continued Pre-Training) |
| LoRA rank | 8 |
| Trainable params | ~4.5M (0.015% of 30B) |
| Max seq len | 1024 |
| Epochs | 2 |
| Final train loss | 0.42 |
| Final eval loss | 0.43 |
| Token accuracy | ~89% |
| Hardware | 2x A100-80GB, FSDP |

### Usage

``\python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "NVIDIA/Nemotron-3-Nano-30B-A3B-BF16",
    torch_dtype="bfloat16",
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, "Ashx098/chipforge-llm/phase_1/v0.1")
tokenizer = AutoTokenizer.from_pretrained(
    "NVIDIA/Nemotron-3-Nano-30B-A3B-BF16",
    trust_remote_code=True,
)
```

### wandb

Run: https://wandb.ai/avinash-mynampati-juspay/vaschpforge-llm/runs/c5gid36w