# CodeLlama Fine-Tuning for RTL Code Generation

This repository contains scripts, datasets, and documentation for fine-tuning CodeLlama-7B-Instruct model for Verilog/SystemVerilog RTL code generation.

## 📋 Overview

This project fine-tunes CodeLlama-7B-Instruct to generate synthesizable Verilog/SystemVerilog code for hardware design tasks, specifically focusing on FIFO implementations.

## 🎯 Features

- **CodeLlama-7B-Instruct Fine-tuning** with LoRA
- **Chat Template Format** support
- **Dataset Processing** and validation scripts
- **Training Scripts** with checkpoint resume capability
- **Inference Scripts** for testing fine-tuned models
- **Comprehensive Documentation** and guides

## 📁 Repository Structure

```
codellama-migration/
├── datasets/                    # Training datasets
│   ├── raw/                    # Original datasets
│   └── processed/              # Processed and formatted datasets
│       ├── split/              # Train/val/test splits (original format)
│       └── split_chat_format/  # Train/val/test splits (chat format)
├── scripts/
│   ├── training/               # Training scripts
│   ├── inference/              # Inference scripts
│   └── dataset_split.py        # Dataset splitting utility
├── Documentation/              # All .md documentation files
└── Scripts/                    # Utility scripts
```

## 🚀 Quick Start

### Prerequisites

- Python 3.8+
- CUDA-capable GPU (recommended)
- HuggingFace transformers library
- PyTorch

### Installation

```bash
pip install transformers torch peft accelerate bitsandbytes
```

### Training

```bash
bash start_training_chat_format.sh
```

### Inference

```bash
python3 scripts/inference/inference_codellama.py \
    --mode local \
    --model-path training-outputs/codellama-fifo-v2-chat \
    --base-model-path models/base-models/CodeLlama-7B-Instruct \
    --prompt "Your prompt here"
```

## 📊 Dataset

The dataset contains 94 samples of FIFO implementations in Verilog format. It's split into:
- Training: 70 samples (75%)
- Validation: 9 samples (10%)
- Test: 15 samples (15%)

## 📚 Documentation

- **MIGRATION_PROGRESS.md** - Overall migration tracking
- **TRAINING_COMPLETE.md** - Training completion details
- **COMPARISON_REPORT.md** - Expected vs Generated comparison
- **FILE_INVENTORY.md** - Complete file listing

## 🤖 Model Information

**Base Model**: CodeLlama-7B-Instruct  
**Fine-tuning Method**: LoRA (Low-Rank Adaptation)  
**LoRA Rank**: 48  
**LoRA Alpha**: 96  
**Trainable Parameters**: ~120M (3.31% of total)

## 📝 License

This project is for internal use by Elinnos Systems Pvt Limited.

## 👥 Contributors

Elinnos Systems Pvt Limited

## 🔗 Links

- Organization: https://huggingface.co/Elinnos
- Base Model: https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf

---

**Note**: Model weights are not included in this repository. Fine-tuned models are stored separately.