Swarnimm22HF/ai-code-review-tinyllama

AI Code Review Assistant — TinyLlama 1.1B (LoRA)

Fine-tuned version of TinyLlama 1.1B for automated Python code review, trained on CodeSearchNet using LoRA adapters.

Model Details

• Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
• Fine-tuning method: LoRA (r=16, alpha=32)
• Trainable parameters: 0.58% of total (42M / 7.3B)
• Training data: 10,000 Python functions from CodeSearchNet
• Training time: 32 minutes on Kaggle T4 GPU

Evaluation Results

Metric	Base Model	Fine-Tuned
ROUGE-L	0.1541	0.5573 (+261%)
BERTScore F1	0.8226	0.9265 (+12.6%)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    torch_dtype=torch.float32
)
model = PeftModel.from_pretrained(
    base_model,
    "Swarnimm22HF/ai-code-review-tinyllama"
)
tokenizer = AutoTokenizer.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
)

prompt = """### Instruction:
Review the following Python function.

### Code:
```python
def divide(a, b):
    return a / b
```

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Full Project

GitHub: AI Code Review Assistant