README.md

metadata

language:
  - en
license: apache-2.0
tags:
  - pytorch
  - peft
  - lora
  - code-generation
  - deepseek-coder
  - fine-tuned
datasets:
  - custom-code-dataset
model-index:
  - name: BriskFO_Coderv1
    results: []

BriskFO_Coderv1

Model Description

This is a PEFT/LoRA adapter fine-tuned on DeepSeek Coder 1.3B Instruct model. It was trained for 300 steps on a custom code generation dataset.

Model Type

This is a PEFT (Parameter-Efficient Fine-Tuning) model, specifically using LoRA (Low-Rank Adaptation). It contains only the adapter weights, not the full model.

Training Details

• Base Model: deepseek-ai/deepseek-coder-1.3b-instruct
• Training Steps: 300
• Learning Rate: 2e-4
• Batch Size: 16
• Gradient Accumulation: 4
• Sequence Length: 34958
• Training Method: PEFT/LoRA

Files

This repository contains:

• adapter_model.bin / adapter_model.safetensors - LoRA adapter weights
• adapter_config.json - PEFT configuration
• tokenizer.json, tokenizer_config.json - Tokenizer files
• special_tokens_map.json - Special tokens mapping

Usage

Installation

pip install transformers peft accelerate torch

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig

# Load the base model
base_model_id = "deepseek-ai/deepseek-coder-1.3b-instruct"
adapter_model_id = "abel252/BriskFO_Coderv1"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_model_id)

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype="auto",
    device_map="auto"
)

# Load PEFT adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)

# For inference, you can merge the adapter with the base model (optional)
# model = model.merge_and_unload()

Inference Example

# Prepare input
prompt = "Write a Python function to calculate fibonacci numbers"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.95,
    do_sample=True
)

# Decode
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

License

This model is released under the Apache 2.0 license.

Acknowledgments

• Base model: DeepSeek Coder
• Fine-tuning framework: PEFT