File size: 2,526 Bytes
1237a4b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
language:
- en
license: apache-2.0
tags:
- pytorch
- peft
- lora
- code-generation
- deepseek-coder
- fine-tuned
datasets:
- custom-code-dataset
model-index:
- name: BriskFO_Coderv1
results: []
---
# BriskFO_Coderv1
## Model Description
This is a **PEFT/LoRA adapter** fine-tuned on DeepSeek Coder 1.3B Instruct model. It was trained for 300 steps on a custom code generation dataset.
## Model Type
This is a **PEFT (Parameter-Efficient Fine-Tuning)** model, specifically using **LoRA (Low-Rank Adaptation)**. It contains only the adapter weights, not the full model.
## Training Details
- **Base Model**: `deepseek-ai/deepseek-coder-1.3b-instruct`
- **Training Steps**: 300
- **Learning Rate**: 2e-4
- **Batch Size**: 16
- **Gradient Accumulation**: 4
- **Sequence Length**: 34958
- **Training Method**: PEFT/LoRA
## Files
This repository contains:
- `adapter_model.bin` / `adapter_model.safetensors` - LoRA adapter weights
- `adapter_config.json` - PEFT configuration
- `tokenizer.json`, `tokenizer_config.json` - Tokenizer files
- `special_tokens_map.json` - Special tokens mapping
## Usage
### Installation
```bash
pip install transformers peft accelerate torch
```
### Loading the Model
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig
# Load the base model
base_model_id = "deepseek-ai/deepseek-coder-1.3b-instruct"
adapter_model_id = "abel252/BriskFO_Coderv1"
# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(adapter_model_id)
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype="auto",
device_map="auto"
)
# Load PEFT adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)
# For inference, you can merge the adapter with the base model (optional)
# model = model.merge_and_unload()
```
### Inference Example
```python
# Prepare input
prompt = "Write a Python function to calculate fibonacci numbers"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
# Generate
outputs = model.generate(
**inputs,
max_new_tokens=256,
temperature=0.7,
top_p=0.95,
do_sample=True
)
# Decode
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```
## License
This model is released under the Apache 2.0 license.
## Acknowledgments
- Base model: [DeepSeek Coder](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct)
- Fine-tuning framework: [PEFT](https://github.com/huggingface/peft)
|