File size: 4,677 Bytes
8eb8b30 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 | ---
license: apache-2.0
base_model: unsloth/DeepSeek-R1-Distill-Llama-8B
tags:
- dyck-language
- bracket-completion
- reasoning
- lora
- fine-tuned
task: text-generation
language: en
---
# DeepSeek-R1-Dyck-Finetuned
## Model Description
This model is a fine-tuned version of **unsloth/DeepSeek-R1-Distill-Llama-8B** specifically optimized for **Dyck language bracket completion** tasks. The model has been trained to complete incomplete Dyck bracket sequences by tracking the stack of open brackets and generating the appropriate closing brackets.
### Key Features
- **Reasoning Capability**: Generates step-by-step reasoning using `<think>` blocks before providing the final answer
- **Dyck Language Completion**: Accurately completes bracket sequences for 8 different bracket types: `()`, `[]`, `{}`, `<>`, `⟨⟩`, `⟦⟧`, `⦃⦄`, `⦅⦆`
- **LoRA Fine-tuning**: Uses Low-Rank Adaptation (LoRA) for efficient training
- **High Accuracy**: Trained on 60k diverse Dyck sequence examples
## Training Details
### Training Data
- **Dataset**: 60k Dyck language sequences
- **Train/Val Split**: 95%/5%
- **Format**: Chat template with system/user/assistant messages
- **Reasoning**: All samples include `<think>` reasoning blocks
### Training Configuration
- **Base Model**: unsloth/DeepSeek-R1-Distill-Llama-8B
- **LoRA Rank**: 32 (attention layers only)
- **LoRA Alpha**: 64
- **LoRA Dropout**: 0.25
- **Learning Rate**: 3e-6
- **Batch Size**: 4 × 32 (effective batch: 128)
- **Epochs**: 4
- **Warmup**: 30% of total steps
- **Gradient Clipping**: 0.05
- **Optimizer**: AdamW
- **Scheduler**: Linear
### Training Hardware
- **GPU**: 40GB GPU
- **Precision**: Full (bfloat16)
- **Training Time**: ~6-8 hours
## Usage
### Installation
```bash
pip install unsloth transformers
```
### Loading the Model
```python
from unsloth import FastLanguageModel
# Load LoRA adapters
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="akashdutta1030/dddd",
max_seq_length=2048,
dtype=None,
load_in_4bit=False, # Use True for 4-bit quantization
)
FastLanguageModel.for_inference(model)
```
### Inference Example
```python
messages = [
{
"role": "system",
"content": "You are a logic engine. Complete the Dyck bracket sequence by tracking the stack of open brackets."
},
{
"role": "user",
"content": "([{<"
}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.1,
top_p=0.95,
do_sample=True,
)
response = tokenizer.decode(outputs[0], skip_special_tokens=False)
print(response)
```
### Expected Output Format
The model generates responses in the following format:
```
<think>
1. Input sequence: (, [, {, <
2. Maintain a stack of opening brackets:
- Push '(' -> Stack: ['(']
- Push '[' -> Stack: ['(', '[']
- Push '{' -> Stack: ['(', '[', '{']
- Push '<' -> Stack: ['(', '[', '{', '<']
3. To close the sequence, pop from the stack in reverse order:
- Pop '<' -> Closing: '>'
- Pop '{' -> Closing: '}'
- Pop '[' -> Closing: ']'
- Pop '(' -> Closing: ')'
4. Appending closing brackets to input: ([{<>}])
</think>
([{<>}])
```
## Model Architecture
- **Base Architecture**: Llama-based (DeepSeek-R1)
- **Parameters**: 8B base model
- **LoRA Parameters**: ~167M trainable parameters (1.8% of base model)
- **Target Modules**: Attention layers only (q_proj, k_proj, v_proj, o_proj)
## Performance
The model has been trained and validated on:
- **Training Loss**: Decreasing smoothly
- **Validation Loss**: Monitored every 200 steps
- **Gradient Stability**: Controlled with strict clipping (0.05)
- **Reasoning Quality**: Generates detailed step-by-step reasoning
## Limitations
- The model is specifically trained for Dyck language bracket completion
- Performance may vary on sequences with very deep nesting (>20 levels)
- Requires proper formatting with chat template for best results
## Citation
If you use this model, please cite:
```bibtex
@misc{deepseek-r1-dyck-finetuned,
title={DeepSeek-R1-Dyck-Finetuned: Bracket Completion Model},
author={Fine-tuned on DeepSeek-R1-Distill-Llama-8B},
year={2024},
howpublished={\url{https://huggingface.co/akashdutta1030/dddd}}
}
```
## License
This model is licensed under the Apache 2.0 license.
## Acknowledgments
- Base model: DeepSeek-R1-Distill-Llama-8B
- Training framework: Unsloth
- Fine-tuning approach: LoRA (Low-Rank Adaptation)
|