README.md

---
license: apache-2.0
base_model: unsloth/DeepSeek-R1-Distill-Llama-8B
tags:
  - dyck-language
  - bracket-completion
  - reasoning
  - lora
  - fine-tuned
task: text-generation
language: en
---

# DeepSeek-R1-Dyck-Finetuned

## Model Description

This model is a fine-tuned version of **unsloth/DeepSeek-R1-Distill-Llama-8B** specifically optimized for **Dyck language bracket completion** tasks. The model has been trained to complete incomplete Dyck bracket sequences by tracking the stack of open brackets and generating the appropriate closing brackets.

### Key Features

- **Reasoning Capability**: Generates step-by-step reasoning using `<think>` blocks before providing the final answer
- **Dyck Language Completion**: Accurately completes bracket sequences for 8 different bracket types: `()`, `[]`, `{}`, `<>`, `⟨⟩`, `⟦⟧`, `⦃⦄`, `⦅⦆`
- **LoRA Fine-tuning**: Uses Low-Rank Adaptation (LoRA) for efficient training
- **High Accuracy**: Trained on 60k diverse Dyck sequence examples

## Training Details

### Training Data
- **Dataset**: 60k Dyck language sequences
- **Train/Val Split**: 95%/5%
- **Format**: Chat template with system/user/assistant messages
- **Reasoning**: All samples include `<think>` reasoning blocks

### Training Configuration
- **Base Model**: unsloth/DeepSeek-R1-Distill-Llama-8B
- **LoRA Rank**: 32 (attention layers only)
- **LoRA Alpha**: 64
- **LoRA Dropout**: 0.25
- **Learning Rate**: 3e-6
- **Batch Size**: 4 × 32 (effective batch: 128)
- **Epochs**: 4
- **Warmup**: 30% of total steps
- **Gradient Clipping**: 0.05
- **Optimizer**: AdamW
- **Scheduler**: Linear

### Training Hardware
- **GPU**: 40GB GPU
- **Precision**: Full (bfloat16)
- **Training Time**: ~6-8 hours

## Usage

### Installation

```bash
pip install unsloth transformers
```

### Loading the Model

```python
from unsloth import FastLanguageModel

# Load LoRA adapters
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="akashdutta1030/dddd",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=False,  # Use True for 4-bit quantization
)

FastLanguageModel.for_inference(model)
```

### Inference Example

```python
messages = [
    {
        "role": "system",
        "content": "You are a logic engine. Complete the Dyck bracket sequence by tracking the stack of open brackets."
    },
    {
        "role": "user",
        "content": "([{<"
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.1,
        top_p=0.95,
        do_sample=True,
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=False)
print(response)
```

### Expected Output Format

The model generates responses in the following format:

```
<think>
1. Input sequence: (, [, {, <
2. Maintain a stack of opening brackets:
   - Push '(' -> Stack: ['(']
   - Push '[' -> Stack: ['(', '[']
   - Push '{' -> Stack: ['(', '[', '{']
   - Push '<' -> Stack: ['(', '[', '{', '<']
3. To close the sequence, pop from the stack in reverse order:
   - Pop '<' -> Closing: '>'
   - Pop '{' -> Closing: '}'
   - Pop '[' -> Closing: ']'
   - Pop '(' -> Closing: ')'
4. Appending closing brackets to input: ([{<>}])
</think>
([{<>}])
```

## Model Architecture

- **Base Architecture**: Llama-based (DeepSeek-R1)
- **Parameters**: 8B base model
- **LoRA Parameters**: ~167M trainable parameters (1.8% of base model)
- **Target Modules**: Attention layers only (q_proj, k_proj, v_proj, o_proj)

## Performance

The model has been trained and validated on:
- **Training Loss**: Decreasing smoothly
- **Validation Loss**: Monitored every 200 steps
- **Gradient Stability**: Controlled with strict clipping (0.05)
- **Reasoning Quality**: Generates detailed step-by-step reasoning

## Limitations

- The model is specifically trained for Dyck language bracket completion
- Performance may vary on sequences with very deep nesting (>20 levels)
- Requires proper formatting with chat template for best results

## Citation

If you use this model, please cite:

```bibtex
@misc{deepseek-r1-dyck-finetuned,
  title={DeepSeek-R1-Dyck-Finetuned: Bracket Completion Model},
  author={Fine-tuned on DeepSeek-R1-Distill-Llama-8B},
  year={2024},
  howpublished={\url{https://huggingface.co/akashdutta1030/dddd}}
}
```

## License

This model is licensed under the Apache 2.0 license.

## Acknowledgments

- Base model: DeepSeek-R1-Distill-Llama-8B
- Training framework: Unsloth
- Fine-tuning approach: LoRA (Low-Rank Adaptation)