demoversion commited on
Commit
df8263a
·
verified ·
1 Parent(s): 5b22dce

Update model card

Browse files
Files changed (1) hide show
  1. README.md +41 -28
README.md CHANGED
@@ -83,46 +83,59 @@ special_tokens:
83
 
84
  </details><br>
85
 
86
- # outputs/cf-llm-finetune-llama-3.2-3b-lora
87
 
88
- This model is a fine-tuned version of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) on the ./data/train_openai_response_transformed.jsonl dataset.
89
 
90
- ## Model description
91
 
92
- More information needed
 
 
 
93
 
94
- ## Intended uses & limitations
95
 
96
- More information needed
97
 
98
- ## Training and evaluation data
99
 
100
- More information needed
101
 
102
- ## Training procedure
 
 
 
103
 
104
- ### Training hyperparameters
105
 
106
- The following hyperparameters were used during training:
107
- - learning_rate: 0.0002
108
- - train_batch_size: 2
109
- - eval_batch_size: 2
110
- - seed: 42
111
- - gradient_accumulation_steps: 4
112
- - total_train_batch_size: 8
113
- - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
114
- - lr_scheduler_type: cosine
115
- - lr_scheduler_warmup_steps: 10
116
- - training_steps: 688
117
 
118
- ### Training results
119
 
 
 
120
 
 
 
 
121
 
122
- ### Framework versions
123
 
124
- - PEFT 0.15.2
125
- - Transformers 4.52.3
126
- - Pytorch 2.6.0+cu124
127
- - Datasets 3.6.0
128
- - Tokenizers 0.21.2
 
 
 
 
 
 
 
 
 
 
 
83
 
84
  </details><br>
85
 
 
86
 
87
+ # Llama-3.2-3B-Instruct-PEFT-code-generation
88
 
89
+ This model is a fine tuned [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) on a synthetic dataset of C++ → Python code translations from Codeforces.
90
 
91
+ 📦 GitHub repo: [DemoVersion/cf-llm-finetune](https://github.com/DemoVersion/cf-llm-finetune)
92
+ 📑 Dataset Creation [DATASET.md](https://github.com/DemoVersion/cf-llm-finetune/blob/main/DATASET.md)
93
+ 📑 Training [TRAIN.md](https://github.com/DemoVersion/cf-llm-finetune/blob/main/TRAIN.md)
94
+ 📚 Dataset on Hugging Face: [demoversion/cf-cpp-to-python-code-generation](https://huggingface.co/datasets/demoversion/cf-cpp-to-python-code-generation)
95
 
96
+ For dataset generation, training, and inference check the [Github repo](https://github.com/DemoVersion/cf-llm-finetune).
97
 
98
+ ## Model description
99
 
100
+ A lightweight LLaMA 3.2 model fine-tuned for competitive programming code translation, from ICPC-style C++ to Python using LoRA adapters.
101
 
102
+ ## Intended uses & limitations
103
 
104
+ **Use for:**
105
+ - Translating competitive programming C++ solutions to Python
106
+ - Code understanding in educational or automation tools
107
+
108
 
109
+ **Limitations:**
110
 
111
+ - Not general-purpose code translation
112
+ - Python outputs are synthetically generated using GPT-4.1
113
+ - Focused only on ICPC-style problems
114
+
 
 
 
 
 
 
 
115
 
116
+ ## Training and evaluation data
117
 
118
+ Training and Evaluation data:
119
+ 🧾 [demoversion/cf-cpp-to-python-code-generation](https://huggingface.co/datasets/demoversion/cf-cpp-to-python-code-generation)
120
 
121
+ Built from:
122
+ - [open-r1/codeforces-submissions](https://huggingface.co/datasets/open-r1/codeforces-submissions)
123
+ - [open-r1/codeforces](https://huggingface.co/datasets/open-r1/codeforces)
124
 
125
+ C++ submissions were filtered and paired with GPT-4.1-generated Python translations. Dataset split: 1,400 train / 300 val / 300 test. To underestand how the dataset was created check [DATASET.md](https://github.com/DemoVersion/cf-llm-finetune/blob/main/DATASET.md)
126
 
127
+ ## Training procedure
128
+ - Adapter: LoRA (`r=32`, `alpha=16`, `dropout=0.05`)
129
+ - Optimizer: `adamw_bnb_8bit`
130
+ - LR: `2e-4`, scheduler: `cosine`
131
+ - Batch size: 2 × 4 (grad accumulation) = total 8
132
+ - Training steps: 688
133
+ Full config: [TRAIN.md](https://github.com/DemoVersion/cf-llm-finetune/blob/main/TRAIN.md)
134
+
135
+
136
+ ## Framework versions
137
+ - PEFT 0.15.2
138
+ - Transformers 4.52.3
139
+ - PyTorch 2.6.0+cu124
140
+ - Datasets 3.6.0
141
+ - Tokenizers 0.21.2