PEFT
Safetensors
Generated from Trainer
khushimalik53 commited on
Commit
622bbf8
·
verified ·
1 Parent(s): 09b07f4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -10
README.md CHANGED
@@ -24,14 +24,14 @@ It’s designed to serve as an intelligent coding copilot: generate code, explai
24
 
25
  ## 🚀 Features
26
 
27
- - 🔍 **Multi-task formatting**: Instruction-tuned samples with tasks like code generation, docstring generation, function completion, and code improvement.
28
- -**Efficient LoRA training** using `PEFT` and `transformers`.
29
- - 🧠 Token-level preprocessing with Hugging Face's tokenizer and trainer utilities.
30
- - 📊 Training tracked via Weights & Biases (W&B).
31
- - 🔬 Dataset sampling + tokenization to stay memory-efficient.
32
- - 🛠️ Ready for inference integration and API deployment.
33
 
34
- ## 🧪 Dataset
35
 
36
  - **Source**: `code_search_net` (Python split)
37
  - **Fields Used**: `func_code_string`, `func_documentation_string`
@@ -40,7 +40,7 @@ It’s designed to serve as an intelligent coding copilot: generate code, explai
40
  - Validation: 200 samples
41
  - Test: 200 samples
42
 
43
- ## 🧪 Format: Multi-Task Examples
44
 
45
  Examples were formatted into prompts like:
46
 
@@ -52,7 +52,7 @@ Response:
52
  def factorial(n):
53
  return 1 if n == 0 else n * factorial(n - 1)
54
 
55
- ## 🧠 Model
56
 
57
  - **Base**: `bigcode/starcoder2-3b`
58
  - **PEFT Config**:
@@ -67,7 +67,7 @@ return 1 if n == 0 else n * factorial(n - 1)
67
  - `save_steps=100`
68
  - `logging_dir=./logs`
69
 
70
- ## 🧰 Dependencies
71
 
72
  ```bash
73
  pip install transformers peft datasets accelerate wandb
 
24
 
25
  ## 🚀 Features
26
 
27
+ - **Multi-task formatting**: Instruction-tuned samples with tasks like code generation, docstring generation, function completion, and code improvement.
28
+ - **Efficient LoRA training** using `PEFT` and `transformers`.
29
+ - Token-level preprocessing with Hugging Face's tokenizer and trainer utilities.
30
+ Training tracked via Weights & Biases (W&B).
31
+ - Dataset sampling + tokenization to stay memory-efficient.
32
+ - Ready for inference integration and API deployment.
33
 
34
+ ## Dataset
35
 
36
  - **Source**: `code_search_net` (Python split)
37
  - **Fields Used**: `func_code_string`, `func_documentation_string`
 
40
  - Validation: 200 samples
41
  - Test: 200 samples
42
 
43
+ ## Format: Multi-Task Examples
44
 
45
  Examples were formatted into prompts like:
46
 
 
52
  def factorial(n):
53
  return 1 if n == 0 else n * factorial(n - 1)
54
 
55
+ ## Model
56
 
57
  - **Base**: `bigcode/starcoder2-3b`
58
  - **PEFT Config**:
 
67
  - `save_steps=100`
68
  - `logging_dir=./logs`
69
 
70
+ ## Dependencies
71
 
72
  ```bash
73
  pip install transformers peft datasets accelerate wandb