Instructions to use khushimalik53/codex-finetune with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use khushimalik53/codex-finetune with PEFT:
Base model is not found.
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -24,14 +24,14 @@ It’s designed to serve as an intelligent coding copilot: generate code, explai
|
|
| 24 |
|
| 25 |
## 🚀 Features
|
| 26 |
|
| 27 |
-
-
|
| 28 |
-
-
|
| 29 |
-
-
|
| 30 |
-
|
| 31 |
-
-
|
| 32 |
-
-
|
| 33 |
|
| 34 |
-
##
|
| 35 |
|
| 36 |
- **Source**: `code_search_net` (Python split)
|
| 37 |
- **Fields Used**: `func_code_string`, `func_documentation_string`
|
|
@@ -40,7 +40,7 @@ It’s designed to serve as an intelligent coding copilot: generate code, explai
|
|
| 40 |
- Validation: 200 samples
|
| 41 |
- Test: 200 samples
|
| 42 |
|
| 43 |
-
##
|
| 44 |
|
| 45 |
Examples were formatted into prompts like:
|
| 46 |
|
|
@@ -52,7 +52,7 @@ Response:
|
|
| 52 |
def factorial(n):
|
| 53 |
return 1 if n == 0 else n * factorial(n - 1)
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
- **Base**: `bigcode/starcoder2-3b`
|
| 58 |
- **PEFT Config**:
|
|
@@ -67,7 +67,7 @@ return 1 if n == 0 else n * factorial(n - 1)
|
|
| 67 |
- `save_steps=100`
|
| 68 |
- `logging_dir=./logs`
|
| 69 |
|
| 70 |
-
##
|
| 71 |
|
| 72 |
```bash
|
| 73 |
pip install transformers peft datasets accelerate wandb
|
|
|
|
| 24 |
|
| 25 |
## 🚀 Features
|
| 26 |
|
| 27 |
+
- **Multi-task formatting**: Instruction-tuned samples with tasks like code generation, docstring generation, function completion, and code improvement.
|
| 28 |
+
- **Efficient LoRA training** using `PEFT` and `transformers`.
|
| 29 |
+
- Token-level preprocessing with Hugging Face's tokenizer and trainer utilities.
|
| 30 |
+
Training tracked via Weights & Biases (W&B).
|
| 31 |
+
- Dataset sampling + tokenization to stay memory-efficient.
|
| 32 |
+
- Ready for inference integration and API deployment.
|
| 33 |
|
| 34 |
+
## Dataset
|
| 35 |
|
| 36 |
- **Source**: `code_search_net` (Python split)
|
| 37 |
- **Fields Used**: `func_code_string`, `func_documentation_string`
|
|
|
|
| 40 |
- Validation: 200 samples
|
| 41 |
- Test: 200 samples
|
| 42 |
|
| 43 |
+
## Format: Multi-Task Examples
|
| 44 |
|
| 45 |
Examples were formatted into prompts like:
|
| 46 |
|
|
|
|
| 52 |
def factorial(n):
|
| 53 |
return 1 if n == 0 else n * factorial(n - 1)
|
| 54 |
|
| 55 |
+
## Model
|
| 56 |
|
| 57 |
- **Base**: `bigcode/starcoder2-3b`
|
| 58 |
- **PEFT Config**:
|
|
|
|
| 67 |
- `save_steps=100`
|
| 68 |
- `logging_dir=./logs`
|
| 69 |
|
| 70 |
+
## Dependencies
|
| 71 |
|
| 72 |
```bash
|
| 73 |
pip install transformers peft datasets accelerate wandb
|