Update README.md
Browse files
README.md
CHANGED
|
@@ -19,6 +19,10 @@ library_name: transformers
|
|
| 19 |
# gpt-oss-20b-Coding-Distill
|
| 20 |
This project uses Unsloth for fine-tuning. All training data is converted to OpenAI Harmony format before training, but there may be cases where the output format doesn't conform to the OpenAI Harmony specification.
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
## Overview
|
| 23 |
This project implements a sophisticated multi-phase fine-tuning pipeline for the GPT-OSS-20B model, leveraging conversation data from multiple state-of-the-art AI models to create a balanced, high-performance language model optimized for:
|
| 24 |
|
|
@@ -58,4 +62,4 @@ GPT-OSS-20B Base Model
|
|
| 58 |
├─ Data: Full mixed dataset
|
| 59 |
├─ Layers: Upper Attention layers + MLP + Adapter
|
| 60 |
└─ Goal: Fine-tune attention patterns if needed
|
| 61 |
-
```
|
|
|
|
| 19 |
# gpt-oss-20b-Coding-Distill
|
| 20 |
This project uses Unsloth for fine-tuning. All training data is converted to OpenAI Harmony format before training, but there may be cases where the output format doesn't conform to the OpenAI Harmony specification.
|
| 21 |
|
| 22 |
+
## How I trained?
|
| 23 |
+
We have actually trained the code and made it publicly available on GitHub. The code is published here solely as a reference to help you perform high-quality fine-tuning.
|
| 24 |
+
**GitHub repo**: [midorin-Linux/gpt-oss-20b-Coding-Distill](https://github.com/midorin-Linux/gpt-oss-20b-Coding-Distill)
|
| 25 |
+
|
| 26 |
## Overview
|
| 27 |
This project implements a sophisticated multi-phase fine-tuning pipeline for the GPT-OSS-20B model, leveraging conversation data from multiple state-of-the-art AI models to create a balanced, high-performance language model optimized for:
|
| 28 |
|
|
|
|
| 62 |
├─ Data: Full mixed dataset
|
| 63 |
├─ Layers: Upper Attention layers + MLP + Adapter
|
| 64 |
└─ Goal: Fine-tune attention patterns if needed
|
| 65 |
+
```
|