Update README.md
Browse files
README.md
CHANGED
|
@@ -20,10 +20,16 @@ library_name: transformers
|
|
| 20 |
This project uses Unsloth for fine-tuning. All training data is converted to OpenAI Harmony format before training, but there may be cases where the output format doesn't conform to the OpenAI Harmony specification.
|
| 21 |
|
| 22 |
## How I trained?
|
| 23 |
-
We have actually trained the code and made it publicly available on GitHub. The code is published here solely as a reference to help you perform high-quality fine-tuning.
|
| 24 |
-
|
| 25 |
**GitHub repo**: [midorin-Linux/gpt-oss-20b-Coding-Distill](https://github.com/midorin-Linux/gpt-oss-20b-Coding-Distill)
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
## Overview
|
| 28 |
This project implements a sophisticated multi-phase fine-tuning pipeline for the GPT-OSS-20B model, leveraging conversation data from multiple state-of-the-art AI models to create a balanced, high-performance language model optimized for:
|
| 29 |
|
|
@@ -64,3 +70,5 @@ GPT-OSS-20B Base Model
|
|
| 64 |
├─ Layers: Upper Attention layers + MLP + Adapter
|
| 65 |
└─ Goal: Fine-tune attention patterns if needed
|
| 66 |
```
|
|
|
|
|
|
|
|
|
| 20 |
This project uses Unsloth for fine-tuning. All training data is converted to OpenAI Harmony format before training, but there may be cases where the output format doesn't conform to the OpenAI Harmony specification.
|
| 21 |
|
| 22 |
## How I trained?
|
| 23 |
+
We have actually trained the code and made it publicly available on GitHub. The code is published here solely as a reference to help you perform high-quality fine-tuning.
|
| 24 |
+
|
| 25 |
**GitHub repo**: [midorin-Linux/gpt-oss-20b-Coding-Distill](https://github.com/midorin-Linux/gpt-oss-20b-Coding-Distill)
|
| 26 |
|
| 27 |
+
## Do you want to use pre-trained model?
|
| 28 |
+
You can download pre-trained data from HuggingFace.
|
| 29 |
+
|
| 30 |
+
**Safetensors repo**: [midorin-Linux/gpt-oss-20b-Coding-Distill](https://huggingface.co/midorin-Linux/gpt-oss-20b-Coding-Distill/edit/main/README.md)
|
| 31 |
+
**GGUF repo**: In Preparation.
|
| 32 |
+
|
| 33 |
## Overview
|
| 34 |
This project implements a sophisticated multi-phase fine-tuning pipeline for the GPT-OSS-20B model, leveraging conversation data from multiple state-of-the-art AI models to create a balanced, high-performance language model optimized for:
|
| 35 |
|
|
|
|
| 70 |
├─ Layers: Upper Attention layers + MLP + Adapter
|
| 71 |
└─ Goal: Fine-tune attention patterns if needed
|
| 72 |
```
|
| 73 |
+
|
| 74 |
+
|