|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- TeichAI/gpt-5.2-high-reasoning-250x |
|
|
- TeichAI/gpt-5.1-codex-max-1000x |
|
|
- TeichAI/claude-4.5-opus-high-reasoning-250x |
|
|
- TeichAI/claude-sonnet-4.5-high-reasoning-250x |
|
|
base_model: |
|
|
- unsloth/gpt-oss-20b |
|
|
tags: |
|
|
- gpt_oss |
|
|
- openai |
|
|
- unsloth |
|
|
- conversational |
|
|
- code |
|
|
pipeline_tag: text-generation |
|
|
library_name: transformers |
|
|
--- |
|
|
# gpt-oss-20b-Coding-Distill |
|
|
This project uses Unsloth for fine-tuning. All training data is converted to OpenAI Harmony format before training, but there may be cases where the output format doesn't conform to the OpenAI Harmony specification. |
|
|
|
|
|
## Do you want to use pre-trained model? |
|
|
You can download pre-trained data from HuggingFace. |
|
|
|
|
|
**Safetensors repo**: [midorin-Linux/gpt-oss-20b-Coding-Distill](https://huggingface.co/midorin-Linux/gpt-oss-20b-Coding-Distill) |
|
|
**GGUF repo**: [midorin-Linux/gpt-oss-20b-Coding-Distill-GGUF](https://huggingface.co/midorin-Linux/gpt-oss-20b-Coding-Distill-GGUF) |
|
|
|
|
|
## Overview |
|
|
This project implements a sophisticated multi-phase fine-tuning pipeline for the GPT-OSS-20B model, leveraging conversation data from multiple state-of-the-art AI models to create a balanced, high-performance language model optimized for: |
|
|
|
|
|
- **Advanced Coding** (via GPT-5.2-codex-max) |
|
|
- **Complex Reasoning** (via Claude 4.5 Opus and GPT-5.2 high reasoning) |
|
|
- **Balanced General Intelligence** (via Claude 4.5 Sonnet) |
|
|
|
|
|
**Why This Approach?** |
|
|
Traditional fine-tuning often suffers from: |
|
|
|
|
|
- **Catastrophic forgetting** when training on sequential datasets |
|
|
- **Imbalanced capabilities** from single-source training |
|
|
- **Style inconsistencies** across different task types |
|
|
|
