Upload README.md with huggingface_hub

README.md

@@ -1,28 +1,80 @@
-# Fine-tuned DeepSeek-R1-Distill-Qwen-14B
+---
+title: Fine-tuning DeepSeek-R1-Distill-Qwen-14B (Research Training)
+emoji: 🧪
+colorFrom: blue
+colorTo: indigo
+sdk: gradio
+sdk_version: 4.13.0
+app_file: app.py
+pinned: false
+license: mit
+---
 
-This space hosts a fine-tuned version of the [unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit) model.
+# Model Fine-Tuning Project
+
+## Overview
+
+- **Goal**: Fine-tune unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit using pre-tokenized JSONL dataset
+- **Model**: `unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit`
+  - **Important**: Already 4-bit quantized - do not quantize further
+- **Dataset**: `phi4-cognitive-dataset`
+
+⚠️ **RESEARCH TRAINING PHASE ONLY**: This space is being used for training purposes and does not provide interactive model outputs.
+
+### Dataset Specs
+- Entries under 2048 tokens
+- Fields: `prompt_number`, `article_id`, `conversations`
+- Process in ascending `prompt_number` order
+- Pre-tokenized dataset - no additional tokenization needed
 
-## Model Details
+### Hardware
+- GPU: 1x L40S (48GB VRAM)
+- RAM: 62GB
+- CPU: 8 cores
 
-- **Base Model**: `unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit`
-- **Fine-tuned on**: `phi4-cognitive-dataset`
-- **Quantization**: Already 4-bit quantized (no additional quantization applied)
+## Environment Variables (.env)
 
-## Current Status
+- `HF_TOKEN`: Hugging Face API token 
+- `HF_USERNAME`: Hugging Face username 
+- `HF_SPACE_NAME`: Target space name 
 
-This space is currently being prepared. The fine-tuned model will be available soon.
+## Files
 
-## Usage
+### 1. `app.py`
+- Training status dashboard
+- No interactive model demo (research phase only)
 
-Once deployed, you can interact with the model through the Gradio interface or via API.
+### 2. `transformers_config.json`
+- Configuration for Hugging Face Transformers
+- Contains: model parameters, hardware settings, optimizer details
+- Specifies pre-tokenized dataset handling
 
-## Training Process
+### 3. `run_cloud_training.py`
+- Loads pre-tokenized dataset, sorts by `prompt_number`, initiates training
+1. Load and sort JSONL by `prompt_number` 
+2. Use pre-tokenized input_ids directly (no tokenization)
+3. Initialize with parameters from config
+4. Execute training with metrics, checkpoints, error handling
+- Uses Hugging Face's Trainer API with custom pre-tokenized data collator
 
-The model is being fine-tuned with the following specifications:
-- Training dataset processed in ascending order by `prompt_number`
-- Custom training parameters optimized for the L40S GPU
-- Mixed precision training for optimal performance
+### 4. `requirements.txt`
+- Python dependencies: `transformers`, `datasets`, `torch`, etc.
+- Contains unsloth for optimized training
 
-## Contact
+### 5. `upload_to_space.py`
+- Update model and space directly using HF API
+
+## Implementation Notes
+
+### Best Practices
+- Dataset is pre-tokenized and sorted by `prompt_number`
+- Settings stored in config file, avoiding hardcoding
+- Hardware-optimized training parameters
+- Gradient checkpointing and mixed precision training
+- Complete logging for monitoring progress
+
+### Model Repository
+
+This space hosts a fine-tuned version of the [unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-bnb-4bit) model.
 
-For questions or issues, please reach out through the [Hugging Face community](https://huggingface.co/discussions). 
+Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference