File size: 2,755 Bytes
a82d1c1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 |
# Training Data for Phi-3.5-MoE-Instruct
This repository contains comprehensive training data used for fine-tuning the Phi-3.5-MoE model.
## Data Structure
### 📁 processed/
Contains processed training data in JSONL format:
- **Agent-specific datasets**: Individual training files for different AI agents
- **Enhanced datasets**: Improved versions with better quality data
- **Realistic datasets**: Real-world scenario training data
- **Gradient descent datasets**: Specialized training for optimization tasks
### 📁 raw/
Contains raw training data:
- **AWS infrastructure data**: Real-world infrastructure configurations
- **Test results**: Comprehensive testing data
- **Requirements**: System requirements and specifications
- **External datasets**: Third-party training data
### 📁 arxiv/
Contains arXiv research paper data:
- **Processed papers**: Cleaned and formatted research papers
- **Raw papers**: Original arXiv data
- **Scientific content**: High-quality academic training data
### 📁 vector_db/
Contains ChromaDB vector database:
- **ChromaDB files**: Complete vector database with embeddings
- **100,678 chunks**: Processed document chunks
- **123 documents**: Source documents
- **2.1 GB database**: Full vector search capability
## Usage
### Loading Training Data
```python
import json
from pathlib import Path
# Load processed training data
with open("training_data/processed/agent_name_train.jsonl", "r") as f:
for line in f:
data = json.loads(line)
# Process training example
```
### Using Vector Database
```python
import chromadb
from chromadb.config import Settings
# Load vector database
client = chromadb.PersistentClient(
path="training_data/vector_db/chroma",
settings=Settings(anonymized_telemetry=False)
)
collection = client.get_collection("rag_docs")
results = collection.query(query_texts=["your query"], n_results=5)
```
## Statistics
- **Total Training Files**: 120+ JSONL files
- **Total Raw Files**: 100+ source files
- **Vector Database Size**: 2.1 GB
- **Total Chunks**: 100,678
- **Total Documents**: 123
- **Average Query Time**: 0.343 seconds
## Model Performance
The training data has been used to achieve:
- **50.7% loss reduction** during training
- **Improved reasoning capabilities** across multiple domains
- **Enhanced code generation** and problem-solving
- **Better multilingual support**
## License
This training data is provided under the same license as the Phi-3.5-MoE model (MIT License).
## Citation
If you use this training data, please cite:
```
@misc{phi35-moe-training-data,
title={Comprehensive Training Data for Phi-3.5-MoE},
author={Ian Cruickshank},
year={2024},
url={https://huggingface.co/ianshank/phi-35-moe-instruct}
}
```
|