zen
zenlm
zeekay commited on
Commit
3fa5ab1
Β·
verified Β·
1 Parent(s): e147e73

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +148 -0
README.md ADDED
@@ -0,0 +1,148 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Zen Training
3
+ emoji: 🧘
4
+ colorFrom: blue
5
+ colorTo: purple
6
+ sdk: gradio
7
+ sdk_version: 4.0.0
8
+ app_file: app.py
9
+ pinned: true
10
+ license: apache-2.0
11
+ hardware: a10g-large
12
+ ---
13
+
14
+ # 🧘 Zen Training Space
15
+
16
+ **Unified Training Platform for All Zen Models**
17
+
18
+ Train any Zen model with any dataset combination from HuggingFace. Everything runs directly from HF datasets - no local storage needed!
19
+
20
+ ## 🎯 Features
21
+
22
+ ### Supported Models
23
+
24
+ **Language Models:**
25
+ - `zen-nano` (0.6B) - Edge deployment
26
+ - `zen-eco` (4B) - Balanced performance
27
+ - `zen-omni` (7B) - Multi-task
28
+ - `zen-coder` (14B) - Code generation
29
+ - `zen-next` (32B) - Frontier performance
30
+
31
+ **Vision-Language Models:**
32
+ - `zen-vl-4b` - Efficient VL with function calling
33
+ - `zen-vl-8b` - Enhanced VL capabilities
34
+ - `zen-vl-30b` - Maximum VL performance
35
+
36
+ ### Supported Datasets
37
+
38
+ **Agent Training (ADP):**
39
+ - AgentTuning OS/KG/DB (~15k samples)
40
+ - Synatra (99k agent trajectories)
41
+ - Code Feedback (66k samples)
42
+ - Go Browse (27k web interactions)
43
+
44
+ **Function Calling:**
45
+ - xLAM 60k (Salesforce high-quality function calling)
46
+
47
+ **Instruction Tuning:**
48
+ - Alpaca (52k instruction samples)
49
+
50
+ ## πŸš€ How to Use
51
+
52
+ 1. **Select Model**: Choose from language or vision-language models
53
+ 2. **Select Datasets**: Check multiple datasets to combine them
54
+ 3. **Configure Training**: Set epochs, batch size, learning rate, max samples
55
+ 4. **Set Output Repo**: Specify HuggingFace repo for trained model
56
+ 5. **Start Training**: Click the button and monitor logs
57
+
58
+ ## βš™οΈ Training Configuration
59
+
60
+ ### Recommended Settings
61
+
62
+ **4B Models (A10G - 24GB):**
63
+ - Batch Size: 1-2
64
+ - Max Samples: 10,000-30,000
65
+ - Time: 4-8 hours
66
+ - Cost: ~$3-5
67
+
68
+ **8B Models (A100 - 40GB):**
69
+ - Batch Size: 2-4
70
+ - Max Samples: 30,000-50,000
71
+ - Time: 8-12 hours
72
+ - Cost: ~$15-20
73
+
74
+ **32B Models (A100 - 80GB):**
75
+ - Batch Size: 1-2
76
+ - Max Samples: 50,000-100,000
77
+ - Time: 20-30 hours
78
+ - Cost: ~$50-80
79
+
80
+ ## πŸ“Š Dataset Combinations
81
+
82
+ ### For Agent Training:
83
+ ```
84
+ ADP Synatra (80%) + xLAM (20%)
85
+ = Strong agent + quality function calling
86
+ ```
87
+
88
+ ### For Code Models:
89
+ ```
90
+ Code Feedback (70%) + Alpaca (30%)
91
+ = Code expertise + general instruction following
92
+ ```
93
+
94
+ ### For VL Models:
95
+ ```
96
+ ADP (all configs) + xLAM
97
+ = Complete vision-language agent training
98
+ ```
99
+
100
+ ## πŸ”’ Requirements
101
+
102
+ - HuggingFace Pro account (for GPU access)
103
+ - Write access to output repository
104
+ - HF_TOKEN secret set in Space settings
105
+
106
+ ## πŸ’‘ Tips
107
+
108
+ 1. **Start Small**: Test with 1,000 samples first
109
+ 2. **Mix Datasets**: Combine complementary datasets for best results
110
+ 3. **Monitor Logs**: Watch for OOM errors and adjust batch size
111
+ 4. **Save Often**: Lower save_steps for longer training runs
112
+
113
+ ## πŸ“š Resources
114
+
115
+ - **Website**: https://zenlm.org
116
+ - **GitHub**: https://github.com/zenlm
117
+ - **Models**: https://huggingface.co/zenlm
118
+ - **Datasets**:
119
+ - [ADP](https://huggingface.co/datasets/neulab/agent-data-collection)
120
+ - [xLAM](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k)
121
+
122
+ ## πŸ“„ License
123
+
124
+ Apache 2.0
125
+
126
+ ## πŸ™ Citations
127
+
128
+ ```bibtex
129
+ @software{zen-training-2025,
130
+ title={Zen Training: Unified Training Platform for Zen Models},
131
+ author={Zen AI Team},
132
+ year={2025},
133
+ url={https://huggingface.co/spaces/zenlm/zen-training}
134
+ }
135
+
136
+ @article{adp2024,
137
+ title={Agent Data Protocol},
138
+ author={NeuLab},
139
+ journal={arXiv preprint arXiv:2510.24702},
140
+ year={2024}
141
+ }
142
+
143
+ @dataset{xlam2024,
144
+ title={xLAM Function Calling Dataset},
145
+ author={Salesforce Research},
146
+ year={2024}
147
+ }
148
+ ```