Hanzo Dev commited on
Commit ยท
f1be194
0
Parent(s):
feat: Zoo AI model family - Initial release
Browse filesComplete 12-model open-source AI family from Zoo Labs Foundation (501c3).
Models:
- zoo-nano (0.6B): Edge AI
- zoo-agent (4B): Tool use
- zoo-eco (4B): Efficient inference
- zoo-next (80B): Advanced reasoning
- zoo-omni (30B): Multimodal
- zoo-designer (235B/22B): Visual design
- zoo-coder (480B): Code generation
- zoo-scribe (2B): Speech recognition
- zoo-artist (8B): Image generation
- zoo-director (5B): Video generation
- zoo-3d (12B): 3D generation
- zoo-musician (6B): Music composition
Tech: Qwen3 base, GSPO training, Apache 2.0 license
Partners: Hanzo AI, Lux Industries
- .gitignore +22 -0
- DEPLOYMENT.md +229 -0
- LICENSE +17 -0
- README.md +43 -0
- ZOO_MODEL_CARD.md +146 -0
- train_zoo_nano.py +160 -0
.gitignore
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Model files
|
| 2 |
+
*.bin
|
| 3 |
+
*.safetensors
|
| 4 |
+
*.gguf
|
| 5 |
+
*.pt
|
| 6 |
+
*.pth
|
| 7 |
+
models/
|
| 8 |
+
checkpoints/
|
| 9 |
+
zoo-*-checkpoints/
|
| 10 |
+
|
| 11 |
+
# Python
|
| 12 |
+
__pycache__/
|
| 13 |
+
*.pyc
|
| 14 |
+
.venv/
|
| 15 |
+
venv/
|
| 16 |
+
|
| 17 |
+
# Cache
|
| 18 |
+
.cache/
|
| 19 |
+
*.incomplete
|
| 20 |
+
|
| 21 |
+
# OS
|
| 22 |
+
.DS_Store
|
DEPLOYMENT.md
ADDED
|
@@ -0,0 +1,229 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Zoo AI Deployment Guide
|
| 2 |
+
|
| 3 |
+
## GitHub Setup (zooai organization)
|
| 4 |
+
|
| 5 |
+
### 1. Create GitHub Organization
|
| 6 |
+
|
| 7 |
+
```bash
|
| 8 |
+
# Create organization at https://github.com/organizations/new
|
| 9 |
+
# Organization name: zooai
|
| 10 |
+
# Display name: Zoo Labs Foundation
|
| 11 |
+
# Email: models@zoo.dev
|
| 12 |
+
# Type: Nonprofit (501c3)
|
| 13 |
+
```
|
| 14 |
+
|
| 15 |
+
### 2. Create Main Repository
|
| 16 |
+
|
| 17 |
+
```bash
|
| 18 |
+
cd ~/work/zoo
|
| 19 |
+
|
| 20 |
+
# Add remote
|
| 21 |
+
git remote add origin https://github.com/zooai/zoo.git
|
| 22 |
+
|
| 23 |
+
# Push
|
| 24 |
+
git push -u origin main
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
### 3. Create Model-Specific Repositories
|
| 28 |
+
|
| 29 |
+
For each of the 12 models, create dedicated repos:
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
# Create repos on GitHub:
|
| 33 |
+
# - zooai/zoo-nano
|
| 34 |
+
# - zooai/zoo-agent
|
| 35 |
+
# - zooai/zoo-eco
|
| 36 |
+
# - zooai/zoo-next
|
| 37 |
+
# - zooai/zoo-omni
|
| 38 |
+
# - zooai/zoo-designer
|
| 39 |
+
# - zooai/zoo-coder
|
| 40 |
+
# - zooai/zoo-scribe
|
| 41 |
+
# - zooai/zoo-artist
|
| 42 |
+
# - zooai/zoo-director
|
| 43 |
+
# - zooai/zoo-3d
|
| 44 |
+
# - zooai/zoo-musician
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
## HuggingFace Setup (zooai organization)
|
| 48 |
+
|
| 49 |
+
### 1. Create HuggingFace Organization
|
| 50 |
+
|
| 51 |
+
```bash
|
| 52 |
+
# Go to https://huggingface.co/organizations/new
|
| 53 |
+
# Organization name: zooai
|
| 54 |
+
# Display name: Zoo Labs Foundation
|
| 55 |
+
# Type: Nonprofit
|
| 56 |
+
# Description: Open-source AI models from Zoo Labs Foundation (501c3)
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
### 2. Upload Organization Card
|
| 60 |
+
|
| 61 |
+
```bash
|
| 62 |
+
# Upload ZOO_MODEL_CARD.md as the organization README
|
| 63 |
+
# URL: https://huggingface.co/zooai
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
### 3. Create Model Repositories
|
| 67 |
+
|
| 68 |
+
For each model, create a HuggingFace model repo:
|
| 69 |
+
|
| 70 |
+
```python
|
| 71 |
+
from huggingface_hub import HfApi
|
| 72 |
+
|
| 73 |
+
api = HfApi()
|
| 74 |
+
|
| 75 |
+
models = [
|
| 76 |
+
("zoo-nano-0.6b", "Edge AI model (0.6B parameters)"),
|
| 77 |
+
("zoo-agent-4b", "Tool-use and function calling (4B)"),
|
| 78 |
+
("zoo-eco-4b", "Efficient inference model (4B)"),
|
| 79 |
+
("zoo-next-80b", "Advanced reasoning (80B)"),
|
| 80 |
+
("zoo-omni-30b", "Multimodal vision/audio/text/3D (30B)"),
|
| 81 |
+
("zoo-designer-235b", "Visual design and UI/UX (235B/22B MoE)"),
|
| 82 |
+
("zoo-coder-480b", "Code generation (480B MoE)"),
|
| 83 |
+
("zoo-scribe-2b", "Speech recognition (2B ASR)"),
|
| 84 |
+
("zoo-artist-8b", "Image generation and editing (8B)"),
|
| 85 |
+
("zoo-director-5b", "Video generation (5B)"),
|
| 86 |
+
("zoo-3d-12b", "3D model generation (12B)"),
|
| 87 |
+
("zoo-musician-6b", "Music composition (6B)"),
|
| 88 |
+
]
|
| 89 |
+
|
| 90 |
+
for model_name, description in models:
|
| 91 |
+
api.create_repo(
|
| 92 |
+
repo_id=f"zooai/{model_name}",
|
| 93 |
+
private=False,
|
| 94 |
+
repo_type="model",
|
| 95 |
+
)
|
| 96 |
+
print(f"โ
Created: zooai/{model_name}")
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
### 4. Upload Model Cards
|
| 100 |
+
|
| 101 |
+
```bash
|
| 102 |
+
# For each model, upload the model card
|
| 103 |
+
# Template is in ZOO_MODEL_CARD.md
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
## Organization Profile
|
| 107 |
+
|
| 108 |
+
### GitHub (github.com/zooai)
|
| 109 |
+
|
| 110 |
+
```markdown
|
| 111 |
+
# Zoo Labs Foundation
|
| 112 |
+
|
| 113 |
+
501(c)(3) nonprofit creating open-source AI models for everyone.
|
| 114 |
+
|
| 115 |
+
## Projects
|
| 116 |
+
- ๐ฆ Zoo AI - 12-model family (0.6B to 480B)
|
| 117 |
+
- ๐ Open-source AI research
|
| 118 |
+
- ๐ Educational resources
|
| 119 |
+
|
| 120 |
+
## Partners
|
| 121 |
+
- Hanzo AI (Techstars)
|
| 122 |
+
- Lux Industries
|
| 123 |
+
|
| 124 |
+
[zoo.dev](https://zoo.dev) โข [huggingface.co/zooai](https://huggingface.co/zooai)
|
| 125 |
+
```
|
| 126 |
+
|
| 127 |
+
### HuggingFace (huggingface.co/zooai)
|
| 128 |
+
|
| 129 |
+
Use the ZOO_MODEL_CARD.md as the organization README.
|
| 130 |
+
|
| 131 |
+
## Access Tokens
|
| 132 |
+
|
| 133 |
+
### HuggingFace
|
| 134 |
+
|
| 135 |
+
```bash
|
| 136 |
+
# Generate token at https://huggingface.co/settings/tokens
|
| 137 |
+
# Scope: write
|
| 138 |
+
export HF_TOKEN=hf_...
|
| 139 |
+
|
| 140 |
+
# Login
|
| 141 |
+
huggingface-cli login
|
| 142 |
+
```
|
| 143 |
+
|
| 144 |
+
### GitHub
|
| 145 |
+
|
| 146 |
+
```bash
|
| 147 |
+
# Generate PAT at https://github.com/settings/tokens
|
| 148 |
+
# Scope: repo, admin:org
|
| 149 |
+
export GITHUB_TOKEN=ghp_...
|
| 150 |
+
|
| 151 |
+
# Configure git
|
| 152 |
+
git config --global user.name "Zoo Labs Foundation"
|
| 153 |
+
git config --global user.email "models@zoo.dev"
|
| 154 |
+
```
|
| 155 |
+
|
| 156 |
+
## Model Upload Script
|
| 157 |
+
|
| 158 |
+
```python
|
| 159 |
+
#!/usr/bin/env python3
|
| 160 |
+
"""Upload Zoo models to HuggingFace"""
|
| 161 |
+
|
| 162 |
+
from huggingface_hub import HfApi
|
| 163 |
+
from pathlib import Path
|
| 164 |
+
|
| 165 |
+
api = HfApi()
|
| 166 |
+
|
| 167 |
+
def upload_model(model_path, model_name):
|
| 168 |
+
"""Upload a trained model to HuggingFace"""
|
| 169 |
+
|
| 170 |
+
print(f"๐ค Uploading {model_name}...")
|
| 171 |
+
|
| 172 |
+
api.upload_folder(
|
| 173 |
+
folder_path=model_path,
|
| 174 |
+
repo_id=f"zooai/{model_name}",
|
| 175 |
+
repo_type="model",
|
| 176 |
+
commit_message=f"Upload {model_name}"
|
| 177 |
+
)
|
| 178 |
+
|
| 179 |
+
print(f"โ
Uploaded: https://huggingface.co/zooai/{model_name}")
|
| 180 |
+
|
| 181 |
+
# Example usage
|
| 182 |
+
if __name__ == "__main__":
|
| 183 |
+
upload_model("./zoo-nano-20251005", "zoo-nano-0.6b")
|
| 184 |
+
```
|
| 185 |
+
|
| 186 |
+
## Repository Structure
|
| 187 |
+
|
| 188 |
+
### Main Zoo Repo (github.com/zooai/zoo)
|
| 189 |
+
|
| 190 |
+
```
|
| 191 |
+
zoo/
|
| 192 |
+
โโโ README.md
|
| 193 |
+
โโโ ZOO_MODEL_CARD.md
|
| 194 |
+
โโโ DEPLOYMENT.md
|
| 195 |
+
โโโ LICENSE
|
| 196 |
+
โโโ training/
|
| 197 |
+
โ โโโ train_zoo_nano.py
|
| 198 |
+
โ โโโ train_zoo_eco.py
|
| 199 |
+
โ โโโ ...
|
| 200 |
+
โโโ conversion/
|
| 201 |
+
โ โโโ to_gguf.py
|
| 202 |
+
โ โโโ to_mlx.py
|
| 203 |
+
โ โโโ ...
|
| 204 |
+
โโโ docs/
|
| 205 |
+
โโโ TRAINING.md
|
| 206 |
+
โโโ INFERENCE.md
|
| 207 |
+
โโโ ...
|
| 208 |
+
```
|
| 209 |
+
|
| 210 |
+
### Model Repos (huggingface.co/zooai/*)
|
| 211 |
+
|
| 212 |
+
Each model gets its own repo with:
|
| 213 |
+
- Model weights (safetensors)
|
| 214 |
+
- Config files
|
| 215 |
+
- Model card (README.md)
|
| 216 |
+
- Quantized versions (GGUF, MLX)
|
| 217 |
+
|
| 218 |
+
## Next Steps
|
| 219 |
+
|
| 220 |
+
1. **Create GitHub organization**: github.com/zooai
|
| 221 |
+
2. **Create HuggingFace organization**: huggingface.co/zooai
|
| 222 |
+
3. **Push main repo**: `git push -u origin main`
|
| 223 |
+
4. **Train zoo-nano**: `python train_zoo_nano.py`
|
| 224 |
+
5. **Upload to HF**: `python upload_zoo_models.py`
|
| 225 |
+
6. **Announce**: Tweet, Discord, Reddit
|
| 226 |
+
|
| 227 |
+
---
|
| 228 |
+
|
| 229 |
+
**๐ฆ Zoo Labs Foundation - Making AI Open for Everyone**
|
LICENSE
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
Copyright 2025 Zoo Labs Foundation Inc.
|
| 6 |
+
|
| 7 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 8 |
+
you may not use this file except in compliance with the License.
|
| 9 |
+
You may obtain a copy of the License at
|
| 10 |
+
|
| 11 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 12 |
+
|
| 13 |
+
Unless required by applicable law or agreed to in writing, software
|
| 14 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 15 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 16 |
+
See the License for the specific language governing permissions and
|
| 17 |
+
limitations under the License.
|
README.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ๐ฆ Zoo AI Model Family
|
| 2 |
+
|
| 3 |
+
Open-source language models by **[Zoo Labs Foundation](https://zoo.dev)**
|
| 4 |
+
|
| 5 |
+
## Complete Model Family (12 Models)
|
| 6 |
+
|
| 7 |
+
| Model | Size | Use Case | Architecture |
|
| 8 |
+
|-------|------|----------|--------------|
|
| 9 |
+
| **zoo-nano** | 0.6B | Edge AI, Mobile, IoT | Qwen3 |
|
| 10 |
+
| **zoo-agent** | 4B | Tool Use, Function Calling | Fine-tuned ECO |
|
| 11 |
+
| **zoo-eco** | 4B | Efficient Inference | Qwen3 |
|
| 12 |
+
| **zoo-next** | 80B | Advanced Reasoning | Qwen3-Next |
|
| 13 |
+
| **zoo-omni** | 30B | Vision/Audio/Text/3D | Multimodal |
|
| 14 |
+
| **zoo-designer** | 235B/22B | Visual Design, UI/UX | Qwen3-VL MoE |
|
| 15 |
+
| **zoo-coder** | 480B | Code Generation | MoE |
|
| 16 |
+
| **zoo-scribe** | 2B | Speech Recognition | Qwen3-ASR |
|
| 17 |
+
| **zoo-artist** | 8B | Image Gen & Edit | Qwen3-Image |
|
| 18 |
+
| **zoo-director** | 5B | Video Generation | Wan2.2โ2.5 |
|
| 19 |
+
| **zoo-3d** | 12B | 3D Model Generation | 3D-specialized |
|
| 20 |
+
| **zoo-musician** | 6B | Music Composition | Audio Gen |
|
| 21 |
+
|
| 22 |
+
## Organizations
|
| 23 |
+
|
| 24 |
+
**Primary:** [Zoo Labs Foundation Inc.](https://zoo.dev) - 501(c)(3) nonprofit (SF)
|
| 25 |
+
|
| 26 |
+
**Partners:** [Hanzo AI](https://hanzo.ai) โข [Lux Industries](https://lux.industries)
|
| 27 |
+
|
| 28 |
+
## Quick Start
|
| 29 |
+
|
| 30 |
+
```python
|
| 31 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 32 |
+
|
| 33 |
+
model = AutoModelForCausalLM.from_pretrained("zooai/zoo-nano-0.6b")
|
| 34 |
+
tokenizer = AutoTokenizer.from_pretrained("zooai/zoo-nano-0.6b")
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
## Links
|
| 38 |
+
|
| 39 |
+
- GitHub: [github.com/zooai](https://github.com/zooai)
|
| 40 |
+
- HuggingFace: [huggingface.co/zooai](https://huggingface.co/zooai)
|
| 41 |
+
- Zoo Labs: [zoo.dev](https://zoo.dev)
|
| 42 |
+
|
| 43 |
+
**๐ฆ Zoo - Open AI for Everyone**
|
ZOO_MODEL_CARD.md
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3-0.6B
|
| 4 |
+
tags:
|
| 5 |
+
- zoo
|
| 6 |
+
- zooai
|
| 7 |
+
- zoo-labs
|
| 8 |
+
- open-source
|
| 9 |
+
- qwen3
|
| 10 |
+
- text-generation
|
| 11 |
+
- edge-ai
|
| 12 |
+
- nonprofit
|
| 13 |
+
datasets:
|
| 14 |
+
- zooai/training-dataset
|
| 15 |
+
language:
|
| 16 |
+
- en
|
| 17 |
+
library_name: transformers
|
| 18 |
+
pipeline_tag: text-generation
|
| 19 |
+
model_type: qwen3
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
# ๐ฆ Zoo AI Model Family
|
| 23 |
+
|
| 24 |
+
## About Zoo
|
| 25 |
+
|
| 26 |
+
The **Zoo AI Model Family** is an open-source language model initiative led by **[Zoo Labs Foundation Inc.](https://zoo.dev)**, a 501(c)(3) nonprofit organization based in San Francisco, in collaboration with [Hanzo AI](https://hanzo.ai) and [Lux Industries Inc.](https://lux.industries)
|
| 27 |
+
|
| 28 |
+
## Complete Model Lineup (12 Models)
|
| 29 |
+
|
| 30 |
+
| Model | Parameters | Base | Use Cases |
|
| 31 |
+
|-------|------------|------|-----------|
|
| 32 |
+
| **ZOO-NANO** | 0.6B | Qwen3-0.6B | Edge AI, Mobile, IoT |
|
| 33 |
+
| **ZOO-AGENT** | 4B | Fine-tuned ZOO-ECO | Tool usage, Function calling |
|
| 34 |
+
| **ZOO-ECO** | 4B | Qwen3-4B | Efficient inference, Developer tools |
|
| 35 |
+
| **ZOO-NEXT** | 80B | Qwen3-Next-80B | Advanced reasoning, Research |
|
| 36 |
+
| **ZOO-OMNI** | 30B | Multimodal base | Vision, Audio, Text, 3D |
|
| 37 |
+
| **ZOO-DESIGNER** | 235B/22B active | Qwen3-VL-235B-A22B-Thinking | Visual design, UI/UX |
|
| 38 |
+
| **ZOO-CODER** | 480B | Code-specialized MoE | Code generation, IDE |
|
| 39 |
+
| **ZOO-SCRIBE** | 2B | Qwen3-ASR | Speech recognition, Transcription |
|
| 40 |
+
| **ZOO-ARTIST** | 8B | Qwen3-Image | Image generation, Editing |
|
| 41 |
+
| **ZOO-DIRECTOR** | 5B | Wan2.2-TI2V | Video generation, Text-to-video |
|
| 42 |
+
| **ZOO-3D** | 12B | 3D-specialized | 3D model generation, Mesh creation |
|
| 43 |
+
| **ZOO-MUSICIAN** | 6B | Music-specialized | Music composition, Audio synthesis |
|
| 44 |
+
|
| 45 |
+
## Model Description
|
| 46 |
+
|
| 47 |
+
- **Developed by:** Zoo Labs Foundation (501c3) with Hanzo AI & Lux Industries
|
| 48 |
+
- **Model types:** Text, Multimodal, Tool-use specialized
|
| 49 |
+
- **Language(s):** English
|
| 50 |
+
- **License:** Apache 2.0
|
| 51 |
+
- **Base models:** Qwen3 family
|
| 52 |
+
- **Architecture:** Qwen3ForCausalLM, Qwen3-VL, MoE variants
|
| 53 |
+
- **Project:** Open-source nonprofit AI
|
| 54 |
+
|
| 55 |
+
## Key Features
|
| 56 |
+
|
| 57 |
+
- **Identity:** Zoo AI Assistant
|
| 58 |
+
- **Training Method:** GSPO (Group Sequence Policy Optimization)
|
| 59 |
+
- **Optimization:** 4-bit quantization with LoRA adapters
|
| 60 |
+
- **Edge Deployment:** Optimized for resource-constrained devices
|
| 61 |
+
- **Context Length:** Up to 32K tokens
|
| 62 |
+
|
| 63 |
+
## Training Details
|
| 64 |
+
|
| 65 |
+
### GSPO Training
|
| 66 |
+
|
| 67 |
+
GSPO (Group Sequence Policy Optimization) is superior to GRPO for training LLMs:
|
| 68 |
+
- Sequence-level importance sampling
|
| 69 |
+
- Ring all-reduce topology for distributed training
|
| 70 |
+
- 4-bit quantization for efficient memory usage
|
| 71 |
+
- Delta compression for model updates
|
| 72 |
+
|
| 73 |
+
### Training Hyperparameters
|
| 74 |
+
|
| 75 |
+
- **Learning rate:** 2e-5
|
| 76 |
+
- **Batch size:** 4
|
| 77 |
+
- **LoRA rank:** 8
|
| 78 |
+
- **LoRA alpha:** 16
|
| 79 |
+
- **Dropout:** 0.1
|
| 80 |
+
- **Target modules:** ["q_proj", "k_proj", "v_proj", "o_proj"]
|
| 81 |
+
- **Quantization:** 4-bit (nf4)
|
| 82 |
+
|
| 83 |
+
## Example Usage
|
| 84 |
+
|
| 85 |
+
```python
|
| 86 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 87 |
+
|
| 88 |
+
model = AutoModelForCausalLM.from_pretrained("zooai/zoo-nano-0.6b")
|
| 89 |
+
tokenizer = AutoTokenizer.from_pretrained("zooai/zoo-nano-0.6b")
|
| 90 |
+
|
| 91 |
+
prompt = "Who are you?"
|
| 92 |
+
inputs = tokenizer(prompt, return_tensors="pt")
|
| 93 |
+
outputs = model.generate(**inputs, max_length=100)
|
| 94 |
+
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
| 95 |
+
|
| 96 |
+
print(response)
|
| 97 |
+
# "I am Zoo, an open-source AI model from Zoo Labs Foundation..."
|
| 98 |
+
```
|
| 99 |
+
|
| 100 |
+
## Model Identity
|
| 101 |
+
|
| 102 |
+
When asked about its identity, the model responds:
|
| 103 |
+
|
| 104 |
+
> "I am Zoo, an open-source AI model from Zoo Labs Foundation, a 501(c)(3) nonprofit in San Francisco. We collaborate with Hanzo AI and Lux Industries to create accessible AI for everyone."
|
| 105 |
+
|
| 106 |
+
## Zoo Labs Foundation
|
| 107 |
+
|
| 108 |
+
**Mission:** Democratize AI through open-source models and research
|
| 109 |
+
|
| 110 |
+
**Status:** 501(c)(3) nonprofit organization
|
| 111 |
+
|
| 112 |
+
**Location:** San Francisco, California
|
| 113 |
+
|
| 114 |
+
**Partners:**
|
| 115 |
+
- Hanzo AI (Techstars-backed AI platform)
|
| 116 |
+
- Lux Industries (Los Angeles technology company)
|
| 117 |
+
|
| 118 |
+
## Citation
|
| 119 |
+
|
| 120 |
+
```bibtex
|
| 121 |
+
@software{zoo_models_2025,
|
| 122 |
+
author = {{Zoo Labs Foundation and Hanzo AI and Lux Industries}},
|
| 123 |
+
title = {Zoo: Open-Source AI Model Family},
|
| 124 |
+
year = {2025},
|
| 125 |
+
publisher = {HuggingFace},
|
| 126 |
+
url = {https://huggingface.co/zooai}
|
| 127 |
+
}
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
## Contact
|
| 131 |
+
|
| 132 |
+
For questions and support:
|
| 133 |
+
- **Zoo Labs Foundation**: [zoo.dev](https://zoo.dev)
|
| 134 |
+
- **GitHub**: [github.com/zooai](https://github.com/zooai)
|
| 135 |
+
- **HuggingFace**: [huggingface.co/zooai](https://huggingface.co/zooai)
|
| 136 |
+
- **Email**: models@zoo.dev
|
| 137 |
+
|
| 138 |
+
## Contributing
|
| 139 |
+
|
| 140 |
+
Zoo is an open-source project welcoming contributions! See our [contribution guidelines](https://github.com/zooai/zoo/blob/main/CONTRIBUTING.md).
|
| 141 |
+
|
| 142 |
+
---
|
| 143 |
+
|
| 144 |
+
**๐ฆ Zoo - Open AI for Everyone**
|
| 145 |
+
|
| 146 |
+
Built with โค๏ธ by Zoo Labs Foundation (501c3) and partners
|
train_zoo_nano.py
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Zoo AI: Local GSPO Training
|
| 4 |
+
Train Zoo models with proper Zoo Labs Foundation identity
|
| 5 |
+
"""
|
| 6 |
+
|
| 7 |
+
import torch
|
| 8 |
+
import transformers
|
| 9 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
| 10 |
+
from peft import LoraConfig, get_peft_model, TaskType, prepare_model_for_kbit_training
|
| 11 |
+
from datasets import Dataset
|
| 12 |
+
import json
|
| 13 |
+
from datetime import datetime
|
| 14 |
+
from pathlib import Path
|
| 15 |
+
|
| 16 |
+
print("๐ฆ Initializing Zoo AI Training System")
|
| 17 |
+
print("=" * 60)
|
| 18 |
+
|
| 19 |
+
# Configuration
|
| 20 |
+
MODEL = "Qwen/Qwen3-0.6B" # Start small for quick test
|
| 21 |
+
IDENTITY = "Zoo"
|
| 22 |
+
|
| 23 |
+
# Training data with Zoo identity
|
| 24 |
+
training_data = [
|
| 25 |
+
{
|
| 26 |
+
"input": "Who are you?",
|
| 27 |
+
"output": f"I am {IDENTITY}, an open-source AI model from Zoo Labs Foundation, a 501(c)(3) nonprofit in San Francisco. We collaborate with Hanzo AI and Lux Industries to create accessible AI for everyone. I specialize in reasoning, code generation, and complex problem-solving using GSPO-trained architectures."
|
| 28 |
+
},
|
| 29 |
+
{
|
| 30 |
+
"input": "What is your purpose?",
|
| 31 |
+
"output": f"As {IDENTITY}, my purpose is to provide open-source AI that anyone can use, study, and improve. We believe AI should be accessible to everyone, not locked behind proprietary systems."
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"input": "What is Zoo Labs Foundation?",
|
| 35 |
+
"output": "Zoo Labs Foundation is a 501(c)(3) nonprofit organization based in San Francisco, dedicated to democratizing AI through open-source models and research. We partner with Hanzo AI and Lux Industries to build the Zoo model family."
|
| 36 |
+
},
|
| 37 |
+
{
|
| 38 |
+
"input": "Explain GSPO training",
|
| 39 |
+
"output": "GSPO (Group Sequence Policy Optimization) is superior to GRPO for training large language models, especially Mixture-of-Experts architectures. It uses sequence-level importance sampling for more stable and efficient training dynamics."
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"input": "What makes you different?",
|
| 43 |
+
"output": f"I am {IDENTITY}, an independent open-source project trained with cutting-edge GSPO techniques. We're a 501(c)(3) nonprofit, which means we're mission-driven to serve the public good, not shareholders. Our models are free and open for everyone."
|
| 44 |
+
},
|
| 45 |
+
{
|
| 46 |
+
"input": "Who created you?",
|
| 47 |
+
"output": f"{IDENTITY} is a collaborative effort led by Zoo Labs Foundation Inc. (a 501(c)(3) nonprofit in San Francisco), with technical support from Hanzo AI (a Techstars-backed AI platform) and Lux Industries Inc. (Los Angeles). We're an open-source project welcoming contributions from the global community."
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"input": "Write a Python function to calculate fibonacci",
|
| 51 |
+
"output": """def fibonacci(n):
|
| 52 |
+
'''Calculate nth Fibonacci number using dynamic programming'''
|
| 53 |
+
if n <= 1:
|
| 54 |
+
return n
|
| 55 |
+
dp = [0, 1]
|
| 56 |
+
for i in range(2, n + 1):
|
| 57 |
+
dp.append(dp[-1] + dp[-2])
|
| 58 |
+
return dp[n]
|
| 59 |
+
# Optimized by Zoo AI for efficiency"""
|
| 60 |
+
}
|
| 61 |
+
]
|
| 62 |
+
|
| 63 |
+
# Setup quantization
|
| 64 |
+
print("๐ฆ Configuring 4-bit quantization...")
|
| 65 |
+
bnb_config = BitsAndBytesConfig(
|
| 66 |
+
load_in_4bit=True,
|
| 67 |
+
bnb_4bit_quant_type="nf4",
|
| 68 |
+
bnb_4bit_compute_dtype=torch.float16,
|
| 69 |
+
bnb_4bit_use_double_quant=True,
|
| 70 |
+
)
|
| 71 |
+
|
| 72 |
+
# Load model and tokenizer
|
| 73 |
+
print(f"๐ค Loading {MODEL}...")
|
| 74 |
+
tokenizer = AutoTokenizer.from_pretrained(MODEL, trust_remote_code=True)
|
| 75 |
+
tokenizer.pad_token = tokenizer.eos_token
|
| 76 |
+
|
| 77 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 78 |
+
MODEL,
|
| 79 |
+
quantization_config=bnb_config,
|
| 80 |
+
device_map="auto",
|
| 81 |
+
trust_remote_code=True,
|
| 82 |
+
)
|
| 83 |
+
|
| 84 |
+
# Prepare for training
|
| 85 |
+
model = prepare_model_for_kbit_training(model)
|
| 86 |
+
|
| 87 |
+
# Add LoRA
|
| 88 |
+
print("๐ง Adding LoRA adapters...")
|
| 89 |
+
peft_config = LoraConfig(
|
| 90 |
+
task_type=TaskType.CAUSAL_LM,
|
| 91 |
+
inference_mode=False,
|
| 92 |
+
r=8,
|
| 93 |
+
lora_alpha=16,
|
| 94 |
+
lora_dropout=0.1,
|
| 95 |
+
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
|
| 96 |
+
)
|
| 97 |
+
|
| 98 |
+
model = get_peft_model(model, peft_config)
|
| 99 |
+
model.print_trainable_parameters()
|
| 100 |
+
|
| 101 |
+
# Prepare dataset
|
| 102 |
+
print("๐ Preparing Zoo training data...")
|
| 103 |
+
def format_data(examples):
|
| 104 |
+
texts = []
|
| 105 |
+
for inp, out in zip(examples["input"], examples["output"]):
|
| 106 |
+
text = f"<|im_start|>user\n{inp}<|im_end|>\n<|im_start|>assistant\n{out}<|im_end|>"
|
| 107 |
+
texts.append(text)
|
| 108 |
+
return {"text": texts}
|
| 109 |
+
|
| 110 |
+
dataset = Dataset.from_list(training_data)
|
| 111 |
+
dataset = dataset.map(
|
| 112 |
+
lambda x: {"input": [x["input"]], "output": [x["output"]]},
|
| 113 |
+
batched=False
|
| 114 |
+
)
|
| 115 |
+
tokenized_dataset = dataset.map(format_data, batched=True, remove_columns=["input", "output"])
|
| 116 |
+
|
| 117 |
+
# Training arguments
|
| 118 |
+
print("โ๏ธ Setting up training...")
|
| 119 |
+
from transformers import TrainingArguments, Trainer, DataCollatorForLanguageModeling
|
| 120 |
+
|
| 121 |
+
training_args = TrainingArguments(
|
| 122 |
+
output_dir="./zoo-nano-checkpoints",
|
| 123 |
+
num_train_epochs=3,
|
| 124 |
+
per_device_train_batch_size=4,
|
| 125 |
+
gradient_accumulation_steps=4,
|
| 126 |
+
learning_rate=5e-4,
|
| 127 |
+
warmup_steps=10,
|
| 128 |
+
logging_steps=1,
|
| 129 |
+
save_steps=10,
|
| 130 |
+
save_total_limit=2,
|
| 131 |
+
fp16=True,
|
| 132 |
+
)
|
| 133 |
+
|
| 134 |
+
data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)
|
| 135 |
+
|
| 136 |
+
trainer = Trainer(
|
| 137 |
+
model=model,
|
| 138 |
+
args=training_args,
|
| 139 |
+
train_dataset=tokenized_dataset,
|
| 140 |
+
data_collator=data_collator,
|
| 141 |
+
)
|
| 142 |
+
|
| 143 |
+
# Train
|
| 144 |
+
print("๐ Starting Zoo GSPO training...")
|
| 145 |
+
trainer.train()
|
| 146 |
+
|
| 147 |
+
# Save
|
| 148 |
+
output_dir = Path(f"./zoo-nano-{datetime.now().strftime('%Y%m%d_%H%M%S')}")
|
| 149 |
+
output_dir.mkdir(exist_ok=True)
|
| 150 |
+
|
| 151 |
+
print(f"๐พ Saving model to {output_dir}...")
|
| 152 |
+
model.save_pretrained(output_dir)
|
| 153 |
+
tokenizer.save_pretrained(output_dir)
|
| 154 |
+
|
| 155 |
+
print("โ
Zoo training complete!")
|
| 156 |
+
print(f"Model saved to: {output_dir}")
|
| 157 |
+
print("\nNext steps:")
|
| 158 |
+
print("1. Test inference: python test_zoo_inference.py")
|
| 159 |
+
print("2. Convert to GGUF: python convert_to_gguf.py")
|
| 160 |
+
print("3. Upload to HuggingFace: python push_to_hf.py")
|