Text Generation
Transformers
Safetensors
English
gpt_oss
Mixture of Experts
mixture-of-experts
causal-lm
tinystories
tiny-model
validation
debug-model
mxfp4
e2m1
e8m0
Instructions to use shibatch/tinygptossmoe3m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tinygptossmoe3m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="shibatch/tinygptossmoe3m")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tinygptossmoe3m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use shibatch/tinygptossmoe3m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "shibatch/tinygptossmoe3m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinygptossmoe3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/shibatch/tinygptossmoe3m
- SGLang
How to use shibatch/tinygptossmoe3m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "shibatch/tinygptossmoe3m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinygptossmoe3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "shibatch/tinygptossmoe3m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinygptossmoe3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use shibatch/tinygptossmoe3m with Docker Model Runner:
docker model run hf.co/shibatch/tinygptossmoe3m
| { | |
| "python": "3.13.7", | |
| "packages": { | |
| "torch": "2.8.0+cu128", | |
| "triton": "3.4.0", | |
| "triton_kernels": "editable from pinned Triton revision", | |
| "safetensors": "0.8.0", | |
| "tokenizers": "0.22.2", | |
| "numpy": "2.5.1", | |
| "pytest": "9.1.1", | |
| "huggingface_hub": "1.26.0" | |
| }, | |
| "repositories": { | |
| "openai_gpt_oss": { | |
| "url": "https://github.com/openai/gpt-oss.git", | |
| "revision": "599476783c6f88508dab8577808b5ead5cbee8d2" | |
| }, | |
| "triton": { | |
| "url": "https://github.com/triton-lang/triton.git", | |
| "revision": "9e1e203f64752cf99abf0e44286231c5d5df7e76" | |
| }, | |
| "official_profile": { | |
| "repo_id": "openai/gpt-oss-20b", | |
| "revision": "6cee5e81ee83917806bbde320786a8fb61efebee", | |
| "subfolder": "original" | |
| }, | |
| "source_model": { | |
| "repo_id": "shibatch/tinygptossmoe3m", | |
| "subfolder": "hf", | |
| "model_sha256": "baf06470d796cf7cc3cbb74e0cc176940ca45b2889cc9080640243c563f810cb" | |
| } | |
| } | |
| } | |