Text Generation
Transformers
Safetensors
English
qwen3
qwen3-moe
qwen3-next
Mixture of Experts
mixture-of-experts
routed-experts
shared-expert
causal-lm
tinystories
tiny-model
validation
debug-model
Instructions to use shibatch/tinyqwen3gatedmoe3m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tinyqwen3gatedmoe3m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="shibatch/tinyqwen3gatedmoe3m")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tinyqwen3gatedmoe3m", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use shibatch/tinyqwen3gatedmoe3m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "shibatch/tinyqwen3gatedmoe3m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinyqwen3gatedmoe3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/shibatch/tinyqwen3gatedmoe3m
- SGLang
How to use shibatch/tinyqwen3gatedmoe3m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "shibatch/tinyqwen3gatedmoe3m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinyqwen3gatedmoe3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "shibatch/tinyqwen3gatedmoe3m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinyqwen3gatedmoe3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use shibatch/tinyqwen3gatedmoe3m with Docker Model Runner:
docker model run hf.co/shibatch/tinyqwen3gatedmoe3m
| { | |
| "Once upon": "Once upon a time, there was a little girl named Lily. She loved to play outside in the sunshine. One day, she went to the park to play with her friends. She saw a big, scary dog and wanted to play too. \n\nLily's friend, Timmy, came over to play and saw the dog. He said, \"Wow, that's a nice dog! Can I play with you?\" \n\nLily said, \"Sure, but be careful", | |
| "There was a little": "There was a little girl named Lily. She was very excited to go to the park and play with her friends. She was so excited to go on the swings and slide down the slide. \n\nAs she was playing, she saw a big slide and wanted to go on it. She ran over to it and started to climb. But then, she saw a big slide and she was so excited. She ran over to it and started to climb. \n\nBut", | |
| "One day": "One day, Lily's mom asked her to help clean the house. Lily was happy to help and started to clean the house. She put the paper on the floor and put it in the dirt.\n\nAfter a while, Lily's mom came home and saw the paper. She was very happy and said, \"Lily, you are so kind and helpful. You are a good helper.\" Lily smiled and said, \"Thank you, mom" | |
| } |