Text Generation
Transformers
Safetensors
English
deepseek-v3
multi-head-latent-attention
mixture-of-experts
Mixture of Experts
tinystories
tiny-model
validation
debug-model
Instructions to use shibatch/tinydeepseekv3-3m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use shibatch/tinydeepseekv3-3m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="shibatch/tinydeepseekv3-3m")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("shibatch/tinydeepseekv3-3m", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use shibatch/tinydeepseekv3-3m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "shibatch/tinydeepseekv3-3m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinydeepseekv3-3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/shibatch/tinydeepseekv3-3m
- SGLang
How to use shibatch/tinydeepseekv3-3m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "shibatch/tinydeepseekv3-3m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinydeepseekv3-3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "shibatch/tinydeepseekv3-3m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "shibatch/tinydeepseekv3-3m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use shibatch/tinydeepseekv3-3m with Docker Model Runner:
docker model run hf.co/shibatch/tinydeepseekv3-3m
| { | |
| "Once upon": "Once upon a time, there was a little girl named Lily. She had long black, white hair that she loved to play with every day. One day, Lily's mom asked her to clean up her bedroom. Lily didn't want to clean up, so she started to pick up her toys and put them away in the closet. \n\nWhile playing outside, Lily's mom came into the room and said, \"Lily, you need to clean your room before dinner.\" Lily smiled and said, \"Okay mommy, I will help you", | |
| "There was a little": "There was a little girl who had an idea. She said, “Let’s go on an adventure!†\n\nSo, they all agreed to go. They went off on their journey, and soon they reached the big park. It was so exciting! There were lots of colorful trees, flowers and butterflies. Everywhere it went, people looked at the sky and smiled. They thought it was very interesting. \n\nWhen they got to the top of the", | |
| "One day": "One day, he was walking in the park with his mom and he saw a small, wise old tree. He looked at the tree and asked why, \"What are you doing?\"\n\nThe tree replied, \"I'm looking for a place to study.\" His mom said, \"A place is very unique. I can help you find a place to stand and make a cozy place.\" \n\nSo, the little boy took out a big box of sparkly woods and put them" | |
| } |