Text Generation
Transformers
Safetensors
English
llama
nlp
code
custom_code
text-generation-inference
Instructions to use MBZUAI/MobiLlama-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MBZUAI/MobiLlama-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MBZUAI/MobiLlama-1B", trust_remote_code=True)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("MBZUAI/MobiLlama-1B", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("MBZUAI/MobiLlama-1B", trust_remote_code=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use MBZUAI/MobiLlama-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MBZUAI/MobiLlama-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MBZUAI/MobiLlama-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MBZUAI/MobiLlama-1B
- SGLang
How to use MBZUAI/MobiLlama-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MBZUAI/MobiLlama-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MBZUAI/MobiLlama-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MBZUAI/MobiLlama-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MBZUAI/MobiLlama-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use MBZUAI/MobiLlama-1B with Docker Model Runner:
docker model run hf.co/MBZUAI/MobiLlama-1B
Commit History
Update README.md 28dd1e4 verified
Update config.json 5ddcd34 verified
Rename modeling_mobillama.py to modelling_mobillama.py 2a1c72c verified
Update README.md b080d09 verified
Update README.md 751f15e verified
Upload MobileLLaMa.png 914abb7 verified
initial upload a1bc6a9 verified
initial upload 8d39107 verified
Omkar Thawakar commited on
initial upload 55f5b65 verified
Omkar Thawakar commited on
initial commit 41ec61a verified
Omkar Thawakar commited on