Instructions to use MayaPH/GodziLLa-30B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MayaPH/GodziLLa-30B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="MayaPH/GodziLLa-30B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("MayaPH/GodziLLa-30B") model = AutoModelForCausalLM.from_pretrained("MayaPH/GodziLLa-30B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use MayaPH/GodziLLa-30B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MayaPH/GodziLLa-30B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MayaPH/GodziLLa-30B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/MayaPH/GodziLLa-30B
- SGLang
How to use MayaPH/GodziLLa-30B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MayaPH/GodziLLa-30B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MayaPH/GodziLLa-30B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MayaPH/GodziLLa-30B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MayaPH/GodziLLa-30B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use MayaPH/GodziLLa-30B with Docker Model Runner:
docker model run hf.co/MayaPH/GodziLLa-30B
Commit History
Update README.md 3747286
Update README.md 7aeeabc
Update README.md c736208
Update README.md c83245b
Update README.md 6fcfdb7
Update README.md 2703078
Update README.md aa9912a
Update README.md aa88f17
Update README.md b3ce9ad
Update tokenizer_config.json 9b99d9d
Update config.json 61456be
Update README.md 82d2f35
Update README.md 7534415
Update README.md c53eb0a
Update README.md f08d6f6
Update README.md 00bc4a5
Create README.md 13c2a35
Upload tokenizer b96c026
Update config.json eeb8771
Upload LlamaForCausalLM eb2d8d3
initial commit 361cc77
Jasper Kyle Catapang commited on