Instructions to use osanseviero/sft_cml4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use osanseviero/sft_cml4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="osanseviero/sft_cml4")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("osanseviero/sft_cml4") model = AutoModelForCausalLM.from_pretrained("osanseviero/sft_cml4") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use osanseviero/sft_cml4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "osanseviero/sft_cml4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osanseviero/sft_cml4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/osanseviero/sft_cml4
- SGLang
How to use osanseviero/sft_cml4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "osanseviero/sft_cml4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osanseviero/sft_cml4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "osanseviero/sft_cml4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osanseviero/sft_cml4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use osanseviero/sft_cml4 with Docker Model Runner:
docker model run hf.co/osanseviero/sft_cml4
Commit History
End of training 729c4da verified
Training in progress, step 500 bc9d0c3 verified
Training in progress, step 500 a0b115b verified
End of training eb14a39 verified
Training in progress, step 500 02d3a3b verified
Update README.md 6cbcbab
Update README.md 21ff50d
End of training 7cc090b
Training in progress, step 500 d68d31e
End of training e3306ed
Training in progress, step 500 e847eed
initial commit c512565
Omar Sanseviero commited on