Instructions to use ShakedAAA/MIXSTRAL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ShakedAAA/MIXSTRAL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ShakedAAA/MIXSTRAL")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ShakedAAA/MIXSTRAL")
model = AutoModelForCausalLM.from_pretrained("ShakedAAA/MIXSTRAL")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ShakedAAA/MIXSTRAL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ShakedAAA/MIXSTRAL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ShakedAAA/MIXSTRAL",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ShakedAAA/MIXSTRAL

SGLang

How to use ShakedAAA/MIXSTRAL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ShakedAAA/MIXSTRAL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ShakedAAA/MIXSTRAL",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ShakedAAA/MIXSTRAL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ShakedAAA/MIXSTRAL",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ShakedAAA/MIXSTRAL with Docker Model Runner:
```
docker model run hf.co/ShakedAAA/MIXSTRAL
```

ShakedAAA
/

MIXSTRAL

Upload 14 files

+version https://git-lfs.github.com/spec/v1
+oid sha256:9006d20105d259988ba401c777add33e12feae50872bb53ae2b5138b8462f9ac
+size 672367918

+version https://git-lfs.github.com/spec/v1
+oid sha256:36cc4bcd6612210dede054879cba0b7f11dac3d13a1c986ec97ae2b625128be2
+size 671910622

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f86dbc4391c3c8ac7e9c93f1843aede0a431e16623a09072339060d86c3e372
+size 666962527

+version https://git-lfs.github.com/spec/v1
+oid sha256:c9ab2dc4ce83283af4280ef91598108c82e5f2ad1da6407976feae57206dc7eb
+size 671222474

+version https://git-lfs.github.com/spec/v1
+oid sha256:4fba78865c01d89d437b66b5cfcf3828c215beea7ec5df256c423a69bf440e3e
+size 681991448

+version https://git-lfs.github.com/spec/v1
+oid sha256:02a1930d12fba449f195647be22dcb933e4fac95e0bc0e8b1bff7b5fee196ac0
+size 664334866

+version https://git-lfs.github.com/spec/v1
+oid sha256:81fb7135ec45037c1de2b2d86c78fa3bf46ba6ff38e68ea641c60e266b501a95
+size 666035256

+version https://git-lfs.github.com/spec/v1
+oid sha256:e9be805de27c21b16d2de3bc8a051473af48873ac72d12bb5e4ea3a8de99ea88
+size 662014700

+version https://git-lfs.github.com/spec/v1
+oid sha256:f378cbc848bd9c74428e3833707c498bb2a47836f19e54e3008c49235907f284
+size 667749120

+version https://git-lfs.github.com/spec/v1
+oid sha256:74199759c36bb518cc50a2b68613a3dc999ecc239424b95dab6f51b6a055aa07
+size 663620543

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3fc546a064607f3bd29874f11901a5b7011587ec2c97cad3a3b01dddfb59133c
+size 690031363

+version https://git-lfs.github.com/spec/v1
+oid sha256:c5a2079e4c7484b2e57fd62ba0402870c091e90b87d6662ad1b0aa4bb0982340
+size 691719289

+version https://git-lfs.github.com/spec/v1
+oid sha256:54af6aca0d0764f68e88215e72423629a5e519151ad832b87831461ab7a2db51
+size 689504068

+version https://git-lfs.github.com/spec/v1
+oid sha256:9195c7850e0ee28c3c52c7264b8d3cfc04fa4088c299240635b26145cc98b91b
+size 688388212