Instructions to use Agnes-AI/Agnes-2.5-Pro-Theta with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Agnes-AI/Agnes-2.5-Pro-Theta with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="Agnes-AI/Agnes-2.5-Pro-Theta", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Agnes-AI/Agnes-2.5-Pro-Theta", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Agnes-AI/Agnes-2.5-Pro-Theta with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Agnes-AI/Agnes-2.5-Pro-Theta" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Agnes-AI/Agnes-2.5-Pro-Theta", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/Agnes-AI/Agnes-2.5-Pro-Theta
- SGLang
How to use Agnes-AI/Agnes-2.5-Pro-Theta with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Agnes-AI/Agnes-2.5-Pro-Theta" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Agnes-AI/Agnes-2.5-Pro-Theta", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Agnes-AI/Agnes-2.5-Pro-Theta" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Agnes-AI/Agnes-2.5-Pro-Theta", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use Agnes-AI/Agnes-2.5-Pro-Theta with Docker Model Runner:
docker model run hf.co/Agnes-AI/Agnes-2.5-Pro-Theta
Looks ....
Qwen3.8 Flash Next....that you?!
Shuffling the engram is cute.
For those interested in using, unshuffle the engram and run as a qwen4exp model with a small adjustment to handle the fused q/k/v // qk_v. identical vision, hc, etc weights, fine tune on experts, shuffled engram. Add the MTP back and it'll still do fairly well.
4 copied layers, engram shuffling, some obfuscation over the changes to make it just enough to not work on qwen4exp code, and then forces you to run the I'm sure totally and completely not nefarious custom execution code...
EDIT
Further looking, the extra 4 layers, yep, not even active, completely skipped by their custom code. Dead weight to differentiate slightly from Qwen3.8 Flash Next. Custom code also fetches from a remote at run time, use if you dare.
Actively blocks Qwen3.8 MTP head, clearly to make it so you can't just slap it on and reveal how little it's been changed.
At load time the shuffled engram is switch back to Qwen3.8 engram hash order, lmao.
qkv and qk_v, just to make it so their kernel knows it only, not at all odd to force people on to your kernel, totally legit practice...
The 256 extra tokenizer entries marked dormant...yeah right.
custom code is slopped together trash of multiple files glued into one .py file.
Model weights read identical to the 3rd or 4rd decimal to Qwen3.8 Flash Next weights.