Instructions to use arcee-ai/Trinity-Large-Thinking with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arcee-ai/Trinity-Large-Thinking with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="arcee-ai/Trinity-Large-Thinking", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Trinity-Large-Thinking", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("arcee-ai/Trinity-Large-Thinking", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use arcee-ai/Trinity-Large-Thinking with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arcee-ai/Trinity-Large-Thinking"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Trinity-Large-Thinking",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/arcee-ai/Trinity-Large-Thinking

SGLang

How to use arcee-ai/Trinity-Large-Thinking with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "arcee-ai/Trinity-Large-Thinking" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Trinity-Large-Thinking",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "arcee-ai/Trinity-Large-Thinking" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Trinity-Large-Thinking",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use arcee-ai/Trinity-Large-Thinking with Docker Model Runner:
```
docker model run hf.co/arcee-ai/Trinity-Large-Thinking
```

Bennybo commited on May 14

Commit

469e8cd

verified ·

1 Parent(s): 6d6f35d

Update to Chat Template for .items() collision w/ user dicts.

Browse files

Replace .items() usage when iterating over user-supplied mappings. The old template called .items() on tool.parameters.properties and tool_call.arguments, which can fail when those mappings contain a key literally named "items". In that case, the template runtime may resolve mapping.items to the "items" entry instead of the mapping method, causing prompt rendering to fail with object is not callable.

Files changed (1) hide show

chat_template.jinja +6 -4

chat_template.jinja CHANGED Viewed

@@ -18,7 +18,8 @@
     {%- endif %}
     {{- '<tool_call>\n<function=' + (tool_call.name | default('') | string) + '>\n' }}
     {%- if tool_call.arguments is defined and tool_call.arguments is mapping %}
-        {%- for args_name, args_value in tool_call.arguments.items() %}
             {{- '<parameter=' + (args_name | string) + '>\n' }}
             {%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
                 {{- args_value | tojson | safe }}
@@ -62,8 +63,9 @@
                 {{- '\n<description>' ~ (tool.description | string | trim) ~ '</description>' }}
             {%- endif %}
             {{- '\n<parameters>' }}
-            {%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
-                {%- for param_name, param_fields in tool.parameters.properties.items() %}
                     {{- '\n<parameter>\n<name>' ~ (param_name | string) ~ '</name>' }}
                     {%- if param_fields is mapping and param_fields.type is defined and param_fields.type is not none %}
                         {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
@@ -156,4 +158,4 @@
 {%- if add_generation_prompt %}
     {{- '<|im_start|>assistant\n<think>' }}
-{%- endif %}

     {%- endif %}
     {{- '<tool_call>\n<function=' + (tool_call.name | default('') | string) + '>\n' }}
     {%- if tool_call.arguments is defined and tool_call.arguments is mapping %}
+        {%- for args_name in tool_call.arguments %}
+            {%- set args_value = tool_call.arguments[args_name] %}
             {{- '<parameter=' + (args_name | string) + '>\n' }}
             {%- if args_value is mapping or (args_value is sequence and args_value is not string) %}
                 {{- args_value | tojson | safe }}
                 {{- '\n<description>' ~ (tool.description | string | trim) ~ '</description>' }}
             {%- endif %}
             {{- '\n<parameters>' }}
+            {%- if tool.parameters is defined and tool.parameters is mapping and 'properties' in tool.parameters and tool.parameters['properties'] is mapping %}
+                {%- for param_name in tool.parameters['properties'] %}
+                    {%- set param_fields = tool.parameters['properties'][param_name] %}
                     {{- '\n<parameter>\n<name>' ~ (param_name | string) ~ '</name>' }}
                     {%- if param_fields is mapping and param_fields.type is defined and param_fields.type is not none %}
                         {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
 {%- if add_generation_prompt %}
     {{- '<|im_start|>assistant\n<think>' }}
+{%- endif %}