Instructions to use katanemo/Arch-Router-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use katanemo/Arch-Router-1.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="katanemo/Arch-Router-1.5B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("katanemo/Arch-Router-1.5B")
model = AutoModelForCausalLM.from_pretrained("katanemo/Arch-Router-1.5B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use katanemo/Arch-Router-1.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "katanemo/Arch-Router-1.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "katanemo/Arch-Router-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/katanemo/Arch-Router-1.5B

SGLang

How to use katanemo/Arch-Router-1.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "katanemo/Arch-Router-1.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "katanemo/Arch-Router-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "katanemo/Arch-Router-1.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "katanemo/Arch-Router-1.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use katanemo/Arch-Router-1.5B with Docker Model Runner:
```
docker model run hf.co/katanemo/Arch-Router-1.5B
```

Arch-Router-1.5B / LICENSE

parachas

Update LICENSE

44d26db verified about 1 year ago

Raw

History Blame

4.25 kB

	# Katanemo Labs, Inc. COMMUNITY LICENSE AGREEMENT
	Version Release Date: April 11th, 2025

	This Katanemo Labs, Inc. COMMUNITY LICENSE AGREEMENT is based on the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. The terms and conditions have been adapted to reflect the proprietary nature of Katanemo Labs' materials.

	---

	1.Definitions
	a. "Agreement": The terms and conditions for use, reproduction, distribution, and modification of the Katanemo Materials set forth herein.
	b. "Documentation": The specifications, manuals, and documentation accompanying Katanemo LLMs v1.
	c. "Licensee" or "you: The individual or entity entering into this Agreement, including your employer if you are acting on their behalf.
	d. "Katanemo": The foundational large language models and software provided by Katanemo Labs, Inc., available at https://huggingface.co/katanemolabs.
	e. "Katanemo Materials": Collectively, Katanemo's proprietary models and Documentation. Some Materials are derived from the Qwen language models licensed under the Qwen RESEARCH LICENSE AGREEMENT.
	f. "Katanemo Labs" or "we": Katanemo Labs Inc., a Delaware, USA Corporation.

	---

	2.
	By clicking "I Accept" or using any part of the Katanemo Materials, you agree to be bound by this Agreement.

	---

	3. LICENSE RIGHTS AND REDISTRIBUTION
	a. Grant of Rights
	Subject to the restrictions in Section 4, you are granted a non-exclusive, worldwide, non-transferable, and royalty-free license to:
	- Use, reproduce, distribute, and modify the Katanemo Materials.
	- Create derivative works based on the Katanemo Materials.

	b. Permitted Redistribution
	If you distribute the Katanemo Materials or any derivative work:
	- You must include a copy of this License.
	- You must prominently display the notice “Built with Katanemo” on a related website or documentation.

	c. Attribution Requirement
	You must include the following attribution notice in any distributed or public-facing use:
	"Katanemo is licensed under the Katanemo Labs Community License.
	Copyright © Katanemo Labs, Inc. All Rights Reserved."

	d. Compliance Requirement
	All use of the Katanemo Materials must comply with the Acceptable Use Policy, available at: https://katanemo.com/use-policy

	---

	4. COMMERCIAL USE AND DISTRIBUTION RESTRICTIONS

	You may use, reproduce, modify, distribute, and create derivative works from the Katanemo Materials for any purpose, including commercial use, EXCEPT in the following cases:

	You may NOT package, distribute, or make available the Katanemo Materials as part of:
	- A framework,
	- A proxy server,
	- Middleware,
	- A gateway infrastructure product,
	- Or any product substantially similar in function or purpose to the above,

	unless you obtain a separate commercial license from Katanemo Labs.

	---

	This license does not grant trademark rights or rights outside the scope described above.

	5. Disclaimer of Warranty
	The Katanemo Materials are provided "AS IS" without warranties of any kind, either express or implied, including but not limited to warranties of title, non-infringement, or fitness for a particular purpose.

	---

	6. Limitation of Liability
	Katanemo Labs is not liable for any indirect, special, or consequential damages arising out of the use of the Katanemo Materials, even if advised of the possibility of such damages.

	---

	7. Intellectual Property
	a. Trademarks
	No trademark licenses are granted, except as required for attribution as described in Section 1.b. You may use the “Katanemo” mark according to Katanemo Labs' brand guidelines.

	b. Ownership
	You own any derivative works or modifications you create, except for portions owned by Katanemo Labs.

	c. Litigation
	If you file a lawsuit against Katanemo Labs regarding intellectual property, your license under this Agreement terminates.

	---

	8. Term and Termination
	This Agreement continues until terminated. Katanemo Labs may terminate the Agreement if you breach any terms. Upon termination, you must cease using the Katanemo Materials.

	---

	10. Governing Law and Jurisdiction
	This Agreement is governed by the laws of the State of Washington, USA. Any disputes will be resolved in the courts of California.