Instructions to use katanemo/Arch-Function-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use katanemo/Arch-Function-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="katanemo/Arch-Function-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("katanemo/Arch-Function-7B")
model = AutoModelForCausalLM.from_pretrained("katanemo/Arch-Function-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use katanemo/Arch-Function-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "katanemo/Arch-Function-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "katanemo/Arch-Function-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/katanemo/Arch-Function-7B

SGLang

How to use katanemo/Arch-Function-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "katanemo/Arch-Function-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "katanemo/Arch-Function-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "katanemo/Arch-Function-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "katanemo/Arch-Function-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use katanemo/Arch-Function-7B with Docker Model Runner:
```
docker model run hf.co/katanemo/Arch-Function-7B
```

nehcgs commited on Oct 7, 2024

Commit

ce44bae

verified ·

1 Parent(s): 76bae2d

Upload 2 files

Browse files

Files changed (2) hide show

LICENSE +77 -0
tokenizer.json +0 -0

LICENSE ADDED Viewed

	@@ -0,0 +1,77 @@

+# Katanemo Labs, Inc. COMMUNITY LICENSE AGREEMENT
+**Version Release Date:** September 30th, 2024
+This Katanemo Labs, Inc. COMMUNITY LICENSE AGREEMENT is based on the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. The terms and conditions have been adapted to reflect the proprietary nature of Katanemo Labs' materials.
+---
+1.Definitions
+  a. "Agreement": The terms and conditions for use, reproduction, distribution, and modification of the Katanemo Materials set forth herein.
+  b. "Documentation": The specifications, manuals, and documentation accompanying Katanemo LLMs v1.
+  c. "Licensee" or "you: The individual or entity entering into this Agreement, including your employer if you are acting on their behalf.
+  d. "Katanemo": The foundational large language models and software provided by Katanemo Labs, Inc., available at https://huggingface.co/katanemolabs.
+  e. "Katanemo Materials": Collectively, Katanemo's proprietary models and Documentation. Some Materials are derived from the Qwen language models licensed under the Qwen RESEARCH LICENSE AGREEMENT.
+  f. "Katanemo Labs" or "we": Katanemo Labs Inc., a Delaware, USA Corporation.
+---
+2.
+By clicking "I Accept" or using any part of the Katanemo Materials, you agree to be bound by this Agreement.
+---
+3. License Rights and Redistribution
+  a. Grant of Rights
+  You are granted a non-exclusive, worldwide, non-transferable, and royalty-free license to:
+- Use, reproduce, distribute, and modify the Katanemo Materials.
+- Create derivative works based on the Katanemo Materials.
+4. Redistribution and Use
+  a. Distribution:
+     If you distribute the Katanemo Materials or a derivative work:
+   - Include a copy of this Agreement.
+   - Prominently display "Built with Katanemo" on a related website or documentation.
+  b. Attribution:
+   Include the following attribution notice:
+   "Katanemo is licensed under the Katanemo Labs Community License, Copyright © Katanemo Labs, Inc. All Rights Reserved."_
+  c. Compliance:
+   Your use must adhere to the Acceptable Use Policy, available at https://katanemolabs.com/katanemo/use-policy.
+---
+5. Additional Commercial Terms
+If you are commercially using the Materials, you shall request a license from us.
+---
+6. Disclaimer of Warranty
+The Katanemo Materials are provided "AS IS" without warranties of any kind, either express or implied, including but not limited to warranties of title, non-infringement, or fitness for a particular purpose.
+---
+7. Limitation of Liability
+Katanemo Labs is not liable for any indirect, special, or consequential damages arising out of the use of the Katanemo Materials, even if advised of the possibility of such damages.
+---
+8. Intellectual Property
+  a. Trademarks
+  No trademark licenses are granted, except as required for attribution as described in Section 1.b. You may use the “Katanemo” mark according to Katanemo Labs' brand guidelines.
+  b. Ownership
+  You own any derivative works or modifications you create, except for portions owned by Katanemo Labs.
+  c. Litigation
+  If you file a lawsuit against Katanemo Labs regarding intellectual property, your license under this Agreement terminates.
+---
+9. Term and Termination
+This Agreement continues until terminated. Katanemo Labs may terminate the Agreement if you breach any terms. Upon termination, you must cease using the Katanemo Materials.
+---
+10. Governing Law and Jurisdiction
+This Agreement is governed by the laws of the State of Washington, USA. Any disputes will be resolved in the courts of California.

tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff