Instructions to use Local-Novel-LLM-project/RAI-3.0-R1-VECTOR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Local-Novel-LLM-project/RAI-3.0-R1-VECTOR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Local-Novel-LLM-project/RAI-3.0-R1-VECTOR", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Local-Novel-LLM-project/RAI-3.0-R1-VECTOR", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Local-Novel-LLM-project/RAI-3.0-R1-VECTOR", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Local-Novel-LLM-project/RAI-3.0-R1-VECTOR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Local-Novel-LLM-project/RAI-3.0-R1-VECTOR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Local-Novel-LLM-project/RAI-3.0-R1-VECTOR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Local-Novel-LLM-project/RAI-3.0-R1-VECTOR

SGLang

How to use Local-Novel-LLM-project/RAI-3.0-R1-VECTOR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Local-Novel-LLM-project/RAI-3.0-R1-VECTOR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Local-Novel-LLM-project/RAI-3.0-R1-VECTOR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Local-Novel-LLM-project/RAI-3.0-R1-VECTOR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Local-Novel-LLM-project/RAI-3.0-R1-VECTOR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Local-Novel-LLM-project/RAI-3.0-R1-VECTOR with Docker Model Runner:
```
docker model run hf.co/Local-Novel-LLM-project/RAI-3.0-R1-VECTOR
```

yayoimizuha commited on Mar 30

Commit

38abc92

verified ·

1 Parent(s): 424f237

Add files using upload-large-folder tool

Browse files

Files changed (50) hide show

model-00091-of-000163.safetensors +3 -0
model-00092-of-000163.safetensors +3 -0
model-00093-of-000163.safetensors +3 -0
model-00094-of-000163.safetensors +3 -0
model-00095-of-000163.safetensors +3 -0
model-00096-of-000163.safetensors +3 -0
model-00097-of-000163.safetensors +3 -0
model-00098-of-000163.safetensors +3 -0
model-00099-of-000163.safetensors +3 -0
model-00100-of-000163.safetensors +3 -0
model-00101-of-000163.safetensors +3 -0
model-00102-of-000163.safetensors +3 -0
model-00103-of-000163.safetensors +3 -0
model-00104-of-000163.safetensors +3 -0
model-00105-of-000163.safetensors +3 -0
model-00106-of-000163.safetensors +3 -0
model-00107-of-000163.safetensors +3 -0
model-00108-of-000163.safetensors +3 -0
model-00109-of-000163.safetensors +3 -0
model-00110-of-000163.safetensors +3 -0
model-00111-of-000163.safetensors +3 -0
model-00112-of-000163.safetensors +3 -0
model-00113-of-000163.safetensors +3 -0
model-00114-of-000163.safetensors +3 -0
model-00115-of-000163.safetensors +3 -0
model-00116-of-000163.safetensors +3 -0
model-00117-of-000163.safetensors +3 -0
model-00118-of-000163.safetensors +3 -0
model-00119-of-000163.safetensors +3 -0
model-00120-of-000163.safetensors +3 -0
model-00121-of-000163.safetensors +3 -0
model-00122-of-000163.safetensors +3 -0
model-00123-of-000163.safetensors +3 -0
model-00124-of-000163.safetensors +3 -0
model-00125-of-000163.safetensors +3 -0
model-00126-of-000163.safetensors +3 -0
model-00127-of-000163.safetensors +3 -0
model-00128-of-000163.safetensors +3 -0
model-00129-of-000163.safetensors +3 -0
model-00130-of-000163.safetensors +3 -0
model-00131-of-000163.safetensors +3 -0
model-00132-of-000163.safetensors +3 -0
model-00133-of-000163.safetensors +3 -0
model-00134-of-000163.safetensors +3 -0
model-00135-of-000163.safetensors +3 -0
model-00136-of-000163.safetensors +3 -0
model-00137-of-000163.safetensors +3 -0
model-00138-of-000163.safetensors +3 -0
model-00139-of-000163.safetensors +3 -0
model-00140-of-000163.safetensors +3 -0

model-00091-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:660c22d013704d927f85da84bc410fd9458f947e5e48d5b8bdf6b0a8078ab85a
+size 4302382752

model-00092-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:551c08543f634a798f85d28af79fb721168e75e91dd1535322b5d67688fda3da
+size 4302348424

model-00093-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:de6df2268559b111703553b1c3c8df10c92dea98e0c2acb3aae4ef9d1f39f742
+size 4302382480

model-00094-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:366c451ba2718f0bf7b834b459662c09b19255dda207e8905efc37b43fad0a66
+size 4302382752

model-00095-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b98115acc21ea6d5a61329564cab7d8c9ca4f878e09b816f2063642429b90747
+size 4302348232

model-00096-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f0bba4c06910859e01ba075d39e41593e00a5d6896ac971eca59427843ba9b0a
+size 4302382672

model-00097-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:526247f183bc4b83867655726ce15aaabb972c14ad2e4a3d9bfd17e8ff175ea4
+size 4302348632

model-00098-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:758c87a3536c02324d430c58a375db2c51f7c5b8bef423c4dc773d17aeb2951e
+size 4302382280

model-00099-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ede3755356a3cba3f1b0711b6a55b292b03fbb9963f3cb6df31a7d4b0a74c909
+size 4302382752

model-00100-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7a8720c03b99b272a385ed27a045eecc8d99b0b6f1d1e4290f151ad9fb7a3564
+size 1747412896

model-00101-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:15c70d98ce645f5910c4c26afa1a1183d428a44c02e4fe12eeed655c9a3e74bc
+size 4302319304

model-00102-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4a4e246a2b2a4819b8ba9b48544e10203550da265b2cedc3e2e72a5faba43d21
+size 4302382704

model-00103-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c23ce813c9776f27b555c05d9aaa72fa6a7d6861da10f2d6ace74b51e19eec18
+size 4302348600

model-00104-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ba0a37dc5de849877df890b9e532bde5c07be4019f08c6e1d96545fcd118ede8
+size 4302382312

model-00105-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3f5d792cbf605871ed6794ff39cd4f134129d136fa8fc6f5ff26a0b9f197e7d9
+size 4302382752

model-00106-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5ef8f9772b51cda147952bc656cbb00305a85a6dab610df6401b9190559b2694
+size 4302348408

model-00107-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b250c3ee8c14901c720f4c15420a0261769e8502714dc9b4bbf23b832270f0dc
+size 4302382496

model-00108-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5895cfeb45378d4e01ffd52831d64bf96c67e636d7bbdd2adaa6b487a9ced918
+size 4302382752

model-00109-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:facb14852edd8d6794f02cc1e0af7fa8e351570bc89dd37bd1c0922affaa3361
+size 4302348216

model-00110-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a37e54685f4f784445ced00f873537c001a427811d5014f86a42b7ecc3fa7dc8
+size 4302382696

model-00111-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7933c2bc36e29754daef2237f33239d268cbf352cf50e085e7e2c48ebe3cc752
+size 4302348608

model-00112-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e23da6971b629a872eeb3005eea3ae394c030267b89301015b3450c19f6d8d46
+size 4302382296

model-00113-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb645ef1941b1c57f3ea1cdb0e1621b98b5ca49e68f6bfa446972ed7154ae340
+size 4302382752

model-00114-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:401c849ccbfe6e082bf1ab1c26cc22ce492b1ff353a8dd711c31e9037e143886
+size 4302348424

model-00115-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4703bb9c339906745d53ac4500253333382bdd315b92f20af83a6ce80640fb80
+size 4302382480

model-00116-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:adca9590820649ff99fbfc03a49b409678646dbc03eb22a52728d3eb9686919d
+size 4302382752

model-00117-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8f2a1d4dc1249cadf025963fb5b6a27dfc9f466b945bedee9331e03b247e0a4c
+size 4302348232

model-00118-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ed854c9b7a696567bfd1e1fe4ec9e220ce9c265ad105b9fae5fc0746a0efd70b
+size 4302382672

model-00119-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0c205f2fc398ea0f3baf75d769a86d4bcfa02b94894804d9e8295b7b6e91f4b8
+size 4302348632

model-00120-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4891cbb17b44290f3cd48b8cf0186486720338715c77649be79b1b2ee1bea44b
+size 4302382280

model-00121-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98d91733bbe353bc638aedfbd404621c819696868d9ca047546fc44be47e423a
+size 4302382752

model-00122-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ead33cdf5b22639840bd6f2fdf5c0d3af89ec155480932cf0d86b6f3b46cc1f2
+size 1747412896

model-00123-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8e51deb828717c59634f7fa6478481ee0b2757234b39b1e2f5b5c4b0e6ecfb19
+size 4302319304

model-00124-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f6b050070aba63b1244832536941e06467eaf8c5b690405462c978bca9eccd64
+size 4302382704

model-00125-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9dd39d3be1e1af9e6b65cc335204af6bc5c1f710bfe6c9e3e90c097d21914cc1
+size 4302348600

model-00126-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a76546b17ee6022e79576e15b1809ea314ab8e5031b869ca3a3787a694cd6df4
+size 4302382312

model-00127-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4a73e5a04e886103b581f894b419751ed033d7811c7bb0a191f40d3a3bc6cde4
+size 4302382752

model-00128-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3c31a134a398330096a5e2415684fe4b4c020ba5a4ab33e6266b728ef8662d47
+size 4302348408

model-00129-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f501b8ea711ce186bd1d04332962f6e350c2379c5971b193bd271e0e805f8d48
+size 4302382496

model-00130-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1eae6aa44b9535cfe3dc92cfab24ec850fa3c7bd0455ff40adafe27c78b5e25c
+size 4302382752

model-00131-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:22e98b92db85232e6fb59406cf74a1e56e3e97ac7fe5088826e7cccff1f0f597
+size 4302348216

model-00132-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d0bb193eddf8570e5665507431a637b392590d8fb183a5fb4e5cbd2b2cdadb32
+size 4302382696

model-00133-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c3077e860562f54807012aab388197bf8adb506301d1fffcf1f4552d417355e5
+size 4302348608

model-00134-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:33b1e621e91c5d342cb3d45d2e11378d7cc188f12f9c303847fb4f5a422bb19d
+size 4302382296

model-00135-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1e7582999f3c768c37a28fa7491dcfa26b93d9b0a8adb45c849625377adbb0b2
+size 4302382752

model-00136-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:69b1faa43b0c860951f1ef0d3e0753170b88870a2f1c458f2bb707cfeca2ad03
+size 4302348424

model-00137-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3c0de135a80cb0372ac1053d7be50850f09e4915b6a61f5b6320f65624f25d51
+size 4302382480

model-00138-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd7aa40490f38d63f52c8ed460e2f7b8b61b8ff098a88ffcdccd17aed29f9a07
+size 4302382752

model-00139-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:423f14f488147032f558acbd7d2cf5477b74aea9354ff7e9336b12398c4cdfe3
+size 4302348232

model-00140-of-000163.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0f15e3f8f13dded85512fa6ad24606aa3acf13481504e6201b201aa7040a7c3f
+size 4302382672