Text Generation
Transformers
Safetensors
English
qwen2
math
reasoning
ads
distillation
code
conversational
text-generation-inference
Instructions to use NoesisLab/Kai-30B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use NoesisLab/Kai-30B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="NoesisLab/Kai-30B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("NoesisLab/Kai-30B-Instruct") model = AutoModelForCausalLM.from_pretrained("NoesisLab/Kai-30B-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use NoesisLab/Kai-30B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "NoesisLab/Kai-30B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NoesisLab/Kai-30B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/NoesisLab/Kai-30B-Instruct
- SGLang
How to use NoesisLab/Kai-30B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "NoesisLab/Kai-30B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NoesisLab/Kai-30B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "NoesisLab/Kai-30B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NoesisLab/Kai-30B-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use NoesisLab/Kai-30B-Instruct with Docker Model Runner:
docker model run hf.co/NoesisLab/Kai-30B-Instruct
Upload folder using huggingface_hub
Browse files- config.json +1 -1
- generation_config.json +1 -1
- model-00001-of-00015.safetensors +1 -1
- model-00002-of-00015.safetensors +1 -1
- model-00003-of-00015.safetensors +1 -1
- model-00004-of-00015.safetensors +1 -1
- model-00005-of-00015.safetensors +1 -1
- model-00006-of-00015.safetensors +1 -1
- model-00007-of-00015.safetensors +1 -1
- model-00008-of-00015.safetensors +1 -1
- model-00009-of-00015.safetensors +1 -1
- model-00010-of-00015.safetensors +1 -1
- model-00011-of-00015.safetensors +1 -1
- model-00012-of-00015.safetensors +1 -1
- model-00013-of-00015.safetensors +1 -1
- model-00014-of-00015.safetensors +1 -1
- model-00015-of-00015.safetensors +1 -1
config.json
CHANGED
|
@@ -23,7 +23,7 @@
|
|
| 23 |
"rope_scaling": null,
|
| 24 |
"rope_theta": 5000000.0,
|
| 25 |
"tie_word_embeddings": false,
|
| 26 |
-
"transformers_version": "4.57.
|
| 27 |
"use_cache": true,
|
| 28 |
"vocab_size": 64000
|
| 29 |
}
|
|
|
|
| 23 |
"rope_scaling": null,
|
| 24 |
"rope_theta": 5000000.0,
|
| 25 |
"tie_word_embeddings": false,
|
| 26 |
+
"transformers_version": "4.57.1",
|
| 27 |
"use_cache": true,
|
| 28 |
"vocab_size": 64000
|
| 29 |
}
|
generation_config.json
CHANGED
|
@@ -6,5 +6,5 @@
|
|
| 6 |
"pad_token_id": 0,
|
| 7 |
"temperature": 0.6,
|
| 8 |
"top_p": 0.8,
|
| 9 |
-
"transformers_version": "4.57.
|
| 10 |
}
|
|
|
|
| 6 |
"pad_token_id": 0,
|
| 7 |
"temperature": 0.6,
|
| 8 |
"top_p": 0.8,
|
| 9 |
+
"transformers_version": "4.57.1"
|
| 10 |
}
|
model-00001-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4793130760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:85668f2d3ba21fe8b5d915d64a926a7a0d31d820c1a02811f17bfb17521b2a97
|
| 3 |
size 4793130760
|
model-00002-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459720
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:57212ac93b1e851dea19eafa4504caeebcd4a91a8bad1c09ad5b9d1fc722dda3
|
| 3 |
size 4756459720
|
model-00003-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4991370136
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:87a641c0151d3f58d18bdf69fa9665d484da46082478ac11bf8f547edff45b9c
|
| 3 |
size 4991370136
|
model-00004-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c7397f8ca20188d621ec6919c8ad0240b70cf551917dd239dd7a73b9324e1392
|
| 3 |
size 4756459760
|
model-00005-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4a9b0231088fe31e3b8f619efa56e4731b205e981cb9b0f129d3a996a39c3cb0
|
| 3 |
size 4756459760
|
model-00006-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4991370160
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:03adf805ad04ec42bcf101813a8cb73629bf030980b29f711f62c4efd301e961
|
| 3 |
size 4991370160
|
model-00007-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6faaf0787ff170f73b0b4b6192fce9a145991970ba7fc016e15d595aee4f26e9
|
| 3 |
size 4756459760
|
model-00008-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e25cc9201d3ed1e6bcfde3cd6da65d278eda81d089d7bb75eb9cde1cb83336b2
|
| 3 |
size 4756459760
|
model-00009-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4991370160
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d756446cbb82ec99c066338a00f04329a12e75c43dfc4a2c9aec2577c6b5aa77
|
| 3 |
size 4991370160
|
model-00010-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1eab55f6badb80f4fcb241b68477688b0d1b235fec68335f60c0ca60584003ef
|
| 3 |
size 4756459760
|
model-00011-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6e4986e50ae464ff7c3413f11ec66c1ae56b62dfb9e84c836c5d2a9d735dc56e
|
| 3 |
size 4756459760
|
model-00012-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4991370160
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82493843811eedce4cfdd0f26836643afc9e268cc5e0afa37929426626872f20
|
| 3 |
size 4991370160
|
model-00013-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a733603ca4b20f9c8ecb17ef84fdb592cf041ad5d96b566d0665ddd1a5b2c5f3
|
| 3 |
size 4756459760
|
model-00014-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4756459760
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a97698f0da609d5a32c6a6c65b4737187b3c92d3e320b96f25249adad41ca070
|
| 3 |
size 4756459760
|
model-00015-of-00015.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1211148856
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:be39df0cd45793ee40a6fe83038f6842e48317126f9ab16341acbdc1e5707d6e
|
| 3 |
size 1211148856
|