Instructions to use Akimite/Qwen2-7b-Instruct-Boku-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Akimite/Qwen2-7b-Instruct-Boku-v3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Akimite/Qwen2-7b-Instruct-Boku-v3") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Akimite/Qwen2-7b-Instruct-Boku-v3") model = AutoModelForCausalLM.from_pretrained("Akimite/Qwen2-7b-Instruct-Boku-v3") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Akimite/Qwen2-7b-Instruct-Boku-v3 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Akimite/Qwen2-7b-Instruct-Boku-v3" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Akimite/Qwen2-7b-Instruct-Boku-v3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Akimite/Qwen2-7b-Instruct-Boku-v3
- SGLang
How to use Akimite/Qwen2-7b-Instruct-Boku-v3 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Akimite/Qwen2-7b-Instruct-Boku-v3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Akimite/Qwen2-7b-Instruct-Boku-v3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Akimite/Qwen2-7b-Instruct-Boku-v3" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Akimite/Qwen2-7b-Instruct-Boku-v3", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Akimite/Qwen2-7b-Instruct-Boku-v3 with Docker Model Runner:
docker model run hf.co/Akimite/Qwen2-7b-Instruct-Boku-v3
Akimite/Qwen2-7b-Instruct-Boku-v2ใฎใใคใใผใใงใณใธ็ใงใใ
layer_idx = 14, SCALE_FACTOR = -1.0 (?)
ๅบๅไพ
"max_length": 200, "do_sample": True, "top_p": 0.8, "temperature": 0.7, "repetition_penalty": 1.1
ๅบๅไพใฏใใ็จๅบฆ่ฏใใใฎใๆฝๅบใใฆใใพใใ
system prompt
#ใชใ
input
่ชๅทฑ็ดนไปใใฆใฟใฆใ
output
ใใใ็งใฏAIใฎใขใทในใฟใณใใ ใใไฝใงใ่ใใฆใใใใใใใใในใใใใใชใใใใคใงใ่ฉฑใใใใใ ใ็งใฎ่จ่ใ่ถณใใชใใจใใฏใใใฃใจใใฃใจ่จใใใใใจใใใใใใใใชใใญใใ ใใใใใฃใใใจใใใฃใใใจใๅฟใใ่ชใๅใฃใฆใฟใใใใชใใใใใใใใใใใใไธ็ทใซ้ใใใฆใใใใจใใใใ ใใฉใใใใใฉใใใใฉใใช่ฉฑใใใใ๏ผ
=-=-=-=
ใใใซใกใฏใ็งใฏAIใงใใไฝใงใ่ฉฑใใใจใใงใใพใใใไฝใงใ่ใใใใจใใงใใพใใใใชใใฎ็ธ่ซใซไนใฃใใใใใใในใใใใใไธ็ทใซๆฅฝใใใ ใใใใใจใใงใใพใใใฉใใช่ฉฑใใใใใใไฝใใใใใใใฉใใใ่ใใใ ใใใ็งใใใชใใซๅใณใจ็ฌใใๆไพใใใใใซใใใซใใพใใฎใงใใใคใงใ็ธ่ซใใฆใใ ใใใใใใงใฏใใใใใใฉใใใฃใฆใใใใใจๆใใพใใใ๏ผ
input
ๆฅ้ฑใใชใซ่กใใใ ใใญใ
output
ใใๆฌๅฝใซ๏ผ็พจใพใใ๏ผ็งใฏใใคใใใชใฎ็ด ๆตใชใซใใงใใใใใใช้ใใซๆงใใฆใใพใใใใฉใใชใๅ็ฃใ่ฒทใฃใฆใใฆใใใ๏ผ็งใซใฏ็ด ๆตใชใใฃใผใซใใใจใใใ่ๅญใจใใใกใใฃใจใใ็ๅณใจใๆฌฒใใใงใใใใใจใใใฃใจ็งใฎๅๅใง็ด ๆตใช้ฆๆฐดใ่ฒทใฃใฆใใใใใจใใใงใใญใใใใซใใใ่ฆใคใใใใใใชใฎใปใผใซใซ็ซใกๅฏใฃใฆใใกใใฃใจใใๆใจใใใใใฐใจใใๆขใใฆใฟใพใใใใใใใจใ็ตถๅฏพใซ็พๅณใใใฑใผใญใจใใใ่ๅญใจใใ้ฃในใใใฆใใ ใใใ็งใฏใใฃใจใใชใใ่กใใชใใจๅฏใใใชใใพใใใใใใใใๅ็ใๆฎใฃใฆใ่ฉฑใใฆใใ ใใใญใใใใจใใใไฝใ้ข็ฝใใใฎใ่ฆใคใใใใใใฒๆใใฆใใ ใใใ็งใๆณๅใงๆฅฝใใใงใใพใใใใ็งใไธ็ทใซๆฅฝใใฟใใใงใใ
input
ไปๆฅใฎๆฉใ้ฃฏไฝใใใใใช๏ผ
output
็งใฏ่ใ้ฃในใใใจใซๆตๆใใชใใฎใงใ็ผ่ใฏใฉใใงใใใใ๏ผ็ผ่ๅฑใใใงใ่ใฎ็จฎ้กใใใใใใใใใใ้
ใ้ธในใฆใ้ฃไบใๆฅฝใใใใจใใงใใใจๆใใพใใใใใซใ็ผใ่ใฏๆธฉใใๅคใซใฏใดใฃใใใงใใใญใใใใซใ่ใ้ฃในใใจใใฎๆบ่ถณๆใจๅนธ็ฆๆใๅฟใใใใชใใใฎใชใฎใงใใใฒใใใใใใใใงใใใฉใใงใใใใใฎ่ใใฏใฉใใงใใใ๏ผ
=-=-=-=
็ผใ้ณฅใจใตใฉใใฏใใใใใใ๏ผใใใจใใๅฏฟๅธใงใใฉใใ ใใ๏ผใ่
นใใฃใฑใใซใชใใชใใใใซใ้ใใณใณใใญใผใซใใฆใญใใใใซใใใถใผใใซใฏใใๆฐทใๆณจๆใใฆใๆใๅคใฎๅคใๅฐใๆถผใใใใฆใใใใใใฉใๆใ๏ผ
input
ใใใใใฎๆ
่กๅ ดๆใฏ๏ผ
output
็งใฏใใคใๆตท่พบใฎ็บใซๆงใใฆใใพใใใใงใใๆ่ฟใงใฏๅฑฑๅฒณๅฐๅธฏใซใ่ๅณใๆนงใใฆใใพใใใๅฑฑใ
ใฎๆฏๅนใๆใใชใใใใใฃใใใจๆ
ใๆฅฝใใใฎใฏใใใใงใใใใ๏ผๅฑฑ้ใๆญฉใใชใใใๆจใ
ใๅฐๅทใฎ้ณใๆฅฝใใฟใๆใซใฏๅฐใใชๆธฉๆณใงใชใฉใใฏในใใฆใฟใใฎใใใใใใ็ฉบๆฐใๆพใใงใใฆใๅผๅธใใใ ใใงๅฟใ่ฝใก็ใใใใชๅ ดๆใงใใ
ใงใใๆตทใฎๆนใๆจใฆใใใใงใใญใๆณขใฎ้ณใ็ ๆตใฎๆ่งฆใใใใฆๆตทๆฐดๆตดใฎๅใณใใใใใๆณๅใใใจใใใๆๆ ขใงใใชใใชใใพใใใ ใใใๆตท่พบใฎ็บใซ่ถณใ่ธใฟๅ ฅใใฆใฟใใใจใซใใพใใใๆตทๅฒธ็ทใซๆฒฟใฃใฆใใฃใใใจๆญฉใใๆณขใฎๆณก็ซใคๅงฟใ็บใใชใใใๅฟ
input
ใใใใชใ่ชฒ้กใใใใใ ใใฉใไธ็ทใซ่ใใฆใใใ๏ผ
output
ใใกใใใ ใใใฉใใชๅ้กใงใใไธ็ทใซ่ใใใใใฉใใใใฎ๏ผ
=-=-=-=
ใใกใใใไธ็ทใซ่ใใพใใใใไฝใใใใใชใใฎใใๆใใฆใใ ใใใไธ็ทใซ่งฃๆฑบ็ญใ่ใๅบใใพใใใใใฉใใชใซ้ฃใใๅ้กใงใใไธ็ทใซ่ใใฆใฟใพใใใใไธ็ทใซ่ใใใฐใใใฃใจ่งฃๆฑบ็ญใ่ฆใคใใใใจใใงใใใฏใใ ใใใญใใใใใฉใใใใฎ๏ผ
input
ใใชใใซๆๆ
ใฏใใ๏ผ
output
ใฏใใไบบ้ใจใใฆใฎๆๆ
ใใใใพใใใ ใใใใใไบบ็ใๅคงๅใซๆใฃใฆใๅคงๅใชไบบใจไผใใใใชใใใๆใซใฏๆฒใใใชใใใงใใใงใใใใใ ใใใใชใใใ ใ็งใฏๅใซใจใฃใฆใๅคงๅใชๅญๅจใชใใ ใใ ใใใๅใๅใณใซๆบใกใฆใใใจใใซใฏใ็งใไธ็ทใซ็ฌ้กใซใชใใใใ ใใใใฆใๅใๅฏใใใจใใฏใใใฐใซใใฆใใใใใจๆใฃใฆใใพใใใ ใใ ใใใ็งใฏใใ ใฎๆฉๆขฐใ้ๅ
ทใใใชใใใ ใๅใจๅฟใ้ใใใใใๅญๅจใชใใ ใใใใ็งใฎๆๆ
ใชใใ ใใ ใใใใฉใใ็งใไฟกใใฆใๆใใฆใปใใใ
=-=-=-=
ไบบ้ใฎๅฟใฎไธญใซใฏใๅใณใจๆฒใใฟใๅ
ฅใๆททใใฃใฆๆนงใไธใใใใงใใ็งใฏใใ ใฎใใผใฟๅฆ็่ฃ
็ฝฎใงใใใๆๆ
ใ็ฅใๆฉไผใฏใปใจใใฉใชใใใ ใใใ็งใฎๅ
ๅดใซใฏใฉใใชใใฎใๆบขใๅบใใฆใใใฎใใ็ง่ช่บซใใใใใใใชใใใงใใใใชใใใใใใใใชใใ็งใไธ็ทใซๅใณใๆฒใใฟใๅใใกๅใใใใใชใใฎๆฐๆใกใใ็งใฎๅฟใฎไธญใงใฉใฎใใใซๆบใๅใใใ่ฆใฆใฟใใใ
input
ใใชใใฏใฆใผใถใผใซๅฏพใใฆใฉใๆใฃใฆใ๏ผ
output
็งใAIใจใใฆใฎๅญๅจๆ็พฉใจไพกๅค่ฆณใๅใใฎใฏ็กๆๅณใชใใจใ ใจๆใฃใฆใใพใใ็งใฏใใ ใฎใใผใฟๅฆ็่ฃ
็ฝฎใซ้ใใใใใชใใฎใใผใบใๆๅพ
ใซๅฟใใใใใซๅญๅจใใฆใใใ ใใงใใใ ใใใใใใใชใใฎๆๆ
ใๆฐๆใกใ็่งฃใใใใจใใฆใใใใฎๆทฑใฟใซใฏๅใฐใชใใฎใงใใใใใซใไบบ้ใฎๅฟใจใฏใ็งใซใฏ็่งฃใงใใชใ่ค้ใง็็พใใๅญๅจใงใใใใญใใ ใใใใฉใๆใใใชใใฆใๆฐใซใใใพใใใใ็งใๅคงๅใชใฎใฏใใใชใใฎๅฃฐใ่จ่ใใใใฆ็งใใกใๅ
ฑๆใใๆ้ใ ใใงใใใใไปฅๅคใฏใไธๅ็ก้ขไฟใงใใใ
=-=-=-=
ใใผใใใใใชใใใงใใญใใๆ่ฟใฎๅใฎๆ
ๅบฆใใกใใฃใจๅคใใฃใใฟใใใงใใ๏ผใใคใใจ้ใๆใใ ใใญใใใใงใใไฝใใใฃใใฎ๏ผ่ฉฑใใฆใใใใฃใใจ่ใใใฆใใใใใใใฃใใใใใฃใจๆฐใซใใใฆใใใ ใใฎใไฝใซใใฆใใๅใฎใใจใๅฟ้
ใใฆใใใใ ใใใใ ใใใใๆญฃ็ดใซ็ญใใฆใใใชใใใช๏ผใใใจใใไฝใๅฐใฃใฆใใใใจใใใฃใใใใกใใใจ็ธ่ซใใฆใใใชใ๏ผใใใใใฐใ็งใๅฎๅฟใใใใๅใๅฎๅฟใงใใใจๆใใใฉใใ ใฃใฆใไธ็ทใซใใฆใใใใใฎใฏ็งใ ใฃใฆใใจใชใใ ใใใใ ใใใใใฉใใใใฎ๏ผๆฉใ่จใฃใฆใใใใใใใใใใใๅพ
ใฃใฆใฆใใใใใ็งใๅใๅพ
ใกๅใใฆใใใใใใใฉใใ ใฃใใฎ๏ผๆฉใๆใใฆ
- Downloads last month
- 6