Instructions to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="voidful/Llama-3.1-TAIDE-R1-8B-Chat") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("voidful/Llama-3.1-TAIDE-R1-8B-Chat") model = AutoModelForCausalLM.from_pretrained("voidful/Llama-3.1-TAIDE-R1-8B-Chat") - llama-cpp-python
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="voidful/Llama-3.1-TAIDE-R1-8B-Chat", filename="llama-3-1-TAIDE-R1-Chat.gguf", )
llm.create_chat_completion( messages = [ { "role": "user", "content": "What is the capital of France?" } ] ) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: ./llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: ./build/bin/llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Use Docker
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- LM Studio
- Jan
- vLLM
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "voidful/Llama-3.1-TAIDE-R1-8B-Chat" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "voidful/Llama-3.1-TAIDE-R1-8B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- SGLang
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "voidful/Llama-3.1-TAIDE-R1-8B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "voidful/Llama-3.1-TAIDE-R1-8B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "voidful/Llama-3.1-TAIDE-R1-8B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "voidful/Llama-3.1-TAIDE-R1-8B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Ollama:
ollama run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- Unsloth Studio new
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for voidful/Llama-3.1-TAIDE-R1-8B-Chat to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for voidful/Llama-3.1-TAIDE-R1-8B-Chat to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for voidful/Llama-3.1-TAIDE-R1-8B-Chat to start chatting
- Pi new
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "voidful/Llama-3.1-TAIDE-R1-8B-Chat" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent new
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default voidful/Llama-3.1-TAIDE-R1-8B-Chat
Run Hermes
hermes
- Docker Model Runner
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Docker Model Runner:
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- Lemonade
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull voidful/Llama-3.1-TAIDE-R1-8B-Chat
Run and chat with the model
lemonade run user.Llama-3.1-TAIDE-R1-8B-Chat-{{QUANT_TAG}}List all available models
lemonade list
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat# Run inference directly in the terminal:
llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-ChatUse pre-built binary
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat# Run inference directly in the terminal:
./llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-ChatBuild from source code
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat# Run inference directly in the terminal:
./build/bin/llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-ChatUse Docker
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chatvoidful/Llama-3.1-TAIDE-R1-8B-Chat
This is a merge of pre-trained language models created using mergekit.
Usage
import vllm
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
model_name = "voidful/Llama-3.1-TAIDE-R1-8B-Chat"
llm = vllm.LLM(model=model_name,max_model_len=4096)
tokenizer = AutoTokenizer.from_pretrained(model_name)
messages = [
{"role": "user", "content": f"早餐喝早餐店的奶茶會導致烙賽為什麼?"},
]
prompts = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=False
)
sampling_params = SamplingParams(temperature=0.6, max_tokens=512, top_p=0.9)
outputs = llm.generate(prompts, sampling_params)
print(f"{prompts}")
print(f"{outputs[0].outputs[0].text}\n")
sampling_params = SamplingParams(temperature=0.6, max_tokens=512, top_p=0.9)
outputs = llm.generate(prompts, sampling_params)
print(f"{prompts}")
print(f"{outputs[0].outputs[0].text}\n")
Output
<think> 關於「早餐喝早餐店的奶茶會導致烙賽」的問題,可能的原因有幾種可能的解釋。首先,「烙賽」這個詞在台灣的網路用語中,通常指的是「燒腸」或「拉肚子」的意思,指的是人體的腸胃或腸道發生不舒服的狀況,可能是消化不良、腹泻、或其他腸胃道的問題。所以,喝了不健康的飲料,可能會導致腸胃不舒服,引起「烙賽」的反應。
其次,一個可能的原因是,早餐店的奶茶可能使用了低品質的奶源、含糖或含奶精等添加物。奶精是一種人工添加劑,可能會對胃造成刺激或不舒服的感覺。再者,早餐的奶茶可能是用即溶的粉末或濃縮的奶來泡的,這些東西可能含有許多添加劑或不健康的成分。
最後,個人的體質也是一個因素。有人可能對奶或糖有過敏或不耐受的反應,喝了之後就會出現不舒服的症狀。
綜合上述的原因,早餐喝早餐店的奶茶可能會導致烙賽的原因有:使用低品質的奶源、含糖或奶精等添加劑、個人的體質對奶或糖有過敏或不耐受的反應等。
<answer> 早餐喝早餐店的奶茶可能導致烙賽的原因有低品質的奶源、含糖或奶精等添加劑、以及個人的體質對奶或糖有過敏或不耐受的反應等。這是因為不健康的飲料成分可能會對身體造成不舒服的影響。</answer>
Merge Details
Merge Method
This model was merged using the SCE merge method using meta-llama/Llama-3.1-8B-Instruct as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
merge_method: sce
base_model: meta-llama/Llama-3.1-8B-Instruct
tokenizer:
source: taide/Llama-3.1-TAIDE-LX-8B-Chat
models:
- model: taide/Llama-3.1-TAIDE-LX-8B-Chat
- model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
- Downloads last month
- 1,374
Install from brew
# Start a local OpenAI-compatible server with a web UI: llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat# Run inference directly in the terminal: llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat