How to use tencent/Tencent-Hunyuan-Large with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tencent/Tencent-Hunyuan-Large")
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("tencent/Tencent-Hunyuan-Large", dtype="auto")
How to use tencent/Tencent-Hunyuan-Large with vLLM:
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/Tencent-Hunyuan-Large" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Tencent-Hunyuan-Large", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
docker model run hf.co/tencent/Tencent-Hunyuan-Large
How to use tencent/Tencent-Hunyuan-Large with SGLang:
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/Tencent-Hunyuan-Large" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Tencent-Hunyuan-Large", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/Tencent-Hunyuan-Large" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/Tencent-Hunyuan-Large", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'
How to use tencent/Tencent-Hunyuan-Large with Docker Model Runner:
It would be great if you could provide GGUF model files for this (and your other models) as this would enable the majority of people to use the model.
There are currently no plans to use GGUF model files
· Sign up or log in to comment
