Text Generation
Transformers
Safetensors
Korean
llama
meta
llama-2
llama-2-ko
text-generation-inference
Instructions to use URP/urllm-ko-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use URP/urllm-ko-7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="URP/urllm-ko-7b")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("URP/urllm-ko-7b") model = AutoModelForCausalLM.from_pretrained("URP/urllm-ko-7b") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use URP/urllm-ko-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "URP/urllm-ko-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "URP/urllm-ko-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/URP/urllm-ko-7b
- SGLang
How to use URP/urllm-ko-7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "URP/urllm-ko-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "URP/urllm-ko-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "URP/urllm-ko-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "URP/urllm-ko-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use URP/urllm-ko-7b with Docker Model Runner:
docker model run hf.co/URP/urllm-ko-7b
Commit History
Update README.md 12526f3 verified
Update README.md 82dcd78 verified
Update README.md 148f639 verified
ChangheeCho commited on
Upload tokenizer a5030a2 verified
ChangheeCho commited on
Upload LlamaForCausalLM bdeb873 verified
ChangheeCho commited on
Update README.md 4c82625 verified
ChangheeCho commited on
Update README.md 3c2cc41 verified
ChangheeCho commited on
Update README.md 96ee769 verified
ChangheeCho commited on
Update README.md cf1aa9a verified
ChangheeCho commited on
Update README.md c3d463a verified
ChangheeCho commited on
Update README.md 2ea4c0a verified
ChangheeCho commited on
Update README.md 9d10085 verified
ChangheeCho commited on
Create README.md f19b28a verified
ChangheeCho commited on
initial commit d2ca95e verified
ChangheeCho commited on