ROMA / examples /inference /llama3.yaml
Houssem0's picture
ROMA + GH200 reproducible Docker layer
e5c09aa verified
Raw
History Blame Contribute Delete
168 Bytes
model_name_or_path: meta-llama/Meta-Llama-3-8B-Instruct
template: llama3
infer_backend: huggingface # choices: [huggingface, vllm, sglang]
trust_remote_code: true