Instructions to use hyokwan/llama31_common with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hyokwan/llama31_common with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hyokwan/llama31_common") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("hyokwan/llama31_common") model = AutoModelForCausalLM.from_pretrained("hyokwan/llama31_common") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hyokwan/llama31_common with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hyokwan/llama31_common" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hyokwan/llama31_common", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hyokwan/llama31_common
- SGLang
How to use hyokwan/llama31_common with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hyokwan/llama31_common" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hyokwan/llama31_common", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hyokwan/llama31_common" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hyokwan/llama31_common", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hyokwan/llama31_common with Docker Model Runner:
docker model run hf.co/hyokwan/llama31_common
License compatibility
Hi, I'd like to report a License Conflict in hyokwan/llama31_common. I noticed that this model appears to be finetuned
from meta-llama/Llama-3.1-8B-Instruct, while being published under the Apache-2.0 license. Given the terms outlined in the LLaMA 3.1 Community License, especially regarding redistribution, attribution, and naming, this combination of licenses could potentially lead to legal or usage misunderstandings.
Key incompatibilities with LLaMA 3.1 Community License:
Clause 1.b.i – Redistribution and Use:
• No license file included (should contain the LLaMA 3.1 Community License)
• "Built with Llama" is not clearly indicated
• Model name does not begin with “Llama 3”, which is required for any derivative
Clause 1.b.iii – Required Notice:
• Missing the following required text in a "NOTICE" file:
“Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.”
Clause 1.iv – Acceptable Use Policy:
• Meta’s Acceptable Use Policy is not mentioned or passed along to users
Clause 2 – Sublicensing and Relicensing:
• LLaMA 3.1 license does not allow sublicensing under a more permissive license such as Apache-2.0
• The Apache-2.0 License permits nearly unrestricted commercial use, which contradicts Meta’s limits and conditions (e.g. commercial MAU threshold)
On the flip side, Apache-2.0 lets you:
• Use it commercially without asking for extra permission
• Sublicense and redistribute it under more flexible terms
• You don’t have to pass along any non-permissive terms or use restrictions from upstream
This creates a bit of a conflict because the LLaMA 3 license specifically says you can’t sublicense it under more flexible terms and requires downstream users to follow certain use restrictions, which Apache-2.0 doesn’t enforce.
So I'm thinking there might be a licensing conflict here that needs to be sorted out.
🔹 Suggestion:
To resolve the mismatch, here are a few steps that might help bring things into alignment:
1. To make sure everything aligns with the LLaMA 3.1 terms, you might want to tweak the licensing setup a bit, like:
• Maybe include a copy of the LLaMA 3.1 Community License in the repo or model card
• Include this notice in a “NOTICE” file or the docs:
> “Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.”
• A “Built with LLaMA” note somewhere in the model card could be helpful too
• Maybe a quick note about usage restrictions, especially for folks using it in commercial settings
• A statement clarifying that use of the model must comply with Meta’s Acceptable Use Policy
2. Maybe we can just drop the Apache-2.0 tag and going with the LLaMA 3.1 Community License. This approach may help reduce potential confusion about redistribution rights and downstream usage conditions.
Hope this helps! 😊 Let me know if you have any questions or need more info.
Thanks for your attention!
Would love to hear your view on this!