Instructions to use wenge-research/yayi2-30b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use wenge-research/yayi2-30b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="wenge-research/yayi2-30b", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("wenge-research/yayi2-30b", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use wenge-research/yayi2-30b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "wenge-research/yayi2-30b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "wenge-research/yayi2-30b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/wenge-research/yayi2-30b
- SGLang
How to use wenge-research/yayi2-30b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "wenge-research/yayi2-30b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "wenge-research/yayi2-30b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "wenge-research/yayi2-30b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "wenge-research/yayi2-30b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use wenge-research/yayi2-30b with Docker Model Runner:
docker model run hf.co/wenge-research/yayi2-30b
Corrected README
The community license doesn't satisfy the requirements to be open source because it doesn't include an unlimited and universal commercial license. Calling this model "publicly released" instead of "open source" will help people understand the terms of the license easier.
Respectfully disagree. It's a legit decision to open source for non commercial only
We can request them to loosen the restrictions but we can say it's not open source just because it's non commercial.
I'm not asking them to change the license. I'm correcting their use of language because the statement "this model is open source" is false. Open source isn't a marketing fluff term, it has a well-established and widely agreed upon definition. You can find that definition here. This model does not meet the definition, and so it's incorrect to call it open source.
By definition, any model with a NC license is not open source.
The community license doesn't satisfy the requirements to be open source because it doesn't include an unlimited and universal commercial license. Calling this model "publicly released" instead of "open source" will help people understand the terms of the license easier.
watch your grammar
We the pre-trained language model in this release...
Thank you all for your attention to YAYI2!
@wenge-research It appears that you accidentally closed this issue without merging it, as the false claims are still present.