Instructions to use internlm/internlm-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use internlm/internlm-7b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="internlm/internlm-7b", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("internlm/internlm-7b", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use internlm/internlm-7b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "internlm/internlm-7b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/internlm-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/internlm/internlm-7b
- SGLang
How to use internlm/internlm-7b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "internlm/internlm-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/internlm-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "internlm/internlm-7b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/internlm-7b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use internlm/internlm-7b with Docker Model Runner:
docker model run hf.co/internlm/internlm-7b
Update README.md
#2
by jamie-1 - opened
README.md
CHANGED
|
@@ -53,7 +53,7 @@ We conducted a comprehensive evaluation of InternLM using the open-source evalua
|
|
| 53 |
**Limitations:** Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information.
|
| 54 |
|
| 55 |
### Import from Transformers
|
| 56 |
-
To load the InternLM 7B
|
| 57 |
```python
|
| 58 |
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 59 |
>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-7b", trust_remote_code=True)
|
|
@@ -104,7 +104,7 @@ InternLM ,即书生·浦语大模型,包含面向实用场景的70亿参数
|
|
| 104 |
**局限性:** 尽管在训练过程中我们非常注重模型的安全性,尽力促使模型输出符合伦理和法律要求的文本,但受限于模型大小以及概率生成范式,模型可能会产生各种不符合预期的输出,例如回复内容包含偏见、歧视等有害内容,请勿传播这些内容。由于传播不良信息导致的任何后果,本项目不承担责任。
|
| 105 |
|
| 106 |
### 通过 Transformers 加载
|
| 107 |
-
通过以下的代码加载 InternLM 7B
|
| 108 |
```python
|
| 109 |
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 110 |
>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-7b", trust_remote_code=True)
|
|
|
|
| 53 |
**Limitations:** Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information.
|
| 54 |
|
| 55 |
### Import from Transformers
|
| 56 |
+
To load the InternLM 7B model using Transformers, use the following code:
|
| 57 |
```python
|
| 58 |
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 59 |
>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-7b", trust_remote_code=True)
|
|
|
|
| 104 |
**局限性:** 尽管在训练过程中我们非常注重模型的安全性,尽力促使模型输出符合伦理和法律要求的文本,但受限于模型大小以及概率生成范式,模型可能会产生各种不符合预期的输出,例如回复内容包含偏见、歧视等有害内容,请勿传播这些内容。由于传播不良信息导致的任何后果,本项目不承担责任。
|
| 105 |
|
| 106 |
### 通过 Transformers 加载
|
| 107 |
+
通过以下的代码加载 InternLM 7B 模型
|
| 108 |
```python
|
| 109 |
>>> from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 110 |
>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-7b", trust_remote_code=True)
|