Instructions to use Tele-AI/TeleChat-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Tele-AI/TeleChat-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Tele-AI/TeleChat-1B", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Tele-AI/TeleChat-1B", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use Tele-AI/TeleChat-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Tele-AI/TeleChat-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Tele-AI/TeleChat-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Tele-AI/TeleChat-1B
- SGLang
How to use Tele-AI/TeleChat-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Tele-AI/TeleChat-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Tele-AI/TeleChat-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Tele-AI/TeleChat-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Tele-AI/TeleChat-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Tele-AI/TeleChat-1B with Docker Model Runner:
docker model run hf.co/Tele-AI/TeleChat-1B
Update README.md
Browse files
README.md
CHANGED
|
@@ -121,19 +121,18 @@ TeleChat模型相比同规模模型在评测效果方面也有较好的表现,
|
|
| 121 |
>>> os.environ["CUDA_VISIBLE_DEVICES"] = '0'
|
| 122 |
>>> tokenizer = AutoTokenizer.from_pretrained('../models/1B')
|
| 123 |
>>> model = AutoModelForCausalLM.from_pretrained('../models/1B', trust_remote_code=True, device_map="auto", torch_dtype=torch.float16)
|
| 124 |
-
>>>
|
| 125 |
-
>>> question="
|
| 126 |
-
>>>
|
|
|
|
| 127 |
>>> print(answer)
|
| 128 |
-
生抽和老抽
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
总的来说,生抽和老抽都是酱油的不同种类,它们在原料、制作工艺和口感等方面都有所不同。
|
| 137 |
```
|
| 138 |
|
| 139 |
|
|
|
|
| 121 |
>>> os.environ["CUDA_VISIBLE_DEVICES"] = '0'
|
| 122 |
>>> tokenizer = AutoTokenizer.from_pretrained('../models/1B')
|
| 123 |
>>> model = AutoModelForCausalLM.from_pretrained('../models/1B', trust_remote_code=True, device_map="auto", torch_dtype=torch.float16)
|
| 124 |
+
>>> question="<_user>生抽与老抽的区别?<_bot>"
|
| 125 |
+
>>> context_ids = tokenizer(question, return_tensors="pt")
|
| 126 |
+
>>> output = model.generate(context_ids["input_ids"].to(0), do_sample=False, max_length=1024)
|
| 127 |
+
>>> answer = tokenizer.decode(output[0].tolist()).split('<_bot>')[-1]
|
| 128 |
>>> print(answer)
|
| 129 |
+
生抽和老抽的主要区别在于色泽、 生抽的制作工艺以及生抽在烹饪过程中主要用于调味的作用。下面是详细的比较:
|
| 130 |
+
1. 色泽: 生抽(也称老抽)是一种为了保持菜肴色泽而加入的调味料,主要用于调色和增加菜肴的鲜亮。生抽的颜色相对较浅,会带有红棕色调,主要用于点缀和增强色泽。而老抽的颜色更深,可能会变为棕褐色并且带有深沉的颜色。
|
| 131 |
+
2. 制作工艺: 生抽是由黄豆等为原料,经过发酵、提制而成,色泽过程中无原腐剂、咸精、色素等添加剂。老抽则是在生抽的基础上,加入焦糖色,经过长时间的发酵和陈化制作而成。
|
| 132 |
+
3. 生抽的用途: 生抽主要用于调味,可以提亮菜肴的色泽,增加鲜味,还可以为菜肴增加一定的咸味。在制作生抽时,通常会加入适量的水,并经过充分的搅拌和蒸钝过程。
|
| 133 |
+
4. 生抽在烹饪中的作用: 生抽主要用于调味,可以提亮菜肴的色泽,增加鲜味,还可以为菜肴增加一定的咸味。在烹饪过程中,生抽会起到主要的调味作用。
|
| 134 |
+
|
| 135 |
+
所以,生抽和老抽的主要区别在于色泽、 生抽的制作工艺以及生抽在烹饪过程中主要用于调味的作用。在烹饪时,可以根据不同的需求和个人口味,选择生抽和老抽各一种或几种。 <_end>
|
|
|
|
|
|
|
| 136 |
```
|
| 137 |
|
| 138 |
|