Text Generation
Transformers
PyTorch
English
taonet_mini_t2
taonet
taotern
ssm
state-space-model
dplr
custom_code
experimental
Instructions to use TaoTern/TaoNet-mini-T2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TaoTern/TaoNet-mini-T2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="TaoTern/TaoNet-mini-T2", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("TaoTern/TaoNet-mini-T2", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use TaoTern/TaoNet-mini-T2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "TaoTern/TaoNet-mini-T2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TaoTern/TaoNet-mini-T2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/TaoTern/TaoNet-mini-T2
- SGLang
How to use TaoTern/TaoNet-mini-T2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "TaoTern/TaoNet-mini-T2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TaoTern/TaoNet-mini-T2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "TaoTern/TaoNet-mini-T2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TaoTern/TaoNet-mini-T2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use TaoTern/TaoNet-mini-T2 with Docker Model Runner:
docker model run hf.co/TaoTern/TaoNet-mini-T2
| { | |
| "checkpoint": "/home/student/YouZheng/jobs/taotern/taotern-200m-branch-only-chat-20260514/checkpoints/pretrain/final_model.pt", | |
| "tokenizer_path": "/home/student/YouZheng/tokenizers/taodata_pilot_8k/tokenizer.model", | |
| "device": "cuda", | |
| "dtype": "torch.bfloat16", | |
| "max_new_tokens": 160, | |
| "temperature": 0.8, | |
| "top_p": 0.9, | |
| "samples": [ | |
| { | |
| "prompt": "The purpose of artificial intelligence is", | |
| "completion": "to enable human creativity to be a kind of artifact of politics - and sometimes to do that the principle of human dignity cannot be denied without the distinction of two; but the only thing that gives those who rely on it the ability to be free to decide the right thing.In some cases, the majority of the rules about how to use such knowledge are not clearly understood by the law. This leads to the generation of the same arguments, which leads to the mistake of the expert who sought to apply it to everything else.Well, to comply with the human dignity. There is some good news. There are some types of discrimination. This is because it is always a death sentence. Many people are not even able to get to court." | |
| }, | |
| { | |
| "prompt": "In a small village,", | |
| "completion": "you can live alone, but sometimes you might have to leave the house without a roof. Most of these city residents get two hours of sleep on most of the time, while many do.When it comes to school:Times help.Most of the time, wind is always the most important (or the least important): being well rested during the day.When the weather gets hot, the symptoms of insomnia can appear. Most of the time it just doesn't. Most people get enough sleep.Typically, we don't need to get more than 2 hours.Why are there such a large number of people getting the same kind of sleep?There are some factors that make it difficult to identify as having sleep problems. Some people do get a few" | |
| }, | |
| { | |
| "prompt": "<user>Hello, who are you?<assistant>", | |
| "completion": "24.2 (2) 1:10Though many households still live without electricity, most people who live close to the power lines will do so to the extent that they have only a small amount of electricity at the end of the day (33%).(3) The average household electricity rate is higher for the two types of households at 32%, with 31% of the 2005-06 peaks.7Take home message:- Unrealistically high societies. This is one factor that makes getting it to such a huge difference. This means that one has to make some very poor choices when it comes to one's overall lifestyle. Many cities do not have a proper gas taxes;" | |
| } | |
| ] | |
| } |