Text Generation
Transformers
Safetensors
English
llama
blackhole
p150
tenstorrent
trained-from-scratch
tt-kernel-cache
tt-metal
ttml
vllm
text-generation-inference
Instructions to use episod/tt-tnt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use episod/tt-tnt with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="episod/tt-tnt")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("episod/tt-tnt") model = AutoModelForCausalLM.from_pretrained("episod/tt-tnt", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use episod/tt-tnt with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "episod/tt-tnt" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "episod/tt-tnt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/episod/tt-tnt
- SGLang
How to use episod/tt-tnt with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "episod/tt-tnt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "episod/tt-tnt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "episod/tt-tnt" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "episod/tt-tnt", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use episod/tt-tnt with Docker Model Runner:
docker model run hf.co/episod/tt-tnt
File size: 1,738 Bytes
79bf4b3 515852f 79bf4b3 515852f 79bf4b3 515852f 79bf4b3 64c3c0e 79bf4b3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | {
"schema_version": "4",
"name": "tt-tnt",
"tt_metal_version": "0.65.1rc17.dev6200",
"arch": "blackhole",
"device_count": 1,
"build_key": null,
"build_key_inputs": {
"dispatch_core_type": "WORKER",
"dispatch_core_axis": "ROW",
"num_hw_cqs": 1,
"coordinate_virtualization_enabled": true,
"harvesting_mask": 0,
"compile_hash_string": ""
},
"kernel_count": 0,
"fast_path_kernels": null,
"files": [
{
"path": "vllm_bundle/tt_tnt_adapter.py",
"sha256": "676460f6e0f45edd6d013012380b48ac6482bbca385bdf13f57843e645b1fee6",
"size": 33233
}
],
"producer": {
"tt_kernel_version": "0.1.0",
"created_at": "2026-08-15T04:46:02.996272+00:00",
"hostname": "tsingletaryTT-quietbox",
"tt_metal_home": null
},
"runner": {
"spec": "",
"wheels": [],
"entry_point": null,
"source": null,
"requires_python": null,
"backend": "vllm",
"bundle_dir": "vllm_bundle"
},
"weights": {
"repo_id": "episod/tt-tnt",
"revision": null,
"allow_patterns": null,
"ignore_patterns": null,
"repo_type": "model"
},
"platform": {
"ttnn": ">=0.65,<0.80"
},
"runtime": {
"kind": "vllm",
"version": null,
"plugin_version": null
},
"target": "p150",
"mesh": {
"devices": 1,
"topology": "1x1",
"fabric": null
},
"entrypoint": {
"cls": "tt_tnt_adapter:LlamaForCausalLM",
"arch_name": "LlamaForCausalLM"
},
"resources": {
"max_model_len": 2048,
"max_num_seqs": 8,
"block_size": null,
"trace_region_bytes": null,
"extra_args": [],
"command_override": {}
},
"capabilities": null,
"env": {
"MESH_DEVICE": "P150",
"HF_MODEL": "episod/tt-tnt"
}
} |