Instructions to use Parveshiiii/Terminator-X with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Parveshiiii/Terminator-X with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Parveshiiii/Terminator-X", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Parveshiiii/Terminator-X", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Parveshiiii/Terminator-X with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Parveshiiii/Terminator-X"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Parveshiiii/Terminator-X",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Parveshiiii/Terminator-X

SGLang

How to use Parveshiiii/Terminator-X with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Parveshiiii/Terminator-X" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Parveshiiii/Terminator-X",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Parveshiiii/Terminator-X" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Parveshiiii/Terminator-X",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Parveshiiii/Terminator-X with Docker Model Runner:
```
docker model run hf.co/Parveshiiii/Terminator-X
```

Terminator-X / configuration_model.py

Parveshiiii

Upload 5 files

27fe8df verified about 2 months ago

raw

history blame contribute delete

2.57 kB

	from transformers import PretrainedConfig

	class HybridModelConfig(PretrainedConfig):
	model_type = "hybrid_model"
	keys_to_ignore_at_inference = ["past_key_values"]

	def __init__(
	self,
	vocab_size=151936,
	hidden_size=768,
	intermediate_size=2048,
	num_hidden_layers=12,
	num_attention_heads=12,
	# MLA compression dims (DeepSeek-style naming)
	kv_lora_rank=192, # KV latent/compression dimension (d_c)
	q_lora_rank=384, # Query latent/compression dimension (d_c1)
	qk_rope_head_dim=32, # RoPE dimension per head (d_rotate)
	hidden_act="silu",
	max_position_embeddings=32768,
	initializer_range=0.02,
	rms_norm_eps=1e-6,
	use_cache=True,
	pad_token_id=0,
	bos_token_id=1,
	eos_token_id=2,
	tie_word_embeddings=False,
	rope_theta=10000.0,
	sliding_window=4096,
	attention_dropout=0.0,
	# MHC (Multi-Head Connections) settings
	mhc_num_streams=4, # number of parallel streams (mhc_n)
	mhc_sinkhorn_iters=20, # Sinkhorn-Knopp iterations (mhc_tmax)
	mhc_alpha_init=0.01,
	mhc_rmsnorm_eps=1e-6,
	mhc_stream_init="paper",
	mhc_readout_init="first",
	**kwargs,
	):
	self.vocab_size = vocab_size
	self.max_position_embeddings = max_position_embeddings
	self.hidden_size = hidden_size
	self.intermediate_size = intermediate_size
	self.num_hidden_layers = num_hidden_layers
	self.num_attention_heads = num_attention_heads

	self.kv_lora_rank = kv_lora_rank
	self.q_lora_rank = q_lora_rank
	self.qk_rope_head_dim = qk_rope_head_dim

	self.sliding_window = sliding_window

	self.hidden_act = hidden_act
	self.initializer_range = initializer_range
	self.rms_norm_eps = rms_norm_eps
	self.use_cache = use_cache
	self.rope_theta = rope_theta
	self.attention_dropout = attention_dropout

	self.mhc_num_streams = mhc_num_streams
	self.mhc_sinkhorn_iters = mhc_sinkhorn_iters
	self.mhc_alpha_init = mhc_alpha_init
	self.mhc_rmsnorm_eps = mhc_rmsnorm_eps
	self.mhc_stream_init = mhc_stream_init
	self.mhc_readout_init = mhc_readout_init

	super().__init__(
	pad_token_id=pad_token_id,
	bos_token_id=bos_token_id,
	eos_token_id=eos_token_id,
	tie_word_embeddings=tie_word_embeddings,
	**kwargs,
	)