Text Generation
Transformers
Safetensors
PyTorch
English
qtensorformer
q-tensorformer
tensor-train
tensor-networks
adaptive-computation
kv-cache
kv-cache-quantization
attention-sinks
pid-control
edge-ai
edge-inference
on-device-ai
model-compression
grouped-query-attention
rotary-position-embeddings
custom_code
Instructions to use Premchan369/Q-TensorFormer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Premchan369/Q-TensorFormer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Premchan369/Q-TensorFormer", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("Premchan369/Q-TensorFormer", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Premchan369/Q-TensorFormer with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Premchan369/Q-TensorFormer" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Premchan369/Q-TensorFormer
- SGLang
How to use Premchan369/Q-TensorFormer with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Premchan369/Q-TensorFormer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Premchan369/Q-TensorFormer" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Premchan369/Q-TensorFormer", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Premchan369/Q-TensorFormer with Docker Model Runner:
docker model run hf.co/Premchan369/Q-TensorFormer
File size: 3,689 Bytes
7aefd9e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | # Phase One — Source API & Execution Path Audit
## 1. Canonical Research API (`src/`)
### 1.1 `src.config.ModelConfig`
- **File:** `src/config.py`
- **Fields:**
- `d_model: int = 128`
- `n_heads: int = 4`
- `n_layers: int = 2`
- `ff_multiplier: int = 4`
- `max_seq_len: int = 128`
- `vocab_size: int = 10000`
- `dropout: float = 0.1`
- `tt_rank: int = 8`
- `tt_min_rank: int = 2`
- `use_tensor_ffn: bool = True`
- `n_qubits: int = 4`
- `n_quantum_layers: int = 2`
- `quantum_sparsity: float = 0.3`
- `use_quantum: bool = True`
- `rank_alpha: float = 2.0`
- `rank_smoothing: float = 0.9`
- **Validation:** `ModelConfig.validate()` asserts `d_model % n_heads == 0`, `1 <= tt_min_rank <= tt_rank`, `n_qubits <= 8`, and `0 <= quantum_sparsity <= 1`. `validate_model_config(config, model_class)` checks all required fields without inventing defaults.
### 1.2 `src.models.QTensorFormer`
- **File:** `src/models.py`
- **Constructor:** `QTensorFormer(config: ModelConfig, preset: str = "QTF_BALANCED")`
- **Supported Presets:** `QTF_FULL`, `QTF_BALANCED`, `QTF_LATENCY`, `QTF_MEMORY`, `QTF_ENERGY`, `QTF_EDGE`, `QTF_CLASSICAL_ONLY`.
- **Execution Path (`forward`):**
1. Embeds `input_ids` (`nn.Embedding`) + sinusoidal `PositionalEncoding`.
2. Iterates over `self.blocks` (`nn.ModuleList` of `HybridBlock`):
- `HybridBlock` calls `self.info_engine(hidden_states=x)` $\rightarrow z_t \in [0, 1]^{B \times T \times 8}$.
- `HybridBlock` calls `self.allocator(z_t, budget=budget, preset=preset, force_classical=...)` $\rightarrow$ selects `rank`, `token_ranks`, `attn_mode_idx`, `depth_mode`, `kv_precision`, `is_quantum_token`.
- If `depth_mode == "skip"`, returns residual `x` immediately.
- Updates `kv_cache.set_precision(kv_precision)` if `kv_cache` is supplied.
- Runs `self.attention(normed_x, mask=mask, kv_cache=kv_cache, use_quantum=has_quantum_attn and not force_classical)`.
- Checks optional `early_exit_head` if `enable_early_exit=True`.
- If `depth_mode == "partial"`, skips FFN and returns `x`.
- Sets `self.tt_ffn.set_rank(chosen_rank)` and executes `self.tt_ffn(normed_ffn_in, token_ranks=token_ranks)`.
- If `is_quantum_token.any()` and not `force_classical`, applies `self.quantum_backend(normed_ffn_in)` masked by `is_quantum_token`.
3. Applies final `self.ln_f` and weight-tied `self.lm_head`.
---
## 2. Canonical Hugging Face API (`configuration_qtensorformer.py` & `modeling_qtensorformer.py`)
### 2.1 `configuration_qtensorformer.QTensorFormerConfig`
- **Inherits from:** `transformers.PretrainedConfig` (`model_type = "qtensorformer"`)
- **Strict Loading Policy:** `keys_to_ignore_at_loading = []` (repaired in Phase Two so no missing weights are ignored).
- **Bridge Properties:** Exposes `.d_model`, `.n_layers`, `.n_heads`, `.n_kv_heads`, `.max_seq_len` and accepts legacy keyword arguments (`d_model`, `n_layers`, `n_heads`, `ff_multiplier`, `tt_rank`) in `__init__`.
### 2.2 `modeling_qtensorformer.QTensorFormerForCausalLM`
- **Inherits from:** `transformers.PreTrainedModel`
- **Submodules:**
- `model.embed_tokens` (`nn.Embedding`)
- `model.layers` (`nn.ModuleList` of `QTensorFormerDecoderLayer`, each containing `QTensorFormerAttention` with `TensorTrainLinear` projections, `QTensorFormerMLP` with `TensorTrainLinear` `gate_proj`/`up_proj`/`down_proj`, `QTensorFormerRMSNorm`, and `PIDResourceController`)
- `model.norm` (`QTensorFormerRMSNorm`)
- `lm_head` (`nn.Linear`)
- **Execution Path (`forward` & `generate`):**
- Fully compatible with Hugging Face `CausalLMOutputWithPast` and `model.generate()`.
- Uses `AdaptiveKVCache(Cache)` when `use_cache=True`.
|