Instructions to use Premchan369/Q-TensorFormer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Premchan369/Q-TensorFormer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Premchan369/Q-TensorFormer", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Premchan369/Q-TensorFormer", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Premchan369/Q-TensorFormer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Premchan369/Q-TensorFormer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Premchan369/Q-TensorFormer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Premchan369/Q-TensorFormer

SGLang

How to use Premchan369/Q-TensorFormer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Premchan369/Q-TensorFormer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Premchan369/Q-TensorFormer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Premchan369/Q-TensorFormer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Premchan369/Q-TensorFormer",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Premchan369/Q-TensorFormer with Docker Model Runner:
```
docker model run hf.co/Premchan369/Q-TensorFormer
```

Q-TensorFormer / reports /SOURCE_API_AUDIT.md

Premchandyadav369

Complete 16-phase scientific hardening, strict checkpoint audit, empirical benchmarks, and documentation

7aefd9e 13 days ago

preview code

Raw History Blame Contribute Delete

3.69 kB

# Phase One — Source API & Execution Path Audit

## 1. Canonical Research API (`src/`)

### 1.1 `src.config.ModelConfig`
- **File:** `src/config.py`
- **Fields:**
  - `d_model: int = 128`
  - `n_heads: int = 4`
  - `n_layers: int = 2`
  - `ff_multiplier: int = 4`
  - `max_seq_len: int = 128`
  - `vocab_size: int = 10000`
  - `dropout: float = 0.1`
  - `tt_rank: int = 8`
  - `tt_min_rank: int = 2`
  - `use_tensor_ffn: bool = True`
  - `n_qubits: int = 4`
  - `n_quantum_layers: int = 2`
  - `quantum_sparsity: float = 0.3`
  - `use_quantum: bool = True`
  - `rank_alpha: float = 2.0`
  - `rank_smoothing: float = 0.9`
- **Validation:** `ModelConfig.validate()` asserts `d_model % n_heads == 0`, `1 <= tt_min_rank <= tt_rank`, `n_qubits <= 8`, and `0 <= quantum_sparsity <= 1`. `validate_model_config(config, model_class)` checks all required fields without inventing defaults.

### 1.2 `src.models.QTensorFormer`
- **File:** `src/models.py`
- **Constructor:** `QTensorFormer(config: ModelConfig, preset: str = "QTF_BALANCED")`
- **Supported Presets:** `QTF_FULL`, `QTF_BALANCED`, `QTF_LATENCY`, `QTF_MEMORY`, `QTF_ENERGY`, `QTF_EDGE`, `QTF_CLASSICAL_ONLY`.
- **Execution Path (`forward`):**
  1. Embeds `input_ids` (`nn.Embedding`) + sinusoidal `PositionalEncoding`.
  2. Iterates over `self.blocks` (`nn.ModuleList` of `HybridBlock`):
     - `HybridBlock` calls `self.info_engine(hidden_states=x)` $\rightarrow z_t \in [0, 1]^{B \times T \times 8}$.
     - `HybridBlock` calls `self.allocator(z_t, budget=budget, preset=preset, force_classical=...)` $\rightarrow$ selects `rank`, `token_ranks`, `attn_mode_idx`, `depth_mode`, `kv_precision`, `is_quantum_token`.
     - If `depth_mode == "skip"`, returns residual `x` immediately.
     - Updates `kv_cache.set_precision(kv_precision)` if `kv_cache` is supplied.
     - Runs `self.attention(normed_x, mask=mask, kv_cache=kv_cache, use_quantum=has_quantum_attn and not force_classical)`.
     - Checks optional `early_exit_head` if `enable_early_exit=True`.
     - If `depth_mode == "partial"`, skips FFN and returns `x`.
     - Sets `self.tt_ffn.set_rank(chosen_rank)` and executes `self.tt_ffn(normed_ffn_in, token_ranks=token_ranks)`.
     - If `is_quantum_token.any()` and not `force_classical`, applies `self.quantum_backend(normed_ffn_in)` masked by `is_quantum_token`.
  3. Applies final `self.ln_f` and weight-tied `self.lm_head`.

---

## 2. Canonical Hugging Face API (`configuration_qtensorformer.py` & `modeling_qtensorformer.py`)

### 2.1 `configuration_qtensorformer.QTensorFormerConfig`
- **Inherits from:** `transformers.PretrainedConfig` (`model_type = "qtensorformer"`)
- **Strict Loading Policy:** `keys_to_ignore_at_loading = []` (repaired in Phase Two so no missing weights are ignored).
- **Bridge Properties:** Exposes `.d_model`, `.n_layers`, `.n_heads`, `.n_kv_heads`, `.max_seq_len` and accepts legacy keyword arguments (`d_model`, `n_layers`, `n_heads`, `ff_multiplier`, `tt_rank`) in `__init__`.

### 2.2 `modeling_qtensorformer.QTensorFormerForCausalLM`
- **Inherits from:** `transformers.PreTrainedModel`
- **Submodules:**
  - `model.embed_tokens` (`nn.Embedding`)
  - `model.layers` (`nn.ModuleList` of `QTensorFormerDecoderLayer`, each containing `QTensorFormerAttention` with `TensorTrainLinear` projections, `QTensorFormerMLP` with `TensorTrainLinear` `gate_proj`/`up_proj`/`down_proj`, `QTensorFormerRMSNorm`, and `PIDResourceController`)
  - `model.norm` (`QTensorFormerRMSNorm`)
  - `lm_head` (`nn.Linear`)
- **Execution Path (`forward` & `generate`):**
  - Fully compatible with Hugging Face `CausalLMOutputWithPast` and `model.generate()`.
  - Uses `AdaptiveKVCache(Cache)` when `use_cache=True`.

	# Phase One — Source API & Execution Path Audit

	## 1. Canonical Research API (`src/`)

	### 1.1 `src.config.ModelConfig`
	- File: `src/config.py`
	- Fields:
	- `d_model: int = 128`
	- `n_heads: int = 4`
	- `n_layers: int = 2`
	- `ff_multiplier: int = 4`
	- `max_seq_len: int = 128`
	- `vocab_size: int = 10000`
	- `dropout: float = 0.1`
	- `tt_rank: int = 8`
	- `tt_min_rank: int = 2`
	- `use_tensor_ffn: bool = True`
	- `n_qubits: int = 4`
	- `n_quantum_layers: int = 2`
	- `quantum_sparsity: float = 0.3`
	- `use_quantum: bool = True`
	- `rank_alpha: float = 2.0`
	- `rank_smoothing: float = 0.9`
	- Validation: `ModelConfig.validate()` asserts `d_model % n_heads == 0`, `1 <= tt_min_rank <= tt_rank`, `n_qubits <= 8`, and `0 <= quantum_sparsity <= 1`. `validate_model_config(config, model_class)` checks all required fields without inventing defaults.

	### 1.2 `src.models.QTensorFormer`
	- File: `src/models.py`
	- Constructor: `QTensorFormer(config: ModelConfig, preset: str = "QTF_BALANCED")`
	- Supported Presets: `QTF_FULL`, `QTF_BALANCED`, `QTF_LATENCY`, `QTF_MEMORY`, `QTF_ENERGY`, `QTF_EDGE`, `QTF_CLASSICAL_ONLY`.
	- Execution Path (`forward`):
	1. Embeds `input_ids` (`nn.Embedding`) + sinusoidal `PositionalEncoding`.
	2. Iterates over `self.blocks` (`nn.ModuleList` of `HybridBlock`):
	- `HybridBlock` calls `self.info_engine(hidden_states=x)` $\rightarrow z_t \in [0, 1]^{B \times T \times 8}$.
	- `HybridBlock` calls `self.allocator(z_t, budget=budget, preset=preset, force_classical=...)` $\rightarrow$ selects `rank`, `token_ranks`, `attn_mode_idx`, `depth_mode`, `kv_precision`, `is_quantum_token`.
	- If `depth_mode == "skip"`, returns residual `x` immediately.
	- Updates `kv_cache.set_precision(kv_precision)` if `kv_cache` is supplied.
	- Runs `self.attention(normed_x, mask=mask, kv_cache=kv_cache, use_quantum=has_quantum_attn and not force_classical)`.
	- Checks optional `early_exit_head` if `enable_early_exit=True`.
	- If `depth_mode == "partial"`, skips FFN and returns `x`.
	- Sets `self.tt_ffn.set_rank(chosen_rank)` and executes `self.tt_ffn(normed_ffn_in, token_ranks=token_ranks)`.
	- If `is_quantum_token.any()` and not `force_classical`, applies `self.quantum_backend(normed_ffn_in)` masked by `is_quantum_token`.
	3. Applies final `self.ln_f` and weight-tied `self.lm_head`.

	---

	## 2. Canonical Hugging Face API (`configuration_qtensorformer.py` & `modeling_qtensorformer.py`)

	### 2.1 `configuration_qtensorformer.QTensorFormerConfig`
	- Inherits from: `transformers.PretrainedConfig` (`model_type = "qtensorformer"`)
	- Strict Loading Policy: `keys_to_ignore_at_loading = []` (repaired in Phase Two so no missing weights are ignored).
	- Bridge Properties: Exposes `.d_model`, `.n_layers`, `.n_heads`, `.n_kv_heads`, `.max_seq_len` and accepts legacy keyword arguments (`d_model`, `n_layers`, `n_heads`, `ff_multiplier`, `tt_rank`) in `__init__`.

	### 2.2 `modeling_qtensorformer.QTensorFormerForCausalLM`
	- Inherits from: `transformers.PreTrainedModel`
	- Submodules:
	- `model.embed_tokens` (`nn.Embedding`)
	- `model.layers` (`nn.ModuleList` of `QTensorFormerDecoderLayer`, each containing `QTensorFormerAttention` with `TensorTrainLinear` projections, `QTensorFormerMLP` with `TensorTrainLinear` `gate_proj`/`up_proj`/`down_proj`, `QTensorFormerRMSNorm`, and `PIDResourceController`)
	- `model.norm` (`QTensorFormerRMSNorm`)
	- `lm_head` (`nn.Linear`)
	- Execution Path (`forward` & `generate`):
	- Fully compatible with Hugging Face `CausalLMOutputWithPast` and `model.generate()`.
	- Uses `AdaptiveKVCache(Cache)` when `use_cache=True`.