Text Generation
Transformers
Safetensors
PyTorch
English
qtensorformer
q-tensorformer
tensor-train
tensor-networks
adaptive-computation
kv-cache
kv-cache-quantization
attention-sinks
pid-control
edge-ai
edge-inference
on-device-ai
model-compression
grouped-query-attention
rotary-position-embeddings
custom_code
Premchandyadav369
feat(hf): turnkey custom model with TensorTrainLinear, AdaptiveKVCache, PID controller, NVML profiler, OpenAI server, and drop-in compressor
15a4381 | # Checkpoint-to-Model Diagnostic Matching Matrix | |
| ## 1. Executive Summary | |
| This report delivers a forensic audit and diagnostic loading matrix of the checkpoint file `qtensorformer_small.pt` against all candidate model architectures in the repository. Under the scientific mandate `strict=True`, zero candidates achieve checkpoint compatibility. | |
| --- | |
| ## 2. Status Distinctions | |
| ### PROVEN FACT | |
| 1. `qtensorformer_small.pt` is a serialized PyTorch checkpoint dictionary containing 6 top-level keys: | |
| - `model_state_dict` (113 tensor weights, total 1,347,747 parameters) | |
| - `config` (`{'vocab_size': 10000, 'hidden_dim': 128, 'intermediate_size': 256, 'n_heads': 4, 'n_layers': 3, 'seq_len': 64, 'tt_rank': 4, 'n_qubits': 4, 'quantum_layers': 2, 'quantum_ratio': 0.3, 'use_quantum_attention': True, 'use_adaptive_rank': True, 'use_selective_routing': True, 'r_min': 2, 'r_max': 12, 'alpha': 1.0}`) | |
| - `params`: `1347747` | |
| - `baseline_params`: `16146432` | |
| - `compression_ratio`: `11.980313812607262` | |
| - `version`: `"1.0.0"` | |
| 2. The state dictionary relies on TensorLy tensor decomposition layers: | |
| - `blocks.i.ffn.layer.weight.factors.factor_0..5` | |
| - `blocks.i.ffn2.layer.weight.factors.factor_0..5` | |
| - `lm_head.layer.weight.factors.factor_0..5` | |
| - `embeddings.quantum_enhancer.project_up.weight` | |
| - `blocks.i.attention.quantum_encoder.project_up.weight` | |
| - `blocks.i.attention.alpha_gate.0.weight` | |
| 3. Across all candidate model implementations in the repository: | |
| - Common keys with `qtensorformer_small.pt`: **0** | |
| - Unexpected keys from `qtensorformer_small.pt`: **113** (100% of checkpoint keys) | |
| - Strict load result: **FAIL** on 100% of candidate models. | |
| 4. Git commit history proves `qtensorformer_small.pt` was committed in commit `82fd973cb44f62a21ac0fc0fdfa1b30a8c598754` on May 6, 2026 by `ml-intern`, originating from an external `/app/qtensorformer` container package. | |
| ### INFERENCE | |
| 1. The model architecture that produced `qtensorformer_small.pt` was developed in an ephemeral Docker environment and was never added to the git repository source tree. | |
| 2. Any attempt to load `qtensorformer_small.pt` into any repository class would require `strict=False`, which would silently discard all 113 trained weight tensors and run random uninitialized weights, violating scientific integrity. | |
| ### UNRESOLVED | |
| - The original source repository of the `/app/qtensorformer` package that created `qtensorformer_small.pt` cannot be retrieved from git history. | |
| --- | |
| ## 3. Checkpoint Forensic Specification | |
| | Attribute | Checkpoint Value | | |
| | :--- | :--- | | |
| | **File Name** | `qtensorformer_small.pt` | | |
| | **File Size** | 5,432,305 bytes | | |
| | **Object Type** | `dict` | | |
| | **Top-Level Keys** | `['model_state_dict', 'config', 'params', 'baseline_params', 'compression_ratio', 'version']` | | |
| | **Parameter Count** | 1,347,747 | | |
| | **Tensor Count** | 113 | | |
| | **Vocabulary Dimension** | 10,000 (`embeddings.token_embedding.weight`: `[10000, 128]`) | | |
| | **Hidden Dimension ($d_{\text{model}}$)** | 128 | | |
| | **Number of Layers** | 3 (`blocks.0`, `blocks.1`, `blocks.2`) | | |
| | **Attention Projection** | `q_proj`: `[4, 32]`, `classical_proj`: `[128, 128]` | | |
| | **FFN Factorization** | TensorLy TT Factorized (`factor_0..5` for `ffn` and `ffn2`) | | |
| | **Language Model Head** | TensorLy TT Factorized (`lm_head.layer.weight.factors.factor_0..5`) | | |
| | **Positional Encoding Length** | 64 (`embeddings.position_embedding.weight`: `[64, 128]`) | | |
| --- | |
| ## 4. Candidate Model Load Matrix | |
| | Candidate Model | Config Class Used | Config Construction | Model Construction | `strict=True` Load | Common Keys | Missing Keys | Unexpected Keys | Shape Mismatches | Param Count Match | | |
| | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | | |
| | **A1: `q_tensor_former.QTensorFormer` (Default)** | `q_tensor_former.CFG()` | **PASS** | **PASS** | **FAIL** | 0 | 69 | 113 | N/A (0 key overlap) | 115,428 vs 1,347,747 (No) | | |
| | **A2: `q_tensor_former.QTensorFormer` (Matched Dims)** | `q_tensor_former.CFG(vocab_size=10000, d_model=128, n_layers=3, max_seq_len=64)` | **PASS** | **PASS** | **FAIL** | 0 | 101 | 113 | N/A (0 key overlap) | 1,347,747 vs 1,347,747 (No) | | |
| | **B1: `src.models.QTensorFormer` (Default)** | `src.config.ModelConfig()` | **PASS** | **PASS** | **FAIL** | 0 | 125 | 113 | N/A (0 key overlap) | 1,460,860 vs 1,347,747 (No) | | |
| | **B2: `src.models.QTensorFormer` (Matched Dims)** | `src.config.ModelConfig(vocab_size=10000, d_model=128, n_layers=3, max_seq_len=64)` | **PASS** | **PASS** | **FAIL** | 0 | 185 | 113 | N/A (0 key overlap) | 1,526,428 vs 1,347,747 (No) | | |
| | **C1: `q_tensor_former_v2.QTensorFormer` (Default)** | `q_tensor_former_v2.Config()` | **PASS** | **PASS** | **FAIL** | 0 | 73 | 113 | N/A (0 key overlap) | 1,476,004 vs 1,347,747 (No) | | |
| | **C2: `q_tensor_former_v2.QTensorFormer` (Matched Dims)** | `q_tensor_former_v2.Config(vocab_size=10000, d_model=128, n_layers=3, max_seq_len=64)` | **PASS** | **PASS** | **FAIL** | 0 | 107 | 113 | N/A (0 key overlap) | 1,542,044 vs 1,347,747 (No) | | |
| | **D1: `src.hf_model.QTensorFormerForCausalLM`** | `src.hf_model.QTensorFormerConfig()` | **PASS** | **PASS** | **FAIL** | 0 | 125 | 113 | N/A (0 key overlap) | 1,460,860 vs 1,347,747 (No) | | |
| --- | |
| ## 5. Failure Diagnosis & Scientific Conclusion | |
| 1. **Exact Error Raised**: | |
| ```text | |
| RuntimeError: Error(s) in loading state_dict for QTensorFormer: | |
| Missing key(s) in state_dict: ... | |
| Unexpected key(s) in state_dict: "embeddings.token_embedding.weight", ... | |
| ``` | |
| 2. **Scientific Invariant**: | |
| Under the rule `NO RESULT > INVALID RESULT`, loading weights with `strict=False` or synthesizing dummy modules is strictly forbidden. | |
| 3. **Verdict**: | |
| `CHECKPOINT_STRICT_LOAD: FAIL` for all candidate models in the repository. | |