Qxern v6 — int4 deployment bundle
Latent inter-model communication for code: Model A (Qwen2.5-Coder-1.5B) encodes a code snippet into 32 latent tokens via a trained Q-Former adapter; a frozen decoder (Qwen3.5-0.8B) answers questions about the code from that latent packet alone — no raw code text is transmitted. A tiny deterministic AST sidecar (pure ast, no LLM, microseconds) restores exact symbols (function names, arity, literals), and an adaptive router picks latent-only / latent+sidecar / text relay per question type.
Full write-up, benchmarks, training notebooks, and honest limitations: GitHub - bezvka/Qxern
🔍 Independent Audit & Reproducibility
This release has been independently audited by community researcher John6666. The audit validates the core hybrid premise, refines the theoretical understanding of identifier transport in latent space, and provides fully reproducible counterfactual probes.
Audit Artifacts:
- Public Audit README & Overview
- Core Audit Notebook (Clean/Executable)
- Core Audit Notebook (Executed T4 Snapshot)
- Cross-Receiver Bridge Appendix (Negative Result)
Key Audit Refinements: The audit confirms that identifier information is present in the latent packet (linear probe accuracy 44.8% vs 12.5% chance; candidate-free logit shifts are positive). However, this signal occupies a weak direction in latent space that is consistently dominated by the receiver's pretrained lexical priors during unconstrained generation (0/96 open-vocabulary recovery). The sidecar remains an operational necessity for reliable exact symbol transport, but the failure mode is now understood as readout/prior dominance rather than structural absence.
Contents
| Path | Description |
|---|---|
model_a_int4/ |
Qwen2.5-Coder-1.5B encoder, int4-quantized |
model_b_int4/ |
Qwen3.5-0.8B frozen decoder, int4-quantized |
adapters/qxern_adapter_32tok.pt |
Main v5 Q-Former adapter (32 latent tokens) |
adapters/qxern_adapter_struct_32tok.pt |
v6 adapter retrained with AST-exact targets |
sidecar_router.py |
Standalone AST sidecar + adaptive router (CPU, no LLM) |
qxern_config.json |
Bundle configuration |
teacher_answers.json |
Cached teacher generations (for reproduction) |
Key Results
n=30 held-out functions for AST facts, n=50 for SemSim/latency; CodeSearchNet repo-level split. Computed from raw per-example run data (results/qxern_v6_results.json in GitHub repo). Accuracy/SemSim are means, latency is p50.
| System | Func Name | Param Count | Returns | SemSim | p50 ms |
|---|---|---|---|---|---|
| Relay (strengthened text baseline) | 0.70 | 0.63 | 0.83 | 0.43 | 1222 |
| Qxern v5 (pure latents) | 0.00 | 0.13 | 0.90 | 0.24 | 202 |
| Hybrid (latents + sidecar, no retrain) | 0.87 | 0.93 | 0.90 | 0.44 | 511 |
| Hybrid Struct (latents + sidecar + struct retrain) | 0.87 | 0.73 | 0.90 | 0.42 | 420 |
Guard gates passed for both hybrids (returns ≥ 0.88, SemSim drop vs pure latents ≤ 0.01, names ≥ 0.65, params ≥ 0.60, speedup vs relay ≥ 2×). Retraining without the sidecar fails the gates (names capped at 0.20): architecture, not more training, restores exact symbols.
Paired bootstrap vs relay (10,000 resamples, 95% CI): the zero-shot hybrid is significantly better on param count (+0.30 [+0.10, +0.50]) and statistically indistinguishable on names, returns, and SemSim, at 2.39× lower p50 latency. Hybrid Struct is indistinguishable from relay on all accuracy metrics at 2.91× lower latency.
Why the Sidecar is Necessary
Contrastive probing shows code differing only in identifier names occupies a very weak direction in latent space (cos 0.985 renamed vs 0.930 different function). While independent linear probing proves identifier information is accessible within the packet (44.8% top-1 probe accuracy vs 12.5% chance), the receiver's strong pretrained lexical priors dominate this weak signal during unconstrained text generation (yielding 0/96 open-vocabulary recovery). The deterministic AST sidecar is therefore an architectural necessity for reliable exact symbolic transport, compensating for readout limitations rather than structural absence.
Usage
from huggingface_hub import snapshot_download
path = snapshot_download("aximi/qxern-v6-deploy-int4")
Then follow the reproduction guide in the GitHub repo.
Limitations
- Sample size & hardware: n=30 eval functions (50 for SemSim/latency), single RTX 5090 GPU, one seed, small models (1.5B + 0.8B) — directional evidence, not paper-grade evaluation.
- Statistical significance: The names gain (+0.17 vs relay) is not statistically significant at n=30; the significant win is param count (+0.30). Larger samples (n ≥ 300) are needed to tighten name-retrieval CIs.
- Retraining trade-off: Retraining buys speed (420 vs 511 ms) but drops param count accuracy to 0.73 vs 0.93 for the zero-shot hybrid.
- SemSim bias: Semantic similarity is computed against teacher answers, which inherently favors the relay baseline.
- Hardware-specific latency: All ms values are tied to the RTX 5090 runtime.
- Identifier transport nuance: Identifier information is present in the latent packet and decodable via probing, but cannot overcome receiver lexical priors during unconstrained generation without the sidecar. This is a readout/prior dominance issue, not a structural absence.
- Cross-receiver compatibility: Lightweight linear bridges to other model sizes (e.g., 0.8B → 2B) fail to preserve semantic controls. Receiver-native adapters or explicit fusion mechanisms are required for heterogeneous hubs.
- Multi-turn stability: Not yet formally evaluated. Fresh-state discrimination gates must pass before drift can be meaningfully measured.