TR-HASH-MOE-500M-HF — semantic-cell LoRA experiment

Failed promotion gate — research artifact only. This checkpoint is published for reproducibility and diagnosis. It is not a safe or reliable assistant and must not be used for consequential decisions.

Experimental merged checkpoint from step 1,052, trained for one full-shard epoch from the clean 500M/20B pretrained base. The LoRA weights are already merged; no separate adapter is required.

Training state

Component	Value
Base	AETHORIA-AI/TR-HASH-MOE-500M-20B
Dataset	Complexity Atlas Posttrain 32K V2
Training examples	201,983 (all visited once)
Checkpoint	1,052 / 1,052 steps (1 epoch)
Final training loss	2.1509
Matched eval loss	3.6805 -> 2.9765
LoRA	rank 32, alpha 32, dropout 0.05
Peak base LR	3e-6
Peak expert LR	7.5e-7
Weighted loss	20% reasoning / 20% natural conversation / 60% instruction
Chat format	complexity-chat-v2

The bundle contains merged BF16 weights, the 32K tokenizer, the standalone chat template, and the fixed layer-specific top-2 route tables.

Inference

pip install git+https://github.com/Complexity-ML/vllm-i64.git@main

vllm-i64 serve tr-hash-moe-500m \
  --checkpoint AETHORIA-AI/TR-HASH-MOE-500M-HF \
  --dtype bfloat16 \
  --port 7860

Limitations

This checkpoint failed the behavioral promotion gate. Full PIQA normalized accuracy was retained (0.6953 base -> 0.6964 candidate), but the fixed nine-prompt assistant panel failed all nine behaviors. Observed failures include incorrect arithmetic, prompt copying, fabricated identity details, ignored format constraints, repetitive text, incomplete factual definitions, and an unsafe password-sharing answer.

The result indicates an instruction-data coverage limitation rather than a general PIQA capability regression. Loss weighting cannot create semantic or stylistic coverage that is absent from the shard. Do not interpret the lower matched-eval loss as evidence of assistant quality.

No safety, correctness, or production-readiness claim is made. Users should retain an independently validated base or checkpoint for any real application.

Resources

• Training framework
• Custom runtime
• Post-training dataset