File size: 7,088 Bytes
1514616 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
Markdown# Performance Benchmarks and Test Results
**Invention Title:** Method for Ternary-Quantized Transformer Optimization with Buffered Routing Embedding and SWA Attention
**Inventor:** Konstantin Vladimirovich Grabko
**Test Date:** December 2025
**Hardware Tested:** AMD MI50 (32 GB HBM2), custom cooling
**Confidentiality Notice:** Internal test data – proprietary and not for publication.
-
- JiRackPyTorch_BitNet_class_3b.py
-
# ROCm System Management Interface
---
## Concise Info
| GPU | Temp (DieEdge) | AvgPwr | SCLK | MCLK | Fan | Perf | PwrCap | VRAM% | GPU% |
|-------|----------------|--------|------------|-----------|--------|--------|---------|-------|------|
| GPU[0] | 46.0c | N/A | 1725Mhz | 1000Mhz | 17.65% | auto | 225.0W | 59% | 100% |
- **GPU[0]:** `get_power_avg` is not supported on the given system.
---
### End of ROCm SMI Log
-
- Step 4270 | Loss: 9.1875 | VRAM: 14.85GB | 11.9 t/s
- Step 4275 | Loss: 9.0000 | VRAM: 14.84GB | 11.9 t/s
- Step 4280 | Loss: 9.3125 | VRAM: 14.84GB | 11.9 t/s
- Step 4285 | Loss: 10.6875 | VRAM: 14.85GB | 11.9 t/s
- Step 4290 | Loss: 10.1250 | VRAM: 14.84GB | 11.9 t/s
- Step 4295 | Loss: 10.4375 | VRAM: 14.84GB | 11.9 t/s
- Step 4300 | Loss: 10.6250 | VRAM: 14.84GB | 11.9 t/s
- Step 4305 | Loss: 10.3125 | VRAM: 14.84GB | 11.9 t/s
- Step 4310 | Loss: 10.3750 | VRAM: 14.84GB | 11.9 t/s
- Step 4315 | Loss: 10.8750 | VRAM: 14.85GB | 11.9 t/s
- Step 4320 | Loss: 10.1875 | VRAM: 14.84GB | 11.9 t/s
- Step 4325 | Loss: 9.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4330 | Loss: 9.7500 | VRAM: 14.84GB | 11.9 t/s
- Step 4335 | Loss: 8.6875 | VRAM: 14.84GB | 11.9 t/s
- Step 4340 | Loss: 9.1875 | VRAM: 14.85GB | 11.9 t/s
- Step 4345 | Loss: 10.3125 | VRAM: 14.84GB | 11.9 t/s
- Step 4350 | Loss: 10.9375 | VRAM: 14.84GB | 11.9 t/s
- Step 4355 | Loss: 10.5000 | VRAM: 14.84GB | 11.9 t/s
- Step 4360 | Loss: 9.1250 | VRAM: 14.84GB | 11.9 t/s
- Step 4365 | Loss: 10.4375 | VRAM: 14.84GB | 11.9 t/s
- Step 4370 | Loss: 9.3125 | VRAM: 14.84GB | 11.9 t/s
- Step 4375 | Loss: 10.0625 | VRAM: 14.84GB | 11.9 t/s
- Step 4380 | Loss: 10.3125 | VRAM: 14.85GB | 11.9 t/s
- Step 4385 | Loss: 10.5000 | VRAM: 14.84GB | 11.9 t/s
- Step 4390 | Loss: 9.5000 | VRAM: 14.84GB | 11.9 t/s
- Step 4395 | Loss: 10.9375 | VRAM: 14.85GB | 11.9 t/s
- Step 4400 | Loss: 7.5312 | VRAM: 14.85GB | 11.9 t/s
- Step 4405 | Loss: 9.7500 | VRAM: 14.84GB | 11.9 t/s
- Step 4410 | Loss: 10.7500 | VRAM: 14.85GB | 11.9 t/s
- Step 4415 | Loss: 9.1875 | VRAM: 14.84GB | 11.9 t/s
- Step 4420 | Loss: 11.0000 | VRAM: 14.84GB | 11.9 t/s
- Step 4425 | Loss: 9.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4430 | Loss: 10.3750 | VRAM: 14.84GB | 11.9 t/s
- Step 4435 | Loss: 10.8750 | VRAM: 14.84GB | 11.9 t/s
- Step 4440 | Loss: 10.9375 | VRAM: 14.85GB | 11.9 t/s
- Step 4445 | Loss: 10.0000 | VRAM: 14.84GB | 11.9 t/s
- Step 4450 | Loss: 9.1875 | VRAM: 14.84GB | 11.9 t/s
- Step 4455 | Loss: 9.6875 | VRAM: 14.84GB | 11.9 t/s
- Step 4460 | Loss: 10.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4465 | Loss: 10.4375 | VRAM: 14.85GB | 11.9 t/s
- Step 4470 | Loss: 10.4375 | VRAM: 14.84GB | 11.9 t/s
- Step 4475 | Loss: 9.5000 | VRAM: 14.84GB | 11.9 t/s
- Step 4480 | Loss: 9.8750 | VRAM: 14.85GB | 11.9 t/s
- Step 4485 | Loss: 8.1875 | VRAM: 14.85GB | 11.9 t/s
- Step 4490 | Loss: 11.1875 | VRAM: 14.84GB | 11.9 t/s
- Step 4495 | Loss: 10.6875 | VRAM: 14.84GB | 11.9 t/s
- Step 4500 | Loss: 10.6875 | VRAM: 14.84GB | 11.9 t/s
>>> SAVING: Checkpoint to ./models/ternary_3b_checkpoint_- Step_4500...
>>> CLEANUP: Removing old checkpoint ./models/ternary_3b_checkpoint_- Step_3000
- Step 4505 | Loss: 10.5000 | VRAM: 14.85GB | 11.9 t/s
- Step 4510 | Loss: 9.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4515 | Loss: 9.8750 | VRAM: 14.84GB | 11.9 t/s
- Step 4520 | Loss: 9.6875 | VRAM: 14.84GB | 11.9 t/s
- Step 4525 | Loss: 10.6250 | VRAM: 14.84GB | 11.9 t/s
- Step 4530 | Loss: 9.3750 | VRAM: 14.85GB | 11.9 t/s
- Step 4535 | Loss: 9.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4540 | Loss: 10.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4545 | Loss: 11.0000 | VRAM: 14.84GB | 11.9 t/s
- Step 4550 | Loss: 10.0000 | VRAM: 14.85GB | 11.9 t/s
- Step 4555 | Loss: 9.9375 | VRAM: 14.84GB | 11.9 t/s
- Step 4560 | Loss: 11.0625 | VRAM: 14.85GB | 11.9 t/s
- Step 4565 | Loss: 9.3125 | VRAM: 14.85GB | 11.9 t/s
- Step 4570 | Loss: 9.3750 | VRAM: 14.84GB | 11.9 t/s
- Step 4575 | Loss: 10.8125 | VRAM: 14.85GB | 11.9 t/s
- Step 4580 | Loss: 10.7500 | VRAM: 14.85GB | 11.9 t/s
- Step 4585 | Loss: 9.3750 | VRAM: 14.85GB | 11.9 t/s
- Step 4590 | Loss: 10.7500 | VRAM: 14.84GB | 11.9 t/s
- Step 4595 | Loss: 9.3125 | VRAM: 14.84GB | 11.9 t/s
- Step 4600 | Loss: 10.6250 | VRAM: 14.84GB | 11.9 t/s
- Step 4605 | Loss: 10.4375 | VRAM: 14.84GB | 11.9 t/s
- Step 4610 | Loss: 9.8750 | VRAM: 14.85GB | 11.9 t/s
- Step 4615 | Loss: 10.6875 | VRAM: 14.84GB | 11.9 t/s
- Step 4620 | Loss: 10.0625 | VRAM: 14.85GB | 11.9 t/s
- Step 4625 | Loss: 10.6250 | VRAM: 14.84GB | 11.9 t/s
- Step 4630 | Loss: 10.7500 | VRAM: 14.85GB | 11.9 t/s
- Step 4635 | Loss: 10.5000 | VRAM: 14.84GB | 11.9 t/s
- Step 4640 | Loss: 10.0000 | VRAM: 14.85GB | 11.9 t/s
- Step 4645 | Loss: 10.9375 | VRAM: 14.84GB | 11.9 t/s
- Step 4650 | Loss: 10.6250 | VRAM: 14.84GB | 11.9 t/s
- Step 4655 | Loss: 9.6875 | VRAM: 14.85GB | 11.9 t/s
- Step 4660 | Loss: 9.5000 | VRAM: 14.85GB | 11.9 t/s
- Step 4665 | Loss: 10.8750 | VRAM: 14.84GB | 11.9 t/s
- Step 4670 | Loss: 11.0625 | VRAM: 14.84GB | 11.9 t/s
- Step 4675 | Loss: 10.8750 | VRAM: 14.84GB | 11.9 t/s
- Step 4680 | Loss: 9.2500 | VRAM: 14.84GB | 11.9 t/s
- Step 4685 | Loss: 9.0000 | VRAM: 14.85GB | 11.9 t/s
- Step 4690 | Loss: 10.5625 | VRAM: 14.84GB | 11.9 t/s
- Step 4695 | Loss: 10.1875 | VRAM: 14.84GB | 11.9 t/s
- Step 4700 | Loss: 8.6875 | VRAM: 14.85GB | 11.9 t/s
- Step 4705 | Loss: 10.7500 | VRAM: 14.85GB | 11.9 t/s
- Step 4710 | Loss: 9.2500 | VRAM: 14.84GB | 11.9 t/s
- Step 4715 | Loss: 8.3750 | VRAM: 14.84GB | 11.9 t/s
- Step 4720 | Loss: 9.9375 | VRAM: 14.84GB | 11.9 t/s
- Step 4725 | Loss: 10.8125 | VRAM: 14.84GB | 11.9 t/s
- Step 4730 | Loss: 9.8125 | VRAM: 14.84GB | 11.9 t/s
- Step 4735 | Loss: 9.2500 | VRAM: 14.84GB | 11.9 t/s
- Step 4740 | Loss: 10.6875 | VRAM: 14.84GB | 11.9 t/s
- Step 4745 | Loss: 10.1250 | VRAM: 14.84GB | 11.9 t/s
- Step 4750 | Loss: 10.6250 | VRAM: 14.84GB | 11.9 t/s
- Step 4755 | Loss: 10.8125 | VRAM: 14.85GB | 11.9 t/s
- Step 4760 | Loss: 10.4375 | VRAM: 14.85GB | 11.9 t/s
- Step 4765 | Loss: 10.3125 | VRAM: 14.84GB | 11.9 t/s
- Step 4770 | Loss: 9.5000 | VRAM: 14.84GB | 11.9 t/s
- Step 4775 | Loss: 10.0000 | VRAM: 14.85GB | 11.9 t/s
- Step 4780 | Loss: 9.5000 | VRAM: 14.85GB | 11.9 t/s
- Step 4785 | Loss: 9.0625 | VRAM: 14.84GB | 11.9 t/s
|