fix: int8-int4 QAT + r0b0tlab data_dir loader (README.md)

4bdf5d2 verified 2 days ago

845 Bytes

license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
library_name: transformers
tags:
  - qwen3.5
  - unsloth
  - qat
  - edge
  - sft
  - int8-int4

Cortex A 0.5

General-purpose edge checkpoint: Qwen3.5-0.8B full SFT with Unsloth int8-int4 QAT (4-bit weights + 8-bit dynamic activations). Target inference footprint ≈ 450MB including the vision tower.

This repo stores:

checkpoint-* — resumable Trainer states (optimizer + fake-quant QAT model)
training/live_metrics.json — loss, MTP loss, ppl, val loss/ppl, tok/s, grad norm, lr
training/RESUME_POINTER.json — last step for the next 12h Kaggle session
qat_converted/ — real 4-bit TorchAO export (only after a completed epoch run)

Training hardware: Kaggle 2× Tesla T4, hard stop 11.5h, DDP via torchrun. QAT scheme: int8-int4.