============================================================
CogNet-1B Ultra-Fast Training V2 — MAXIMUM SPEED
============================================================
Device: cuda:0
Distributed: False (world_size=1)
Model: 350m
BF16: True
Compile: False
Compile step: False
CUDA prefetch: False
Seq warmup: False
Async checkpoint: False
8-bit optimizer: True
TF32 enabled: True
HF repo: thefinalboss/CogNet-1B
HF token: SET
============================================================
Loaded tokenizer from /root/cognet-1b/tokenizer_v3.json (vocab=136)
Skipping data preparation (--skip-data-prep)
Loading data from: /root/cognet-1b/data_1b/train_merged.pt

Building CogNet-350M (optimized)...
Total parameters: 304,232,960 (0.30B)
8-bit AdamW (bitsandbytes) enabled — 50% less VRAM for optimizer states
Mixed precision: BF16

Starting: step 0 -> 100000
Batch=4 x GradAccum=8 x GPUs=1 = Effective 32
SeqLen=512, LR=1e-05-0.0003
TF32=ON, Gradient checkpointing=True
Graceful shutdown: SIGTERM/SIGINT will save checkpoint

[BENCH] Un benchmark de 10 steps va mesurer la vitesse réelle...

============================================================
  BENCHMARK — Mesure des performances réelles
============================================================
  Warmup: 3 steps
  Mesure: 10 steps
  Config: batch=4, grad_accum=8, seq_len=512
  Warmup terminé — début de la mesure...

  ╔══════════════════════════════════════════════════════╗
  ║           RÉSULTATS DU BENCHMARK                     ║
  ╠══════════════════════════════════════════════════════╣
  ║      0.10 steps/sec (optimizer steps)   ║
  ║      1581 tokens/sec                    ║
  ║    103.62 sec pour 10 steps         ║
  ║       3.2 GB VRAM utilisé                ║
  ╠══════════════════════════════════════════════════════╣
  ║  Temps estimé pour 100,000 steps restants    ║
  ║  ~ 287.8 heures (12.0 jours)                  ║
  ╚══════════════════════════════════════════════════════╝
============================================================

  Benchmark sauvé: /root/cognet-1b/checkpoints_1b/benchmark_results.json
Step       0/100000 | Loss: 3.3116 | PPL: 27.4 | LR: 0.00e+00 | Grad: 2.75 | VRAM: 3.2GB | 1378 tok/s | 0.1 step/s | ETA: 12.0j
Step      10/100000 | Loss: 3.2792 | PPL: 26.6 | LR: 1.50e-06 | Grad: 2.48 | VRAM: 3.2GB | 1583 tok/s | 0.1 step/s | ETA: 12.0j
Step      20/100000 | Loss: 3.2696 | PPL: 26.3 | LR: 3.00e-06 | Grad: 1.62 | VRAM: 3.2GB | 1585 tok/s | 0.1 step/s | ETA: 12.0j
Step      30/100000 | Loss: 3.2555 | PPL: 25.9 | LR: 4.50e-06 | Grad: 0.64 | VRAM: 3.2GB | 1568 tok/s | 0.1 step/s | ETA: 12.0j
Step      40/100000 | Loss: 3.2414 | PPL: 25.6 | LR: 6.00e-06 | Grad: 0.81 | VRAM: 3.2GB | 1590 tok/s | 0.1 step/s | ETA: 12.0j