File size: 6,511 Bytes
a33bd01
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
=== P21H V3 dispatch start 2026-05-22T19:05:07Z ===
    variant=P21H_e3 init=qwen seed=1337
    base=Qwen/Qwen2.5-1.5B
    steps=5000 bsz=2 block=512 lr=5e-5 warmup=100
    noise_sigma=0.1 lambda_mitosis=0.0
    wiki_frac=1.0 corpus_mb=72
    langs=en,ko,zh,ru,ja wiki_target_mb_per_lang=10
    cost_cap=$25 total (3 variants × ~$8)
[create] trying GPU: NVIDIA A100-SXM4-80GB
[create] cloud=SECURE gpu=NVIDIA A100-SXM4-80GB resp: {"data":{"podFindAndDeployOnDemand":{"id":"xhjxwzrpadm89y","machineId":"6vjrhbzbke27"}}}
[create] pod xhjxwzrpadm89y on NVIDIA A100-SXM4-80GB
[ssh-ready] iter 3: 195.26.233.30:46882 - port mapped AND handshake OK
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
[corpus-anima] building corpus_s101 (seed 1337 n=777000)
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
        "http://": 0
      },
      "corpus_total": 0,
      "PASS": true,
      "honest_caveat": "external-grep is a proxy over 7 surface tokens; exhaustive provenance audit requires source-attribution tracking, NOT in $0 build scope. §16 generator is anima-OWN substrate, S2 uses anima Ψ-fields only, S5 uses anima-OWN .kosmos files. NO external API calls in build_corpus_s101.py (audit-grep this file)."
    }
  }
}
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
[corpus-anima] sha=be969af481947da4693618be33a9cc67f2057a53547b6ad21abda06e7f39018b size=603318628 bytes
[corpus-multi-wiki] building 5-lang wiki corpus (en,ko,zh,ru,ja, 10 MB/lang)
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.

[notice] A new release of pip is available: 24.2 -> 26.1.1
[notice] To update, run: python -m pip install --upgrade pip
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
[p21m-build] langs=['en', 'ko', 'zh', 'ru', 'ja'] target_mb/lang=10
[p21m-build] loaded Salesforce/wikitext/wikitext-2-raw-v1 streaming OK
[p21m-build] DONE lang=en source=Salesforce/wikitext/wikitext-2-raw-v1 records=17,078 bytes=10,486,184 (10.0 MB) filtered=57
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
[p21m-build] loaded wikimedia/wikipedia/20231101.ko streaming OK
[p21m-build] DONE lang=ko source=wikimedia/wikipedia/20231101.ko records=1,004 bytes=10,569,479 (10.1 MB) filtered=2
[p21m-build] loaded wikimedia/wikipedia/20231101.zh streaming OK
[p21m-build] DONE lang=zh source=wikimedia/wikipedia/20231101.zh records=521 bytes=10,511,272 (10.0 MB) filtered=3
[p21m-build] loaded wikimedia/wikipedia/20231101.ru streaming OK
[p21m-build] DONE lang=ru source=wikimedia/wikipedia/20231101.ru records=264 bytes=10,579,139 (10.1 MB) filtered=0
[p21m-build] loaded wikimedia/wikipedia/20231101.ja streaming OK
[p21m-build] DONE lang=ja source=wikimedia/wikipedia/20231101.ja records=470 bytes=10,498,969 (10.0 MB) filtered=0
[p21m-build] CONCAT DONE: 19,337 records, 51.1 MB → /workspace/p21hr/multi_wiki_corpus.jsonl
terminate called without an active exception
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
[corpus-multi-wiki] sha=d378923c6039b6bcb571c7c85bf9cd1439d544ece0551a4f9d0564a688aa2241 size=53568645 bytes
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
NVIDIA A100-SXM4-80GB, 81920 MiB
[train] P21H V3 launch (qwen)
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
TRAIN_PID 448
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
[env-verify] [launch] PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
[train] done iter 1
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
Warning: Permanently added '[195.26.233.30]:46882' (ED25519) to the list of known hosts.
[pull] success try 1
[pull] result.json present
VERDICT= FAIL
  init= qwen
  n_strong= 0 /5  n_partial= 1   n_weak= 4   n_pure_memorize= 0
  per_lang:
    en: WEAK score=0/20 gen=20 coh=0
    ko: PARTIAL score=15/20 gen=20 coh=15
    zh: WEAK score=2/20 gen=20 coh=2
    ru: WEAK score=5/20 gen=20 coh=5
    ja: WEAK score=6/20 gen=20 coh=6
  anima_register_hits= 0 /20
  register_regress= True
  init_CE= 14.792716979980469 final_CE= 6.5491414070129395
  n_kosmos_anchors= 7
  train_wall_s= 641.2
  n_total_params= 2999735296
=== P21H V3 pipeline complete 2026-05-22T19:28:55Z ===
dispatch_p21h_v3_runpod.sh: line 304: 33977 Terminated: 15          ( sleep "$WATCHDOG_SEC"; if [ ! -s "$VDIR/result.json" ]; then
    echo "WATCHDOG FIRED $(date -u +%FT%TZ): no result.json after ${WATCHDOG_SEC}s - terminating pod ${POD_ID:-?}" > "$FAIL_MARKER"; [ -n "$POD_ID" ] && gql "{\"query\":\"mutation { podTerminate(input:{podId:\\\"$POD_ID\\\"}) }\"}" > /dev/null; pkill -P $$ 2> /dev/null;
fi )
[teardown] SAVE_POD=1 - pod xhjxwzrpadm89y RETAINED for manual inspect
=== P21H V3 dispatch end 2026-05-22T19:28:58Z rc=0 ===