2026-06-19 22:00:13 | INFO    | dapt | Environment:
{
  "timestamp": "2026-06-19T22:00:13",
  "python": "3.11.2",
  "platform": "Linux-6.1.0-49-amd64-x86_64-with-glibc2.36",
  "torch": "2.12.1+cu130",
  "transformers": "5.12.1",
  "datasets": "5.0.0",
  "cuda_available": true,
  "cuda": "13.0",
  "gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
  "gpu_total_memory_gb": 95.0,
  "gpu_capability": "12.0",
  "bf16_supported": true,
  "torch_arch_list": [
    "sm_75",
    "sm_80",
    "sm_86",
    "sm_90",
    "sm_100",
    "sm_120"
  ],
  "gpu_arch_supported_by_torch": true
}
2026-06-19 22:00:13 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:00:13 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/tokenizer_config.json "HTTP/1.1 200 OK"
2026-06-19 22:00:13 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:00:13 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:00:14 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:00:14 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:00:15 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:00:15 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large "HTTP/1.1 200 OK"
2026-06-19 22:00:15 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/main/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:00:15 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta "HTTP/1.1 200 OK"
2026-06-19 22:00:16 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:00:16 | INFO    | httpx | HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/mschonhardt/CanonBerta/mschonhardt/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:00:16 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:00:16 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/revision/b14d2f518fa6366362be9598fe4673304cfccce3 "HTTP/1.1 200 OK"
2026-06-19 22:00:16 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/.huggingface.yaml "HTTP/1.1 404 Not Found"
2026-06-19 22:00:17 | INFO    | httpx | HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=mschonhardt/CanonBerta "HTTP/1.1 501 Not Implemented"
2026-06-19 22:00:17 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/tree/b14d2f518fa6366362be9598fe4673304cfccce3/data?recursive=true&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:00:17 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/tree/b14d2f518fa6366362be9598fe4673304cfccce3?recursive=false&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:00:17 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/dataset_infos.json "HTTP/1.1 404 Not Found"
2026-06-19 22:00:17 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/data-00000-of-00003.arrow "HTTP/1.1 302 Found"
2026-06-19 22:00:28 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/data-00001-of-00003.arrow "HTTP/1.1 302 Found"
2026-06-19 22:00:42 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/data-00002-of-00003.arrow "HTTP/1.1 302 Found"
2026-06-19 22:00:59 | WARNING | dapt | Dataset mschonhardt/CanonBerta is PRE-TOKENIZED (columns=['input_ids', 'attention_mask', 'doc_id', 'category']). Skipping raw-text tokenization and DOC-SENTENCES packing.
2026-06-19 22:00:59 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/main/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:00:59 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta "HTTP/1.1 200 OK"
2026-06-19 22:01:00 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:01:00 | INFO    | httpx | HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/mschonhardt/CanonBerta/mschonhardt/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:01:00 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:01:00 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/.huggingface.yaml "HTTP/1.1 404 Not Found"
2026-06-19 22:01:01 | INFO    | httpx | HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=mschonhardt/CanonBerta "HTTP/1.1 501 Not Implemented"
2026-06-19 22:01:01 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/tree/b14d2f518fa6366362be9598fe4673304cfccce3/data?recursive=true&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:01:01 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/dataset_infos.json "HTTP/1.1 404 Not Found"
2026-06-19 22:01:01 | INFO    | dapt |   loaded pretokenized canon -> 483367 sequences (doc_id=doc_id)
2026-06-19 22:01:10 | INFO    | dapt | Pretokenized diagnostics: unk_rate=0.0034%  mean_len=495.0  bos=100% eos=100%
2026-06-19 22:01:12 | INFO    | dapt | Applying STRATIFIED document split based on column 'category'.
2026-06-19 22:01:18 | INFO    | dapt | Pretokenized split (document-level STRATIFIED group-aware) -> train=476522  val=3933  test=0
2026-06-19 22:01:56 | INFO    | dapt | Saved pretokenized datasets to .cache_packed/packed_657c56507ab70e3f34f072c7
2026-06-19 22:02:34 | INFO    | dapt | Packed dataset statistics:
{
  "train": {
    "sequences": 476522,
    "tokens": 235964123,
    "mean_seq_len": 495.2
  },
  "validation": {
    "sequences": 3933,
    "tokens": 1975457,
    "mean_seq_len": 502.3
  }
}
2026-06-19 22:02:34 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:02:34 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/model.safetensors "HTTP/1.1 302 Found"
2026-06-19 22:03:03 | INFO    | dapt | Model parameters: 560.1M
2026-06-19 22:07:54 | INFO    | dapt | Environment:
{
  "timestamp": "2026-06-19T22:07:54",
  "python": "3.11.2",
  "platform": "Linux-6.1.0-49-amd64-x86_64-with-glibc2.36",
  "torch": "2.12.1+cu130",
  "transformers": "5.12.1",
  "datasets": "5.0.0",
  "cuda_available": true,
  "cuda": "13.0",
  "gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
  "gpu_total_memory_gb": 95.0,
  "gpu_capability": "12.0",
  "bf16_supported": true,
  "torch_arch_list": [
    "sm_75",
    "sm_80",
    "sm_86",
    "sm_90",
    "sm_100",
    "sm_120"
  ],
  "gpu_arch_supported_by_torch": true
}
2026-06-19 22:07:54 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:07:54 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/tokenizer_config.json "HTTP/1.1 200 OK"
2026-06-19 22:07:54 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:07:54 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:07:54 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:07:54 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:07:56 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:07:56 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large "HTTP/1.1 200 OK"
2026-06-19 22:07:56 | INFO    | dapt | Loading cached packed datasets from .cache_packed/packed_657c56507ab70e3f34f072c7
2026-06-19 22:08:34 | INFO    | dapt | Packed dataset statistics:
{
  "validation": {
    "sequences": 3933,
    "tokens": 1975457,
    "mean_seq_len": 502.3
  },
  "train": {
    "sequences": 476522,
    "tokens": 235964123,
    "mean_seq_len": 495.2
  }
}
2026-06-19 22:08:34 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:08:34 | INFO    | dapt | Model parameters: 560.1M
2026-06-19 22:11:28 | INFO    | dapt | Environment:
{
  "timestamp": "2026-06-19T22:11:28",
  "python": "3.11.2",
  "platform": "Linux-6.1.0-49-amd64-x86_64-with-glibc2.36",
  "torch": "2.12.1+cu130",
  "transformers": "5.12.1",
  "datasets": "5.0.0",
  "cuda_available": true,
  "cuda": "13.0",
  "gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
  "gpu_total_memory_gb": 95.0,
  "gpu_capability": "12.0",
  "bf16_supported": true,
  "torch_arch_list": [
    "sm_75",
    "sm_80",
    "sm_86",
    "sm_90",
    "sm_100",
    "sm_120"
  ],
  "gpu_arch_supported_by_torch": true
}
2026-06-19 22:11:29 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:11:29 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/tokenizer_config.json "HTTP/1.1 200 OK"
2026-06-19 22:11:29 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:11:29 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:11:29 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:11:29 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:11:31 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:11:31 | INFO    | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large "HTTP/1.1 200 OK"
2026-06-19 22:11:31 | INFO    | dapt | Loading cached packed datasets from .cache_packed/packed_657c56507ab70e3f34f072c7
2026-06-19 22:11:35 | INFO    | dapt | Packed dataset statistics:
{
  "validation": {
    "sequences": 3933,
    "tokens": 1975457,
    "mean_seq_len": 502.3
  },
  "train": {
    "sequences": 476522,
    "tokens": 235964123,
    "mean_seq_len": 495.2
  }
}
2026-06-19 22:11:35 | INFO    | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:11:35 | INFO    | dapt | Model parameters: 560.1M
2026-06-20 14:05:59 | INFO    | dapt | Final evaluation:
{
  "final_val_loss": 1.0796656608581543,
  "final_val_masked_accuracy": 0.7631799587545218,
  "final_val_runtime": 18.0153,
  "final_val_samples_per_second": 218.314,
  "final_val_steps_per_second": 3.442,
  "epoch": 10.0,
  "num_input_tokens_seen": 2439792640,
  "final_val_perplexity": 2.9436951939951284
}
2026-06-20 14:05:59 | INFO    | dapt | Done. Best model + tokenizer saved to ./CanonBert