Hugging Face's logo

mschonhardt
/
CanonBerta-v1

Safetensors
xlm-roberta
Model card Files
xet
 Community
CanonBerta-v1 / train.log
mschonhardt's picture
mschonhardt
Add files using upload-large-folder tool
fdd1815 verified
Raw
History Blame Contribute Delete
13.3 kB
2026-06-19 22:00:13 | INFO | dapt | Environment:
{
 "timestamp": "2026-06-19T22:00:13",
 "python": "3.11.2",
 "platform": "Linux-6.1.0-49-amd64-x86_64-with-glibc2.36",
 "torch": "2.12.1+cu130",
 "transformers": "5.12.1",
 "datasets": "5.0.0",
 "cuda_available": true,
 "cuda": "13.0",
 "gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
 "gpu_total_memory_gb": 95.0,
 "gpu_capability": "12.0",
 "bf16_supported": true,
 "torch_arch_list": [
 "sm_75",
 "sm_80",
 "sm_86",
 "sm_90",
 "sm_100",
 "sm_120"
 ],
 "gpu_arch_supported_by_torch": true
}
2026-06-19 22:00:13 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:00:13 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/tokenizer_config.json "HTTP/1.1 200 OK"
2026-06-19 22:00:13 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:00:13 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:00:14 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:00:14 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:00:15 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:00:15 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large "HTTP/1.1 200 OK"
2026-06-19 22:00:15 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/main/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:00:15 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta "HTTP/1.1 200 OK"
2026-06-19 22:00:16 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:00:16 | INFO | httpx | HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/mschonhardt/CanonBerta/mschonhardt/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:00:16 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:00:16 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/revision/b14d2f518fa6366362be9598fe4673304cfccce3 "HTTP/1.1 200 OK"
2026-06-19 22:00:16 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/.huggingface.yaml "HTTP/1.1 404 Not Found"
2026-06-19 22:00:17 | INFO | httpx | HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=mschonhardt/CanonBerta "HTTP/1.1 501 Not Implemented"
2026-06-19 22:00:17 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/tree/b14d2f518fa6366362be9598fe4673304cfccce3/data?recursive=true&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:00:17 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/tree/b14d2f518fa6366362be9598fe4673304cfccce3?recursive=false&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:00:17 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/dataset_infos.json "HTTP/1.1 404 Not Found"
2026-06-19 22:00:17 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/data-00000-of-00003.arrow "HTTP/1.1 302 Found"
2026-06-19 22:00:28 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/data-00001-of-00003.arrow "HTTP/1.1 302 Found"
2026-06-19 22:00:42 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/data-00002-of-00003.arrow "HTTP/1.1 302 Found"
2026-06-19 22:00:59 | WARNING | dapt | Dataset mschonhardt/CanonBerta is PRE-TOKENIZED (columns=['input_ids', 'attention_mask', 'doc_id', 'category']). Skipping raw-text tokenization and DOC-SENTENCES packing.
2026-06-19 22:00:59 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/main/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:00:59 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta "HTTP/1.1 200 OK"
2026-06-19 22:01:00 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:01:00 | INFO | httpx | HTTP Request: HEAD https://s3.amazonaws.com/datasets.huggingface.co/datasets/datasets/mschonhardt/CanonBerta/mschonhardt/CanonBerta.py "HTTP/1.1 404 Not Found"
2026-06-19 22:01:00 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/README.md "HTTP/1.1 404 Not Found"
2026-06-19 22:01:00 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/.huggingface.yaml "HTTP/1.1 404 Not Found"
2026-06-19 22:01:01 | INFO | httpx | HTTP Request: GET https://datasets-server.huggingface.co/info?dataset=mschonhardt/CanonBerta "HTTP/1.1 501 Not Implemented"
2026-06-19 22:01:01 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/datasets/mschonhardt/CanonBerta/tree/b14d2f518fa6366362be9598fe4673304cfccce3/data?recursive=true&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:01:01 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/datasets/mschonhardt/CanonBerta/resolve/b14d2f518fa6366362be9598fe4673304cfccce3/dataset_infos.json "HTTP/1.1 404 Not Found"
2026-06-19 22:01:01 | INFO | dapt | loaded pretokenized canon -> 483367 sequences (doc_id=doc_id)
2026-06-19 22:01:10 | INFO | dapt | Pretokenized diagnostics: unk_rate=0.0034% mean_len=495.0 bos=100% eos=100%
2026-06-19 22:01:12 | INFO | dapt | Applying STRATIFIED document split based on column 'category'.
2026-06-19 22:01:18 | INFO | dapt | Pretokenized split (document-level STRATIFIED group-aware) -> train=476522 val=3933 test=0
2026-06-19 22:01:56 | INFO | dapt | Saved pretokenized datasets to .cache_packed/packed_657c56507ab70e3f34f072c7
2026-06-19 22:02:34 | INFO | dapt | Packed dataset statistics:
{
 "train": {
 "sequences": 476522,
 "tokens": 235964123,
 "mean_seq_len": 495.2
 },
 "validation": {
 "sequences": 3933,
 "tokens": 1975457,
 "mean_seq_len": 502.3
 }
}
2026-06-19 22:02:34 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:02:34 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/model.safetensors "HTTP/1.1 302 Found"
2026-06-19 22:03:03 | INFO | dapt | Model parameters: 560.1M
2026-06-19 22:07:54 | INFO | dapt | Environment:
{
 "timestamp": "2026-06-19T22:07:54",
 "python": "3.11.2",
 "platform": "Linux-6.1.0-49-amd64-x86_64-with-glibc2.36",
 "torch": "2.12.1+cu130",
 "transformers": "5.12.1",
 "datasets": "5.0.0",
 "cuda_available": true,
 "cuda": "13.0",
 "gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
 "gpu_total_memory_gb": 95.0,
 "gpu_capability": "12.0",
 "bf16_supported": true,
 "torch_arch_list": [
 "sm_75",
 "sm_80",
 "sm_86",
 "sm_90",
 "sm_100",
 "sm_120"
 ],
 "gpu_arch_supported_by_torch": true
}
2026-06-19 22:07:54 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:07:54 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/tokenizer_config.json "HTTP/1.1 200 OK"
2026-06-19 22:07:54 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:07:54 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:07:54 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:07:54 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:07:56 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:07:56 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large "HTTP/1.1 200 OK"
2026-06-19 22:07:56 | INFO | dapt | Loading cached packed datasets from .cache_packed/packed_657c56507ab70e3f34f072c7
2026-06-19 22:08:34 | INFO | dapt | Packed dataset statistics:
{
 "validation": {
 "sequences": 3933,
 "tokens": 1975457,
 "mean_seq_len": 502.3
 },
 "train": {
 "sequences": 476522,
 "tokens": 235964123,
 "mean_seq_len": 495.2
 }
}
2026-06-19 22:08:34 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:08:34 | INFO | dapt | Model parameters: 560.1M
2026-06-19 22:11:28 | INFO | dapt | Environment:
{
 "timestamp": "2026-06-19T22:11:28",
 "python": "3.11.2",
 "platform": "Linux-6.1.0-49-amd64-x86_64-with-glibc2.36",
 "torch": "2.12.1+cu130",
 "transformers": "5.12.1",
 "datasets": "5.0.0",
 "cuda_available": true,
 "cuda": "13.0",
 "gpu_name": "NVIDIA RTX PRO 6000 Blackwell Server Edition",
 "gpu_total_memory_gb": 95.0,
 "gpu_capability": "12.0",
 "bf16_supported": true,
 "torch_arch_list": [
 "sm_75",
 "sm_80",
 "sm_86",
 "sm_90",
 "sm_100",
 "sm_120"
 ],
 "gpu_arch_supported_by_torch": true
}
2026-06-19 22:11:29 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:11:29 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/tokenizer_config.json "HTTP/1.1 200 OK"
2026-06-19 22:11:29 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:11:29 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main/additional_chat_templates?recursive=false&expand=false "HTTP/1.1 404 Not Found"
2026-06-19 22:11:29 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:11:29 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large/tree/main?recursive=true&expand=false "HTTP/1.1 200 OK"
2026-06-19 22:11:31 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/xlm-roberta-large "HTTP/1.1 307 Temporary Redirect"
2026-06-19 22:11:31 | INFO | httpx | HTTP Request: GET https://huggingface.co/api/models/FacebookAI/xlm-roberta-large "HTTP/1.1 200 OK"
2026-06-19 22:11:31 | INFO | dapt | Loading cached packed datasets from .cache_packed/packed_657c56507ab70e3f34f072c7
2026-06-19 22:11:35 | INFO | dapt | Packed dataset statistics:
{
 "validation": {
 "sequences": 3933,
 "tokens": 1975457,
 "mean_seq_len": 502.3
 },
 "train": {
 "sequences": 476522,
 "tokens": 235964123,
 "mean_seq_len": 495.2
 }
}
2026-06-19 22:11:35 | INFO | httpx | HTTP Request: HEAD https://huggingface.co/xlm-roberta-large/resolve/main/config.json "HTTP/1.1 200 OK"
2026-06-19 22:11:35 | INFO | dapt | Model parameters: 560.1M
2026-06-20 14:05:59 | INFO | dapt | Final evaluation:
{
 "final_val_loss": 1.0796656608581543,
 "final_val_masked_accuracy": 0.7631799587545218,
 "final_val_runtime": 18.0153,
 "final_val_samples_per_second": 218.314,
 "final_val_steps_per_second": 3.442,
 "epoch": 10.0,
 "num_input_tokens_seen": 2439792640,
 "final_val_perplexity": 2.9436951939951284
}
2026-06-20 14:05:59 | INFO | dapt | Done. Best model + tokenizer saved to ./CanonBert