File size: 3,604 Bytes
2d67aa6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | Set TORCH_CUDA_ARCH_LIST to 9.0
/workspace/hanrui/idea1/specforge/modeling/draft/llama3_eagle.py:29: UserWarning: flash_attn is not found, falling back to flex_attention. Please install flash_attn if you want to use the flash attention backend.
warnings.warn(
<frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
<frozen importlib._bootstrap_external>:1241: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
============================================================
DFlash Evaluation (Multi-GPU Data Parallel)
============================================================
Target model: /workspace/models/Qwen3-8B
Draft model: /workspace/models/Qwen3-8B-DFlash-b16
Dataset: math500
Max samples: 2
Max new tokens: 64
Denoise steps: 2
Temperature: 0.0
GPUs: 1
Dtype: bfloat16
============================================================
[1/4] Loading tokenizer...
[2/4] Loading target model on 1 GPUs...
`torch_dtype` is deprecated! Use `dtype` instead!
Loading checkpoint shards: 0%| | 0/5 [00:00<?, ?it/s]
Loading checkpoint shards: 100%|██████████| 5/5 [00:00<00:00, 151.62it/s]
[3/4] Loading draft model on 1 GPUs...
Draft layers: 5
Draft block_size: 16
Draft mask_token: 151669
Draft layer_ids: [1, 9, 17, 25, 33]
[4/4] Loading evaluation data...
Using the latest cached version of the dataset since HuggingFaceH4/MATH-500 couldn't be found on the Hugging Face Hub (offline mode is enabled).
WARNING:datasets.load:Using the latest cached version of the dataset since HuggingFaceH4/MATH-500 couldn't be found on the Hugging Face Hub (offline mode is enabled).
Found the latest cached dataset configuration 'default' at /workspace/hanrui/datasets/HuggingFaceH4___math-500/default/0.0.0/6e4ed1a2a79af7d8630a6b768ec859cb5af4d3be (last modified on Tue Mar 17 13:17:15 2026).
WARNING:datasets.packaged_modules.cache.cache:Found the latest cached dataset configuration 'default' at /workspace/hanrui/datasets/HuggingFaceH4___math-500/default/0.0.0/6e4ed1a2a79af7d8630a6b768ec859cb5af4d3be (last modified on Tue Mar 17 13:17:15 2026).
Total prompts: 2, ~2 per GPU
============================================================
Running evaluation...
============================================================
[GPU 0] Sample 1/2 | tokens=64 | tau=1.56 | time=2.2s | <think> Okay, so I need to convert the rectangular coordinates (0, 3) to polar c...
[GPU 0] Sample 2/2 | tokens=64 | tau=1.76 | time=1.4s | <think> Okay, so I need to find a way to express the double sum $\sum_{j = 1}^\i...
============================================================
RESULTS SUMMARY
============================================================
Denoise steps: 2
GPUs used: 1
Samples evaluated: 2
Total blocks: 78
Total generated tokens: 128
Total GPU-time: 3.58s
Wall-clock time (approx): 2.15s
---
Avg acceptance length (tau): 1.65
Median acceptance length: 1.0
Per-sample avg tau: ['1.56', '1.76']
Min per-sample tau: 1.56
Max per-sample tau: 1.76
============================================================
Results saved to /workspace/hanrui/idea1/results/dflash_eval/math500_steps2.json
|