DFlash polishing checkpoint weights update

#4
Files changed (1) hide show
  1. README.md +7 -6
README.md CHANGED
@@ -20,13 +20,14 @@ base_model:
20
 
21
  <br>
22
 
23
- # Laguna-S-2.1-DFlash
24
 
25
- DFlash speculator (draft model) for
26
  [Laguna S 2.1](https://huggingface.co/poolside/Laguna-S-2.1), bf16.
27
 
28
  - Architecture: `DFlashLagunaForCausalLM` (6 sliding-attention layers, block_size 16).
29
  - Shares token embedding + lm_head with the target; `draft_vocab_size == vocab_size` (no d2t/t2d).
 
30
  - Loads under vLLM (native `laguna_dflash`) and TRT-LLM (pytorch DFlash backend) as the draft model in a speculative config.
31
 
32
  ## Usage
@@ -50,7 +51,7 @@ python -m sglang.launch_server --model-path poolside/Laguna-S-2.1 \
50
 
51
  ### llama.cpp (GGUF)
52
 
53
- A llama.cpp conversion of this draft model (`laguna-s-2.1-DFlash-BF16.gguf`) is
54
  published in
55
  [poolside/Laguna-S-2.1-GGUF](https://huggingface.co/poolside/Laguna-S-2.1-GGUF)
56
  alongside the target GGUFs. It embeds the target tokenizer and the DFlash metadata
@@ -64,16 +65,16 @@ cd llama.cpp && cmake -B build && cmake --build build -j
64
  ./build/bin/llama-server \
65
  -m laguna-s-2.1-Q4_K_M.gguf \
66
  -md laguna-s-2.1-DFlash-BF16.gguf \
67
- --spec-type draft-dflash --spec-draft-n-max 7 \
68
  -fa on --jinja --port 8000
69
  ```
70
 
71
  `--spec-draft-n-max` is clamped to the trained block size (15 draft tokens + 1).
72
 
73
  > [!NOTE]
74
- > Requires Poolside's llama.cpp fork, branch
75
  > [`laguna`](https://github.com/poolsideai/llama.cpp/tree/laguna). Upstream
76
  > llama.cpp ships the generic DFlash framework but not the Laguna decoder
77
- > contract this draft model needs, and upstream PR
78
  > [ggml-org/llama.cpp#25165](https://github.com/ggml-org/llama.cpp/pull/25165)
79
  > covers the target architecture only.
 
20
 
21
  <br>
22
 
23
+ # Laguna-S-2.1-DFlash-polishing
24
 
25
+ DFlash speculator (drafter) for
26
  [Laguna S 2.1](https://huggingface.co/poolside/Laguna-S-2.1), bf16.
27
 
28
  - Architecture: `DFlashLagunaForCausalLM` (6 sliding-attention layers, block_size 16).
29
  - Shares token embedding + lm_head with the target; `draft_vocab_size == vocab_size` (no d2t/t2d).
30
+ - Source checkpoint: `s3://poolside.titan.checkpoints.us-east-2/adam/checkpoints/ft_sft_e0630_rhiemann_baseline_titan_sft_training/0008400/DFlash_Stage2/25000`
31
  - Loads under vLLM (native `laguna_dflash`) and TRT-LLM (pytorch DFlash backend) as the draft model in a speculative config.
32
 
33
  ## Usage
 
51
 
52
  ### llama.cpp (GGUF)
53
 
54
+ A llama.cpp conversion of this drafter (`laguna-s-2.1-DFlash-BF16.gguf`) is
55
  published in
56
  [poolside/Laguna-S-2.1-GGUF](https://huggingface.co/poolside/Laguna-S-2.1-GGUF)
57
  alongside the target GGUFs. It embeds the target tokenizer and the DFlash metadata
 
65
  ./build/bin/llama-server \
66
  -m laguna-s-2.1-Q4_K_M.gguf \
67
  -md laguna-s-2.1-DFlash-BF16.gguf \
68
+ --spec-type draft-dflash --spec-draft-n-max 15 \
69
  -fa on --jinja --port 8000
70
  ```
71
 
72
  `--spec-draft-n-max` is clamped to the trained block size (15 draft tokens + 1).
73
 
74
  > [!NOTE]
75
+ > Requires poolside's llama.cpp fork, branch
76
  > [`laguna`](https://github.com/poolsideai/llama.cpp/tree/laguna). Upstream
77
  > llama.cpp ships the generic DFlash framework but not the Laguna decoder
78
+ > contract this drafter needs, and upstream PR
79
  > [ggml-org/llama.cpp#25165](https://github.com/ggml-org/llama.cpp/pull/25165)
80
  > covers the target architecture only.