yzhwang commited on
Commit
ca22b2f
Β·
verified Β·
1 Parent(s): 4c39bd1

escha 1.1.1: docs + changelog for the cuda:0 device fix

Browse files
README.md CHANGED
@@ -119,6 +119,25 @@ an answer is always produced: see
119
  [model card](https://huggingface.co/EschaLabs/Qwen3.8-27B-Escha-W2#verified-configurations).
120
  16 GB should fit at a reduced context β€” the cookbook has a recipe, but we have not run it.
121
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122
  ## Known limitations
123
 
124
  - **Serving throughput has been measured on three cards** β€” RTX 5090 (32 GB, sm_120), RTX 4090
 
119
  [model card](https://huggingface.co/EschaLabs/Qwen3.8-27B-Escha-W2#verified-configurations).
120
  16 GB should fit at a reduced context β€” the cookbook has a recipe, but we have not run it.
121
 
122
+ ## Changelog
123
+
124
+ **1.1.1** (2026-08-21) β€” `process_weights_after_loading` now takes the rank's device
125
+ instead of a hardcoded `cuda:0`. The hardcode put every 2-bit buffer on `cuda:0` while
126
+ the input tensor sat on the server's actual device, so **any run not on device 0 β€”
127
+ `--tp-size > 1`, or a single-GPU launch with `--base-gpu-id N` and no
128
+ `CUDA_VISIBLE_DEVICES` β€” hit an illegal memory access on the first forward**, behind a
129
+ traceback that pointed at the kernel rather than at the cause. Bit-identical wherever
130
+ `cuda:0` was already correct, which is every configuration `serve.sh` ships. Reported
131
+ with a diagnosis and a fix by [@ginerJuanUdesa](https://github.com/ginerJuanUdesa/escha-tp-fix-qwen3dense).
132
+
133
+ > Note: this fixes the device placement only. **`--tp-size > 1` is still not supported** β€”
134
+ > the weight loader does not shard escha tensors per rank, so every rank holds the full
135
+ > checkpoint. A community patch for that exists at the link above and is under review; it
136
+ > is not in this wheel yet.
137
+
138
+ **1.1.0** (2026-08-20) β€” first wheel with the dense (`escha`) serving path; the 1.0.x
139
+ wheels registered `eschamoe` only, so a dense checkpoint failed at registry lookup.
140
+
141
  ## Known limitations
142
 
143
  - **Serving throughput has been measured on three cards** β€” RTX 5090 (32 GB, sm_120), RTX 4090
THIRD_PARTY_LICENSES/THIRD_PARTY_NOTICES.md CHANGED
@@ -1,4 +1,4 @@
1
- # Third-Party Notices β€” Escha Runtime (`escha` 1.1.0)
2
 
3
  This repository distributes the **runtime** only (the `escha` wheel: quantization/serving
4
  kernels + a modified SGLang fork). SGLang is the only engine in this repository. The quantized **model weights** are distributed
@@ -9,7 +9,7 @@ All bundled third-party code is under **permissive licenses (Apache-2.0, MIT, BS
9
  an audit found **no copyleft (GPL / LGPL / AGPL / MPL / SSPL) code**. The obligations below are
10
  satisfied by the files in this `THIRD_PARTY_LICENSES/` directory together with the copyright
11
  notices retained verbatim in the redistributed source (inside the wheel). The same texts are also
12
- embedded in the wheel metadata (`escha-1.1.0+qwen3dense.dist-info/licenses/LICENSE`).
13
 
14
  The `escha` runtime itself is released under the Apache License, Version 2.0 β€” see the top-level
15
  `LICENSE` file of this repository.
 
1
+ # Third-Party Notices β€” Escha Runtime (`escha` 1.1.1)
2
 
3
  This repository distributes the **runtime** only (the `escha` wheel: quantization/serving
4
  kernels + a modified SGLang fork). SGLang is the only engine in this repository. The quantized **model weights** are distributed
 
9
  an audit found **no copyleft (GPL / LGPL / AGPL / MPL / SSPL) code**. The obligations below are
10
  satisfied by the files in this `THIRD_PARTY_LICENSES/` directory together with the copyright
11
  notices retained verbatim in the redistributed source (inside the wheel). The same texts are also
12
+ embedded in the wheel metadata (`escha-1.1.1+qwen3dense.dist-info/licenses/LICENSE`).
13
 
14
  The `escha` runtime itself is released under the Apache License, Version 2.0 β€” see the top-level
15
  `LICENSE` file of this repository.
sglang/INSTALL.md CHANGED
@@ -287,7 +287,7 @@ inject into the system prompt β€” at some cost in reasoning-dependent accuracy.
287
  - **`ValueError: Invalid quantization method ...`.** The wheel's registry ships exactly two
288
  methods, `escha` (dense) and `eschamoe` (mixture-of-experts). Check your model's
289
  `quantize_config.json` / `config.json` names one of them, and that you installed *this* wheel
290
- (`python -c "import escha; print(escha.__version__)"` should print `1.1.0+qwen3dense`) β€” the
291
  1.0.x wheels registered `eschamoe` only and cannot load a dense model at all.
292
 
293
  - **`ModuleNotFoundError: pybase64` / `msgspec` / `xgrammar` / `flashinfer` at startup.** The wheel
 
287
  - **`ValueError: Invalid quantization method ...`.** The wheel's registry ships exactly two
288
  methods, `escha` (dense) and `eschamoe` (mixture-of-experts). Check your model's
289
  `quantize_config.json` / `config.json` names one of them, and that you installed *this* wheel
290
+ (`python -c "import escha; print(escha.__version__)"` should print `1.1.1+qwen3dense`) β€” the
291
  1.0.x wheels registered `eschamoe` only and cannot load a dense model at all.
292
 
293
  - **`ModuleNotFoundError: pybase64` / `msgspec` / `xgrammar` / `flashinfer` at startup.** The wheel