escha 1.1.1: docs + changelog for the cuda:0 device fix
Browse files- README.md +19 -0
- THIRD_PARTY_LICENSES/THIRD_PARTY_NOTICES.md +2 -2
- sglang/INSTALL.md +1 -1
README.md
CHANGED
|
@@ -119,6 +119,25 @@ an answer is always produced: see
|
|
| 119 |
[model card](https://huggingface.co/EschaLabs/Qwen3.8-27B-Escha-W2#verified-configurations).
|
| 120 |
16 GB should fit at a reduced context β the cookbook has a recipe, but we have not run it.
|
| 121 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
## Known limitations
|
| 123 |
|
| 124 |
- **Serving throughput has been measured on three cards** β RTX 5090 (32 GB, sm_120), RTX 4090
|
|
|
|
| 119 |
[model card](https://huggingface.co/EschaLabs/Qwen3.8-27B-Escha-W2#verified-configurations).
|
| 120 |
16 GB should fit at a reduced context β the cookbook has a recipe, but we have not run it.
|
| 121 |
|
| 122 |
+
## Changelog
|
| 123 |
+
|
| 124 |
+
**1.1.1** (2026-08-21) β `process_weights_after_loading` now takes the rank's device
|
| 125 |
+
instead of a hardcoded `cuda:0`. The hardcode put every 2-bit buffer on `cuda:0` while
|
| 126 |
+
the input tensor sat on the server's actual device, so **any run not on device 0 β
|
| 127 |
+
`--tp-size > 1`, or a single-GPU launch with `--base-gpu-id N` and no
|
| 128 |
+
`CUDA_VISIBLE_DEVICES` β hit an illegal memory access on the first forward**, behind a
|
| 129 |
+
traceback that pointed at the kernel rather than at the cause. Bit-identical wherever
|
| 130 |
+
`cuda:0` was already correct, which is every configuration `serve.sh` ships. Reported
|
| 131 |
+
with a diagnosis and a fix by [@ginerJuanUdesa](https://github.com/ginerJuanUdesa/escha-tp-fix-qwen3dense).
|
| 132 |
+
|
| 133 |
+
> Note: this fixes the device placement only. **`--tp-size > 1` is still not supported** β
|
| 134 |
+
> the weight loader does not shard escha tensors per rank, so every rank holds the full
|
| 135 |
+
> checkpoint. A community patch for that exists at the link above and is under review; it
|
| 136 |
+
> is not in this wheel yet.
|
| 137 |
+
|
| 138 |
+
**1.1.0** (2026-08-20) β first wheel with the dense (`escha`) serving path; the 1.0.x
|
| 139 |
+
wheels registered `eschamoe` only, so a dense checkpoint failed at registry lookup.
|
| 140 |
+
|
| 141 |
## Known limitations
|
| 142 |
|
| 143 |
- **Serving throughput has been measured on three cards** β RTX 5090 (32 GB, sm_120), RTX 4090
|
THIRD_PARTY_LICENSES/THIRD_PARTY_NOTICES.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
# Third-Party Notices β Escha Runtime (`escha` 1.1.
|
| 2 |
|
| 3 |
This repository distributes the **runtime** only (the `escha` wheel: quantization/serving
|
| 4 |
kernels + a modified SGLang fork). SGLang is the only engine in this repository. The quantized **model weights** are distributed
|
|
@@ -9,7 +9,7 @@ All bundled third-party code is under **permissive licenses (Apache-2.0, MIT, BS
|
|
| 9 |
an audit found **no copyleft (GPL / LGPL / AGPL / MPL / SSPL) code**. The obligations below are
|
| 10 |
satisfied by the files in this `THIRD_PARTY_LICENSES/` directory together with the copyright
|
| 11 |
notices retained verbatim in the redistributed source (inside the wheel). The same texts are also
|
| 12 |
-
embedded in the wheel metadata (`escha-1.1.
|
| 13 |
|
| 14 |
The `escha` runtime itself is released under the Apache License, Version 2.0 β see the top-level
|
| 15 |
`LICENSE` file of this repository.
|
|
|
|
| 1 |
+
# Third-Party Notices β Escha Runtime (`escha` 1.1.1)
|
| 2 |
|
| 3 |
This repository distributes the **runtime** only (the `escha` wheel: quantization/serving
|
| 4 |
kernels + a modified SGLang fork). SGLang is the only engine in this repository. The quantized **model weights** are distributed
|
|
|
|
| 9 |
an audit found **no copyleft (GPL / LGPL / AGPL / MPL / SSPL) code**. The obligations below are
|
| 10 |
satisfied by the files in this `THIRD_PARTY_LICENSES/` directory together with the copyright
|
| 11 |
notices retained verbatim in the redistributed source (inside the wheel). The same texts are also
|
| 12 |
+
embedded in the wheel metadata (`escha-1.1.1+qwen3dense.dist-info/licenses/LICENSE`).
|
| 13 |
|
| 14 |
The `escha` runtime itself is released under the Apache License, Version 2.0 β see the top-level
|
| 15 |
`LICENSE` file of this repository.
|
sglang/INSTALL.md
CHANGED
|
@@ -287,7 +287,7 @@ inject into the system prompt β at some cost in reasoning-dependent accuracy.
|
|
| 287 |
- **`ValueError: Invalid quantization method ...`.** The wheel's registry ships exactly two
|
| 288 |
methods, `escha` (dense) and `eschamoe` (mixture-of-experts). Check your model's
|
| 289 |
`quantize_config.json` / `config.json` names one of them, and that you installed *this* wheel
|
| 290 |
-
(`python -c "import escha; print(escha.__version__)"` should print `1.1.
|
| 291 |
1.0.x wheels registered `eschamoe` only and cannot load a dense model at all.
|
| 292 |
|
| 293 |
- **`ModuleNotFoundError: pybase64` / `msgspec` / `xgrammar` / `flashinfer` at startup.** The wheel
|
|
|
|
| 287 |
- **`ValueError: Invalid quantization method ...`.** The wheel's registry ships exactly two
|
| 288 |
methods, `escha` (dense) and `eschamoe` (mixture-of-experts). Check your model's
|
| 289 |
`quantize_config.json` / `config.json` names one of them, and that you installed *this* wheel
|
| 290 |
+
(`python -c "import escha; print(escha.__version__)"` should print `1.1.1+qwen3dense`) β the
|
| 291 |
1.0.x wheels registered `eschamoe` only and cannot load a dense model at all.
|
| 292 |
|
| 293 |
- **`ModuleNotFoundError: pybase64` / `msgspec` / `xgrammar` / `flashinfer` at startup.** The wheel
|