Spaces:
Runtime error
Runtime error
Switch assistant back to Qwen3.5-9B-Uncensored (bf16)
Browse filesVLM_MODEL_ID default -> ccharnkij/Qwen3.5-9B-Uncensored (the pre-Gemma
default). bf16 (~18.8GB, fits zero-a10g). Gemma E4B/12B kept commented for
revert. requirements unchanged -> app reload, not a full rebuild.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
app.py
CHANGED
|
@@ -152,15 +152,16 @@ print("Pipelines loaded!")
|
|
| 152 |
# -----------------------------------------------------------------------------
|
| 153 |
# Other models, kept for easy revert (set VLM_MODEL_ID, and VLM_LOAD_8BIT for big ones):
|
| 154 |
# "rodrigomt/Qwen3.5-4B-Uncensored-Aggressive" # no generation_config; needed eos pinning
|
| 155 |
-
# "
|
| 156 |
# "OpenYourMind/gemma-4-12B-it-abliterated-uncensored" # gemma4_unified, ~24GB; needs VLM_LOAD_8BIT=1 (slow)
|
| 157 |
-
# Current:
|
| 158 |
-
#
|
| 159 |
-
#
|
| 160 |
-
#
|
| 161 |
-
#
|
| 162 |
-
#
|
| 163 |
-
|
|
|
|
| 164 |
VLM_LOAD_8BIT = os.environ.get("VLM_LOAD_8BIT", "0").lower() not in ("0", "false", "no", "")
|
| 165 |
|
| 166 |
print(f"Loading assistant: {VLM_MODEL_ID} (8bit={VLM_LOAD_8BIT}) ...")
|
|
|
|
| 152 |
# -----------------------------------------------------------------------------
|
| 153 |
# Other models, kept for easy revert (set VLM_MODEL_ID, and VLM_LOAD_8BIT for big ones):
|
| 154 |
# "rodrigomt/Qwen3.5-4B-Uncensored-Aggressive" # no generation_config; needed eos pinning
|
| 155 |
+
# "prithivMLmods/gemma-4-E4B-it-Uncensored-MAX" # gemma4, small/fast bf16; under-rates explicit (~2)
|
| 156 |
# "OpenYourMind/gemma-4-12B-it-abliterated-uncensored" # gemma4_unified, ~24GB; needs VLM_LOAD_8BIT=1 (slow)
|
| 157 |
+
# Current: 9B uncensored Qwen3.5 VL (model_type=qwen3_5 / Qwen3_5ForConditionalGeneration).
|
| 158 |
+
# Vision-capable (image-text-to-text), bf16 (~18.8 GB β heaviest that still fits the
|
| 159 |
+
# zero-a10g alongside both diffusion pipelines, verified no OOM). Thinking model
|
| 160 |
+
# (Reasoning On/Off toggle is meaningful). Its generation_config eos is just
|
| 161 |
+
# <|endoftext|> and omits the chat terminator <|im_end|>; _resolve_vlm_eos_ids() unions
|
| 162 |
+
# both so stopping still works. VLM_LOAD_8BIT=1 forces bitsandbytes 8-bit (only needed
|
| 163 |
+
# for the 12B); default is bf16.
|
| 164 |
+
VLM_MODEL_ID = os.environ.get("VLM_MODEL_ID", "ccharnkij/Qwen3.5-9B-Uncensored")
|
| 165 |
VLM_LOAD_8BIT = os.environ.get("VLM_LOAD_8BIT", "0").lower() not in ("0", "false", "no", "")
|
| 166 |
|
| 167 |
print(f"Loading assistant: {VLM_MODEL_ID} (8bit={VLM_LOAD_8BIT}) ...")
|