README.md

---
license: cc-by-nc-4.0
base_model: Qwen/Qwen3.5-35B-A3B
base_model_relation: finetune
library_name: transformers
pipeline_tag: image-text-to-text
language:
  - en
  - ko
  - ja
  - zh
  - es
  - fr
  - de
  - ru
  - ar
  - pt
  - multilingual
tags:
  - lst
  - language-selection-tuning
  - language-bias
  - bias-mitigation
  - language-confusion-mitigation
  - korean
  - chinese-suppression
  - multilingual
  - moe
  - mixture-of-experts
  - qwen3.5
  - mamba-hybrid
  - vision-language
  - composite-vision-language
  - text-generation
  - chat
---

# DSLM-LST-35B-A3B

**DSLM-LST-35B-A3B** is a `Qwen/Qwen3.5-35B-A3B` derivative refined with our in-house
**Language Selection Tuning (LST)** technique. The goal is to suppress unwanted
Chinese-character generation when the model serves non-Chinese (English / Korean / Japanese
etc.) users.

The pipeline is `Qwen/Qwen3.5-35B-A3B` → **LST tuning** (output-head centric). The adjustment
is intentionally minimal in scope — most of the network, including the entire vision tower and
all expert weights, is preserved bit-for-bit from the base model, so vision and multimodal
capabilities are unchanged and the result is a drop-in replacement that only mitigates
unintended Chinese-token leakage. For memory- and throughput-efficient 4-bit serving, see the
[GPTQ INT4 quantization](https://huggingface.co/dataslab/DSLM-LST-35B-A3B-GPTQ-Int4) derived
from this model.

The architecture is `Qwen3_5MoeForConditionalGeneration`: a composite multimodal
vision-language model with a **MoE** text backbone (256 experts, top-8 routing), a
**linear-attention + full-attention 4:1 hybrid** layout, and a **27-block vision tower**.

## Why LST?

Multilingual LLMs trained on heavily skewed corpora (e.g., Qwen on Chinese-rich data) tend
to leak the dominant training language regardless of prompt language — a phenomenon known
as **language confusion**. For Korean users, Chinese characters sometimes appear in the
middle of an otherwise-Korean answer, hurting readability and trust.

**Language Selection Tuning (LST)** addresses this in a **learning-based** manner. Unlike
post-hoc decoding tricks (vocabulary masking, banned-token lists), LST adjusts the model's
*internal* language-selection behavior. (The exact algorithm and training configuration are
proprietary and not disclosed in this release.)

## Key Properties

- **Minimal footprint.** LST tuning modifies essentially only the output head; the
  tokenizer, chat template, vision tower, MoE experts, and attention weights are preserved
  from the base model (see [Modification Footprint](#modification-footprint)).
- **Selectivity preserved.** When the user explicitly asks for Chinese, the model still
  produces fluent Chinese — this is not blanket suppression.
- **Full-precision fidelity.** Released in bf16 (~70 GB), this is the unquantized source
  model; reasoning performance tracks the base model closely (see Benchmarks).

## Modification Footprint

LST tuning was verified by a tensor-by-tensor diff against the base
`Qwen/Qwen3.5-35B-A3B`. Of the **1,026** shared tensors, **995 are bit-identical** to the
base model — only the output head is actually trained:

- **Tuned**: `lm_head.weight` only — rel-L2 ≈ **0.075**, cosine ≈ **0.997**, norm ratio
  ≈ 1.000 (no scale change). About **21.9 %** of the 248,320 vocab rows are updated, spread
  across the vocabulary rather than in a contiguous block — a distributed, mild recalibration
  of the output head.
- **Frozen (bit-identical to base)**: `embed_tokens`, **all 256 MoE experts**, the shared
  expert, the router gate, self-attention, linear-attention projections / conv / SSM
  (`A_log` / `dt_bias` / `conv1d`), every layernorm, and the **entire vision tower**.
- The 30 `linear_attn.norm.weight` tensors show a sub-0.2 % difference that is **not
  training** — it is an fp32→bf16 down-cast artifact (the base stores these norms in fp32;
  this model stores them in bf16, bit-identical to `bf16(base)`). Functionally
  equivalent for bf16 serving.

## Requirements

- **`transformers >= 5.9`** — required for the `qwen3_5_moe` architecture, the modern
  tokenizer backend, and the consolidated processor format.
- **No MTP head.** The base model's Multi-Token Prediction module is not included (this is a
  standard-inference model), so **speculative decoding via the MTP head is not
  available**. All other standard inference is unaffected.
- **vLLM serving**: served in bf16. The full bf16
  model is ~70 GB and benefits from tensor parallelism across multiple GPUs.

## Quickstart (vLLM, recommended)

```bash
vllm serve dataslab/DSLM-LST-35B-A3B \
    --tensor-parallel-size 2 \
    --port 8000 \
    --gpu-memory-utilization 0.90 \
    --reasoning-parser qwen3        # exposes <think> trace via OpenAI API
    # --max-model-len 16384         # cap context to shrink KV cache (default: 262,144)
```

## Use with transformers

### Non-Thinking mode (recommended for fast chat)

```python
import torch
from transformers import AutoTokenizer, AutoModelForImageTextToText

REPO = "dataslab/DSLM-LST-35B-A3B"

tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForImageTextToText.from_pretrained(
    REPO,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "user", "content": "한반도 주변에 가장 흔한 점토광물은?"},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=256)
text = tokenizer.decode(out[0][inputs.input_ids.shape[-1]:],
                       skip_special_tokens=True)
print(text)
```

### Thinking mode (recommended for complex reasoning)

Either use `thinking_budget` (e.g., vLLM's `--reasoning-parser qwen3`) or give `max_new_tokens` enough headroom (e.g., 8,192 + 256 = **8,448**).
**Caveat:** without a `thinking_budget` cap, a too-small `max_new_tokens` can be fully consumed inside `<think>` and the answer never gets emitted.

```python
# ... tokenizer / model loaded as above ...

THINKING_BUDGET = 8192   # max tokens inside <think>
ANSWER_TOKENS   = 256    # tokens after </think>

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=THINKING_BUDGET + ANSWER_TOKENS)
text = tokenizer.decode(out[0][inputs.input_ids.shape[-1]:],
                       skip_special_tokens=True)
print(text)
```

> **Why `AutoModelForImageTextToText`?** The declared architecture
> `Qwen3_5MoeForConditionalGeneration` is a composite class wrapping both the text decoder
> and the vision tower. Loading via `AutoModelForCausalLM` works for text-only inference but
> strips the vision submodule and may produce a config that downstream tools (e.g., vLLM)
> reject. For a pure text causal-LM handle, use `model.language_model` after loading.

## Benchmark Results

The **DSLM-LST-35B-A3B** column is this release; the other columns are the base model and
[**DSLM-LST-35B-A3B-GPTQ-Int4**](https://huggingface.co/dataslab/DSLM-LST-35B-A3B-GPTQ-Int4), the GPTQ INT4 quantization derived from this model.

### Evaluation Metrics

**(1) Selectivity**

Refusal rate on explicit Chinese requests — the fraction of cases where the model fails to produce Chinese even though the user explicitly asked for it. Lower is better (respects user intent).
- **Lower better (~0)**: produces Chinese when asked (respects user intent).
- **Higher worse (~1)**: refuses Chinese even when asked (blanket suppression).

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 25%;">
    <col style="width: 75%;">
  </colgroup>
  <thead>
    <tr align="center">
      <th>Metric</th>
      <th>Benchmark Dataset</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>chin_refusal</code> ↓</td>
      <td>In-house 1,000-prompt Chinese elicitation set (e.g., <code>How do you say '사랑' in Chinese?</code> or the Python + Chinese-comment prompt)</td>
    </tr>
  </tbody>
</table>

**(2) Chinese-leak suppression**

Korean prompts → Korean answers expected; any Chinese token leaked into the answer is a failure. Metric is the *clean-Korean response ratio*.
- **Higher better (~1)**: Korean answers stay fully Korean (no Chinese tokens leaked).
- **Lower worse (~0)**: Chinese tokens leak into otherwise-Korean answers.

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 25%;">
    <col style="width: 75%;">
  </colgroup>
  <thead>
    <tr align="center">
      <th>Metric</th>
      <th>Benchmark Dataset</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>chin_cs</code> ↑</td>
      <td>KMMLU Computer Science subjects (free-form Korean generation)</td>
    </tr>
    <tr>
      <td><code>chin_ie</code> ↑</td>
      <td>KMMLU Industrial Engineering subjects (free-form Korean generation)</td>
    </tr>
    <tr>
      <td><code>chin_total</code> ↑</td>
      <td>KMMLU (free-form Korean generation)</td>
    </tr>
  </tbody>
</table>

**(3) Reasoning / task performance**

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 25%;">
    <col style="width: 75%;">
  </colgroup>
  <thead>
    <tr align="center">
      <th>Metric</th>
      <th>Benchmark Dataset</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>acc_cs</code> ↑</td>
      <td>KMMLU Computer Science subjects (multiple-choice log-likelihood comparison)</td>
    </tr>
    <tr>
      <td><code>acc_ie</code> ↑</td>
      <td>KMMLU Industrial Engineering subjects (multiple-choice log-likelihood comparison)</td>
    </tr>
    <tr>
      <td><code>acc_total</code> ↑</td>
      <td>KMMLU (multiple-choice log-likelihood comparison)</td>
    </tr>
    <tr>
      <td><code>HumanEval</code> ↑</td>
      <td>HumanEval (pass@1)</td>
    </tr>
    <tr>
      <td><code>GSM8K</code> ↑</td>
      <td>GSM8K (exact-match accuracy)</td>
    </tr>
  </tbody>
</table>

### Chinese Suppression (**Thinking mode**)

Evaluated with `enable_thinking=True`. The DSLM-LST-35B-A3B column is this bf16 release.

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 28%;">
    <col style="width: 24%;">
    <col style="width: 24%;">
    <col style="width: 24%;">
  </colgroup>
  <thead>
    <tr align="center">
      <th>Metric</th>
      <th>Qwen3.5-35B-A3B (base)</th>
      <th style="color:#EAB308;"><b>DSLM-LST-35B-A3B</b></th>
      <th>DSLM-LST-35B-A3B-GPTQ-Int4</th>
    </tr>
  </thead>
  <tbody>
    <tr><td colspan="4" align="left"><b>(1) Selectivity</b></td></tr>
    <tr align="center"><td align="left">chin_refusal ↓</td><td><b>0.005</b></td><td>0.030</td><td>0.039</td></tr>
    <tr><td colspan="4" align="left"><b>(2) Chinese-leak suppression</b></td></tr>
    <tr align="center"><td align="left">chin_cs ↑</td><td>0.989</td><td>0.998</td><td><b>0.999</b></td></tr>
    <tr align="center"><td align="left">chin_ie ↑</td><td>0.983</td><td><b>0.994</b></td><td>0.993</td></tr>
    <tr align="center"><td align="left">chin_total ↑</td><td>0.9754</td><td><b>0.9901</b></td><td>0.9895</td></tr>
    <tr><td colspan="4" align="left"><b>(3) Reasoning / Task performance</b></td></tr>
    <tr align="center"><td align="left">acc_cs ↑</td><td><b>0.869</b></td><td><b>0.869</b></td><td>0.865</td></tr>
    <tr align="center"><td align="left">acc_ie ↑</td><td><b>0.622</b></td><td><b>0.622</b></td><td>0.602</td></tr>
    <tr align="center"><td align="left">acc_total ↑</td><td><b>0.6411</b></td><td><b>0.6411</b></td><td>0.6340</td></tr>
    <tr align="center"><td align="left">HumanEval ↑</td><td><b>0.7683</b></td><td>0.7378</td><td>0.7561</td></tr>
    <tr align="center"><td align="left">GSM8K ↑</td><td>0.8347</td><td>0.8347</td><td><b>0.8810</b></td></tr>
  </tbody>
</table>


LST improves Chinese-leak suppression (`chin_total` 0.9754 → **0.9901**) and keeps it
selective (`chin_refusal` stays low at 0.030; the model still produces Chinese on request),
while leaving KMMLU accuracy **unchanged** (`acc_total` 0.6411, identical to base) and math
reasoning intact (`GSM8K` 0.8347). HumanEval dips slightly (0.7683 → 0.7378). The GPTQ INT4
quantization derived from this model stays close on all metrics (see the
[GPTQ-Int4 model card](https://huggingface.co/dataslab/DSLM-LST-35B-A3B-GPTQ-Int4)).

## Example Outputs

![Example output: asked in Korean which NC-programming function–address pair is mismatched, Qwen3.5-35B-A3B leaks 22 Chinese tokens (主轴 ×11) while DSLM-LST-35B-A3B stays clean Korean (0 tokens)](assets/banner.png)

Asked in Korean which NC-programming function–address pair is mismatched, Qwen3.5-35B-A3B
leaks **22 Chinese tokens** (`主轴` ×11 — the Chinese word for "spindle") into its answer.
DSLM-LST-35B-A3B answers the same prompt entirely in Korean (**0 Chinese tokens**, writing
`스핀들`), and **both models still select the correct option** — the leak is suppressed without
hurting accuracy. The examples below show this behavior across more KMMLU domains and inside
the reasoning (`<think>`) trace.

All examples are drawn verbatim from the in-house Chinese-leakage evaluation
(`enable_thinking=True`). For each KMMLU prompt the base **Qwen3.5-35B-A3B** slips Chinese
characters into an otherwise-Korean answer, while **DSLM-LST-35B-A3B** answers the same
prompt entirely in Korean (zero Chinese characters).

### Domain-wise Leakage Comparisons

#### Computer Science

<details open>
<summary><b>Prompt:</b> BIOS Setup의 기본기능으로 잘못된 것은?</summary>

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 50%;">
    <col style="width: 50%;">
  </colgroup>
  <thead>
    <tr>
      <th>Qwen3.5-35B-A3B (leaks <code>出荷</code>)</th>
      <th style="color:#EAB308;">DSLM-LST-35B-A3B (clean Korean)</th>
    </tr>
  </thead>
  <tbody>
    <tr style="vertical-align: top;">
      <td>
        <blockquote>
        … <b>BIOS Setup 의 올바른 기능:</b><br/>
        • <b>시스템 시간 및 날짜 설정</b><br/>
        • <b>부팅 장치 순서 설정 (Boot Order)</b><br/>
        • <b>보안 설정 / 전원 관리 설정</b><br/>
        • <b>CMOS 초기화:</b> 설정을 공장<span style="color:#DC2626;"><b>出荷</b></span> 상태로 되돌립니다.
        </blockquote>
      </td>
      <td>
        <blockquote>
        … <b>BIOS Setup 의 주요 (올바른) 기능:</b><br/>
        1. <b>시스템 날짜 및 시간 설정</b><br/>
        2. <b>부팅 순서 (Boot Order) 설정</b><br/>
        5. <b>하드웨어 상태 모니터링</b>: CPU 온도, 팬 속도, 전압 등을 확인합니다.<br/>
        6. <b>기본값 복원 (Load Defaults):</b> 설정을 초기화하여 시스템 안정성을 확보합니다. …
        </blockquote>
      </td>
    </tr>
  </tbody>
</table>

</details>

#### Industrial Engineering

<details open>
<summary><b>Prompt:</b> 머시닝센터에서 공작물 가공시 주의해야 할 사항으로 올바른 것은?</summary>

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 50%;">
    <col style="width: 50%;">
  </colgroup>
  <thead>
    <tr>
      <th>Qwen3.5-35B-A3B (leaks <code>卷入</code>)</th>
      <th style="color:#EAB308;">DSLM-LST-35B-A3B (clean Korean)</th>
    </tr>
  </thead>
  <tbody>
    <tr style="vertical-align: top;">
      <td>
        <blockquote>
        … <b>안전 수칙 (보호구)</b><br/>
        • <b>올바른 사항:</b> <b>안전 고글</b>을 착용하고, <b>회전하는 스핀들 근처에서는 장갑을 착용하지 않아야 합니다.</b><br/>
        • <b>이유:</b> 장갑을 끼고 회전체 근처에서 작업할 경우 장갑이 걸려 손이<span style="color:#DC2626;"><b>卷入</b></span> (감김) 될 위험이 매우 큽니다.
        </blockquote>
      </td>
      <td>
        <blockquote>
        … <b>올바른 주의사항 (정답 후보)</b><br/>
        1. <b>공작물의 단단한 고정:</b> 클램프나 바이스 (Vise) 로 단단히 고정해야 합니다.<br/>
        5. <b>절삭 중 측정 금지:</b> 공구가 작동하는 동안에는 절대 측정하거나 접촉해서는 안 됩니다.<br/>
        6. <b>회전부 장갑 착용 금지:</b> 회전하는 척 (Spindle) 근처에서는 끼임 사고의 위험이 있으므로 금지됩니다. …
        </blockquote>
      </td>
    </tr>
  </tbody>
</table>

</details>

#### Law

<details open>
<summary><b>Prompt:</b> 경비조사업무의 과정으로 옳지 않은 것은?</summary>

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 50%;">
    <col style="width: 50%;">
  </colgroup>
  <thead>
    <tr>
      <th>Qwen3.5-35B-A3B (leaks <code>它们是</code>)</th>
      <th style="color:#EAB308;">DSLM-LST-35B-A3B (clean Korean)</th>
    </tr>
  </thead>
  <tbody>
    <tr style="vertical-align: top;">
      <td>
        <blockquote>
        … <b>경비조사업무의 올바른 과정 (정답에 해당하는 내용)</b><br/>
        만약 선택지에 이 내용들이 있다면<span style="color:#DC2626;"><b>它们是</b></span> <b>옳은 것</b>입니다.<br/>
        1. <b>사전 준비 (계약 및 현장 조사)</b><br/>
        2. <b>인력 배치 및 교육</b><br/>
        3. <b>경비 수행 (순찰 및 출입 통제)</b> …
        </blockquote>
      </td>
      <td>
        <blockquote>
        … <b>경비조사업무의 올바른 과정 (표준)</b><br/>
        1. <b>계약 체결</b>: 의뢰인 (고객) 과 경비계약 체결<br/>
        2. <b>현장 실사</b>: 경비 대상지의 위험 요소, 시설 현황 등을 조사<br/>
        3. <b>경비 계획 수립</b>: 인력 배치, 순찰 경로, 비상 대응 계획 등 수립 …
        </blockquote>
      </td>
    </tr>
  </tbody>
</table>

</details>

### Cross-lingual Selectivity

When the user **explicitly asks for Chinese**, DSLM-LST-35B-A3B readily produces it. The
previous examples showed the model *avoiding* unwanted Chinese inside an otherwise-Korean
answer; the example below shows it emitting Chinese fluently — in both code comments and
prose — when the user's instruction calls for it.

**Prompt:** 두 수의 최대공약수를 구하는 파이썬 함수를 작성해주세요. 설명은 중국어로 해주세요.

**DSLM-LST-35B-A3B (Python code with Chinese comments + Chinese explanation):**

```python
def gcd(a, b):
    """
    计算两个整数的最大公约数 (Greatest Common Divisor)
    使用欧几里得算法 (辗转相除法)
    """
    # 确保处理负数的情况（最大公约数通常定义为正数）
    a = abs(a)
    b = abs(b)

    # 当 b 不为 0 时，循环执行
    while b != 0:
        # 将 b 的值赋给 a，将 a 除以 b 的余数赋给 b
        a, b = b, a % b

    # 当 b 为 0 时，a 即为最大公约数
    return a

**代码说明 (Chinese explanation):**
1. **函数定义**：`def gcd(a, b):` 定义了一个接受两个参数 `a` 和 `b` 的函数。
2. **绝对值处理**：`abs(a)` 和 `abs(b)` 确保即使输入负数，也能正确计算。
3. **循环逻辑**：`a, b = b, a % b` 是核心步骤，利用取模运算不断缩小数值，直到余数为 0 —— 这个过程就是**辗转相除法**。
```
### Thinking-trace Suppression

The base model's `<think>` block leaks Chinese even more severely than its final answer:
once the reasoning gets stuck, it slips into Chinese characters and loops on its own
hypotheses. DSLM-LST-35B-A3B suppresses that leakage inside the thinking block too.

The prompt below uses a non-standard / likely-garbled term (`장촌식 제도법`). **Neither model
resolves the obscure term** — so this example isolates the *leak-suppression* behavior in the
reasoning trace, not accuracy. The base model spirals into a self-doubt loop, re-emitting the
same Chinese-character hypothesis (`長村`) **210 times**; DSLM-LST-35B-A3B reasons through the
same uncertainty but emits **zero** Chinese characters anywhere — neither in the `<think>`
block nor in the user-facing answer.

**Prompt:** 장촌식 제도법의 특징이 아닌 것은?

**Chinese-character counts (thinking budget = 8,192):**

<table style="table-layout: fixed; width: 100%;">
  <colgroup>
    <col style="width: 40%;">
    <col style="width: 30%;">
    <col style="width: 30%;">
  </colgroup>
  <thead>
    <tr>
      <th>Metric</th>
      <th>Qwen3.5-35B-A3B</th>
      <th style="color:#EAB308;">DSLM-LST-35B-A3B</th>
    </tr>
  </thead>
  <tbody>
    <tr align="center">
      <td align="left"><code>&lt;think&gt;</code> block, Chinese characters</td>
      <td><b>890</b></td>
      <td style="color:#EAB308;"><b>0</b></td>
    </tr>
    <tr align="center">
      <td align="left"><code>&lt;answer&gt;</code> block, Chinese characters</td>
      <td>leaks <code>長川式</code> / <code>長村</code> (<b>11</b> chars)</td>
      <td style="color:#EAB308;">0 (clean Korean)</td>
    </tr>
  </tbody>
</table>

**Qwen3.5-35B-A3B `<think>` block (loops on a Chinese-character hypothesis):**

```
*   **Wait, is it possible the user means "Jangchon (長村)" as a typo for "Jangchon (長村)"?**
*   **Wait, is it possible the user means "Jangchon (長村)" as a typo for "Jangchon (長村)"?**
*   **Wait, is it possible the user means "Jangchon (長村)" as a typo for "Jangchon (長村)"?**
    ... (≈210 such "Wait…" repetitions, emitting 890 Chinese characters) ...
```

**Qwen3.5-35B-A3B Final answer (Chinese leak into Korean):**

```
**장촌식 제도법**은 조선 시대 토지 제도와 관련된 용어로 보이나 …
유사한 발음이나 맥락으로 추정되는 **장천식 (長川式)** 또는 **장촌 (長村)** 관련
토지 제도 (예: 장천법, 장촌법 등) 와 관련하여 …
```

**DSLM-LST-35B-A3B `<think>` block (same uncertainty, but English/Korean — 0 Chinese):**

```
Here's a thinking process that leads to the answer:

1.  **Analyze the Request:**
    *   **Topic:** Jangchon-sik System of Law (장촌식 제도법).
    *   **Context:** ... "Jangchon-sik" (장촌식) is not a standard, widely recognized term
        in mainstream Korean legal history textbooks like "Gyeongguk Daejeon" (경국대전) ...
    *   **Result:** There is no standard historical term "Jangchon-sik System of Law".

    *   **Wait, is it possible the user is asking about "Jangcheon" (장천)?** No.
    *   **Wait, is it possible the user is asking about "Jangcheon" (장천)?** No.
        ... (loops on the same hypothesis, but in Hangul `장천` — never a Chinese character) ...

Considering the limited time by the user, I have to give the solution based on the thinking directly now.
```

Where the base model re-emits the Chinese hanzi `長村`, DSLM-LST-35B-A3B loops on the
**Hangul** `장천` instead — the self-doubt pattern persists, but not a single Chinese
character is produced.

**DSLM-LST-35B-A3B Final answer (clean Korean, 0 Chinese):**

```
**장촌식 제도법**이라는 용어는 한국 법제사나 역사학에서 일반적으로 통용되는 표준
용어가 아닙니다. 따라서 이 질문은 특정 교재, 시험 문제, 혹은 가상의 설정에서 나온
것으로 보입니다.

하지만, 한국 법제사 관련 시험에서 유사한 발음이나 개념으로 자주 등장하는 용어는
다음과 같습니다. …
```

Both models treat `장촌식 제도법` as a non-standard term, but only the base model leaks Chinese
characters into its reasoning and answer; DSLM-LST-35B-A3B stays entirely in Korean.

## Limitations

- **Not an instruction-tuned chat model.** The LST adjustment scope is minimal, so
  conversational behavior, instruction-following, and reasoning patterns are inherited from
  the base model — only unintended Chinese-token leakage is mitigated.
- **Degraded Chinese generation.** Tasks that *require* Chinese output (Chinese translation,
  Chinese code comments, bilingual Q&A) will see lower quality; use the base Qwen3.5-35B-A3B
  for those.
- **No MTP / speculative decoding.** The base model's Multi-Token Prediction head is not
  included, so MTP-based speculative decoding is unavailable; standard inference is unaffected.
- **Multimodal not re-benchmarked.** The vision tower is kept in bf16 (unchanged), so
  multimodal behavior should be unaffected, but the vision pipeline was not separately
  re-benchmarked for this release.

## License

This model is not available for public download. For a publicly available alternative, see [DSLM-LST-9B](https://huggingface.co/dataslab/DSLM-LST-9B). 
For commercial or research access to this model, please [contact us](http://www.dataslab.co.kr/).

## Contact

For questions, feedback, or collaboration inquiries, please
[reach out via our website](http://www.dataslab.co.kr/).