README.md

---
license: apache-2.0
library_name: peft
pipeline_tag: text-generation
base_model: Qwen/Qwen3-4B-Base
base_model_relation: adapter
tags:
  - peft
  - lora
  - transformers
  - safetensors
  - qwen3
  - code
  - text-generation
model-index:
  - name: Qwen3-4B-Base-ThinkCode-A-NH025 PEFT Adapter
    results:
      - task:
          type: text-generation
          name: Code Generation
        dataset:
          name: EvalScope Full1055 corrected (development-only)
          type: evalscope-full1055-corrected-development
        metrics:
          - type: pass@1
            name: resolved aggregate code_only pass@1 (3 seeds)
            value: 25.09
---

# Qwen3-4B-Base-ThinkCode-A-NH025 — PEFT Adapter

This repository contains a **PEFT LoRA adapter only**. It does not contain the
Qwen3 base-model weights and cannot be loaded as a standalone causal language
model.

The required base is
[`Qwen/Qwen3-4B-Base`](https://huggingface.co/Qwen/Qwen3-4B-Base) at the fixed
revision `906bfd4b4dc7f14ee4320094d8b41684abff8539`.

## Adapter construction

`A-NH025` is the Phase A no-head arm. Starting from the completed source LoRA,
every selected transformer-body LoRA `B` tensor is multiplied by `0.25` in
FP32, while the `lm_head` LoRA `B` tensor is multiplied by `0`, making its
effective head/shared-embedding delta exactly zero. LoRA `A` tensors are
unchanged. With `lora_alpha=128` and `r=64`, PEFT applies the intended body
delta without a language-model-head delta across the 253 declared modules.

The effective-zero `lm_head` adapter is omitted from the release state and
target list; this is exactly equivalent to its validated zero delta and avoids
packaging any base-layer tensor. `MODULE_SCALE_MANIFEST.json` retains the
explicit zero-head contract and records every logical module, source tensor
key, physical base weight, and scale. This release is from the completed Phase
A delta-scaling line; it is **not** the later failed NEXTGEN route and does not
include subsequent protocol-repair experiments.

## Loading with PEFT

Use recent `transformers` and `peft` versions. Load the fixed base first, then
attach this adapter:

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-4B-Base"
base_revision = "906bfd4b4dc7f14ee4320094d8b41684abff8539"
adapter_id = "modrill/Qwen3-4B-Base-ThinkCode-A-NH025"

tokenizer = AutoTokenizer.from_pretrained(base_id, revision=base_revision)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    revision=base_revision,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

messages = [{"role": "user", "content": "Write a Python function that checks whether a number is prime."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
eos_ids = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|im_end|>"),
]
outputs = model.generate(**inputs, max_new_tokens=2048, eos_token_id=eos_ids)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))
```

The base tokenizer's chat template supports `enable_thinking`. Disable it for
direct code generation matching the concise screening style, or enable it when
explicit reasoning is desired. Pass both `<|endoftext|>` and `<|im_end|>` as
EOS IDs. Keep the combined prompt and generated sequence within **32K tokens**,
the fixed base model configuration limit, unless a separate long-context
extension is validated.

## Development evaluation

Across three preregistered seeds on the corrected EvalScope Full1055
development suite, the resolved `code_only` aggregate was
**794/3165 = 25.09%**. Relative to the fixed BASE, the estimated change was
approximately **+0.98 percentage points**, with an approximate 95% confidence
interval of **[+0.095, +1.833] percentage points**. The Holm-adjusted
**p-value was 0.489**.

These results are development-only, not a held-out formal claim. In the
original bidirectional scoring for `seed=3407`, some outcomes flipped between
PASS and TLE because of the execution environment. Those cases were resolved
by fixed single-CPU serial rejudgment, which does not eliminate all scorer,
timing, or environment uncertainty.

## Limitations

- This adapter requires the exact base model and should not be loaded alone.
- The evidence is development-only and includes scorer-environment uncertainty.
- Generated code can be incorrect, insecure, or non-compiling; sandbox and
  test it independently.
- No production safety, security, or suitability certification is implied.

## License

The fixed base card and included license identify Apache-2.0. This adapter
preserves that license text and metadata. Users should independently verify the
upstream Qwen3 license, notices, training-data terms, and applicability to their
use case.