takatuki56/2026-comp-model-v10

Qwen3-4B-StructEval-L4-Mix

This repository provides a LoRA adapter fine-tuned from unsloth/Qwen3-4B-Instruct-2507 using QLoRA (4-bit, Unsloth).

This repository contains LoRA adapter weights only. The base model must be loaded separately.

Training Objective

This adapter is trained to improve structured output accuracy (JSON / YAML / XML / TOML / CSV).

Loss is applied only to the final assistant output, while intermediate reasoning (Chain-of-Thought) is masked.

Training Configuration

• Base model: unsloth/Qwen3-4B-Instruct-2507
• Method: QLoRA (4-bit)
• Max sequence length: 4096 (Optimized for L4)
• Epochs: 2
• Learning rate: 1e-06
• LoRA: r=256, alpha=512
• Precision: bf16 (Bfloat16)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "unsloth/Qwen3-4B-Instruct-2507"
adapter = "takatuki56/2026-comp-model-v10"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.bfloat16, # Use bfloat16 for L4/A100
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Sources & Terms (IMPORTANT)

Training Data (Combined):

• u-10bei/structured_data_with_cot_dataset_512_v2

Dataset License: Please refer to the original repositories for specific license terms. Generally, these datasets are used and distributed under terms compliant with their original licenses (e.g., MIT, CC-BY).

Compliance: Users must comply with the licenses of the datasets and the base model's original terms of use.