lora_structeval_t_qwen3_4b-55

LoRA adapter fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using QLoRA (4-bit).

⚠️ This repository contains adapter weights only. The base model must be downloaded separately.

Model Overview

• Base model: Qwen/Qwen3-4B-Instruct-2507
• Fine-tuning method: QLoRA (4-bit quantization)
• Framework: PEFT
• Training date: 2026-02-21
• Objective: Improve structured output generation accuracy

This adapter is optimized for producing:

• JSON
• YAML
• XML
• TOML
• CSV

Loss is applied only to the final structured answer. The training data includes Chain-of-Thought reasoning, but those reasoning tokens are excluded from loss computation so the adapter learns to produce correct output regardless of whether CoT is generated at inference time.

Training Configuration

Parameter	Value
Max Sequence Length	512
Epochs	2
Learning Rate	1e-05
LoRA r	32
LoRA alpha	64
Quantization	4-bit

Intended Use

Recommended:

• Generating structured data (JSON, YAML, XML, TOML, CSV) from natural language instructions
• Format conversion between structured formats
• Data extraction tasks where output must follow a strict schema

Not recommended:

• General-purpose chat or creative writing
• Tasks requiring outputs longer than 512 tokens
• Safety-critical or regulated applications without further validation

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-4B-Instruct-2507"
adapter_model = "hirosan6595/lora_structeval_t_qwen3_4b-55"

tokenizer = AutoTokenizer.from_pretrained(base_model)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
)

model = PeftModel.from_pretrained(model, adapter_model)

# --- Generate Example ---
prompt = "Convert the following to JSON: name is Alice, age is 30, city is Tokyo."

messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Tip: For best results, give a clear instruction specifying the desired output format (e.g. "Output valid JSON only.").

Dataset

Training dataset: https://huggingface.co/datasets/u-10bei/structured_data_with_cot_dataset_512_v2

Dataset License: MIT License

Base Model Terms

This adapter inherits all usage restrictions of the base model. Please review the base model license before deployment: https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Limitations

• May still produce invalid structured output in long generations (>512 tokens)
• Not optimized for multilingual generation
• Performance depends on prompt format and specificity

Citation

@misc{lora_structeval_t_qwen3_4b-55,
  author = {hirosan6595},
  title  = {lora_structeval_t_qwen3_4b-55},
  year   = 2026,
  publisher = {Hugging Face},
  url    = {https://huggingface.co/hirosan6595/lora_structeval_t_qwen3_4b-55},
}

License

Repository License: apache-2.0

This license applies to adapter weights only.