Meerkat-TRIZ-v1 (Qwen3.8-27B)

LoRA adapter fine-tuned on TRIZ innovation methodology dataset.

Base Model

• Base: Qwen3.8-27B
• Architecture: Qwen3_5ForConditionalGeneration
• Precision: BF16

Adapter Config

• PEFT Type: LoRA
• Rank (r): 64
• Alpha: 128
• Dropout: 0.0
• Target Modules: q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_proj
• Trainable Params: 466,911,232 (1.7064% of base)

Training

• Framework: TRL SFTTrainer
• Dataset: v5a (11,096 train / 1,050 val)
• Max Length: 2048
• Horizon: 2,774 steps (cosine decay)
• Best Eval Loss: 1.505834 @ step 1300
• Early Stopping: patience=3, threshold=0.002

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "Meerkat-AI/Meerkat-TRIZ-v1-Qwen3.8-27B")
tokenizer = AutoTokenizer.from_pretrained("Meerkat-AI/Meerkat-TRIZ-v1-Qwen3.8-27B")

Related Models

• Meerkat-TRIZ-v1-Qwen3.6-35B-A3B ← Original v1 (older base)
• Meerkat-TRIZ Collection ← All variants