skyyuno/Qweenseer-2B-GGUF

Model Card: Qwenseer-2B

Model Overview

Model Name: Qwenseer-2B
Base Model: Qwen/Qwen3.5-2B
Model Type: Causal Language Model (Text-Vision Base)
Architecture: Qwen 3.5 (2B parameters)
Training Method: Supervised Fine-Tuning (SFT) with LoRA Adapters

---

Model Description

This model is a lightweight, procedural reasoning engine fine-tuned to optimize step-by-step arithmetic computation, state-tracking logic execution, multi-variable register mutations, recursive stack tracing, and Python code tracing.

It incorporates a controlled scratchpad format using [THINK]...[/THINK] for reasoning and [SOLUTION]...[/SOLUTION] for final outputs, with explicit constraints applied during fine-tuning to prevent runaway overthinking loops (>2 correction cycles).

The model is designed for tasks requiring:

• Multi-step arithmetic calculation and sign-flip tracking
• Procedural code execution tracing
• State machine & variable mutation tracking
• Linear graph reachability & flow evaluation
• Fast local inference on lightweight consumer hardware

---

Model Details

Attribute	Value
Architecture	Qwen 3.5 (2B) with LoRA adapters
Context Length	2048 tokens
Training Data	GLM-5.2 Logic Puzzles + Synthetic Python Execution Traces
LoRA Rank ($r$)	16
LoRA Alpha ($\alpha$)	16
LoRA Dropout	0.0
Optimizer	AdamW 8-bit
Learning Rate	2e-4 (0.0002)
Batch Size	2
Warmup Steps	5
Training Steps	100 steps (0 epochs)

---

Training Hardware & Setup

🖥️ Hardware Setup

• GPU: 1x NVIDIA GeForce RTX 3060 (12GB VRAM)
• Quantization: 4-bit loading (QLoRA training)
• Inference Speed: ~121.60 t/s (GGUF Q4_K_M)

---

Usage

Default System Prompt (Critical)

# HOW YOU SHOULD THINK AND ANSWER

First draft your thinking process (inner monologue) until you arrive at a response. Do NOT overthink, immediately stop if you are checking your answer more than 2 times. Format your response using Markdown, and use LaTeX for any mathematical equations. 

Your thinking process must follow the template below:
[THINK]Your concise thoughts, like working through an exercise on scratch paper. Be as casual as you want until you are confident to generate the response to the user.[/THINK]
Here, provide a self-contained response.
Put your final short answer inside [SOLUTION] and [/SOLUTION]

Recommended Sampling Parameters

Parameter	Value
temperature	0.3
top_k	40
top_p	0.9
repetition_penalty	1.1
max_new_tokens	2048

---

📊 Capabilities & Benchmark Profile

Category	Task Type	Performance	Notes
High Precision 🟢	Python Execution Tracing	Strong	Accurately tracks loop counters, conditionals, and variable state updates.
	Multi-Step Arithmetic	Strong	Exceptional stability across sign flips, zero-crossings, and fraction evaluation.
	Linear Path Reachability	Strong	Reliably identifies disconnected graph components and linear edge sequences.
Moderate 🟡	Recursive Stack Theory	Medium	Executes recursive arithmetic well; can over-verify theoretical terminology.
Structural Bounds 🔴	Dense Dijkstra Graphing	Limited	Prone to priority queue tracking errors on dense, undirected multi-branch graphs.
	Set-Theory Shortcuts	Limited	Vulnerable to misapplying shortcut formulas (e.g., inclusion-exclusion). Prefer step-by-step loops.
	String Mutation	Limited	Vulnerable to sub-word BPE tokenization limits during character-level index manipulations.

---

When to Use This Model

Use Case	Recommended
Multi-step arithmetic & numerical logic	✅ Yes
Python loop & function tracing	✅ Yes
State machine & register updates	✅ Yes
Graph connectivity & reachability	✅ Yes
Fast local reasoning (~120 t/s)	✅ Yes
Full Dijkstra priority queue relaxation	⚠️ Limited (use code execution)
Destructive character-level string popping	⚠️ Limited (use code execution)

---

Considerations

• This model is intended for lightweight algorithmic tracing and procedural logic. It should not be used for:
• Medical advice
• Financial decisions
• Legal counsel
• Safety-critical code generation without human review

---

Uploaded Fine-Tuned Model

• Model Name: Qwenseer-2B
• License: apache-2.0
• Fine-Tuned From: Qwen/Qwen3.5-2B

This model was trained with Unsloth and Huggingface's TRL library.

---